public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v21 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
70+ messages / 4 participants
[nested] [flat]
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v32 04/11] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonably complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 2 +-
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/expected/tablespace.out | 4 ++--
5 files changed, 18 insertions(+), 17 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 7948e6ec06c..7589ef7ac8a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -27427,16 +27427,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 0728547ec2b..d6be8cef6e3 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -709,7 +709,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c99e1633bbf..fa293eca393 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11615,13 +11615,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '9858',
descr => 'list of files in the pg_logical/snapshots directory',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index b3a9d11b5c0..a2bec47d76c 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -270,8 +270,8 @@ select * from pg_stat_file('.') limit 0;
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select filename, isdir from pg_ls_dir_metadata('.') where filename='.';
diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out
index 21f7f19b67f..7d8398b0ca1 100644
--- a/src/test/regress/expected/tablespace.out
+++ b/src/test/regress/expected/tablespace.out
@@ -20,8 +20,8 @@ CREATE TABLESPACE regress_tblspace LOCATION :'testtablespace';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--Bne5rrxQd65beI7a
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v32-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v30 04/11] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonably complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 2 +-
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 18 insertions(+), 17 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index dd9b1afde4..2adc8f7d83 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26936,16 +26936,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 8efb798b2f..243e86a2a3 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -708,7 +708,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b405aa341c..0a4caf8aed 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11539,13 +11539,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '8450', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index b4112ec298..0d3e88fccf 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -241,8 +241,8 @@ select * from pg_stat_file('.') limit 0;
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select filename, isdir from pg_ls_dir_metadata('.') where filename='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 5f7ce3d09f..4183a77b23 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--ZwgA9U+XZDXt4+m+
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v30-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v26 05/11] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonably complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 2 +-
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 18 insertions(+), 17 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 79d30a1568..c68c9d548b 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26491,16 +26491,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 198a99391a..da7d4a7c11 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -706,7 +706,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 17b03d1abd..d2594c2e45 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11258,13 +11258,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '9979', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index fd7c3c791f..7279c83d89 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -228,8 +228,8 @@ select * from pg_stat_file('.') limit 0;
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select filename, isdir from pg_ls_dir_metadata('.') where filename='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--yKkOmjQZXRsvHRX8
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v26-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v19 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 2931f9d7d5..24dc90a74d 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25813,16 +25813,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 5b0147ee2d..08c7189470 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10894,13 +10894,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '9979', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--S0GG+JvAI2G0KxBG
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v19-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v34 04/15] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonably complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 2 +-
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 8 ++++----
src/test/regress/expected/tablespace.out | 4 ++--
5 files changed, 20 insertions(+), 19 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c2ff924d021..6697ab14b14 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -27550,16 +27550,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 1878bac7eb3..1710ebe489b 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -668,7 +668,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 43ee0cdc454..19d062a36dc 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11647,13 +11647,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '9858',
descr => 'list of files in the pg_logical/snapshots directory',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index d18f401248a..6c64cd6f998 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -438,8 +438,8 @@ select * from pg_ls_replslotdir('') limit 0;
(0 rows)
select * from pg_ls_tmpdir() limit 0;
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select * from pg_ls_waldir() limit 0;
@@ -455,8 +455,8 @@ select * from pg_stat_file('.') limit 0;
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select filename, isdir from pg_ls_dir_metadata('.') where filename='.';
diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out
index c6b9074ab02..20a7245d5e1 100644
--- a/src/test/regress/expected/tablespace.out
+++ b/src/test/regress/expected/tablespace.out
@@ -28,8 +28,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.1
--smOfPzt+Qjm5bNGJ
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v34-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v25 05/11] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonably complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 226923a134..4fb4ec9f33 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25829,16 +25829,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 7351615f8a..da7d4a7c11 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -517,7 +517,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if MISSING_OK.
* Other unreadable-directory cases throw an error.
@@ -706,7 +706,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 8e88c549d9..a0148e6130 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10968,13 +10968,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '9979', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index fd7c3c791f..7279c83d89 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -228,8 +228,8 @@ select * from pg_stat_file('.') limit 0;
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select filename, isdir from pg_ls_dir_metadata('.') where filename='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--SBikYMzjhZGK9d4p
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v20 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index eead88419f..3c419672fc 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25843,16 +25843,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 3324ffe16a..8390626e49 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -517,7 +517,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -706,7 +706,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 186ced4a35..381f2e196f 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10911,13 +10911,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '9979', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--Z1Z8UV8BNhgCynIS
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v20-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v35 4/7] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonably complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 2 +-
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 8 ++++----
src/test/regress/expected/tablespace.out | 4 ++--
5 files changed, 20 insertions(+), 19 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index d2a455a3e27..3931411c01f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -27561,16 +27561,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 75b7bf99849..3775cae225d 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 7307f44f371..142b4b8d37d 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11739,13 +11739,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '9858',
descr => 'list of files in the pg_logical/snapshots directory',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index e54e38f54ad..77bf661627b 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -461,8 +461,8 @@ select * from pg_ls_replslotdir('') limit 0;
(0 rows)
select * from pg_ls_tmpdir() limit 0;
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select * from pg_ls_waldir() limit 0;
@@ -478,8 +478,8 @@ select * from pg_stat_file('.') limit 0;
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select filename, isdir from pg_ls_dir_metadata('.') where filename='.';
diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out
index 8159c9f18f1..ce5d6f73e01 100644
--- a/src/test/regress/expected/tablespace.out
+++ b/src/test/regress/expected/tablespace.out
@@ -37,8 +37,8 @@ SELECT regexp_replace(pg_tablespace_location(oid), '(pg_tblspc)/(\d+)', '\1/NNN'
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.1
--olLTNZSltDMg5Vbm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v35-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v33 04/11] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonably complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 2 +-
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 8 ++++----
src/test/regress/expected/tablespace.out | 4 ++--
5 files changed, 20 insertions(+), 19 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 14b8fba3123..e0b0a5ab121 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -27495,16 +27495,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 3921723fda6..4829b34f0f6 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -710,7 +710,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b58987b713c..95746fc0dcf 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11631,13 +11631,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '9858',
descr => 'list of files in the pg_logical/snapshots directory',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index a627839bc28..0305f038369 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -295,8 +295,8 @@ select * from pg_ls_replslotdir('') limit 0;
(0 rows)
select * from pg_ls_tmpdir() limit 0;
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select * from pg_ls_waldir() limit 0;
@@ -312,8 +312,8 @@ select * from pg_stat_file('.') limit 0;
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select filename, isdir from pg_ls_dir_metadata('.') where filename='.';
diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out
index c6b9074ab02..20a7245d5e1 100644
--- a/src/test/regress/expected/tablespace.out
+++ b/src/test/regress/expected/tablespace.out
@@ -28,8 +28,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.1
--9CzcV6dAFIr7O1Ie
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v37 04/11] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonably complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 2 +-
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 8 ++++----
src/test/regress/expected/tablespace.out | 4 ++--
5 files changed, 20 insertions(+), 19 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index a80d794a43a..cff999032bf 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -27630,16 +27630,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index c041c7630c8..5d374177a04 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -740,7 +740,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 89892647808..e2f3361bf37 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11728,13 +11728,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '6270',
descr => 'list of files in the pg_logical/snapshots directory',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index a656f9ad07e..bc77ad62958 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -525,8 +525,8 @@ select * from pg_ls_replslotdir('') limit 0;
(0 rows)
select * from pg_ls_tmpdir() limit 0;
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select * from pg_ls_waldir() limit 0;
@@ -542,8 +542,8 @@ select * from pg_stat_file('.') limit 0;
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select filename, isdir from pg_ls_dir_metadata('.') where filename='.';
diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out
index 8159c9f18f1..ce5d6f73e01 100644
--- a/src/test/regress/expected/tablespace.out
+++ b/src/test/regress/expected/tablespace.out
@@ -37,8 +37,8 @@ SELECT regexp_replace(pg_tablespace_location(oid), '(pg_tblspc)/(\d+)', '\1/NNN'
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.25.1
--Pk/CTwBz1VvfPIDp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v37-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v24 05/11] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonably complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 5e267c4f8d..dcd0aa84d2 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25836,16 +25836,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 7351615f8a..da7d4a7c11 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -517,7 +517,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if MISSING_OK.
* Other unreadable-directory cases throw an error.
@@ -706,7 +706,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f35a684dcf..e2aea6f48b 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10967,13 +10967,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '9979', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index fd7c3c791f..7279c83d89 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -228,8 +228,8 @@ select * from pg_stat_file('.') limit 0;
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select filename, isdir from pg_ls_dir_metadata('.') where filename='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--mPTHnM80CEnHQ2WJ
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v22 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 34c92903d8..4308a50ab2 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25849,16 +25849,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 054b8d4b1a..454348a5e3 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -517,7 +517,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -706,7 +706,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 02b1436d83..149dfacf4b 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10949,13 +10949,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '9147', descr => 'check pages of a relation',
proname => 'pg_relation_check_pages', procost => '10000', prorows => '20',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 20bfe71822..0144f611c2 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select filename, isdir from pg_ls_dir_metadata('.') where filename='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--d6Gm4EdcadzBjdND
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v22-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v36 4/7] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonably complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 2 +-
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 8 ++++----
src/test/regress/expected/tablespace.out | 4 ++--
5 files changed, 20 insertions(+), 19 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index ba30faa8441..8767ef0d7a2 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -28591,16 +28591,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index c7322ed405a..eb74130e721 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 0e5c977e69f..381eb0e277f 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11785,13 +11785,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '6270',
descr => 'list of files in the pg_logical/snapshots directory',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index e54e38f54ad..77bf661627b 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -461,8 +461,8 @@ select * from pg_ls_replslotdir('') limit 0;
(0 rows)
select * from pg_ls_tmpdir() limit 0;
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select * from pg_ls_waldir() limit 0;
@@ -478,8 +478,8 @@ select * from pg_stat_file('.') limit 0;
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select filename, isdir from pg_ls_dir_metadata('.') where filename='.';
diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out
index 8159c9f18f1..ce5d6f73e01 100644
--- a/src/test/regress/expected/tablespace.out
+++ b/src/test/regress/expected/tablespace.out
@@ -37,8 +37,8 @@ SELECT regexp_replace(pg_tablespace_location(oid), '(pg_tblspc)/(\d+)', '\1/NNN'
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.1
--4ybNbZnZ8tziJ7D6
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v36-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v27 04/11] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonably complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 2 +-
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 18 insertions(+), 17 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 76ae2f2dd8..09a5b5b48a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26922,16 +26922,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index f59f853983..a657d8d431 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -709,7 +709,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b85bee2d1e..10ffab88c5 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11509,13 +11509,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '8450', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index b4112ec298..0d3e88fccf 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -241,8 +241,8 @@ select * from pg_stat_file('.') limit 0;
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select filename, isdir from pg_ls_dir_metadata('.') where filename='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 5f7ce3d09f..4183a77b23 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--19uQFt6ulqmgNgg1
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v23 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 0a924fdcc3..ee23bb763c 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25840,16 +25840,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 6f003bbf67..ec016149b6 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -517,7 +517,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -706,7 +706,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index add3c7dc62..cffcd03634 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10963,13 +10963,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '9979', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 20bfe71822..0144f611c2 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select filename, isdir from pg_ls_dir_metadata('.') where filename='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--mhjHhnbe5PrRcwjY
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v23-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v21 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index eead88419f..3c419672fc 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25843,16 +25843,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 24d858d1bc..cd7f052692 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -517,7 +517,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -706,7 +706,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 83927b510a..85ac8fef61 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10911,13 +10911,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '9979', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 20bfe71822..0144f611c2 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select filename, isdir from pg_ls_dir_metadata('.') where filename='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--Tcb1KvpfnM4LxW2s
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v21-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v18 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8ea064144f..33f7137e43 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 5b0147ee2d..08c7189470 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10894,13 +10894,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '9979', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--5I6of5zJg18YgZEa
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v18-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v31 04/11] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonably complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 2 +-
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 18 insertions(+), 17 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index e0099e77df..9ddd23f90e 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -27342,16 +27342,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 0728547ec2..d6be8cef6e 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -709,7 +709,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 39fbaacbc9..3ffdcaf3a7 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11615,13 +11615,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '9858',
descr => 'list of files in the pg_logical/snapshots directory',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index b3a9d11b5c..a2bec47d76 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -270,8 +270,8 @@ select * from pg_stat_file('.') limit 0;
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select filename, isdir from pg_ls_dir_metadata('.') where filename='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index c2f6d64c5c..c146a4c129 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--qZVVwWJgpX9Jzs7f
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v31-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v28 04/11] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonably complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 2 +-
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 18 insertions(+), 17 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 507a6d73f8..940c653807 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26950,16 +26950,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return the file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index f59f853983..a657d8d431 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -709,7 +709,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 550fbf734a..441cbb2cde 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11526,13 +11526,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '8450', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index b4112ec298..0d3e88fccf 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -241,8 +241,8 @@ select * from pg_stat_file('.') limit 0;
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select filename, isdir from pg_ls_dir_metadata('.') where filename='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 5f7ce3d09f..4183a77b23 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--vk/v8fjDPiDepTtA
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 22 +++++++++++---------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 +++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9d617f95b9..891b990859 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</entry>
<entry><type>setof record</type></entry>
<entry>
- List the name, size, and last modification time of files in the
- temporary directory for <parameter>tablespace</parameter>. If
- <parameter>tablespace</parameter> is not provided, the
- <literal>pg_default</literal> tablespace is used. Access is granted
- to members of the <literal>pg_monitor</literal> role and may be
- granted to other non-superuser roles.
+ For the temporary directory within <parameter>tablespace</parameter>,
+ list each file's name, size, last modification time, and a boolean
+ indicating if it is a directory. If <parameter>tablespace</parameter>
+ is not provided, the <literal>pg_default</literal> tablespace is used.
+ Access is granted to members of the <literal>pg_monitor</literal> role
+ and may be granted to other non-superuser roles.
</entry>
</row>
<row>
@@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<primary>pg_ls_tmpdir</primary>
</indexterm>
<para>
- <function>pg_ls_tmpdir</function> returns the name, size, and last modified
- time (mtime) of each file in the temporary file directory for the specified
- <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is
+ <function>pg_ls_tmpdir</function> lists each file in the temporary file
+ directory for the specified <parameter>tablespace</parameter>, along with
+ its size, last modified time (mtime) and a boolean indicating if the file is a
+ directory. Directories are used for temporary files shared by parallel
+ processes. If <parameter>tablespace</parameter> is
not provided, the <literal>pg_default</literal> tablespace is used. By
default only superusers and members of the <literal>pg_monitor</literal>
role can use this function. Access may be granted to others using
<command>GRANT</command>.
- Filenames beginning with a dot, directories, and other special files are not shown.
+ Filenames beginning with a dot and special file types are not shown.
</para>
<indexterm>
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument..
@ 2020-03-09 03:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw)
similar to pg_stat_file().
It's worth breaking the function's return type, since core postgres creates
"shared filesets" underneath the temp dirs, and it's unreasonable to not show
them here, and the alternative query to show them is unreasaonbly complicated.
See following commit which also adds these columns to the other pg_ls_*
functions. Although I don't think it matters that they're easily UNIONed, it'd
still make great sense if they returned the same columns.
Need catversion bump
---
doc/src/sgml/func.sgml | 17 +++++++++--------
src/backend/utils/adt/genfile.c | 4 ++--
src/include/catalog/pg_proc.dat | 8 ++++----
src/test/regress/expected/misc_functions.out | 4 ++--
src/test/regress/output/tablespace.source | 4 ++--
5 files changed, 19 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc1b4ac98c..c9d1857182 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>size</parameter> <type>bigint</type>,
- <parameter>modification</parameter> <type>timestamp with time zone</type> )
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
</para>
<para>
- Returns the name, size, and last modification time (mtime) of each
- ordinary file in the temporary file directory for the
- specified <parameter>tablespace</parameter>.
- If <parameter>tablespace</parameter> is not provided,
- the <literal>pg_default</literal> tablespace is examined. Filenames
- beginning with a dot, directories, and other special files are
- excluded.
+ For each file in the temporary directory within the given
+ <parameter>tablespace</parameter>, return each file's name, size, last
+ modification time (mtime) and a boolean indicating if the file is a directory.
+ Directories are used for temporary files shared by parallel processes.
+ If <parameter>tablespace</parameter> is not provided, the
+ <literal>pg_default</literal> tablespace is examined.
+ Filenames beginning with a dot and special file types are excluded.
</para>
<para>
This function is restricted to superusers and members of
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 4824a55480..023f1ad605 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS)
}
/*
- * Generic function to return a directory listing of files.
+ * Generic function to return a directory listing of files (and optionally dirs).
*
* If the directory isn't there, silently return an empty set if missing_ok.
* Other unreadable-directory cases throw an error.
@@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc)
TempTablespacePath(path, tblspc);
return pg_ls_dir_files(fcinfo, path,
- LS_DIR_HISTORIC | LS_DIR_MISSING_OK);
+ LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK);
}
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f46cba5ed..dc3b497587 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10896,13 +10896,13 @@
{ oid => '5029', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}',
- proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' },
+ proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}',
+ proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' },
{ oid => '5030', descr => 'list files in the pgsql_tmp directory',
proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => 'oid',
- proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}',
- proargnames => '{tablespace,name,size,modification}',
+ proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
+ proargnames => '{tablespace,name,size,modification,isdir}',
prosrc => 'pg_ls_tmpdir_1arg' },
{ oid => '5032', descr => 'list directory with metadata',
proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7930909f02..38493de732 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory
-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
select name, isdir from pg_ls_dir_metadata('.') where name='.';
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index a42714bf40..1c88e914e3 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
- name | size | modification
-------+------+--------------
+ name | size | modification | isdir
+------+------+--------------+-------
(0 rows)
-- try setting and resetting some properties for the new tablespace
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Allow pg_archivecleanup to remove backup history files
@ 2023-06-13 15:49 torikoshia <[email protected]>
2023-06-15 06:20 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
0 siblings, 1 reply; 70+ messages in thread
From: torikoshia @ 2023-06-13 15:49 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Fujii Masao <[email protected]>; Kyotaro Horiguchi <[email protected]>; Bharath Rupireddy <[email protected]>; pgsql-hackers
On 2023-06-12 16:33, Michael Paquier wrote:
> On Fri, Jun 09, 2023 at 12:32:15AM +0900, Fujii Masao wrote:
Thanks for reviewing!
>> printf(_(" -n, --dry-run dry run, show the names of
>> the files that would be removed\n"));
>> + printf(_(" -b, --clean-backup-history clean up files including
>> backup history files\n"));
>>
>> Shouldn't -b option be placed in alphabetical order?
>
> +1.
Modified the place.
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v6-0001-Introduce-pg_archivecleanup-into-getopt_long.patch (3.9K, ../../[email protected]/2-v6-0001-Introduce-pg_archivecleanup-into-getopt_long.patch)
download | inline diff:
From 06b5664d1093aba445332ad5420bd442b99a8f98 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Tue, 14 Jun 2023 00:30:37 +0900
Subject: [PATCH v5 1/3] Introduce pg_archivecleanup into getopt_long
This patch is a preliminary step to add an easy-to-understand option
to delete backup history files, but it also adds long options to
the existing options.
---
doc/src/sgml/ref/pgarchivecleanup.sgml | 5 ++++-
src/bin/pg_archivecleanup/pg_archivecleanup.c | 20 ++++++++++++-------
2 files changed, 17 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/ref/pgarchivecleanup.sgml b/doc/src/sgml/ref/pgarchivecleanup.sgml
index 635e7c7685..09991c2fcd 100644
--- a/doc/src/sgml/ref/pgarchivecleanup.sgml
+++ b/doc/src/sgml/ref/pgarchivecleanup.sgml
@@ -95,6 +95,7 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
<varlistentry>
<term><option>-d</option></term>
+ <term><option>--debug</option></term>
<listitem>
<para>
Print lots of debug logging output on <filename>stderr</filename>.
@@ -104,6 +105,7 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
<varlistentry>
<term><option>-n</option></term>
+ <term><option>--dry-run</option></term>
<listitem>
<para>
Print the names of the files that would have been removed on <filename>stdout</filename> (performs a dry run).
@@ -122,7 +124,8 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
</varlistentry>
<varlistentry>
- <term><option>-x</option> <replaceable>extension</replaceable></term>
+ <term><option>-x <replaceable class="parameter">extension</replaceable></option></term>
+ <term><option>--strip-extension=<replaceable class="parameter">extension</replaceable></option></term>
<listitem>
<para>
Provide an extension
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index 7726d05149..be62ffe75d 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -17,7 +17,7 @@
#include "access/xlog_internal.h"
#include "common/logging.h"
-#include "pg_getopt.h"
+#include "getopt_long.h"
const char *progname;
@@ -252,11 +252,11 @@ usage(void)
printf(_("Usage:\n"));
printf(_(" %s [OPTION]... ARCHIVELOCATION OLDESTKEPTWALFILE\n"), progname);
printf(_("\nOptions:\n"));
- printf(_(" -d generate debug output (verbose mode)\n"));
- printf(_(" -n dry run, show the names of the files that would be removed\n"));
- printf(_(" -V, --version output version information, then exit\n"));
- printf(_(" -x EXT clean up files if they have this extension\n"));
- printf(_(" -?, --help show this help, then exit\n"));
+ printf(_(" -d, --debug generate debug output (verbose mode)\n"));
+ printf(_(" -n, --dry-run dry run, show the names of the files that would be removed\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -x, --strip-extension=EXT strip this extention before identifying files fo clean up\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"For use as archive_cleanup_command in postgresql.conf:\n"
" archive_cleanup_command = 'pg_archivecleanup [OPTION]... ARCHIVELOCATION %%r'\n"
@@ -274,6 +274,12 @@ usage(void)
int
main(int argc, char **argv)
{
+ static struct option long_options[] = {
+ {"debug", no_argument, NULL, 'd'},
+ {"dry-run", no_argument, NULL, 'n'},
+ {"strip-extension", required_argument, NULL, 'x'},
+ {NULL, 0, NULL, 0}
+ };
int c;
pg_logging_init(argv[0]);
@@ -294,7 +300,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt(argc, argv, "dnx:")) != -1)
+ while ((c = getopt_long(argc, argv, "dnx:", long_options, NULL)) != -1)
{
switch (c)
{
--
2.39.2
[text/x-diff] v6-0002-Allow-pg_archivecleanup-to-remove-backup-history-.patch (10.8K, ../../[email protected]/3-v6-0002-Allow-pg_archivecleanup-to-remove-backup-history-.patch)
download | inline diff:
From ef234fe06f32b0b3b0b3d54cefc78ee7fbf40328 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Tue, 14 Jun 2023 00:31:39 +0900
Subject: [PATCH v5 2/3] Allow pg_archivecleanup to remove backup history files
Backup history files are just few bytes, but it can be noisy for the
eye to see a large accumulation of history files mixed with the WAL
segments.
This patch adds a new option to remove files including backup
history files older oldestkeptwalfile.
---
doc/src/sgml/ref/pgarchivecleanup.sgml | 11 +++
src/bin/pg_archivecleanup/pg_archivecleanup.c | 85 ++++++++++++-------
.../t/010_pg_archivecleanup.pl | 79 +++++++++++------
3 files changed, 117 insertions(+), 58 deletions(-)
diff --git a/doc/src/sgml/ref/pgarchivecleanup.sgml b/doc/src/sgml/ref/pgarchivecleanup.sgml
index 09991c2fcd..8fac233db6 100644
--- a/doc/src/sgml/ref/pgarchivecleanup.sgml
+++ b/doc/src/sgml/ref/pgarchivecleanup.sgml
@@ -113,6 +113,17 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>-b</option></term>
+ <term><option>--clean-backup-history</option></term>
+ <listitem>
+ <para>
+ Remove backup history files.
+ For details about backup history file, please refer to the <xref linkend="backup-base-backup"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-V</option></term>
<term><option>--version</option></term>
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index be62ffe75d..a7f77d9158 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -23,6 +23,8 @@ const char *progname;
/* Options and defaults */
bool dryrun = false; /* are we performing a dry-run operation? */
+bool cleanBackupHistory = false; /* remove files including
+ * backup history files */
char *additional_ext = NULL; /* Extension to remove from filenames */
char *archiveLocation; /* where to find the archive? */
@@ -97,15 +99,31 @@ CleanupPriorWALFiles(void)
{
while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
+ char WALFilePath[MAXPGPATH * 2]; /* the file path
+ * including archive */
/*
- * Truncation is essentially harmless, because we skip names of
- * length other than XLOG_FNAME_LEN. (In principle, one could use
- * a 1000-character additional_ext and get trouble.)
+ * Truncation is essentially harmless, because we check the file
+ * format including the length immediately after this.
+ * (In principle, one could use a 1000-character additional_ext
+ * and get trouble.)
*/
strlcpy(walfile, xlde->d_name, MAXPGPATH);
TrimExtension(walfile, additional_ext);
/*
+ * Check file name.
+ *
+ * We skip files which are not WAL file or partial WAL file.
+ * Also we skip backup history files when --clean-backup-history
+ * is not specified.
+ */
+ if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile) &&
+ !(cleanBackupHistory && IsBackupHistoryFileName(walfile)))
+ continue;
+
+ /*
+ * Check cutoff point.
+ *
* We ignore the timeline part of the XLOG segment identifiers in
* deciding whether a segment is still needed. This ensures that
* we won't prematurely remove a segment from a parent timeline.
@@ -118,39 +136,35 @@ CleanupPriorWALFiles(void)
* file. Note that this means files are not removed in the order
* they were originally written, in case this worries you.
*/
- if ((IsXLogFileName(walfile) || IsPartialXLogFileName(walfile)) &&
- strcmp(walfile + 8, exclusiveCleanupFileName + 8) < 0)
- {
- char WALFilePath[MAXPGPATH * 2]; /* the file path
- * including archive */
+ if (strcmp(walfile + 8, exclusiveCleanupFileName + 8) >= 0)
+ continue;
+ /*
+ * Use the original file name again now, including any
+ * extension that might have been chopped off before testing
+ * the sequence.
+ */
+ snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
+ archiveLocation, xlde->d_name);
+
+ if (dryrun)
+ {
/*
- * Use the original file name again now, including any
- * extension that might have been chopped off before testing
- * the sequence.
+ * Prints the name of the file to be removed and skips the
+ * actual removal. The regular printout is so that the
+ * user can pipe the output into some other program.
*/
- snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
- archiveLocation, xlde->d_name);
-
- if (dryrun)
- {
- /*
- * Prints the name of the file to be removed and skips the
- * actual removal. The regular printout is so that the
- * user can pipe the output into some other program.
- */
- printf("%s\n", WALFilePath);
- pg_log_debug("file \"%s\" would be removed", WALFilePath);
- continue;
- }
-
- pg_log_debug("removing file \"%s\"", WALFilePath);
-
- rc = unlink(WALFilePath);
- if (rc != 0)
- pg_fatal("could not remove file \"%s\": %m",
- WALFilePath);
+ printf("%s\n", WALFilePath);
+ pg_log_debug("file \"%s\" would be removed", WALFilePath);
+ continue;
}
+
+ pg_log_debug("removing file \"%s\"", WALFilePath);
+
+ rc = unlink(WALFilePath);
+ if (rc != 0)
+ pg_fatal("could not remove file \"%s\": %m",
+ WALFilePath);
}
if (errno)
@@ -254,6 +268,7 @@ usage(void)
printf(_("\nOptions:\n"));
+ printf(_(" -b, --clean-backup-history clean up files including backup history files\n"));
printf(_(" -d, --debug generate debug output (verbose mode)\n"));
printf(_(" -n, --dry-run dry run, show the names of the files that would be removed\n"));
printf(_(" -V, --version output version information, then exit\n"));
printf(_(" -x, --strip-extension=EXT strip this extention before identifying files fo clean up\n"));
printf(_(" -?, --help show this help, then exit\n"));
@@ -275,6 +290,7 @@ int
main(int argc, char **argv)
{
static struct option long_options[] = {
+ {"clean-backup-history", no_argument, NULL, 'b'},
{"debug", no_argument, NULL, 'd'},
{"dry-run", no_argument, NULL, 'n'},
{"strip-extension", required_argument, NULL, 'x'},
@@ -300,10 +316,13 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "dnx:", long_options, NULL)) != -1)
+ while ((c = getopt_long(argc, argv, "bdnx:", long_options, NULL)) != -1)
{
switch (c)
{
+ case 'b': /* Remove backup history files too */
+ cleanBackupHistory = true;
+ break;
case 'd': /* Debug mode */
pg_logging_increase_verbosity();
break;
diff --git a/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl b/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
index cc3386d146..030a82e7fa 100644
--- a/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
+++ b/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
@@ -12,13 +12,23 @@ program_options_handling_ok('pg_archivecleanup');
my $tempdir = PostgreSQL::Test::Utils::tempdir;
-my @walfiles = (
- '00000001000000370000000C.gz', '00000001000000370000000D',
- '00000001000000370000000E', '00000001000000370000000F.partial',);
+my @walfiles_with_gz = (
+ { name => '00000001000000370000000C.gz', present => 0 },
+ { name => '00000001000000370000000D', present => 0 },
+ { name => '00000001000000370000000E', present => 1 },
+ { name => '00000001000000370000000F.partial', present => 1 },
+ { name => 'unrelated_file', present => 1 });
+
+my @walfiles_with_backup = (
+ { name => '00000001000000370000000D', present => 0 },
+ { name => '00000001000000370000000D.00000028.backup', present => 0 },
+ { name => '00000001000000370000000E', present => 1 },
+ { name => '00000001000000370000000F.partial', present => 1 },
+ { name => 'unrelated_file', present => 1 });
sub create_files
{
- foreach my $fn (@walfiles, 'unrelated_file')
+ foreach my $fn (@_)
{
open my $file, '>', "$tempdir/$fn";
print $file 'CONTENT';
@@ -27,7 +37,18 @@ sub create_files
return;
}
-create_files();
+sub get_walfiles
+{
+ my @walfiles;
+
+ for my $walpair (@_)
+ {
+ push @walfiles, $walpair->{name};
+ }
+ return @walfiles;
+}
+
+create_files(get_walfiles(@walfiles_with_gz));
command_fails_like(
['pg_archivecleanup'],
@@ -59,14 +80,14 @@ command_fails_like(
my $stderr;
my $result =
IPC::Run::run [ 'pg_archivecleanup', '-d', '-n', $tempdir,
- $walfiles[2] ],
+ $walfiles_with_gz[2]->{name} ],
'2>', \$stderr;
ok($result, "pg_archivecleanup dry run: exit code 0");
like(
$stderr,
- qr/$walfiles[1].*would be removed/,
+ qr/$walfiles_with_gz[1]->{name}.*would be removed/,
"pg_archivecleanup dry run: matches");
- foreach my $fn (@walfiles)
+ foreach my $fn (get_walfiles(@walfiles_with_gz))
{
ok(-f "$tempdir/$fn", "$fn not removed");
}
@@ -76,32 +97,40 @@ sub run_check
{
local $Test::Builder::Level = $Test::Builder::Level + 1;
- my ($suffix, $test_name) = @_;
+ my ($testdata, $oldestkeptwalfile, $test_name, @options) = @_;
- create_files();
+ create_files(get_walfiles(@$testdata));
command_ok(
[
- 'pg_archivecleanup', '-x', '.gz', $tempdir,
- $walfiles[2] . $suffix
+ 'pg_archivecleanup', @options, $tempdir,
+ $oldestkeptwalfile
],
"$test_name: runs");
- ok(!-f "$tempdir/$walfiles[0]",
- "$test_name: first older WAL file was cleaned up");
- ok(!-f "$tempdir/$walfiles[1]",
- "$test_name: second older WAL file was cleaned up");
- ok(-f "$tempdir/$walfiles[2]",
- "$test_name: restartfile was not cleaned up");
- ok(-f "$tempdir/$walfiles[3]",
- "$test_name: newer WAL file was not cleaned up");
- ok(-f "$tempdir/unrelated_file",
- "$test_name: unrelated file was not cleaned up");
+ for my $walpair (@$testdata)
+ {
+ if ($walpair->{present})
+ {
+ ok(-f "$tempdir/$walpair->{name}",
+ "$test_name:$walpair->{name} was not cleaned up");
+ }
+ else
+ {
+ ok(!-f "$tempdir/$walpair->{name}",
+ "$test_name:$walpair->{name} was cleaned up");
+ }
+ }
return;
}
-run_check('', 'pg_archivecleanup');
-run_check('.partial', 'pg_archivecleanup with .partial file');
-run_check('.00000020.backup', 'pg_archivecleanup with .backup file');
+run_check(\@walfiles_with_gz, '00000001000000370000000E',
+ 'pg_archivecleanup', '-x.gz');
+run_check(\@walfiles_with_gz, '00000001000000370000000E.partial',
+ 'pg_archivecleanup with .partial file', '-x.gz');
+run_check(\@walfiles_with_gz, '00000001000000370000000E.00000020.backup',
+ 'pg_archivecleanup with .backup file', '-x.gz');
+run_check(\@walfiles_with_backup, '00000001000000370000000E',
+ 'pg_archivecleanup with --clean-backup-history', '-b');
done_testing();
--
2.39.2
[text/x-diff] v6-0003-Refactored-to-reduce-the-nesting-level.patch (5.9K, ../../[email protected]/4-v6-0003-Refactored-to-reduce-the-nesting-level.patch)
download | inline diff:
From bef376b2766923339dedbf3fae9e516262b433a1 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Tue, 14 Jun 2023 00:33:10 +0900
Subject: [PATCH v5 3/3] Refactored to reduce the nesting level.
---
src/bin/pg_archivecleanup/pg_archivecleanup.c | 146 +++++++++---------
1 file changed, 72 insertions(+), 74 deletions(-)
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index a7f77d9158..f475177edb 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -95,87 +95,85 @@ CleanupPriorWALFiles(void)
struct dirent *xlde;
char walfile[MAXPGPATH];
- if ((xldir = opendir(archiveLocation)) != NULL)
+ if ((xldir = opendir(archiveLocation)) == NULL)
+ pg_fatal("could not open archive location \"%s\": %m",
+ archiveLocation);
+
+ while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
- while (errno = 0, (xlde = readdir(xldir)) != NULL)
+ char WALFilePath[MAXPGPATH * 2]; /* the file path
+ * including archive */
+ /*
+ * Truncation is essentially harmless, because we check the file
+ * format including the length immediately after this.
+ * (In principle, one could use a 1000-character additional_ext
+ * and get trouble.)
+ */
+ strlcpy(walfile, xlde->d_name, MAXPGPATH);
+ TrimExtension(walfile, additional_ext);
+
+ /*
+ * Check file name.
+ *
+ * We skip files which are not WAL file or partial WAL file.
+ * Also we skip backup history files when --clean-backup-history
+ * is not specified.
+ */
+ if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile) &&
+ !(cleanBackupHistory && IsBackupHistoryFileName(walfile)))
+ continue;
+
+ /*
+ * Check cutoff point.
+ *
+ * We ignore the timeline part of the XLOG segment identifiers in
+ * deciding whether a segment is still needed. This ensures that
+ * we won't prematurely remove a segment from a parent timeline.
+ * We could probably be a little more proactive about removing
+ * segments of non-parent timelines, but that would be a whole lot
+ * more complicated.
+ *
+ * We use the alphanumeric sorting property of the filenames to
+ * decide which ones are earlier than the exclusiveCleanupFileName
+ * file. Note that this means files are not removed in the order
+ * they were originally written, in case this worries you.
+ */
+ if (strcmp(walfile + 8, exclusiveCleanupFileName + 8) >= 0)
+ continue;
+
+ /*
+ * Use the original file name again now, including any
+ * extension that might have been chopped off before testing
+ * the sequence.
+ */
+ snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
+ archiveLocation, xlde->d_name);
+
+ if (dryrun)
{
- char WALFilePath[MAXPGPATH * 2]; /* the file path
- * including archive */
- /*
- * Truncation is essentially harmless, because we check the file
- * format including the length immediately after this.
- * (In principle, one could use a 1000-character additional_ext
- * and get trouble.)
- */
- strlcpy(walfile, xlde->d_name, MAXPGPATH);
- TrimExtension(walfile, additional_ext);
-
/*
- * Check file name.
- *
- * We skip files which are not WAL file or partial WAL file.
- * Also we skip backup history files when --clean-backup-history
- * is not specified.
+ * Prints the name of the file to be removed and skips the
+ * actual removal. The regular printout is so that the
+ * user can pipe the output into some other program.
*/
- if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile) &&
- !(cleanBackupHistory && IsBackupHistoryFileName(walfile)))
- continue;
-
- /*
- * Check cutoff point.
- *
- * We ignore the timeline part of the XLOG segment identifiers in
- * deciding whether a segment is still needed. This ensures that
- * we won't prematurely remove a segment from a parent timeline.
- * We could probably be a little more proactive about removing
- * segments of non-parent timelines, but that would be a whole lot
- * more complicated.
- *
- * We use the alphanumeric sorting property of the filenames to
- * decide which ones are earlier than the exclusiveCleanupFileName
- * file. Note that this means files are not removed in the order
- * they were originally written, in case this worries you.
- */
- if (strcmp(walfile + 8, exclusiveCleanupFileName + 8) >= 0)
- continue;
-
- /*
- * Use the original file name again now, including any
- * extension that might have been chopped off before testing
- * the sequence.
- */
- snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
- archiveLocation, xlde->d_name);
-
- if (dryrun)
- {
- /*
- * Prints the name of the file to be removed and skips the
- * actual removal. The regular printout is so that the
- * user can pipe the output into some other program.
- */
- printf("%s\n", WALFilePath);
- pg_log_debug("file \"%s\" would be removed", WALFilePath);
- continue;
- }
-
- pg_log_debug("removing file \"%s\"", WALFilePath);
-
- rc = unlink(WALFilePath);
- if (rc != 0)
- pg_fatal("could not remove file \"%s\": %m",
- WALFilePath);
+ printf("%s\n", WALFilePath);
+ pg_log_debug("file \"%s\" would be removed", WALFilePath);
+ continue;
}
- if (errno)
- pg_fatal("could not read archive location \"%s\": %m",
- archiveLocation);
- if (closedir(xldir))
- pg_fatal("could not close archive location \"%s\": %m",
- archiveLocation);
+ pg_log_debug("removing file \"%s\"", WALFilePath);
+
+ rc = unlink(WALFilePath);
+ if (rc != 0)
+ pg_fatal("could not remove file \"%s\": %m",
+ WALFilePath);
}
- else
- pg_fatal("could not open archive location \"%s\": %m",
+
+ if (errno)
+ pg_fatal("could not read archive location \"%s\": %m",
+ archiveLocation);
+ if (closedir(xldir))
+ pg_fatal("could not close archive location \"%s\": %m",
archiveLocation);
}
--
2.39.2
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Allow pg_archivecleanup to remove backup history files
2023-06-13 15:49 Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
@ 2023-06-15 06:20 ` Kyotaro Horiguchi <[email protected]>
2023-06-15 12:38 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
0 siblings, 1 reply; 70+ messages in thread
From: Kyotaro Horiguchi @ 2023-06-15 06:20 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; pgsql-hackers
At Wed, 14 Jun 2023 00:49:39 +0900, torikoshia <[email protected]> wrote in
> On 2023-06-12 16:33, Michael Paquier wrote:
> > On Fri, Jun 09, 2023 at 12:32:15AM +0900, Fujii Masao wrote:
> Thanks for reviewing!
>
> >> printf(_(" -n, --dry-run dry run, show the names of the files that
> >> would be removed\n"));
> >> + printf(_(" -b, --clean-backup-history clean up files including
> >> backup history files\n"));
> >> Shouldn't -b option be placed in alphabetical order?
> > +1.
>
> Modified the place.
- printf(_(" -d generate debug output (verbose mode)\n"));
- printf(_(" -n dry run, show the names of the files that would be removed\n"));
- printf(_(" -V, --version output version information, then exit\n"));
- printf(_(" -x EXT clean up files if they have this extension\n"));
- printf(_(" -?, --help show this help, then exit\n"));
+ printf(_(" -d, --debug generate debug output (verbose mode)\n"));
+ printf(_(" -n, --dry-run dry run, show the names of the files that would be removed\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -x, --strip-extension=EXT strip this extention before identifying files fo clean up\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
After this change, some of these lines corss the boundary of the 80
columns width. (is that policy viable noadays? I am usually working
using terminal windows with such a width..) It's somewhat unrelated to
this patch, but a help line a few lines further down also exceeds the
width. We could shorten it by removing the "/mnt/server" portion, but
I'm not sure if it's worth doing.
> Or for use as a standalone archive cleaner:
> e.g.
> pg_archivecleanup /mnt/server/archiverdir 000000010000000000000010.00000020.backup
+ printf(_(" -x, --strip-extension=EXT strip this extention before identifying files fo clean up\n"));
s/fo/for/ ?
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Allow pg_archivecleanup to remove backup history files
2023-06-13 15:49 Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-15 06:20 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
@ 2023-06-15 12:38 ` torikoshia <[email protected]>
2023-06-16 02:22 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
0 siblings, 1 reply; 70+ messages in thread
From: torikoshia @ 2023-06-15 12:38 UTC (permalink / raw)
To: Kyotaro Horiguchi <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]; pgsql-hackers
On 2023-06-15 15:20, Kyotaro Horiguchi wrote:
Thanks for your review!
> At Wed, 14 Jun 2023 00:49:39 +0900, torikoshia
> <[email protected]> wrote in
>> On 2023-06-12 16:33, Michael Paquier wrote:
>> > On Fri, Jun 09, 2023 at 12:32:15AM +0900, Fujii Masao wrote:
>> Thanks for reviewing!
>>
>> >> printf(_(" -n, --dry-run dry run, show the names of the files that
>> >> would be removed\n"));
>> >> + printf(_(" -b, --clean-backup-history clean up files including
>> >> backup history files\n"));
>> >> Shouldn't -b option be placed in alphabetical order?
>> > +1.
>>
>> Modified the place.
>
> - printf(_(" -d generate debug output (verbose mode)\n"));
> - printf(_(" -n dry run, show the names of the files that
> would be removed\n"));
> - printf(_(" -V, --version output version information, then
> exit\n"));
> - printf(_(" -x EXT clean up files if they have this
> extension\n"));
> - printf(_(" -?, --help show this help, then exit\n"));
> + printf(_(" -d, --debug generate debug output
> (verbose mode)\n"));
> + printf(_(" -n, --dry-run dry run, show the names of
> the files that would be removed\n"));
> + printf(_(" -V, --version output version information,
> then exit\n"));
> + printf(_(" -x, --strip-extension=EXT strip this extention before
> identifying files fo clean up\n"));
> + printf(_(" -?, --help show this help, then
> exit\n"));
>
> After this change, some of these lines corss the boundary of the 80
> columns width. (is that policy viable noadays? I am usually working
> using terminal windows with such a width..) It's somewhat unrelated to
> this patch, but a help line a few lines further down also exceeds the
> width. We could shorten it by removing the "/mnt/server" portion, but
> I'm not sure if it's worth doing.
I also highlight 80th column according to the wiki[1].
Since usage() in other files like pg_rewind.c and initdb.c also
exceeded the 80th column, I thought that was something like a guide.
>> Or for use as a standalone archive cleaner:
>> e.g.
>> pg_archivecleanup /mnt/server/archiverdir
>> 000000010000000000000010.00000020.backup
>
>
> + printf(_(" -x, --strip-extension=EXT strip this extention before
> identifying files fo clean up\n"));
>
> s/fo/for/ ?
Yeah, it's a typo. Fixed it.
[1]
https://wiki.postgresql.org/wiki/Configuring_vim_for_postgres_development
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v7-0001-Introduce-pg_archivecleanup-into-getopt_long.patch (3.9K, ../../[email protected]/2-v7-0001-Introduce-pg_archivecleanup-into-getopt_long.patch)
download | inline diff:
From 06b5664d1093aba445332ad5420bd442b99a8f98 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Thu, 16 Jun 2023 21:13:02 +0900
Subject: [PATCH v7 1/3] Introduce pg_archivecleanup into getopt_long
This patch is a preliminary step to add an easy-to-understand option
to delete backup history files, but it also adds long options to
the existing options.
---
doc/src/sgml/ref/pgarchivecleanup.sgml | 5 ++++-
src/bin/pg_archivecleanup/pg_archivecleanup.c | 20 ++++++++++++-------
2 files changed, 17 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/ref/pgarchivecleanup.sgml b/doc/src/sgml/ref/pgarchivecleanup.sgml
index 635e7c7685..09991c2fcd 100644
--- a/doc/src/sgml/ref/pgarchivecleanup.sgml
+++ b/doc/src/sgml/ref/pgarchivecleanup.sgml
@@ -95,6 +95,7 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
<varlistentry>
<term><option>-d</option></term>
+ <term><option>--debug</option></term>
<listitem>
<para>
Print lots of debug logging output on <filename>stderr</filename>.
@@ -104,6 +105,7 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
<varlistentry>
<term><option>-n</option></term>
+ <term><option>--dry-run</option></term>
<listitem>
<para>
Print the names of the files that would have been removed on <filename>stdout</filename> (performs a dry run).
@@ -122,7 +124,8 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
</varlistentry>
<varlistentry>
- <term><option>-x</option> <replaceable>extension</replaceable></term>
+ <term><option>-x <replaceable class="parameter">extension</replaceable></option></term>
+ <term><option>--strip-extension=<replaceable class="parameter">extension</replaceable></option></term>
<listitem>
<para>
Provide an extension
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index 7726d05149..be62ffe75d 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -17,7 +17,7 @@
#include "access/xlog_internal.h"
#include "common/logging.h"
-#include "pg_getopt.h"
+#include "getopt_long.h"
const char *progname;
@@ -252,11 +252,11 @@ usage(void)
printf(_("Usage:\n"));
printf(_(" %s [OPTION]... ARCHIVELOCATION OLDESTKEPTWALFILE\n"), progname);
printf(_("\nOptions:\n"));
- printf(_(" -d generate debug output (verbose mode)\n"));
- printf(_(" -n dry run, show the names of the files that would be removed\n"));
- printf(_(" -V, --version output version information, then exit\n"));
- printf(_(" -x EXT clean up files if they have this extension\n"));
- printf(_(" -?, --help show this help, then exit\n"));
+ printf(_(" -d, --debug generate debug output (verbose mode)\n"));
+ printf(_(" -n, --dry-run dry run, show the names of the files that would be removed\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -x, --strip-extension=EXT strip this extention before identifying files for clean up\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"For use as archive_cleanup_command in postgresql.conf:\n"
" archive_cleanup_command = 'pg_archivecleanup [OPTION]... ARCHIVELOCATION %%r'\n"
@@ -274,6 +274,12 @@ usage(void)
int
main(int argc, char **argv)
{
+ static struct option long_options[] = {
+ {"debug", no_argument, NULL, 'd'},
+ {"dry-run", no_argument, NULL, 'n'},
+ {"strip-extension", required_argument, NULL, 'x'},
+ {NULL, 0, NULL, 0}
+ };
int c;
pg_logging_init(argv[0]);
@@ -294,7 +300,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt(argc, argv, "dnx:")) != -1)
+ while ((c = getopt_long(argc, argv, "dnx:", long_options, NULL)) != -1)
{
switch (c)
{
--
2.39.2
[text/x-diff] v7-0002-Allow-pg_archivecleanup-to-remove-backup-history-.patch (10.8K, ../../[email protected]/3-v7-0002-Allow-pg_archivecleanup-to-remove-backup-history-.patch)
download | inline diff:
From ef234fe06f32b0b3b0b3d54cefc78ee7fbf40328 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Thu, 16 Jun 2023 21:15:23 +0900
Subject: [PATCH v7 2/3] Allow pg_archivecleanup to remove backup history files
Backup history files are just few bytes, but it can be noisy for the
eye to see a large accumulation of history files mixed with the WAL
segments.
This patch adds a new option to remove files including backup
history files older oldestkeptwalfile.
---
doc/src/sgml/ref/pgarchivecleanup.sgml | 11 +++
src/bin/pg_archivecleanup/pg_archivecleanup.c | 85 ++++++++++++-------
.../t/010_pg_archivecleanup.pl | 79 +++++++++++------
3 files changed, 117 insertions(+), 58 deletions(-)
diff --git a/doc/src/sgml/ref/pgarchivecleanup.sgml b/doc/src/sgml/ref/pgarchivecleanup.sgml
index 09991c2fcd..8fac233db6 100644
--- a/doc/src/sgml/ref/pgarchivecleanup.sgml
+++ b/doc/src/sgml/ref/pgarchivecleanup.sgml
@@ -113,6 +113,17 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>-b</option></term>
+ <term><option>--clean-backup-history</option></term>
+ <listitem>
+ <para>
+ Remove backup history files.
+ For details about backup history file, please refer to the <xref linkend="backup-base-backup"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-V</option></term>
<term><option>--version</option></term>
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index be62ffe75d..a7f77d9158 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -23,6 +23,8 @@ const char *progname;
/* Options and defaults */
bool dryrun = false; /* are we performing a dry-run operation? */
+bool cleanBackupHistory = false; /* remove files including
+ * backup history files */
char *additional_ext = NULL; /* Extension to remove from filenames */
char *archiveLocation; /* where to find the archive? */
@@ -97,15 +99,31 @@ CleanupPriorWALFiles(void)
{
while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
+ char WALFilePath[MAXPGPATH * 2]; /* the file path
+ * including archive */
/*
- * Truncation is essentially harmless, because we skip names of
- * length other than XLOG_FNAME_LEN. (In principle, one could use
- * a 1000-character additional_ext and get trouble.)
+ * Truncation is essentially harmless, because we check the file
+ * format including the length immediately after this.
+ * (In principle, one could use a 1000-character additional_ext
+ * and get trouble.)
*/
strlcpy(walfile, xlde->d_name, MAXPGPATH);
TrimExtension(walfile, additional_ext);
/*
+ * Check file name.
+ *
+ * We skip files which are not WAL file or partial WAL file.
+ * Also we skip backup history files when --clean-backup-history
+ * is not specified.
+ */
+ if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile) &&
+ !(cleanBackupHistory && IsBackupHistoryFileName(walfile)))
+ continue;
+
+ /*
+ * Check cutoff point.
+ *
* We ignore the timeline part of the XLOG segment identifiers in
* deciding whether a segment is still needed. This ensures that
* we won't prematurely remove a segment from a parent timeline.
@@ -118,39 +136,35 @@ CleanupPriorWALFiles(void)
* file. Note that this means files are not removed in the order
* they were originally written, in case this worries you.
*/
- if ((IsXLogFileName(walfile) || IsPartialXLogFileName(walfile)) &&
- strcmp(walfile + 8, exclusiveCleanupFileName + 8) < 0)
- {
- char WALFilePath[MAXPGPATH * 2]; /* the file path
- * including archive */
+ if (strcmp(walfile + 8, exclusiveCleanupFileName + 8) >= 0)
+ continue;
+ /*
+ * Use the original file name again now, including any
+ * extension that might have been chopped off before testing
+ * the sequence.
+ */
+ snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
+ archiveLocation, xlde->d_name);
+
+ if (dryrun)
+ {
/*
- * Use the original file name again now, including any
- * extension that might have been chopped off before testing
- * the sequence.
+ * Prints the name of the file to be removed and skips the
+ * actual removal. The regular printout is so that the
+ * user can pipe the output into some other program.
*/
- snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
- archiveLocation, xlde->d_name);
-
- if (dryrun)
- {
- /*
- * Prints the name of the file to be removed and skips the
- * actual removal. The regular printout is so that the
- * user can pipe the output into some other program.
- */
- printf("%s\n", WALFilePath);
- pg_log_debug("file \"%s\" would be removed", WALFilePath);
- continue;
- }
-
- pg_log_debug("removing file \"%s\"", WALFilePath);
-
- rc = unlink(WALFilePath);
- if (rc != 0)
- pg_fatal("could not remove file \"%s\": %m",
- WALFilePath);
+ printf("%s\n", WALFilePath);
+ pg_log_debug("file \"%s\" would be removed", WALFilePath);
+ continue;
}
+
+ pg_log_debug("removing file \"%s\"", WALFilePath);
+
+ rc = unlink(WALFilePath);
+ if (rc != 0)
+ pg_fatal("could not remove file \"%s\": %m",
+ WALFilePath);
}
if (errno)
@@ -254,6 +268,7 @@ usage(void)
printf(_("\nOptions:\n"));
+ printf(_(" -b, --clean-backup-history clean up files including backup history files\n"));
printf(_(" -d, --debug generate debug output (verbose mode)\n"));
printf(_(" -n, --dry-run dry run, show the names of the files that would be removed\n"));
printf(_(" -V, --version output version information, then exit\n"));
printf(_(" -x, --strip-extension=EXT strip this extention before identifying files for clean up\n"));
printf(_(" -?, --help show this help, then exit\n"));
@@ -275,6 +290,7 @@ int
main(int argc, char **argv)
{
static struct option long_options[] = {
+ {"clean-backup-history", no_argument, NULL, 'b'},
{"debug", no_argument, NULL, 'd'},
{"dry-run", no_argument, NULL, 'n'},
{"strip-extension", required_argument, NULL, 'x'},
@@ -300,10 +316,13 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "dnx:", long_options, NULL)) != -1)
+ while ((c = getopt_long(argc, argv, "bdnx:", long_options, NULL)) != -1)
{
switch (c)
{
+ case 'b': /* Remove backup history files too */
+ cleanBackupHistory = true;
+ break;
case 'd': /* Debug mode */
pg_logging_increase_verbosity();
break;
diff --git a/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl b/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
index cc3386d146..030a82e7fa 100644
--- a/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
+++ b/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
@@ -12,13 +12,23 @@ program_options_handling_ok('pg_archivecleanup');
my $tempdir = PostgreSQL::Test::Utils::tempdir;
-my @walfiles = (
- '00000001000000370000000C.gz', '00000001000000370000000D',
- '00000001000000370000000E', '00000001000000370000000F.partial',);
+my @walfiles_with_gz = (
+ { name => '00000001000000370000000C.gz', present => 0 },
+ { name => '00000001000000370000000D', present => 0 },
+ { name => '00000001000000370000000E', present => 1 },
+ { name => '00000001000000370000000F.partial', present => 1 },
+ { name => 'unrelated_file', present => 1 });
+
+my @walfiles_with_backup = (
+ { name => '00000001000000370000000D', present => 0 },
+ { name => '00000001000000370000000D.00000028.backup', present => 0 },
+ { name => '00000001000000370000000E', present => 1 },
+ { name => '00000001000000370000000F.partial', present => 1 },
+ { name => 'unrelated_file', present => 1 });
sub create_files
{
- foreach my $fn (@walfiles, 'unrelated_file')
+ foreach my $fn (@_)
{
open my $file, '>', "$tempdir/$fn";
print $file 'CONTENT';
@@ -27,7 +37,18 @@ sub create_files
return;
}
-create_files();
+sub get_walfiles
+{
+ my @walfiles;
+
+ for my $walpair (@_)
+ {
+ push @walfiles, $walpair->{name};
+ }
+ return @walfiles;
+}
+
+create_files(get_walfiles(@walfiles_with_gz));
command_fails_like(
['pg_archivecleanup'],
@@ -59,14 +80,14 @@ command_fails_like(
my $stderr;
my $result =
IPC::Run::run [ 'pg_archivecleanup', '-d', '-n', $tempdir,
- $walfiles[2] ],
+ $walfiles_with_gz[2]->{name} ],
'2>', \$stderr;
ok($result, "pg_archivecleanup dry run: exit code 0");
like(
$stderr,
- qr/$walfiles[1].*would be removed/,
+ qr/$walfiles_with_gz[1]->{name}.*would be removed/,
"pg_archivecleanup dry run: matches");
- foreach my $fn (@walfiles)
+ foreach my $fn (get_walfiles(@walfiles_with_gz))
{
ok(-f "$tempdir/$fn", "$fn not removed");
}
@@ -76,32 +97,40 @@ sub run_check
{
local $Test::Builder::Level = $Test::Builder::Level + 1;
- my ($suffix, $test_name) = @_;
+ my ($testdata, $oldestkeptwalfile, $test_name, @options) = @_;
- create_files();
+ create_files(get_walfiles(@$testdata));
command_ok(
[
- 'pg_archivecleanup', '-x', '.gz', $tempdir,
- $walfiles[2] . $suffix
+ 'pg_archivecleanup', @options, $tempdir,
+ $oldestkeptwalfile
],
"$test_name: runs");
- ok(!-f "$tempdir/$walfiles[0]",
- "$test_name: first older WAL file was cleaned up");
- ok(!-f "$tempdir/$walfiles[1]",
- "$test_name: second older WAL file was cleaned up");
- ok(-f "$tempdir/$walfiles[2]",
- "$test_name: restartfile was not cleaned up");
- ok(-f "$tempdir/$walfiles[3]",
- "$test_name: newer WAL file was not cleaned up");
- ok(-f "$tempdir/unrelated_file",
- "$test_name: unrelated file was not cleaned up");
+ for my $walpair (@$testdata)
+ {
+ if ($walpair->{present})
+ {
+ ok(-f "$tempdir/$walpair->{name}",
+ "$test_name:$walpair->{name} was not cleaned up");
+ }
+ else
+ {
+ ok(!-f "$tempdir/$walpair->{name}",
+ "$test_name:$walpair->{name} was cleaned up");
+ }
+ }
return;
}
-run_check('', 'pg_archivecleanup');
-run_check('.partial', 'pg_archivecleanup with .partial file');
-run_check('.00000020.backup', 'pg_archivecleanup with .backup file');
+run_check(\@walfiles_with_gz, '00000001000000370000000E',
+ 'pg_archivecleanup', '-x.gz');
+run_check(\@walfiles_with_gz, '00000001000000370000000E.partial',
+ 'pg_archivecleanup with .partial file', '-x.gz');
+run_check(\@walfiles_with_gz, '00000001000000370000000E.00000020.backup',
+ 'pg_archivecleanup with .backup file', '-x.gz');
+run_check(\@walfiles_with_backup, '00000001000000370000000E',
+ 'pg_archivecleanup with --clean-backup-history', '-b');
done_testing();
--
2.39.2
[text/x-diff] v7-0003-Refactored-to-reduce-the-nesting-level.patch (5.9K, ../../[email protected]/4-v7-0003-Refactored-to-reduce-the-nesting-level.patch)
download | inline diff:
From bef376b2766923339dedbf3fae9e516262b433a1 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Thu, 16 Jun 2023 21:23:02 +0900
Subject: [PATCH v7 3/3] Refactored to reduce the nesting level.
---
src/bin/pg_archivecleanup/pg_archivecleanup.c | 146 +++++++++---------
1 file changed, 72 insertions(+), 74 deletions(-)
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index a7f77d9158..f475177edb 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -95,87 +95,85 @@ CleanupPriorWALFiles(void)
struct dirent *xlde;
char walfile[MAXPGPATH];
- if ((xldir = opendir(archiveLocation)) != NULL)
+ if ((xldir = opendir(archiveLocation)) == NULL)
+ pg_fatal("could not open archive location \"%s\": %m",
+ archiveLocation);
+
+ while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
- while (errno = 0, (xlde = readdir(xldir)) != NULL)
+ char WALFilePath[MAXPGPATH * 2]; /* the file path
+ * including archive */
+ /*
+ * Truncation is essentially harmless, because we check the file
+ * format including the length immediately after this.
+ * (In principle, one could use a 1000-character additional_ext
+ * and get trouble.)
+ */
+ strlcpy(walfile, xlde->d_name, MAXPGPATH);
+ TrimExtension(walfile, additional_ext);
+
+ /*
+ * Check file name.
+ *
+ * We skip files which are not WAL file or partial WAL file.
+ * Also we skip backup history files when --clean-backup-history
+ * is not specified.
+ */
+ if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile) &&
+ !(cleanBackupHistory && IsBackupHistoryFileName(walfile)))
+ continue;
+
+ /*
+ * Check cutoff point.
+ *
+ * We ignore the timeline part of the XLOG segment identifiers in
+ * deciding whether a segment is still needed. This ensures that
+ * we won't prematurely remove a segment from a parent timeline.
+ * We could probably be a little more proactive about removing
+ * segments of non-parent timelines, but that would be a whole lot
+ * more complicated.
+ *
+ * We use the alphanumeric sorting property of the filenames to
+ * decide which ones are earlier than the exclusiveCleanupFileName
+ * file. Note that this means files are not removed in the order
+ * they were originally written, in case this worries you.
+ */
+ if (strcmp(walfile + 8, exclusiveCleanupFileName + 8) >= 0)
+ continue;
+
+ /*
+ * Use the original file name again now, including any
+ * extension that might have been chopped off before testing
+ * the sequence.
+ */
+ snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
+ archiveLocation, xlde->d_name);
+
+ if (dryrun)
{
- char WALFilePath[MAXPGPATH * 2]; /* the file path
- * including archive */
- /*
- * Truncation is essentially harmless, because we check the file
- * format including the length immediately after this.
- * (In principle, one could use a 1000-character additional_ext
- * and get trouble.)
- */
- strlcpy(walfile, xlde->d_name, MAXPGPATH);
- TrimExtension(walfile, additional_ext);
-
/*
- * Check file name.
- *
- * We skip files which are not WAL file or partial WAL file.
- * Also we skip backup history files when --clean-backup-history
- * is not specified.
+ * Prints the name of the file to be removed and skips the
+ * actual removal. The regular printout is so that the
+ * user can pipe the output into some other program.
*/
- if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile) &&
- !(cleanBackupHistory && IsBackupHistoryFileName(walfile)))
- continue;
-
- /*
- * Check cutoff point.
- *
- * We ignore the timeline part of the XLOG segment identifiers in
- * deciding whether a segment is still needed. This ensures that
- * we won't prematurely remove a segment from a parent timeline.
- * We could probably be a little more proactive about removing
- * segments of non-parent timelines, but that would be a whole lot
- * more complicated.
- *
- * We use the alphanumeric sorting property of the filenames to
- * decide which ones are earlier than the exclusiveCleanupFileName
- * file. Note that this means files are not removed in the order
- * they were originally written, in case this worries you.
- */
- if (strcmp(walfile + 8, exclusiveCleanupFileName + 8) >= 0)
- continue;
-
- /*
- * Use the original file name again now, including any
- * extension that might have been chopped off before testing
- * the sequence.
- */
- snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
- archiveLocation, xlde->d_name);
-
- if (dryrun)
- {
- /*
- * Prints the name of the file to be removed and skips the
- * actual removal. The regular printout is so that the
- * user can pipe the output into some other program.
- */
- printf("%s\n", WALFilePath);
- pg_log_debug("file \"%s\" would be removed", WALFilePath);
- continue;
- }
-
- pg_log_debug("removing file \"%s\"", WALFilePath);
-
- rc = unlink(WALFilePath);
- if (rc != 0)
- pg_fatal("could not remove file \"%s\": %m",
- WALFilePath);
+ printf("%s\n", WALFilePath);
+ pg_log_debug("file \"%s\" would be removed", WALFilePath);
+ continue;
}
- if (errno)
- pg_fatal("could not read archive location \"%s\": %m",
- archiveLocation);
- if (closedir(xldir))
- pg_fatal("could not close archive location \"%s\": %m",
- archiveLocation);
+ pg_log_debug("removing file \"%s\"", WALFilePath);
+
+ rc = unlink(WALFilePath);
+ if (rc != 0)
+ pg_fatal("could not remove file \"%s\": %m",
+ WALFilePath);
}
- else
- pg_fatal("could not open archive location \"%s\": %m",
+
+ if (errno)
+ pg_fatal("could not read archive location \"%s\": %m",
+ archiveLocation);
+ if (closedir(xldir))
+ pg_fatal("could not close archive location \"%s\": %m",
archiveLocation);
}
--
2.39.2
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Allow pg_archivecleanup to remove backup history files
2023-06-13 15:49 Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-15 06:20 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-15 12:38 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
@ 2023-06-16 02:22 ` Kyotaro Horiguchi <[email protected]>
2023-06-16 02:30 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
0 siblings, 1 reply; 70+ messages in thread
From: Kyotaro Horiguchi @ 2023-06-16 02:22 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; pgsql-hackers
At Thu, 15 Jun 2023 21:38:28 +0900, torikoshia <[email protected]> wrote in
> On 2023-06-15 15:20, Kyotaro Horiguchi wrote:
> Thanks for your review!
> > + printf(_(" -x, --strip-extension=EXT strip this extention before
> > identifying files fo clean up\n"));
> > + printf(_(" -?, --help show this help, then exit\n"));
> > After this change, some of these lines corss the boundary of the 80
> > columns width. (is that policy viable noadays? I am usually working
> > using terminal windows with such a width..) It's somewhat unrelated to
> > this patch, but a help line a few lines further down also exceeds the
> > width. We could shorten it by removing the "/mnt/server" portion, but
> > I'm not sure if it's worth doing.
>
> I also highlight 80th column according to the wiki[1].
> Since usage() in other files like pg_rewind.c and initdb.c also
> exceeded the 80th column, I thought that was something like a guide.
I think the page is suggesting about program code, not the messages
that binaries print.
ASAICS the main section of the "pg_rewind --help" fits within 80
columns. However, "initdb --help" does output a few lines exceeding
the 80-column limit. Judging by the surrounding lines, I believe we're
still aiming to maintain these the given width. I think we need to fix
initdb in that regard.
> [1]
> https://wiki.postgresql.org/wiki/Configuring_vim_for_postgres_development
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Allow pg_archivecleanup to remove backup history files
2023-06-13 15:49 Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-15 06:20 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-15 12:38 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-16 02:22 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
@ 2023-06-16 02:30 ` Kyotaro Horiguchi <[email protected]>
2023-06-19 02:24 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
0 siblings, 1 reply; 70+ messages in thread
From: Kyotaro Horiguchi @ 2023-06-16 02:30 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; pgsql-hackers
At Fri, 16 Jun 2023 11:22:31 +0900 (JST), Kyotaro Horiguchi <[email protected]> wrote in
> ASAICS the main section of the "pg_rewind --help" fits within 80
> columns. However, "initdb --help" does output a few lines exceeding
> the 80-column limit. Judging by the surrounding lines, I believe we're
> still aiming to maintain these the given width. I think we need to fix
> initdb in that regard.
Mmm, the message was introduced in 2012 by 8a02339e9b. I haven't
noticed this until now...
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Allow pg_archivecleanup to remove backup history files
2023-06-13 15:49 Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-15 06:20 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-15 12:38 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-16 02:22 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-16 02:30 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
@ 2023-06-19 02:24 ` torikoshia <[email protected]>
2023-06-19 05:37 ` Re: Allow pg_archivecleanup to remove backup history files Michael Paquier <[email protected]>
0 siblings, 1 reply; 70+ messages in thread
From: torikoshia @ 2023-06-19 02:24 UTC (permalink / raw)
To: Kyotaro Horiguchi <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]; pgsql-hackers
On 2023-06-16 11:22, Kyotaro Horiguchi wrote:
> At Thu, 15 Jun 2023 21:38:28 +0900, torikoshia
> <[email protected]> wrote in
>> On 2023-06-15 15:20, Kyotaro Horiguchi wrote:
>> Thanks for your review!
>> > + printf(_(" -x, --strip-extension=EXT strip this extention before
>> > identifying files fo clean up\n"));
>> > + printf(_(" -?, --help show this help, then exit\n"));
>> > After this change, some of these lines corss the boundary of the 80
>> > columns width. (is that policy viable noadays? I am usually working
>> > using terminal windows with such a width..) It's somewhat unrelated to
>> > this patch, but a help line a few lines further down also exceeds the
>> > width. We could shorten it by removing the "/mnt/server" portion, but
>> > I'm not sure if it's worth doing.
>>
>> I also highlight 80th column according to the wiki[1].
>> Since usage() in other files like pg_rewind.c and initdb.c also
>> exceeded the 80th column, I thought that was something like a guide.
>
> I think the page is suggesting about program code, not the messages
> that binaries print.
Thanks, now I understand what you meant.
> ASAICS the main section of the "pg_rewind --help" fits within 80
> columns. However, "initdb --help" does output a few lines exceeding
> the 80-column limit. Judging by the surrounding lines, I believe we're
> still aiming to maintain these the given width. I think we need to fix
> initdb in that regard.
Hmm, it seems some other commands also exceeds 80 columns:
pg_amcheck:
--skip=OPTION do NOT check "all-frozen" or
"all-visible" blocks
--startblock=BLOCK begin checking table(s) at the given
block number
--endblock=BLOCK check table(s) only up to the given
block number
--no-synchronized-snapshots do not use synchronized snapshots in
parallel jobs
pg_isready:
-t, --timeout=SECS seconds to wait when attempting connection, 0
disables (default: 3)
pg_receivewal:
--create-slot create a new replication slot (for the slot's
name see --slot)
--drop-slot drop the replication slot (for the slot's name
see --slot)
If you don't mind, I'm going to create another thread about this point.
I'll also discuss below line since it's unrelated to current thread
as you pointed out:
| pg_archivecleanup /mnt/server/archiverdir
000000010000000000000010.00000020.backup
Attached patch fixes the number of columns per row exceeding 80 by
changing to use getopt_long.
On 2023-06-16 11:30, Kyotaro Horiguchi wrote:
> At Fri, 16 Jun 2023 11:22:31 +0900 (JST), Kyotaro Horiguchi
> <[email protected]> wrote in
>> ASAICS the main section of the "pg_rewind --help" fits within 80
>> columns. However, "initdb --help" does output a few lines exceeding
>> the 80-column limit. Judging by the surrounding lines, I believe we're
>> still aiming to maintain these the given width. I think we need to fix
>> initdb in that regard.
>
> Mmm, the message was introduced in 2012 by 8a02339e9b. I haven't
> noticed this until now...
>
> regards.
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v8-0001-Introduce-pg_archivecleanup-into-getopt_long.patch (4.0K, ../../[email protected]/2-v8-0001-Introduce-pg_archivecleanup-into-getopt_long.patch)
download | inline diff:
From 220bbd866158fd69a3a4affe73136f4699353ecd Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 19 Jun 2023 09:46:41 +0900
Subject: [PATCH v8 1/3] Introduce pg_archivecleanup into getopt_long
This patch is a preliminary step to add an easy-to-understand option
to delete backup history files, but it also adds long options to
the existing options.
---
doc/src/sgml/ref/pgarchivecleanup.sgml | 5 ++++-
src/bin/pg_archivecleanup/pg_archivecleanup.c | 22 +++++++++++++------
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/ref/pgarchivecleanup.sgml b/doc/src/sgml/ref/pgarchivecleanup.sgml
index 635e7c7685..09991c2fcd 100644
--- a/doc/src/sgml/ref/pgarchivecleanup.sgml
+++ b/doc/src/sgml/ref/pgarchivecleanup.sgml
@@ -95,6 +95,7 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
<varlistentry>
<term><option>-d</option></term>
+ <term><option>--debug</option></term>
<listitem>
<para>
Print lots of debug logging output on <filename>stderr</filename>.
@@ -104,6 +105,7 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
<varlistentry>
<term><option>-n</option></term>
+ <term><option>--dry-run</option></term>
<listitem>
<para>
Print the names of the files that would have been removed on <filename>stdout</filename> (performs a dry run).
@@ -122,7 +124,8 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
</varlistentry>
<varlistentry>
- <term><option>-x</option> <replaceable>extension</replaceable></term>
+ <term><option>-x <replaceable class="parameter">extension</replaceable></option></term>
+ <term><option>--strip-extension=<replaceable class="parameter">extension</replaceable></option></term>
<listitem>
<para>
Provide an extension
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index 7726d05149..fc0dca9856 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -17,7 +17,7 @@
#include "access/xlog_internal.h"
#include "common/logging.h"
-#include "pg_getopt.h"
+#include "getopt_long.h"
const char *progname;
@@ -252,11 +252,13 @@ usage(void)
printf(_("Usage:\n"));
printf(_(" %s [OPTION]... ARCHIVELOCATION OLDESTKEPTWALFILE\n"), progname);
printf(_("\nOptions:\n"));
- printf(_(" -d generate debug output (verbose mode)\n"));
- printf(_(" -n dry run, show the names of the files that would be removed\n"));
- printf(_(" -V, --version output version information, then exit\n"));
- printf(_(" -x EXT clean up files if they have this extension\n"));
- printf(_(" -?, --help show this help, then exit\n"));
+ printf(_(" -d, --debug generate debug output (verbose mode)\n"));
+ printf(_(" -n, --dry-run dry run, show the names of the files that would be\n"
+ " removed\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -x, --strip-extension=EXT strip this extention before identifying files for\n"
+ " clean up\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"For use as archive_cleanup_command in postgresql.conf:\n"
" archive_cleanup_command = 'pg_archivecleanup [OPTION]... ARCHIVELOCATION %%r'\n"
@@ -274,6 +276,12 @@ usage(void)
int
main(int argc, char **argv)
{
+ static struct option long_options[] = {
+ {"debug", no_argument, NULL, 'd'},
+ {"dry-run", no_argument, NULL, 'n'},
+ {"strip-extension", required_argument, NULL, 'x'},
+ {NULL, 0, NULL, 0}
+ };
int c;
pg_logging_init(argv[0]);
@@ -294,7 +302,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt(argc, argv, "dnx:")) != -1)
+ while ((c = getopt_long(argc, argv, "dnx:", long_options, NULL)) != -1)
{
switch (c)
{
--
2.39.2
[text/x-diff] v8-0002-Allow-pg_archivecleanup-to-remove-backup-history-.patch (10.7K, ../../[email protected]/3-v8-0002-Allow-pg_archivecleanup-to-remove-backup-history-.patch)
download | inline diff:
From d065cf4981f827e1eea76577f92d39e3669dd816 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 19 Jun 2023 09:51:12 +0900
Subject: [PATCH v8 2/3] Allow pg_archivecleanup to remove backup history files
Backup history files are just few bytes, but it can be noisy for the
eye to see a large accumulation of history files mixed with the WAL
segments.
This patch adds a new option to remove files including backup
history files older oldestkeptwalfile.
---
doc/src/sgml/ref/pgarchivecleanup.sgml | 11 +++
src/bin/pg_archivecleanup/pg_archivecleanup.c | 85 ++++++++++++-------
.../t/010_pg_archivecleanup.pl | 79 +++++++++++------
3 files changed, 117 insertions(+), 58 deletions(-)
diff --git a/doc/src/sgml/ref/pgarchivecleanup.sgml b/doc/src/sgml/ref/pgarchivecleanup.sgml
index 09991c2fcd..8fac233db6 100644
--- a/doc/src/sgml/ref/pgarchivecleanup.sgml
+++ b/doc/src/sgml/ref/pgarchivecleanup.sgml
@@ -113,6 +113,17 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>-b</option></term>
+ <term><option>--clean-backup-history</option></term>
+ <listitem>
+ <para>
+ Remove backup history files.
+ For details about backup history file, please refer to the <xref linkend="backup-base-backup"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-V</option></term>
<term><option>--version</option></term>
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index fc0dca9856..41880e42bb 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -23,6 +23,8 @@ const char *progname;
/* Options and defaults */
bool dryrun = false; /* are we performing a dry-run operation? */
+bool cleanBackupHistory = false; /* remove files including
+ * backup history files */
char *additional_ext = NULL; /* Extension to remove from filenames */
char *archiveLocation; /* where to find the archive? */
@@ -97,15 +99,31 @@ CleanupPriorWALFiles(void)
{
while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
+ char WALFilePath[MAXPGPATH * 2]; /* the file path
+ * including archive */
/*
- * Truncation is essentially harmless, because we skip names of
- * length other than XLOG_FNAME_LEN. (In principle, one could use
- * a 1000-character additional_ext and get trouble.)
+ * Truncation is essentially harmless, because we check the file
+ * format including the length immediately after this.
+ * (In principle, one could use a 1000-character additional_ext
+ * and get trouble.)
*/
strlcpy(walfile, xlde->d_name, MAXPGPATH);
TrimExtension(walfile, additional_ext);
/*
+ * Check file name.
+ *
+ * We skip files which are not WAL file or partial WAL file.
+ * Also we skip backup history files when --clean-backup-history
+ * is not specified.
+ */
+ if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile) &&
+ !(cleanBackupHistory && IsBackupHistoryFileName(walfile)))
+ continue;
+
+ /*
+ * Check cutoff point.
+ *
* We ignore the timeline part of the XLOG segment identifiers in
* deciding whether a segment is still needed. This ensures that
* we won't prematurely remove a segment from a parent timeline.
@@ -118,39 +136,35 @@ CleanupPriorWALFiles(void)
* file. Note that this means files are not removed in the order
* they were originally written, in case this worries you.
*/
- if ((IsXLogFileName(walfile) || IsPartialXLogFileName(walfile)) &&
- strcmp(walfile + 8, exclusiveCleanupFileName + 8) < 0)
- {
- char WALFilePath[MAXPGPATH * 2]; /* the file path
- * including archive */
+ if (strcmp(walfile + 8, exclusiveCleanupFileName + 8) >= 0)
+ continue;
+ /*
+ * Use the original file name again now, including any
+ * extension that might have been chopped off before testing
+ * the sequence.
+ */
+ snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
+ archiveLocation, xlde->d_name);
+
+ if (dryrun)
+ {
/*
- * Use the original file name again now, including any
- * extension that might have been chopped off before testing
- * the sequence.
+ * Prints the name of the file to be removed and skips the
+ * actual removal. The regular printout is so that the
+ * user can pipe the output into some other program.
*/
- snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
- archiveLocation, xlde->d_name);
-
- if (dryrun)
- {
- /*
- * Prints the name of the file to be removed and skips the
- * actual removal. The regular printout is so that the
- * user can pipe the output into some other program.
- */
- printf("%s\n", WALFilePath);
- pg_log_debug("file \"%s\" would be removed", WALFilePath);
- continue;
- }
-
- pg_log_debug("removing file \"%s\"", WALFilePath);
-
- rc = unlink(WALFilePath);
- if (rc != 0)
- pg_fatal("could not remove file \"%s\": %m",
- WALFilePath);
+ printf("%s\n", WALFilePath);
+ pg_log_debug("file \"%s\" would be removed", WALFilePath);
+ continue;
}
+
+ pg_log_debug("removing file \"%s\"", WALFilePath);
+
+ rc = unlink(WALFilePath);
+ if (rc != 0)
+ pg_fatal("could not remove file \"%s\": %m",
+ WALFilePath);
}
if (errno)
@@ -252,6 +266,7 @@ usage(void)
printf(_("Usage:\n"));
printf(_(" %s [OPTION]... ARCHIVELOCATION OLDESTKEPTWALFILE\n"), progname);
printf(_("\nOptions:\n"));
+ printf(_(" -b, --clean-backup-history clean up files including backup history files\n"));
printf(_(" -d, --debug generate debug output (verbose mode)\n"));
printf(_(" -n, --dry-run dry run, show the names of the files that would be\n"
" removed\n"));
@@ -277,6 +292,7 @@ int
main(int argc, char **argv)
{
static struct option long_options[] = {
+ {"clean-backup-history", no_argument, NULL, 'b'},
{"debug", no_argument, NULL, 'd'},
{"dry-run", no_argument, NULL, 'n'},
{"strip-extension", required_argument, NULL, 'x'},
@@ -302,10 +318,13 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "dnx:", long_options, NULL)) != -1)
+ while ((c = getopt_long(argc, argv, "bdnx:", long_options, NULL)) != -1)
{
switch (c)
{
+ case 'b': /* Remove backup history files too */
+ cleanBackupHistory = true;
+ break;
case 'd': /* Debug mode */
pg_logging_increase_verbosity();
break;
diff --git a/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl b/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
index cc3386d146..030a82e7fa 100644
--- a/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
+++ b/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
@@ -12,13 +12,23 @@ program_options_handling_ok('pg_archivecleanup');
my $tempdir = PostgreSQL::Test::Utils::tempdir;
-my @walfiles = (
- '00000001000000370000000C.gz', '00000001000000370000000D',
- '00000001000000370000000E', '00000001000000370000000F.partial',);
+my @walfiles_with_gz = (
+ { name => '00000001000000370000000C.gz', present => 0 },
+ { name => '00000001000000370000000D', present => 0 },
+ { name => '00000001000000370000000E', present => 1 },
+ { name => '00000001000000370000000F.partial', present => 1 },
+ { name => 'unrelated_file', present => 1 });
+
+my @walfiles_with_backup = (
+ { name => '00000001000000370000000D', present => 0 },
+ { name => '00000001000000370000000D.00000028.backup', present => 0 },
+ { name => '00000001000000370000000E', present => 1 },
+ { name => '00000001000000370000000F.partial', present => 1 },
+ { name => 'unrelated_file', present => 1 });
sub create_files
{
- foreach my $fn (@walfiles, 'unrelated_file')
+ foreach my $fn (@_)
{
open my $file, '>', "$tempdir/$fn";
print $file 'CONTENT';
@@ -27,7 +37,18 @@ sub create_files
return;
}
-create_files();
+sub get_walfiles
+{
+ my @walfiles;
+
+ for my $walpair (@_)
+ {
+ push @walfiles, $walpair->{name};
+ }
+ return @walfiles;
+}
+
+create_files(get_walfiles(@walfiles_with_gz));
command_fails_like(
['pg_archivecleanup'],
@@ -59,14 +80,14 @@ command_fails_like(
my $stderr;
my $result =
IPC::Run::run [ 'pg_archivecleanup', '-d', '-n', $tempdir,
- $walfiles[2] ],
+ $walfiles_with_gz[2]->{name} ],
'2>', \$stderr;
ok($result, "pg_archivecleanup dry run: exit code 0");
like(
$stderr,
- qr/$walfiles[1].*would be removed/,
+ qr/$walfiles_with_gz[1]->{name}.*would be removed/,
"pg_archivecleanup dry run: matches");
- foreach my $fn (@walfiles)
+ foreach my $fn (get_walfiles(@walfiles_with_gz))
{
ok(-f "$tempdir/$fn", "$fn not removed");
}
@@ -76,32 +97,40 @@ sub run_check
{
local $Test::Builder::Level = $Test::Builder::Level + 1;
- my ($suffix, $test_name) = @_;
+ my ($testdata, $oldestkeptwalfile, $test_name, @options) = @_;
- create_files();
+ create_files(get_walfiles(@$testdata));
command_ok(
[
- 'pg_archivecleanup', '-x', '.gz', $tempdir,
- $walfiles[2] . $suffix
+ 'pg_archivecleanup', @options, $tempdir,
+ $oldestkeptwalfile
],
"$test_name: runs");
- ok(!-f "$tempdir/$walfiles[0]",
- "$test_name: first older WAL file was cleaned up");
- ok(!-f "$tempdir/$walfiles[1]",
- "$test_name: second older WAL file was cleaned up");
- ok(-f "$tempdir/$walfiles[2]",
- "$test_name: restartfile was not cleaned up");
- ok(-f "$tempdir/$walfiles[3]",
- "$test_name: newer WAL file was not cleaned up");
- ok(-f "$tempdir/unrelated_file",
- "$test_name: unrelated file was not cleaned up");
+ for my $walpair (@$testdata)
+ {
+ if ($walpair->{present})
+ {
+ ok(-f "$tempdir/$walpair->{name}",
+ "$test_name:$walpair->{name} was not cleaned up");
+ }
+ else
+ {
+ ok(!-f "$tempdir/$walpair->{name}",
+ "$test_name:$walpair->{name} was cleaned up");
+ }
+ }
return;
}
-run_check('', 'pg_archivecleanup');
-run_check('.partial', 'pg_archivecleanup with .partial file');
-run_check('.00000020.backup', 'pg_archivecleanup with .backup file');
+run_check(\@walfiles_with_gz, '00000001000000370000000E',
+ 'pg_archivecleanup', '-x.gz');
+run_check(\@walfiles_with_gz, '00000001000000370000000E.partial',
+ 'pg_archivecleanup with .partial file', '-x.gz');
+run_check(\@walfiles_with_gz, '00000001000000370000000E.00000020.backup',
+ 'pg_archivecleanup with .backup file', '-x.gz');
+run_check(\@walfiles_with_backup, '00000001000000370000000E',
+ 'pg_archivecleanup with --clean-backup-history', '-b');
done_testing();
--
2.39.2
[text/x-diff] v8-0003-Refactored-to-reduce-the-nesting-level.patch (5.9K, ../../[email protected]/4-v8-0003-Refactored-to-reduce-the-nesting-level.patch)
download | inline diff:
From 806f1c5770763885137a4fccbfa3394f6cd5f8e9 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 19 Jun 2023 09:58:08 +0900
Subject: [PATCH v8 3/3] Refactored to reduce the nesting level.
---
src/bin/pg_archivecleanup/pg_archivecleanup.c | 146 +++++++++---------
1 file changed, 72 insertions(+), 74 deletions(-)
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index 41880e42bb..ffb4a19e7f 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -95,87 +95,85 @@ CleanupPriorWALFiles(void)
struct dirent *xlde;
char walfile[MAXPGPATH];
- if ((xldir = opendir(archiveLocation)) != NULL)
+ if ((xldir = opendir(archiveLocation)) == NULL)
+ pg_fatal("could not open archive location \"%s\": %m",
+ archiveLocation);
+
+ while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
- while (errno = 0, (xlde = readdir(xldir)) != NULL)
+ char WALFilePath[MAXPGPATH * 2]; /* the file path
+ * including archive */
+ /*
+ * Truncation is essentially harmless, because we check the file
+ * format including the length immediately after this.
+ * (In principle, one could use a 1000-character additional_ext
+ * and get trouble.)
+ */
+ strlcpy(walfile, xlde->d_name, MAXPGPATH);
+ TrimExtension(walfile, additional_ext);
+
+ /*
+ * Check file name.
+ *
+ * We skip files which are not WAL file or partial WAL file.
+ * Also we skip backup history files when --clean-backup-history
+ * is not specified.
+ */
+ if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile) &&
+ !(cleanBackupHistory && IsBackupHistoryFileName(walfile)))
+ continue;
+
+ /*
+ * Check cutoff point.
+ *
+ * We ignore the timeline part of the XLOG segment identifiers in
+ * deciding whether a segment is still needed. This ensures that
+ * we won't prematurely remove a segment from a parent timeline.
+ * We could probably be a little more proactive about removing
+ * segments of non-parent timelines, but that would be a whole lot
+ * more complicated.
+ *
+ * We use the alphanumeric sorting property of the filenames to
+ * decide which ones are earlier than the exclusiveCleanupFileName
+ * file. Note that this means files are not removed in the order
+ * they were originally written, in case this worries you.
+ */
+ if (strcmp(walfile + 8, exclusiveCleanupFileName + 8) >= 0)
+ continue;
+
+ /*
+ * Use the original file name again now, including any
+ * extension that might have been chopped off before testing
+ * the sequence.
+ */
+ snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
+ archiveLocation, xlde->d_name);
+
+ if (dryrun)
{
- char WALFilePath[MAXPGPATH * 2]; /* the file path
- * including archive */
- /*
- * Truncation is essentially harmless, because we check the file
- * format including the length immediately after this.
- * (In principle, one could use a 1000-character additional_ext
- * and get trouble.)
- */
- strlcpy(walfile, xlde->d_name, MAXPGPATH);
- TrimExtension(walfile, additional_ext);
-
/*
- * Check file name.
- *
- * We skip files which are not WAL file or partial WAL file.
- * Also we skip backup history files when --clean-backup-history
- * is not specified.
+ * Prints the name of the file to be removed and skips the
+ * actual removal. The regular printout is so that the
+ * user can pipe the output into some other program.
*/
- if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile) &&
- !(cleanBackupHistory && IsBackupHistoryFileName(walfile)))
- continue;
-
- /*
- * Check cutoff point.
- *
- * We ignore the timeline part of the XLOG segment identifiers in
- * deciding whether a segment is still needed. This ensures that
- * we won't prematurely remove a segment from a parent timeline.
- * We could probably be a little more proactive about removing
- * segments of non-parent timelines, but that would be a whole lot
- * more complicated.
- *
- * We use the alphanumeric sorting property of the filenames to
- * decide which ones are earlier than the exclusiveCleanupFileName
- * file. Note that this means files are not removed in the order
- * they were originally written, in case this worries you.
- */
- if (strcmp(walfile + 8, exclusiveCleanupFileName + 8) >= 0)
- continue;
-
- /*
- * Use the original file name again now, including any
- * extension that might have been chopped off before testing
- * the sequence.
- */
- snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
- archiveLocation, xlde->d_name);
-
- if (dryrun)
- {
- /*
- * Prints the name of the file to be removed and skips the
- * actual removal. The regular printout is so that the
- * user can pipe the output into some other program.
- */
- printf("%s\n", WALFilePath);
- pg_log_debug("file \"%s\" would be removed", WALFilePath);
- continue;
- }
-
- pg_log_debug("removing file \"%s\"", WALFilePath);
-
- rc = unlink(WALFilePath);
- if (rc != 0)
- pg_fatal("could not remove file \"%s\": %m",
- WALFilePath);
+ printf("%s\n", WALFilePath);
+ pg_log_debug("file \"%s\" would be removed", WALFilePath);
+ continue;
}
- if (errno)
- pg_fatal("could not read archive location \"%s\": %m",
- archiveLocation);
- if (closedir(xldir))
- pg_fatal("could not close archive location \"%s\": %m",
- archiveLocation);
+ pg_log_debug("removing file \"%s\"", WALFilePath);
+
+ rc = unlink(WALFilePath);
+ if (rc != 0)
+ pg_fatal("could not remove file \"%s\": %m",
+ WALFilePath);
}
- else
- pg_fatal("could not open archive location \"%s\": %m",
+
+ if (errno)
+ pg_fatal("could not read archive location \"%s\": %m",
+ archiveLocation);
+ if (closedir(xldir))
+ pg_fatal("could not close archive location \"%s\": %m",
archiveLocation);
}
--
2.39.2
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Allow pg_archivecleanup to remove backup history files
2023-06-13 15:49 Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-15 06:20 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-15 12:38 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-16 02:22 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-16 02:30 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-19 02:24 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
@ 2023-06-19 05:37 ` Michael Paquier <[email protected]>
2023-06-20 13:27 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
0 siblings, 1 reply; 70+ messages in thread
From: Michael Paquier @ 2023-06-19 05:37 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; [email protected]; [email protected]; pgsql-hackers
On Mon, Jun 19, 2023 at 11:24:29AM +0900, torikoshia wrote:
> Thanks, now I understand what you meant.
If I may ask, why is the refactoring of 0003 done after the feature in
0002? Shouldn't the order be reversed? That would make for a cleaner
git history.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../ZI%[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Allow pg_archivecleanup to remove backup history files
2023-06-13 15:49 Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-15 06:20 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-15 12:38 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-16 02:22 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-16 02:30 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-19 02:24 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-19 05:37 ` Re: Allow pg_archivecleanup to remove backup history files Michael Paquier <[email protected]>
@ 2023-06-20 13:27 ` torikoshia <[email protected]>
2023-06-21 02:59 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
0 siblings, 1 reply; 70+ messages in thread
From: torikoshia @ 2023-06-20 13:27 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; [email protected]; [email protected]; pgsql-hackers
On 2023-06-19 14:37, Michael Paquier wrote:
> On Mon, Jun 19, 2023 at 11:24:29AM +0900, torikoshia wrote:
>> Thanks, now I understand what you meant.
>
> If I may ask, why is the refactoring of 0003 done after the feature in
> 0002? Shouldn't the order be reversed? That would make for a cleaner
> git history.
> --
> Michael
Agreed.
Reversed the order of patches 0002 and 0003.
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v9-0001-Introduce-pg_archivecleanup-into-getopt_long.patch (4.0K, ../../[email protected]/2-v9-0001-Introduce-pg_archivecleanup-into-getopt_long.patch)
download | inline diff:
From a0770f668bf73477815fb44a7386d95ed91bdec5 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 19 Jun 2023 20:42:07 +0900
Subject: [PATCH v9 1/3] Introduce pg_archivecleanup into getopt_long
This patch is a preliminary step to add an easy-to-understand option
to delete backup history files, but it also adds long options to
the existing options.
---
doc/src/sgml/ref/pgarchivecleanup.sgml | 5 ++++-
src/bin/pg_archivecleanup/pg_archivecleanup.c | 22 +++++++++++++------
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/ref/pgarchivecleanup.sgml b/doc/src/sgml/ref/pgarchivecleanup.sgml
index 635e7c7685..09991c2fcd 100644
--- a/doc/src/sgml/ref/pgarchivecleanup.sgml
+++ b/doc/src/sgml/ref/pgarchivecleanup.sgml
@@ -95,6 +95,7 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
<varlistentry>
<term><option>-d</option></term>
+ <term><option>--debug</option></term>
<listitem>
<para>
Print lots of debug logging output on <filename>stderr</filename>.
@@ -104,6 +105,7 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
<varlistentry>
<term><option>-n</option></term>
+ <term><option>--dry-run</option></term>
<listitem>
<para>
Print the names of the files that would have been removed on <filename>stdout</filename> (performs a dry run).
@@ -122,7 +124,8 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
</varlistentry>
<varlistentry>
- <term><option>-x</option> <replaceable>extension</replaceable></term>
+ <term><option>-x <replaceable class="parameter">extension</replaceable></option></term>
+ <term><option>--strip-extension=<replaceable class="parameter">extension</replaceable></option></term>
<listitem>
<para>
Provide an extension
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index 7726d05149..fc0dca9856 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -17,7 +17,7 @@
#include "access/xlog_internal.h"
#include "common/logging.h"
-#include "pg_getopt.h"
+#include "getopt_long.h"
const char *progname;
@@ -252,11 +252,13 @@ usage(void)
printf(_("Usage:\n"));
printf(_(" %s [OPTION]... ARCHIVELOCATION OLDESTKEPTWALFILE\n"), progname);
printf(_("\nOptions:\n"));
- printf(_(" -d generate debug output (verbose mode)\n"));
- printf(_(" -n dry run, show the names of the files that would be removed\n"));
- printf(_(" -V, --version output version information, then exit\n"));
- printf(_(" -x EXT clean up files if they have this extension\n"));
- printf(_(" -?, --help show this help, then exit\n"));
+ printf(_(" -d, --debug generate debug output (verbose mode)\n"));
+ printf(_(" -n, --dry-run dry run, show the names of the files that would be\n"
+ " removed\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -x, --strip-extension=EXT strip this extention before identifying files for\n"
+ " clean up\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"For use as archive_cleanup_command in postgresql.conf:\n"
" archive_cleanup_command = 'pg_archivecleanup [OPTION]... ARCHIVELOCATION %%r'\n"
@@ -274,6 +276,12 @@ usage(void)
int
main(int argc, char **argv)
{
+ static struct option long_options[] = {
+ {"debug", no_argument, NULL, 'd'},
+ {"dry-run", no_argument, NULL, 'n'},
+ {"strip-extension", required_argument, NULL, 'x'},
+ {NULL, 0, NULL, 0}
+ };
int c;
pg_logging_init(argv[0]);
@@ -294,7 +302,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt(argc, argv, "dnx:")) != -1)
+ while ((c = getopt_long(argc, argv, "dnx:", long_options, NULL)) != -1)
{
switch (c)
{
--
2.39.2
[text/x-diff] v9-0002-Refactored-to-reduce-the-nesting-level.patch (5.3K, ../../[email protected]/3-v9-0002-Refactored-to-reduce-the-nesting-level.patch)
download | inline diff:
From e61734f9d2efd326bc259e3f869207404fe80143 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 19 Jun 2023 20:47:53 +0900
Subject: [PATCH v9 2/3] Refactored to reduce the nesting level
---
src/bin/pg_archivecleanup/pg_archivecleanup.c | 114 +++++++++---------
1 file changed, 56 insertions(+), 58 deletions(-)
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index fc0dca9856..950e27e63b 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -93,75 +93,73 @@ CleanupPriorWALFiles(void)
struct dirent *xlde;
char walfile[MAXPGPATH];
- if ((xldir = opendir(archiveLocation)) != NULL)
+ if ((xldir = opendir(archiveLocation)) == NULL)
+ pg_fatal("could not open archive location \"%s\": %m",
+ archiveLocation);
+
+ while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
- while (errno = 0, (xlde = readdir(xldir)) != NULL)
+ /*
+ * Truncation is essentially harmless, because we skip names of
+ * length other than XLOG_FNAME_LEN. (In principle, one could use
+ * a 1000-character additional_ext and get trouble.)
+ */
+ strlcpy(walfile, xlde->d_name, MAXPGPATH);
+ TrimExtension(walfile, additional_ext);
+
+ /*
+ * We ignore the timeline part of the XLOG segment identifiers in
+ * deciding whether a segment is still needed. This ensures that
+ * we won't prematurely remove a segment from a parent timeline.
+ * We could probably be a little more proactive about removing
+ * segments of non-parent timelines, but that would be a whole lot
+ * more complicated.
+ *
+ * We use the alphanumeric sorting property of the filenames to
+ * decide which ones are earlier than the exclusiveCleanupFileName
+ * file. Note that this means files are not removed in the order
+ * they were originally written, in case this worries you.
+ */
+ if ((IsXLogFileName(walfile) || IsPartialXLogFileName(walfile)) &&
+ strcmp(walfile + 8, exclusiveCleanupFileName + 8) < 0)
{
- /*
- * Truncation is essentially harmless, because we skip names of
- * length other than XLOG_FNAME_LEN. (In principle, one could use
- * a 1000-character additional_ext and get trouble.)
- */
- strlcpy(walfile, xlde->d_name, MAXPGPATH);
- TrimExtension(walfile, additional_ext);
+ char WALFilePath[MAXPGPATH * 2]; /* the file path
+ * including archive */
/*
- * We ignore the timeline part of the XLOG segment identifiers in
- * deciding whether a segment is still needed. This ensures that
- * we won't prematurely remove a segment from a parent timeline.
- * We could probably be a little more proactive about removing
- * segments of non-parent timelines, but that would be a whole lot
- * more complicated.
- *
- * We use the alphanumeric sorting property of the filenames to
- * decide which ones are earlier than the exclusiveCleanupFileName
- * file. Note that this means files are not removed in the order
- * they were originally written, in case this worries you.
+ * Use the original file name again now, including any
+ * extension that might have been chopped off before testing
+ * the sequence.
*/
- if ((IsXLogFileName(walfile) || IsPartialXLogFileName(walfile)) &&
- strcmp(walfile + 8, exclusiveCleanupFileName + 8) < 0)
- {
- char WALFilePath[MAXPGPATH * 2]; /* the file path
- * including archive */
+ snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
+ archiveLocation, xlde->d_name);
+ if (dryrun)
+ {
/*
- * Use the original file name again now, including any
- * extension that might have been chopped off before testing
- * the sequence.
+ * Prints the name of the file to be removed and skips the
+ * actual removal. The regular printout is so that the
+ * user can pipe the output into some other program.
*/
- snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
- archiveLocation, xlde->d_name);
-
- if (dryrun)
- {
- /*
- * Prints the name of the file to be removed and skips the
- * actual removal. The regular printout is so that the
- * user can pipe the output into some other program.
- */
- printf("%s\n", WALFilePath);
- pg_log_debug("file \"%s\" would be removed", WALFilePath);
- continue;
- }
-
- pg_log_debug("removing file \"%s\"", WALFilePath);
-
- rc = unlink(WALFilePath);
- if (rc != 0)
- pg_fatal("could not remove file \"%s\": %m",
- WALFilePath);
+ printf("%s\n", WALFilePath);
+ pg_log_debug("file \"%s\" would be removed", WALFilePath);
+ continue;
}
- }
- if (errno)
- pg_fatal("could not read archive location \"%s\": %m",
- archiveLocation);
- if (closedir(xldir))
- pg_fatal("could not close archive location \"%s\": %m",
- archiveLocation);
+ pg_log_debug("removing file \"%s\"", WALFilePath);
+
+ rc = unlink(WALFilePath);
+ if (rc != 0)
+ pg_fatal("could not remove file \"%s\": %m",
+ WALFilePath);
+ }
}
- else
- pg_fatal("could not open archive location \"%s\": %m",
+
+ if (errno)
+ pg_fatal("could not read archive location \"%s\": %m",
+ archiveLocation);
+ if (closedir(xldir))
+ pg_fatal("could not close archive location \"%s\": %m",
archiveLocation);
}
--
2.39.2
[text/x-diff] v9-0003-Allow-pg_archivecleanup-to-remove-backup-history-.patch (10.6K, ../../[email protected]/4-v9-0003-Allow-pg_archivecleanup-to-remove-backup-history-.patch)
download | inline diff:
From d938f968848b762d3665f7e7cbf6bf56574758d6 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 19 Jun 2023 20:54:04 +0900
Subject: [PATCH v9 3/3] Allow pg_archivecleanup to remove backup history files
Backup history files are just few bytes, but it can be noisy for the
eye to see a large accumulation of history files mixed with the WAL
segments.
This patch adds a new option to remove files including backup
history files older oldestkeptwalfile.
---
doc/src/sgml/ref/pgarchivecleanup.sgml | 11 +++
src/bin/pg_archivecleanup/pg_archivecleanup.c | 85 ++++++++++++-------
.../t/010_pg_archivecleanup.pl | 79 +++++++++++------
3 files changed, 117 insertions(+), 58 deletions(-)
diff --git a/doc/src/sgml/ref/pgarchivecleanup.sgml b/doc/src/sgml/ref/pgarchivecleanup.sgml
index 09991c2fcd..8fac233db6 100644
--- a/doc/src/sgml/ref/pgarchivecleanup.sgml
+++ b/doc/src/sgml/ref/pgarchivecleanup.sgml
@@ -113,6 +113,17 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>-b</option></term>
+ <term><option>--clean-backup-history</option></term>
+ <listitem>
+ <para>
+ Remove backup history files.
+ For details about backup history file, please refer to the <xref linkend="backup-base-backup"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-V</option></term>
<term><option>--version</option></term>
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index 950e27e63b..ffb4a19e7f 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -23,6 +23,8 @@ const char *progname;
/* Options and defaults */
bool dryrun = false; /* are we performing a dry-run operation? */
+bool cleanBackupHistory = false; /* remove files including
+ * backup history files */
char *additional_ext = NULL; /* Extension to remove from filenames */
char *archiveLocation; /* where to find the archive? */
@@ -99,15 +101,31 @@ CleanupPriorWALFiles(void)
while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
+ char WALFilePath[MAXPGPATH * 2]; /* the file path
+ * including archive */
/*
- * Truncation is essentially harmless, because we skip names of
- * length other than XLOG_FNAME_LEN. (In principle, one could use
- * a 1000-character additional_ext and get trouble.)
+ * Truncation is essentially harmless, because we check the file
+ * format including the length immediately after this.
+ * (In principle, one could use a 1000-character additional_ext
+ * and get trouble.)
*/
strlcpy(walfile, xlde->d_name, MAXPGPATH);
TrimExtension(walfile, additional_ext);
/*
+ * Check file name.
+ *
+ * We skip files which are not WAL file or partial WAL file.
+ * Also we skip backup history files when --clean-backup-history
+ * is not specified.
+ */
+ if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile) &&
+ !(cleanBackupHistory && IsBackupHistoryFileName(walfile)))
+ continue;
+
+ /*
+ * Check cutoff point.
+ *
* We ignore the timeline part of the XLOG segment identifiers in
* deciding whether a segment is still needed. This ensures that
* we won't prematurely remove a segment from a parent timeline.
@@ -120,39 +138,35 @@ CleanupPriorWALFiles(void)
* file. Note that this means files are not removed in the order
* they were originally written, in case this worries you.
*/
- if ((IsXLogFileName(walfile) || IsPartialXLogFileName(walfile)) &&
- strcmp(walfile + 8, exclusiveCleanupFileName + 8) < 0)
- {
- char WALFilePath[MAXPGPATH * 2]; /* the file path
- * including archive */
+ if (strcmp(walfile + 8, exclusiveCleanupFileName + 8) >= 0)
+ continue;
+ /*
+ * Use the original file name again now, including any
+ * extension that might have been chopped off before testing
+ * the sequence.
+ */
+ snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
+ archiveLocation, xlde->d_name);
+
+ if (dryrun)
+ {
/*
- * Use the original file name again now, including any
- * extension that might have been chopped off before testing
- * the sequence.
+ * Prints the name of the file to be removed and skips the
+ * actual removal. The regular printout is so that the
+ * user can pipe the output into some other program.
*/
- snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
- archiveLocation, xlde->d_name);
-
- if (dryrun)
- {
- /*
- * Prints the name of the file to be removed and skips the
- * actual removal. The regular printout is so that the
- * user can pipe the output into some other program.
- */
- printf("%s\n", WALFilePath);
- pg_log_debug("file \"%s\" would be removed", WALFilePath);
- continue;
- }
-
- pg_log_debug("removing file \"%s\"", WALFilePath);
-
- rc = unlink(WALFilePath);
- if (rc != 0)
- pg_fatal("could not remove file \"%s\": %m",
- WALFilePath);
+ printf("%s\n", WALFilePath);
+ pg_log_debug("file \"%s\" would be removed", WALFilePath);
+ continue;
}
+
+ pg_log_debug("removing file \"%s\"", WALFilePath);
+
+ rc = unlink(WALFilePath);
+ if (rc != 0)
+ pg_fatal("could not remove file \"%s\": %m",
+ WALFilePath);
}
if (errno)
@@ -250,6 +264,7 @@ usage(void)
printf(_("Usage:\n"));
printf(_(" %s [OPTION]... ARCHIVELOCATION OLDESTKEPTWALFILE\n"), progname);
printf(_("\nOptions:\n"));
+ printf(_(" -b, --clean-backup-history clean up files including backup history files\n"));
printf(_(" -d, --debug generate debug output (verbose mode)\n"));
printf(_(" -n, --dry-run dry run, show the names of the files that would be\n"
" removed\n"));
@@ -275,6 +290,7 @@ int
main(int argc, char **argv)
{
static struct option long_options[] = {
+ {"clean-backup-history", no_argument, NULL, 'b'},
{"debug", no_argument, NULL, 'd'},
{"dry-run", no_argument, NULL, 'n'},
{"strip-extension", required_argument, NULL, 'x'},
@@ -300,10 +316,13 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "dnx:", long_options, NULL)) != -1)
+ while ((c = getopt_long(argc, argv, "bdnx:", long_options, NULL)) != -1)
{
switch (c)
{
+ case 'b': /* Remove backup history files too */
+ cleanBackupHistory = true;
+ break;
case 'd': /* Debug mode */
pg_logging_increase_verbosity();
break;
diff --git a/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl b/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
index cc3386d146..030a82e7fa 100644
--- a/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
+++ b/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
@@ -12,13 +12,23 @@ program_options_handling_ok('pg_archivecleanup');
my $tempdir = PostgreSQL::Test::Utils::tempdir;
-my @walfiles = (
- '00000001000000370000000C.gz', '00000001000000370000000D',
- '00000001000000370000000E', '00000001000000370000000F.partial',);
+my @walfiles_with_gz = (
+ { name => '00000001000000370000000C.gz', present => 0 },
+ { name => '00000001000000370000000D', present => 0 },
+ { name => '00000001000000370000000E', present => 1 },
+ { name => '00000001000000370000000F.partial', present => 1 },
+ { name => 'unrelated_file', present => 1 });
+
+my @walfiles_with_backup = (
+ { name => '00000001000000370000000D', present => 0 },
+ { name => '00000001000000370000000D.00000028.backup', present => 0 },
+ { name => '00000001000000370000000E', present => 1 },
+ { name => '00000001000000370000000F.partial', present => 1 },
+ { name => 'unrelated_file', present => 1 });
sub create_files
{
- foreach my $fn (@walfiles, 'unrelated_file')
+ foreach my $fn (@_)
{
open my $file, '>', "$tempdir/$fn";
print $file 'CONTENT';
@@ -27,7 +37,18 @@ sub create_files
return;
}
-create_files();
+sub get_walfiles
+{
+ my @walfiles;
+
+ for my $walpair (@_)
+ {
+ push @walfiles, $walpair->{name};
+ }
+ return @walfiles;
+}
+
+create_files(get_walfiles(@walfiles_with_gz));
command_fails_like(
['pg_archivecleanup'],
@@ -59,14 +80,14 @@ command_fails_like(
my $stderr;
my $result =
IPC::Run::run [ 'pg_archivecleanup', '-d', '-n', $tempdir,
- $walfiles[2] ],
+ $walfiles_with_gz[2]->{name} ],
'2>', \$stderr;
ok($result, "pg_archivecleanup dry run: exit code 0");
like(
$stderr,
- qr/$walfiles[1].*would be removed/,
+ qr/$walfiles_with_gz[1]->{name}.*would be removed/,
"pg_archivecleanup dry run: matches");
- foreach my $fn (@walfiles)
+ foreach my $fn (get_walfiles(@walfiles_with_gz))
{
ok(-f "$tempdir/$fn", "$fn not removed");
}
@@ -76,32 +97,40 @@ sub run_check
{
local $Test::Builder::Level = $Test::Builder::Level + 1;
- my ($suffix, $test_name) = @_;
+ my ($testdata, $oldestkeptwalfile, $test_name, @options) = @_;
- create_files();
+ create_files(get_walfiles(@$testdata));
command_ok(
[
- 'pg_archivecleanup', '-x', '.gz', $tempdir,
- $walfiles[2] . $suffix
+ 'pg_archivecleanup', @options, $tempdir,
+ $oldestkeptwalfile
],
"$test_name: runs");
- ok(!-f "$tempdir/$walfiles[0]",
- "$test_name: first older WAL file was cleaned up");
- ok(!-f "$tempdir/$walfiles[1]",
- "$test_name: second older WAL file was cleaned up");
- ok(-f "$tempdir/$walfiles[2]",
- "$test_name: restartfile was not cleaned up");
- ok(-f "$tempdir/$walfiles[3]",
- "$test_name: newer WAL file was not cleaned up");
- ok(-f "$tempdir/unrelated_file",
- "$test_name: unrelated file was not cleaned up");
+ for my $walpair (@$testdata)
+ {
+ if ($walpair->{present})
+ {
+ ok(-f "$tempdir/$walpair->{name}",
+ "$test_name:$walpair->{name} was not cleaned up");
+ }
+ else
+ {
+ ok(!-f "$tempdir/$walpair->{name}",
+ "$test_name:$walpair->{name} was cleaned up");
+ }
+ }
return;
}
-run_check('', 'pg_archivecleanup');
-run_check('.partial', 'pg_archivecleanup with .partial file');
-run_check('.00000020.backup', 'pg_archivecleanup with .backup file');
+run_check(\@walfiles_with_gz, '00000001000000370000000E',
+ 'pg_archivecleanup', '-x.gz');
+run_check(\@walfiles_with_gz, '00000001000000370000000E.partial',
+ 'pg_archivecleanup with .partial file', '-x.gz');
+run_check(\@walfiles_with_gz, '00000001000000370000000E.00000020.backup',
+ 'pg_archivecleanup with .backup file', '-x.gz');
+run_check(\@walfiles_with_backup, '00000001000000370000000E',
+ 'pg_archivecleanup with --clean-backup-history', '-b');
done_testing();
--
2.39.2
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Allow pg_archivecleanup to remove backup history files
2023-06-13 15:49 Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-15 06:20 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-15 12:38 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-16 02:22 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-16 02:30 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-19 02:24 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-19 05:37 ` Re: Allow pg_archivecleanup to remove backup history files Michael Paquier <[email protected]>
2023-06-20 13:27 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
@ 2023-06-21 02:59 ` Kyotaro Horiguchi <[email protected]>
2023-06-21 14:41 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
0 siblings, 1 reply; 70+ messages in thread
From: Kyotaro Horiguchi @ 2023-06-21 02:59 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; pgsql-hackers
At Tue, 20 Jun 2023 22:27:36 +0900, torikoshia <[email protected]> wrote in
> On 2023-06-19 14:37, Michael Paquier wrote:
> > On Mon, Jun 19, 2023 at 11:24:29AM +0900, torikoshia wrote:
> >> Thanks, now I understand what you meant.
> > If I may ask, why is the refactoring of 0003 done after the feature in
> > 0002? Shouldn't the order be reversed? That would make for a cleaner
> > git history.
> > --
> > Michael
>
> Agreed.
> Reversed the order of patches 0002 and 0003.
Yeah, that is a possible division. However, I meant that we have room
to refactor and decrease the nesting level even further, considering
that 0003 already does this to some extent, when I suggested it. In
that sense, moving the nest-reduction part of 0003 into 0002 makes us
possible to focus on the point of this patch.
What do you think about the attached version?
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
Attachments:
[text/x-patch] v10-0001-Introduce-pg_archivecleanup-into-getopt_long.patch (4.0K, ../../[email protected]/2-v10-0001-Introduce-pg_archivecleanup-into-getopt_long.patch)
download | inline diff:
From 5777c28c01a6ca3bb48baf7dfabf6e720fcaaf5e Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Wed, 21 Jun 2023 11:34:10 +0900
Subject: [PATCH v10 1/3] Introduce pg_archivecleanup into getopt_long
This patch is a preliminary step to add an easy-to-understand option
to delete backup history files, but it also adds long options to
the existing options.
---
doc/src/sgml/ref/pgarchivecleanup.sgml | 5 ++++-
src/bin/pg_archivecleanup/pg_archivecleanup.c | 22 +++++++++++++------
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/ref/pgarchivecleanup.sgml b/doc/src/sgml/ref/pgarchivecleanup.sgml
index 635e7c7685..09991c2fcd 100644
--- a/doc/src/sgml/ref/pgarchivecleanup.sgml
+++ b/doc/src/sgml/ref/pgarchivecleanup.sgml
@@ -95,6 +95,7 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
<varlistentry>
<term><option>-d</option></term>
+ <term><option>--debug</option></term>
<listitem>
<para>
Print lots of debug logging output on <filename>stderr</filename>.
@@ -104,6 +105,7 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
<varlistentry>
<term><option>-n</option></term>
+ <term><option>--dry-run</option></term>
<listitem>
<para>
Print the names of the files that would have been removed on <filename>stdout</filename> (performs a dry run).
@@ -122,7 +124,8 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
</varlistentry>
<varlistentry>
- <term><option>-x</option> <replaceable>extension</replaceable></term>
+ <term><option>-x <replaceable class="parameter">extension</replaceable></option></term>
+ <term><option>--strip-extension=<replaceable class="parameter">extension</replaceable></option></term>
<listitem>
<para>
Provide an extension
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index 7726d05149..fc0dca9856 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -17,7 +17,7 @@
#include "access/xlog_internal.h"
#include "common/logging.h"
-#include "pg_getopt.h"
+#include "getopt_long.h"
const char *progname;
@@ -252,11 +252,13 @@ usage(void)
printf(_("Usage:\n"));
printf(_(" %s [OPTION]... ARCHIVELOCATION OLDESTKEPTWALFILE\n"), progname);
printf(_("\nOptions:\n"));
- printf(_(" -d generate debug output (verbose mode)\n"));
- printf(_(" -n dry run, show the names of the files that would be removed\n"));
- printf(_(" -V, --version output version information, then exit\n"));
- printf(_(" -x EXT clean up files if they have this extension\n"));
- printf(_(" -?, --help show this help, then exit\n"));
+ printf(_(" -d, --debug generate debug output (verbose mode)\n"));
+ printf(_(" -n, --dry-run dry run, show the names of the files that would be\n"
+ " removed\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -x, --strip-extension=EXT strip this extention before identifying files for\n"
+ " clean up\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"For use as archive_cleanup_command in postgresql.conf:\n"
" archive_cleanup_command = 'pg_archivecleanup [OPTION]... ARCHIVELOCATION %%r'\n"
@@ -274,6 +276,12 @@ usage(void)
int
main(int argc, char **argv)
{
+ static struct option long_options[] = {
+ {"debug", no_argument, NULL, 'd'},
+ {"dry-run", no_argument, NULL, 'n'},
+ {"strip-extension", required_argument, NULL, 'x'},
+ {NULL, 0, NULL, 0}
+ };
int c;
pg_logging_init(argv[0]);
@@ -294,7 +302,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt(argc, argv, "dnx:")) != -1)
+ while ((c = getopt_long(argc, argv, "dnx:", long_options, NULL)) != -1)
{
switch (c)
{
--
2.39.3
[text/x-patch] v10-0002-Preliminary-refactoring-for-a-subsequent-patch.patch (5.8K, ../../[email protected]/3-v10-0002-Preliminary-refactoring-for-a-subsequent-patch.patch)
download | inline diff:
From 060922adc1e529c7f6ce5a7bdf726d6f4acaeab4 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Wed, 21 Jun 2023 11:34:53 +0900
Subject: [PATCH v10 2/3] Preliminary refactoring for a subsequent patch
This is a preparatory patch that doesn't introduce any functional
change. Instead, this carries out preliminary refactoring with the
goal of reducing the overall nesting level. This helps to prevent the
forthcoming patch from further deepening the nesting level.
---
src/bin/pg_archivecleanup/pg_archivecleanup.c | 144 +++++++++---------
1 file changed, 76 insertions(+), 68 deletions(-)
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index fc0dca9856..f7269068cb 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -93,76 +93,84 @@ CleanupPriorWALFiles(void)
struct dirent *xlde;
char walfile[MAXPGPATH];
- if ((xldir = opendir(archiveLocation)) != NULL)
- {
- while (errno = 0, (xlde = readdir(xldir)) != NULL)
- {
- /*
- * Truncation is essentially harmless, because we skip names of
- * length other than XLOG_FNAME_LEN. (In principle, one could use
- * a 1000-character additional_ext and get trouble.)
- */
- strlcpy(walfile, xlde->d_name, MAXPGPATH);
- TrimExtension(walfile, additional_ext);
-
- /*
- * We ignore the timeline part of the XLOG segment identifiers in
- * deciding whether a segment is still needed. This ensures that
- * we won't prematurely remove a segment from a parent timeline.
- * We could probably be a little more proactive about removing
- * segments of non-parent timelines, but that would be a whole lot
- * more complicated.
- *
- * We use the alphanumeric sorting property of the filenames to
- * decide which ones are earlier than the exclusiveCleanupFileName
- * file. Note that this means files are not removed in the order
- * they were originally written, in case this worries you.
- */
- if ((IsXLogFileName(walfile) || IsPartialXLogFileName(walfile)) &&
- strcmp(walfile + 8, exclusiveCleanupFileName + 8) < 0)
- {
- char WALFilePath[MAXPGPATH * 2]; /* the file path
- * including archive */
-
- /*
- * Use the original file name again now, including any
- * extension that might have been chopped off before testing
- * the sequence.
- */
- snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
- archiveLocation, xlde->d_name);
-
- if (dryrun)
- {
- /*
- * Prints the name of the file to be removed and skips the
- * actual removal. The regular printout is so that the
- * user can pipe the output into some other program.
- */
- printf("%s\n", WALFilePath);
- pg_log_debug("file \"%s\" would be removed", WALFilePath);
- continue;
- }
-
- pg_log_debug("removing file \"%s\"", WALFilePath);
-
- rc = unlink(WALFilePath);
- if (rc != 0)
- pg_fatal("could not remove file \"%s\": %m",
- WALFilePath);
- }
- }
-
- if (errno)
- pg_fatal("could not read archive location \"%s\": %m",
- archiveLocation);
- if (closedir(xldir))
- pg_fatal("could not close archive location \"%s\": %m",
- archiveLocation);
- }
- else
+ if ((xldir = opendir(archiveLocation)) == NULL)
pg_fatal("could not open archive location \"%s\": %m",
archiveLocation);
+
+ while (errno = 0, (xlde = readdir(xldir)) != NULL)
+ {
+ char WALFilePath[MAXPGPATH * 2]; /* the file path
+ * including archive */
+ /*
+ * Truncation is essentially harmless, because we skip names of
+ * length other than XLOG_FNAME_LEN. (In principle, one could use
+ * a 1000-character additional_ext and get trouble.)
+ */
+ strlcpy(walfile, xlde->d_name, MAXPGPATH);
+ TrimExtension(walfile, additional_ext);
+
+ /*
+ * Check file name.
+ *
+ * We skip files which are not WAL file or partial WAL file.
+ * Also we skip backup history files when --clean-backup-history
+ * is not specified.
+ */
+ if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile))
+ continue;
+
+ /*
+ * Check cutoff point.
+ *
+ * We ignore the timeline part of the XLOG segment identifiers in
+ * deciding whether a segment is still needed. This ensures that
+ * we won't prematurely remove a segment from a parent timeline.
+ * We could probably be a little more proactive about removing
+ * segments of non-parent timelines, but that would be a whole lot
+ * more complicated.
+ *
+ * We use the alphanumeric sorting property of the filenames to
+ * decide which ones are earlier than the exclusiveCleanupFileName
+ * file. Note that this means files are not removed in the order
+ * they were originally written, in case this worries you.
+ */
+ if (strcmp(walfile + 8, exclusiveCleanupFileName + 8) >= 0)
+ continue;
+
+ /*
+ * Use the original file name again now, including any
+ * extension that might have been chopped off before testing
+ * the sequence.
+ */
+ snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
+ archiveLocation, xlde->d_name);
+
+ if (dryrun)
+ {
+ /*
+ * Prints the name of the file to be removed and skips the
+ * actual removal. The regular printout is so that the
+ * user can pipe the output into some other program.
+ */
+ printf("%s\n", WALFilePath);
+ pg_log_debug("file \"%s\" would be removed", WALFilePath);
+ continue;
+ }
+
+ pg_log_debug("removing file \"%s\"", WALFilePath);
+
+ rc = unlink(WALFilePath);
+ if (rc != 0)
+ pg_fatal("could not remove file \"%s\": %m",
+ WALFilePath);
+ }
+
+ if (errno)
+ pg_fatal("could not read archive location \"%s\": %m",
+ archiveLocation);
+ if (closedir(xldir))
+ pg_fatal("could not close archive location \"%s\": %m",
+ archiveLocation);
}
/*
--
2.39.3
[text/x-patch] v10-0003-Allow-pg_archivecleanup-to-remove-backup-history.patch (8.3K, ../../[email protected]/4-v10-0003-Allow-pg_archivecleanup-to-remove-backup-history.patch)
download | inline diff:
From 110aeb49d8fb0c50777e719ae75fd3faa4b759c3 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Wed, 21 Jun 2023 11:36:28 +0900
Subject: [PATCH v10 3/3] Allow pg_archivecleanup to remove backup history
files
Backup history files are just few bytes, but it can be noisy for the
eye to see a large accumulation of history files mixed with the WAL
segments.
This patch adds a new option to remove files including backup
history files older oldestkeptwalfile.
---
doc/src/sgml/ref/pgarchivecleanup.sgml | 11 +++
src/bin/pg_archivecleanup/pg_archivecleanup.c | 19 +++--
.../t/010_pg_archivecleanup.pl | 79 +++++++++++++------
3 files changed, 79 insertions(+), 30 deletions(-)
diff --git a/doc/src/sgml/ref/pgarchivecleanup.sgml b/doc/src/sgml/ref/pgarchivecleanup.sgml
index 09991c2fcd..8fac233db6 100644
--- a/doc/src/sgml/ref/pgarchivecleanup.sgml
+++ b/doc/src/sgml/ref/pgarchivecleanup.sgml
@@ -113,6 +113,17 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>-b</option></term>
+ <term><option>--clean-backup-history</option></term>
+ <listitem>
+ <para>
+ Remove backup history files.
+ For details about backup history file, please refer to the <xref linkend="backup-base-backup"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-V</option></term>
<term><option>--version</option></term>
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index f7269068cb..ffb4a19e7f 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -23,6 +23,8 @@ const char *progname;
/* Options and defaults */
bool dryrun = false; /* are we performing a dry-run operation? */
+bool cleanBackupHistory = false; /* remove files including
+ * backup history files */
char *additional_ext = NULL; /* Extension to remove from filenames */
char *archiveLocation; /* where to find the archive? */
@@ -102,9 +104,10 @@ CleanupPriorWALFiles(void)
char WALFilePath[MAXPGPATH * 2]; /* the file path
* including archive */
/*
- * Truncation is essentially harmless, because we skip names of
- * length other than XLOG_FNAME_LEN. (In principle, one could use
- * a 1000-character additional_ext and get trouble.)
+ * Truncation is essentially harmless, because we check the file
+ * format including the length immediately after this.
+ * (In principle, one could use a 1000-character additional_ext
+ * and get trouble.)
*/
strlcpy(walfile, xlde->d_name, MAXPGPATH);
TrimExtension(walfile, additional_ext);
@@ -116,7 +119,8 @@ CleanupPriorWALFiles(void)
* Also we skip backup history files when --clean-backup-history
* is not specified.
*/
- if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile))
+ if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile) &&
+ !(cleanBackupHistory && IsBackupHistoryFileName(walfile)))
continue;
/*
@@ -260,6 +264,7 @@ usage(void)
printf(_("Usage:\n"));
printf(_(" %s [OPTION]... ARCHIVELOCATION OLDESTKEPTWALFILE\n"), progname);
printf(_("\nOptions:\n"));
+ printf(_(" -b, --clean-backup-history clean up files including backup history files\n"));
printf(_(" -d, --debug generate debug output (verbose mode)\n"));
printf(_(" -n, --dry-run dry run, show the names of the files that would be\n"
" removed\n"));
@@ -285,6 +290,7 @@ int
main(int argc, char **argv)
{
static struct option long_options[] = {
+ {"clean-backup-history", no_argument, NULL, 'b'},
{"debug", no_argument, NULL, 'd'},
{"dry-run", no_argument, NULL, 'n'},
{"strip-extension", required_argument, NULL, 'x'},
@@ -310,10 +316,13 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "dnx:", long_options, NULL)) != -1)
+ while ((c = getopt_long(argc, argv, "bdnx:", long_options, NULL)) != -1)
{
switch (c)
{
+ case 'b': /* Remove backup history files too */
+ cleanBackupHistory = true;
+ break;
case 'd': /* Debug mode */
pg_logging_increase_verbosity();
break;
diff --git a/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl b/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
index cc3386d146..030a82e7fa 100644
--- a/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
+++ b/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
@@ -12,13 +12,23 @@ program_options_handling_ok('pg_archivecleanup');
my $tempdir = PostgreSQL::Test::Utils::tempdir;
-my @walfiles = (
- '00000001000000370000000C.gz', '00000001000000370000000D',
- '00000001000000370000000E', '00000001000000370000000F.partial',);
+my @walfiles_with_gz = (
+ { name => '00000001000000370000000C.gz', present => 0 },
+ { name => '00000001000000370000000D', present => 0 },
+ { name => '00000001000000370000000E', present => 1 },
+ { name => '00000001000000370000000F.partial', present => 1 },
+ { name => 'unrelated_file', present => 1 });
+
+my @walfiles_with_backup = (
+ { name => '00000001000000370000000D', present => 0 },
+ { name => '00000001000000370000000D.00000028.backup', present => 0 },
+ { name => '00000001000000370000000E', present => 1 },
+ { name => '00000001000000370000000F.partial', present => 1 },
+ { name => 'unrelated_file', present => 1 });
sub create_files
{
- foreach my $fn (@walfiles, 'unrelated_file')
+ foreach my $fn (@_)
{
open my $file, '>', "$tempdir/$fn";
print $file 'CONTENT';
@@ -27,7 +37,18 @@ sub create_files
return;
}
-create_files();
+sub get_walfiles
+{
+ my @walfiles;
+
+ for my $walpair (@_)
+ {
+ push @walfiles, $walpair->{name};
+ }
+ return @walfiles;
+}
+
+create_files(get_walfiles(@walfiles_with_gz));
command_fails_like(
['pg_archivecleanup'],
@@ -59,14 +80,14 @@ command_fails_like(
my $stderr;
my $result =
IPC::Run::run [ 'pg_archivecleanup', '-d', '-n', $tempdir,
- $walfiles[2] ],
+ $walfiles_with_gz[2]->{name} ],
'2>', \$stderr;
ok($result, "pg_archivecleanup dry run: exit code 0");
like(
$stderr,
- qr/$walfiles[1].*would be removed/,
+ qr/$walfiles_with_gz[1]->{name}.*would be removed/,
"pg_archivecleanup dry run: matches");
- foreach my $fn (@walfiles)
+ foreach my $fn (get_walfiles(@walfiles_with_gz))
{
ok(-f "$tempdir/$fn", "$fn not removed");
}
@@ -76,32 +97,40 @@ sub run_check
{
local $Test::Builder::Level = $Test::Builder::Level + 1;
- my ($suffix, $test_name) = @_;
+ my ($testdata, $oldestkeptwalfile, $test_name, @options) = @_;
- create_files();
+ create_files(get_walfiles(@$testdata));
command_ok(
[
- 'pg_archivecleanup', '-x', '.gz', $tempdir,
- $walfiles[2] . $suffix
+ 'pg_archivecleanup', @options, $tempdir,
+ $oldestkeptwalfile
],
"$test_name: runs");
- ok(!-f "$tempdir/$walfiles[0]",
- "$test_name: first older WAL file was cleaned up");
- ok(!-f "$tempdir/$walfiles[1]",
- "$test_name: second older WAL file was cleaned up");
- ok(-f "$tempdir/$walfiles[2]",
- "$test_name: restartfile was not cleaned up");
- ok(-f "$tempdir/$walfiles[3]",
- "$test_name: newer WAL file was not cleaned up");
- ok(-f "$tempdir/unrelated_file",
- "$test_name: unrelated file was not cleaned up");
+ for my $walpair (@$testdata)
+ {
+ if ($walpair->{present})
+ {
+ ok(-f "$tempdir/$walpair->{name}",
+ "$test_name:$walpair->{name} was not cleaned up");
+ }
+ else
+ {
+ ok(!-f "$tempdir/$walpair->{name}",
+ "$test_name:$walpair->{name} was cleaned up");
+ }
+ }
return;
}
-run_check('', 'pg_archivecleanup');
-run_check('.partial', 'pg_archivecleanup with .partial file');
-run_check('.00000020.backup', 'pg_archivecleanup with .backup file');
+run_check(\@walfiles_with_gz, '00000001000000370000000E',
+ 'pg_archivecleanup', '-x.gz');
+run_check(\@walfiles_with_gz, '00000001000000370000000E.partial',
+ 'pg_archivecleanup with .partial file', '-x.gz');
+run_check(\@walfiles_with_gz, '00000001000000370000000E.00000020.backup',
+ 'pg_archivecleanup with .backup file', '-x.gz');
+run_check(\@walfiles_with_backup, '00000001000000370000000E',
+ 'pg_archivecleanup with --clean-backup-history', '-b');
done_testing();
--
2.39.3
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Allow pg_archivecleanup to remove backup history files
2023-06-13 15:49 Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-15 06:20 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-15 12:38 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-16 02:22 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-16 02:30 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-19 02:24 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-19 05:37 ` Re: Allow pg_archivecleanup to remove backup history files Michael Paquier <[email protected]>
2023-06-20 13:27 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-21 02:59 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
@ 2023-06-21 14:41 ` torikoshia <[email protected]>
2023-06-22 07:47 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
0 siblings, 1 reply; 70+ messages in thread
From: torikoshia @ 2023-06-21 14:41 UTC (permalink / raw)
To: Kyotaro Horiguchi <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]; pgsql-hackers
On 2023-06-21 11:59, Kyotaro Horiguchi wrote:
> At Tue, 20 Jun 2023 22:27:36 +0900, torikoshia
> <[email protected]> wrote in
>> On 2023-06-19 14:37, Michael Paquier wrote:
>> > On Mon, Jun 19, 2023 at 11:24:29AM +0900, torikoshia wrote:
>> >> Thanks, now I understand what you meant.
>> > If I may ask, why is the refactoring of 0003 done after the feature in
>> > 0002? Shouldn't the order be reversed? That would make for a cleaner
>> > git history.
>> > --
>> > Michael
>>
>> Agreed.
>> Reversed the order of patches 0002 and 0003.
>
> Yeah, that is a possible division. However, I meant that we have room
> to refactor and decrease the nesting level even further, considering
> that 0003 already does this to some extent, when I suggested it. In
> that sense, moving the nest-reduction part of 0003 into 0002 makes us
> possible to focus on the point of this patch.
Thanks for the comment, it seems better than v9 patch.
> What do you think about the attached version?
--v10-0002-Preliminary-refactoring-for-a-subsequent-patch.patch
+ * Also we skip backup history files when --clean-backup-history
+ * is not specified.
+ */
+ if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile))
+ continue;
I think this comment should be located in 0003.
Attached updated patches.
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v11-0001-Introduce-pg_archivecleanup-into-getopt_long.patch (4.0K, ../../[email protected]/2-v11-0001-Introduce-pg_archivecleanup-into-getopt_long.patch)
download | inline diff:
From eb99f581b2990c80329f2baf96ed4d5e9c00dda5 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Wed, 21 Jun 2023 21:42:08 +0900
Subject: [PATCH v11 1/3] Introduce pg_archivecleanup into getopt_long
This patch is a preliminary step to add an easy-to-understand option
to delete backup history files, but it also adds long options to
the existing options.
---
doc/src/sgml/ref/pgarchivecleanup.sgml | 5 ++++-
src/bin/pg_archivecleanup/pg_archivecleanup.c | 22 +++++++++++++------
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/ref/pgarchivecleanup.sgml b/doc/src/sgml/ref/pgarchivecleanup.sgml
index 635e7c7685..09991c2fcd 100644
--- a/doc/src/sgml/ref/pgarchivecleanup.sgml
+++ b/doc/src/sgml/ref/pgarchivecleanup.sgml
@@ -95,6 +95,7 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
<varlistentry>
<term><option>-d</option></term>
+ <term><option>--debug</option></term>
<listitem>
<para>
Print lots of debug logging output on <filename>stderr</filename>.
@@ -104,6 +105,7 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
<varlistentry>
<term><option>-n</option></term>
+ <term><option>--dry-run</option></term>
<listitem>
<para>
Print the names of the files that would have been removed on <filename>stdout</filename> (performs a dry run).
@@ -122,7 +124,8 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
</varlistentry>
<varlistentry>
- <term><option>-x</option> <replaceable>extension</replaceable></term>
+ <term><option>-x <replaceable class="parameter">extension</replaceable></option></term>
+ <term><option>--strip-extension=<replaceable class="parameter">extension</replaceable></option></term>
<listitem>
<para>
Provide an extension
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index 7726d05149..fc0dca9856 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -17,7 +17,7 @@
#include "access/xlog_internal.h"
#include "common/logging.h"
-#include "pg_getopt.h"
+#include "getopt_long.h"
const char *progname;
@@ -252,11 +252,13 @@ usage(void)
printf(_("Usage:\n"));
printf(_(" %s [OPTION]... ARCHIVELOCATION OLDESTKEPTWALFILE\n"), progname);
printf(_("\nOptions:\n"));
- printf(_(" -d generate debug output (verbose mode)\n"));
- printf(_(" -n dry run, show the names of the files that would be removed\n"));
- printf(_(" -V, --version output version information, then exit\n"));
- printf(_(" -x EXT clean up files if they have this extension\n"));
- printf(_(" -?, --help show this help, then exit\n"));
+ printf(_(" -d, --debug generate debug output (verbose mode)\n"));
+ printf(_(" -n, --dry-run dry run, show the names of the files that would be\n"
+ " removed\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -x, --strip-extension=EXT strip this extention before identifying files for\n"
+ " clean up\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"For use as archive_cleanup_command in postgresql.conf:\n"
" archive_cleanup_command = 'pg_archivecleanup [OPTION]... ARCHIVELOCATION %%r'\n"
@@ -274,6 +276,12 @@ usage(void)
int
main(int argc, char **argv)
{
+ static struct option long_options[] = {
+ {"debug", no_argument, NULL, 'd'},
+ {"dry-run", no_argument, NULL, 'n'},
+ {"strip-extension", required_argument, NULL, 'x'},
+ {NULL, 0, NULL, 0}
+ };
int c;
pg_logging_init(argv[0]);
@@ -294,7 +302,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt(argc, argv, "dnx:")) != -1)
+ while ((c = getopt_long(argc, argv, "dnx:", long_options, NULL)) != -1)
{
switch (c)
{
--
2.39.2
[text/x-diff] v11-0002-Preliminary-refactoring-for-a-subsequent-patch.patch (5.7K, ../../[email protected]/3-v11-0002-Preliminary-refactoring-for-a-subsequent-patch.patch)
download | inline diff:
From d760567e6974a5b3618aba228f92c188529f6a75 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Wed, 21 Jun 2023 22:20:21 +0900
Subject: [PATCH v11 2/3] Preliminary refactoring for a subsequent patch
This is a preparatory patch that doesn't introduce any functional
change. Instead, this carries out preliminary refactoring with the
goal of reducing the overall nesting level. This helps to prevent the
forthcoming patch from further deepening the nesting level.
---
src/bin/pg_archivecleanup/pg_archivecleanup.c | 142 +++++++++---------
1 file changed, 74 insertions(+), 68 deletions(-)
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index fc0dca9856..982fe00f20 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -93,76 +93,82 @@ CleanupPriorWALFiles(void)
struct dirent *xlde;
char walfile[MAXPGPATH];
- if ((xldir = opendir(archiveLocation)) != NULL)
- {
- while (errno = 0, (xlde = readdir(xldir)) != NULL)
- {
- /*
- * Truncation is essentially harmless, because we skip names of
- * length other than XLOG_FNAME_LEN. (In principle, one could use
- * a 1000-character additional_ext and get trouble.)
- */
- strlcpy(walfile, xlde->d_name, MAXPGPATH);
- TrimExtension(walfile, additional_ext);
-
- /*
- * We ignore the timeline part of the XLOG segment identifiers in
- * deciding whether a segment is still needed. This ensures that
- * we won't prematurely remove a segment from a parent timeline.
- * We could probably be a little more proactive about removing
- * segments of non-parent timelines, but that would be a whole lot
- * more complicated.
- *
- * We use the alphanumeric sorting property of the filenames to
- * decide which ones are earlier than the exclusiveCleanupFileName
- * file. Note that this means files are not removed in the order
- * they were originally written, in case this worries you.
- */
- if ((IsXLogFileName(walfile) || IsPartialXLogFileName(walfile)) &&
- strcmp(walfile + 8, exclusiveCleanupFileName + 8) < 0)
- {
- char WALFilePath[MAXPGPATH * 2]; /* the file path
- * including archive */
-
- /*
- * Use the original file name again now, including any
- * extension that might have been chopped off before testing
- * the sequence.
- */
- snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
- archiveLocation, xlde->d_name);
-
- if (dryrun)
- {
- /*
- * Prints the name of the file to be removed and skips the
- * actual removal. The regular printout is so that the
- * user can pipe the output into some other program.
- */
- printf("%s\n", WALFilePath);
- pg_log_debug("file \"%s\" would be removed", WALFilePath);
- continue;
- }
-
- pg_log_debug("removing file \"%s\"", WALFilePath);
-
- rc = unlink(WALFilePath);
- if (rc != 0)
- pg_fatal("could not remove file \"%s\": %m",
- WALFilePath);
- }
- }
-
- if (errno)
- pg_fatal("could not read archive location \"%s\": %m",
- archiveLocation);
- if (closedir(xldir))
- pg_fatal("could not close archive location \"%s\": %m",
- archiveLocation);
- }
- else
+ if ((xldir = opendir(archiveLocation)) == NULL)
pg_fatal("could not open archive location \"%s\": %m",
archiveLocation);
+
+ while (errno = 0, (xlde = readdir(xldir)) != NULL)
+ {
+ char WALFilePath[MAXPGPATH * 2]; /* the file path
+ * including archive */
+ /*
+ * Truncation is essentially harmless, because we skip names of
+ * length other than XLOG_FNAME_LEN. (In principle, one could use
+ * a 1000-character additional_ext and get trouble.)
+ */
+ strlcpy(walfile, xlde->d_name, MAXPGPATH);
+ TrimExtension(walfile, additional_ext);
+
+ /*
+ * Check file name.
+ *
+ * We skip files which are not WAL file or partial WAL file.
+ */
+ if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile))
+ continue;
+
+ /*
+ * Check cutoff point.
+ *
+ * We ignore the timeline part of the XLOG segment identifiers in
+ * deciding whether a segment is still needed. This ensures that
+ * we won't prematurely remove a segment from a parent timeline.
+ * We could probably be a little more proactive about removing
+ * segments of non-parent timelines, but that would be a whole lot
+ * more complicated.
+ *
+ * We use the alphanumeric sorting property of the filenames to
+ * decide which ones are earlier than the exclusiveCleanupFileName
+ * file. Note that this means files are not removed in the order
+ * they were originally written, in case this worries you.
+ */
+ if (strcmp(walfile + 8, exclusiveCleanupFileName + 8) >= 0)
+ continue;
+
+ /*
+ * Use the original file name again now, including any
+ * extension that might have been chopped off before testing
+ * the sequence.
+ */
+ snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s",
+ archiveLocation, xlde->d_name);
+
+ if (dryrun)
+ {
+ /*
+ * Prints the name of the file to be removed and skips the
+ * actual removal. The regular printout is so that the
+ * user can pipe the output into some other program.
+ */
+ printf("%s\n", WALFilePath);
+ pg_log_debug("file \"%s\" would be removed", WALFilePath);
+ continue;
+ }
+
+ pg_log_debug("removing file \"%s\"", WALFilePath);
+
+ rc = unlink(WALFilePath);
+ if (rc != 0)
+ pg_fatal("could not remove file \"%s\": %m",
+ WALFilePath);
+ }
+
+ if (errno)
+ pg_fatal("could not read archive location \"%s\": %m",
+ archiveLocation);
+ if (closedir(xldir))
+ pg_fatal("could not close archive location \"%s\": %m",
+ archiveLocation);
}
/*
--
2.39.2
[text/x-diff] v11-0003-Allow-pg_archivecleanup-to-remove-backup-history.patch (8.4K, ../../[email protected]/4-v11-0003-Allow-pg_archivecleanup-to-remove-backup-history.patch)
download | inline diff:
From 4228f489090161f2f968f83c32fd2e05e013136a Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Wed, 21 Jun 2023 22:53:51 +0900
Subject: [PATCH v11 3/3] Allow pg_archivecleanup to remove backup history
files
Backup history files are just few bytes, but it can be noisy for the
eye to see a large accumulation of history files mixed with the WAL
segments.
This patch adds a new option to remove files including backup
history files older oldestkeptwalfile.
---
doc/src/sgml/ref/pgarchivecleanup.sgml | 11 +++
src/bin/pg_archivecleanup/pg_archivecleanup.c | 21 +++--
.../t/010_pg_archivecleanup.pl | 79 +++++++++++++------
3 files changed, 81 insertions(+), 30 deletions(-)
diff --git a/doc/src/sgml/ref/pgarchivecleanup.sgml b/doc/src/sgml/ref/pgarchivecleanup.sgml
index 09991c2fcd..06c043f268 100644
--- a/doc/src/sgml/ref/pgarchivecleanup.sgml
+++ b/doc/src/sgml/ref/pgarchivecleanup.sgml
@@ -93,6 +93,17 @@ pg_archivecleanup: removing file "archive/00000001000000370000000E"
<variablelist>
+ <varlistentry>
+ <term><option>-b</option></term>
+ <term><option>--clean-backup-history</option></term>
+ <listitem>
+ <para>
+ Remove backup history files.
+ For details about backup history file, please refer to the <xref linkend="backup-base-backup"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-d</option></term>
<term><option>--debug</option></term>
diff --git a/src/bin/pg_archivecleanup/pg_archivecleanup.c b/src/bin/pg_archivecleanup/pg_archivecleanup.c
index 982fe00f20..1ed08518ca 100644
--- a/src/bin/pg_archivecleanup/pg_archivecleanup.c
+++ b/src/bin/pg_archivecleanup/pg_archivecleanup.c
@@ -23,6 +23,8 @@ const char *progname;
/* Options and defaults */
bool dryrun = false; /* are we performing a dry-run operation? */
+bool cleanBackupHistory = false; /* remove files including
+ * backup history files */
char *additional_ext = NULL; /* Extension to remove from filenames */
char *archiveLocation; /* where to find the archive? */
@@ -102,9 +104,10 @@ CleanupPriorWALFiles(void)
char WALFilePath[MAXPGPATH * 2]; /* the file path
* including archive */
/*
- * Truncation is essentially harmless, because we skip names of
- * length other than XLOG_FNAME_LEN. (In principle, one could use
- * a 1000-character additional_ext and get trouble.)
+ * Truncation is essentially harmless, because we check the file
+ * format including the length immediately after this.
+ * (In principle, one could use a 1000-character additional_ext
+ * and get trouble.)
*/
strlcpy(walfile, xlde->d_name, MAXPGPATH);
TrimExtension(walfile, additional_ext);
@@ -113,8 +116,11 @@ CleanupPriorWALFiles(void)
* Check file name.
*
* We skip files which are not WAL file or partial WAL file.
+ * Also we skip backup history files when --clean-backup-history
+ * is not specified.
*/
- if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile))
+ if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile) &&
+ !(cleanBackupHistory && IsBackupHistoryFileName(walfile)))
continue;
/*
@@ -258,6 +264,7 @@ usage(void)
printf(_("Usage:\n"));
printf(_(" %s [OPTION]... ARCHIVELOCATION OLDESTKEPTWALFILE\n"), progname);
printf(_("\nOptions:\n"));
+ printf(_(" -b, --clean-backup-history clean up files including backup history files\n"));
printf(_(" -d, --debug generate debug output (verbose mode)\n"));
printf(_(" -n, --dry-run dry run, show the names of the files that would be\n"
" removed\n"));
@@ -283,6 +290,7 @@ int
main(int argc, char **argv)
{
static struct option long_options[] = {
+ {"clean-backup-history", no_argument, NULL, 'b'},
{"debug", no_argument, NULL, 'd'},
{"dry-run", no_argument, NULL, 'n'},
{"strip-extension", required_argument, NULL, 'x'},
@@ -308,10 +316,13 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "dnx:", long_options, NULL)) != -1)
+ while ((c = getopt_long(argc, argv, "bdnx:", long_options, NULL)) != -1)
{
switch (c)
{
+ case 'b': /* Remove backup history files too */
+ cleanBackupHistory = true;
+ break;
case 'd': /* Debug mode */
pg_logging_increase_verbosity();
break;
diff --git a/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl b/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
index cc3386d146..030a82e7fa 100644
--- a/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
+++ b/src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
@@ -12,13 +12,23 @@ program_options_handling_ok('pg_archivecleanup');
my $tempdir = PostgreSQL::Test::Utils::tempdir;
-my @walfiles = (
- '00000001000000370000000C.gz', '00000001000000370000000D',
- '00000001000000370000000E', '00000001000000370000000F.partial',);
+my @walfiles_with_gz = (
+ { name => '00000001000000370000000C.gz', present => 0 },
+ { name => '00000001000000370000000D', present => 0 },
+ { name => '00000001000000370000000E', present => 1 },
+ { name => '00000001000000370000000F.partial', present => 1 },
+ { name => 'unrelated_file', present => 1 });
+
+my @walfiles_with_backup = (
+ { name => '00000001000000370000000D', present => 0 },
+ { name => '00000001000000370000000D.00000028.backup', present => 0 },
+ { name => '00000001000000370000000E', present => 1 },
+ { name => '00000001000000370000000F.partial', present => 1 },
+ { name => 'unrelated_file', present => 1 });
sub create_files
{
- foreach my $fn (@walfiles, 'unrelated_file')
+ foreach my $fn (@_)
{
open my $file, '>', "$tempdir/$fn";
print $file 'CONTENT';
@@ -27,7 +37,18 @@ sub create_files
return;
}
-create_files();
+sub get_walfiles
+{
+ my @walfiles;
+
+ for my $walpair (@_)
+ {
+ push @walfiles, $walpair->{name};
+ }
+ return @walfiles;
+}
+
+create_files(get_walfiles(@walfiles_with_gz));
command_fails_like(
['pg_archivecleanup'],
@@ -59,14 +80,14 @@ command_fails_like(
my $stderr;
my $result =
IPC::Run::run [ 'pg_archivecleanup', '-d', '-n', $tempdir,
- $walfiles[2] ],
+ $walfiles_with_gz[2]->{name} ],
'2>', \$stderr;
ok($result, "pg_archivecleanup dry run: exit code 0");
like(
$stderr,
- qr/$walfiles[1].*would be removed/,
+ qr/$walfiles_with_gz[1]->{name}.*would be removed/,
"pg_archivecleanup dry run: matches");
- foreach my $fn (@walfiles)
+ foreach my $fn (get_walfiles(@walfiles_with_gz))
{
ok(-f "$tempdir/$fn", "$fn not removed");
}
@@ -76,32 +97,40 @@ sub run_check
{
local $Test::Builder::Level = $Test::Builder::Level + 1;
- my ($suffix, $test_name) = @_;
+ my ($testdata, $oldestkeptwalfile, $test_name, @options) = @_;
- create_files();
+ create_files(get_walfiles(@$testdata));
command_ok(
[
- 'pg_archivecleanup', '-x', '.gz', $tempdir,
- $walfiles[2] . $suffix
+ 'pg_archivecleanup', @options, $tempdir,
+ $oldestkeptwalfile
],
"$test_name: runs");
- ok(!-f "$tempdir/$walfiles[0]",
- "$test_name: first older WAL file was cleaned up");
- ok(!-f "$tempdir/$walfiles[1]",
- "$test_name: second older WAL file was cleaned up");
- ok(-f "$tempdir/$walfiles[2]",
- "$test_name: restartfile was not cleaned up");
- ok(-f "$tempdir/$walfiles[3]",
- "$test_name: newer WAL file was not cleaned up");
- ok(-f "$tempdir/unrelated_file",
- "$test_name: unrelated file was not cleaned up");
+ for my $walpair (@$testdata)
+ {
+ if ($walpair->{present})
+ {
+ ok(-f "$tempdir/$walpair->{name}",
+ "$test_name:$walpair->{name} was not cleaned up");
+ }
+ else
+ {
+ ok(!-f "$tempdir/$walpair->{name}",
+ "$test_name:$walpair->{name} was cleaned up");
+ }
+ }
return;
}
-run_check('', 'pg_archivecleanup');
-run_check('.partial', 'pg_archivecleanup with .partial file');
-run_check('.00000020.backup', 'pg_archivecleanup with .backup file');
+run_check(\@walfiles_with_gz, '00000001000000370000000E',
+ 'pg_archivecleanup', '-x.gz');
+run_check(\@walfiles_with_gz, '00000001000000370000000E.partial',
+ 'pg_archivecleanup with .partial file', '-x.gz');
+run_check(\@walfiles_with_gz, '00000001000000370000000E.00000020.backup',
+ 'pg_archivecleanup with .backup file', '-x.gz');
+run_check(\@walfiles_with_backup, '00000001000000370000000E',
+ 'pg_archivecleanup with --clean-backup-history', '-b');
done_testing();
--
2.39.2
^ permalink raw reply [nested|flat] 70+ messages in thread
* Re: Allow pg_archivecleanup to remove backup history files
2023-06-13 15:49 Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-15 06:20 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-15 12:38 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-16 02:22 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-16 02:30 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-19 02:24 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-19 05:37 ` Re: Allow pg_archivecleanup to remove backup history files Michael Paquier <[email protected]>
2023-06-20 13:27 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-21 02:59 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-21 14:41 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
@ 2023-06-22 07:47 ` Kyotaro Horiguchi <[email protected]>
0 siblings, 0 replies; 70+ messages in thread
From: Kyotaro Horiguchi @ 2023-06-22 07:47 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; pgsql-hackers
At Wed, 21 Jun 2023 23:41:33 +0900, torikoshia <[email protected]> wrote in
> --v10-0002-Preliminary-refactoring-for-a-subsequent-patch.patch
> + * Also we skip backup history files when --clean-backup-history
> + * is not specified.
> + */
> + if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(walfile))
> + continue;
>
> I think this comment should be located in 0003.
Auch! Right.
> Attached updated patches.
Thanks!
0001:
Looks good to me.
0002:
+ * Check file name.
+ *
+ * We skip files which are not WAL file or partial WAL file.
There's no need to spend this many lines describing this, and it's
suggested to avoid restating what the code does. I think a comment
might not be necessary. But if we decide to have one, it could look
like this:
> /* we're only removing specific types of files */
Other than that, it looks good to me.
0003:
+ <para>
+ Remove backup history files.
I might be overthinking it, but I'd phrase it as, "Remove backup
history files *as well*.". (The --help message describes the same
thing using "including".)
+ For details about backup history file, please refer to the <xref linkend="backup-base-backup"/>.
We usually write this as simple as "See <xref...> for details (of the
backup history files)" or "Refer to <xref..> for more information
(about the backup history files)." or such like... (I think)
+bool cleanBackupHistory = false; /* remove files including
+ * backup history files */
The indentation appears to be inconsistent.
- * Truncation is essentially harmless, because we skip names of
- * length other than XLOG_FNAME_LEN. (In principle, one could use
- * a 1000-character additional_ext and get trouble.)
+ * Truncation is essentially harmless, because we check the file
+ * format including the length immediately after this.
+ * (In principle, one could use a 1000-character additional_ext
+ * and get trouble.)
*/
strlcpy(walfile, xlde->d_name, MAXPGPATH);
TrimExtension(walfile, additional_ext);
The revised comment seems to drift from the original point. Except for
a potential exception by a too-long addition_ext, the original comment
asserted that the name truncation was safe since it wouldn't affect
the files we're removing. In other words, it asserted that the
filenames to be removed won't be truncated and any actual truncation
wouldn't lead to accidental deletions.
Hence, I think we should adjust the comment to maintain its original
point, and extend it to include backup history files. A possible
revision could be (very simple):
> * Truncation is essentially harmless, because we skip names of length
> * longer than the length of backup history file. (In principle, one
> * could use a 1000-character additional_ext and get trouble.)
Regarding the TAP test, it checks that the --clean-backup-history does
indeed remove backup history files. However, it doesn't check that
this doesn't occur if the option isn't specified. Shouldn't we test
for the latter scenario as well?
+sub get_walfiles
+{
<snip..>
+
+create_files(get_walfiles(@walfiles_with_gz));
The new function get_walfiels() just increases the line count without
cutting any lines. The following changes are sufficient and easier to
read (at least for me).
> open my $file, '>', "$tempdir/$fn->{name}";
> foreach my $fn (map {$_->{name}} @walfiles_with_gz)
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 70+ messages in thread
end of thread, other threads:[~2023-06-22 07:47 UTC | newest]
Thread overview: 70+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v32 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v30 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v26 05/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v19 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v34 04/15] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v25 05/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v20 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v35 4/7] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v33 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v37 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v24 05/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v22 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v36 4/7] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v27 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v23 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v21 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v18 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v31 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v28 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]>
2023-06-13 15:49 Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-15 06:20 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-15 12:38 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-16 02:22 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-16 02:30 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-19 02:24 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-19 05:37 ` Re: Allow pg_archivecleanup to remove backup history files Michael Paquier <[email protected]>
2023-06-20 13:27 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-21 02:59 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
2023-06-21 14:41 ` Re: Allow pg_archivecleanup to remove backup history files torikoshia <[email protected]>
2023-06-22 07:47 ` Re: Allow pg_archivecleanup to remove backup history files Kyotaro Horiguchi <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox