public inbox for [email protected]help / color / mirror / Atom feed
[PATCH v21 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. 62+ messages / 4 participants [nested] [flat]
* [PATCH v21 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index eead88419f..3c419672fc 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25843,16 +25843,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 24d858d1bc..cd7f052692 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -517,7 +517,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -706,7 +706,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 83927b510a..85ac8fef61 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10911,13 +10911,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9979', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 20bfe71822..0144f611c2 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --Tcb1KvpfnM4LxW2s Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v21-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v18 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 8ea064144f..33f7137e43 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 5b0147ee2d..08c7189470 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10894,13 +10894,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9979', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --5I6of5zJg18YgZEa Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v18-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v20 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index eead88419f..3c419672fc 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25843,16 +25843,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 3324ffe16a..8390626e49 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -517,7 +517,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -706,7 +706,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 186ced4a35..381f2e196f 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10911,13 +10911,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9979', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --Z1Z8UV8BNhgCynIS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v20-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v37 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonably complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 2 +- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 8 ++++---- src/test/regress/expected/tablespace.out | 4 ++-- 5 files changed, 20 insertions(+), 19 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index a80d794a43a..cff999032bf 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -27630,16 +27630,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index c041c7630c8..5d374177a04 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -740,7 +740,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 89892647808..e2f3361bf37 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11728,13 +11728,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '6270', descr => 'list of files in the pg_logical/snapshots directory', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index a656f9ad07e..bc77ad62958 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -525,8 +525,8 @@ select * from pg_ls_replslotdir('') limit 0; (0 rows) select * from pg_ls_tmpdir() limit 0; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select * from pg_ls_waldir() limit 0; @@ -542,8 +542,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out index 8159c9f18f1..ce5d6f73e01 100644 --- a/src/test/regress/expected/tablespace.out +++ b/src/test/regress/expected/tablespace.out @@ -37,8 +37,8 @@ SELECT regexp_replace(pg_tablespace_location(oid), '(pg_tblspc)/(\d+)', '\1/NNN' -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.25.1 --Pk/CTwBz1VvfPIDp Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v37-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v36 4/7] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonably complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 2 +- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 8 ++++---- src/test/regress/expected/tablespace.out | 4 ++-- 5 files changed, 20 insertions(+), 19 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index ba30faa8441..8767ef0d7a2 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -28591,16 +28591,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index c7322ed405a..eb74130e721 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 0e5c977e69f..381eb0e277f 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11785,13 +11785,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '6270', descr => 'list of files in the pg_logical/snapshots directory', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index e54e38f54ad..77bf661627b 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -461,8 +461,8 @@ select * from pg_ls_replslotdir('') limit 0; (0 rows) select * from pg_ls_tmpdir() limit 0; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select * from pg_ls_waldir() limit 0; @@ -478,8 +478,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out index 8159c9f18f1..ce5d6f73e01 100644 --- a/src/test/regress/expected/tablespace.out +++ b/src/test/regress/expected/tablespace.out @@ -37,8 +37,8 @@ SELECT regexp_replace(pg_tablespace_location(oid), '(pg_tblspc)/(\d+)', '\1/NNN' -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.1 --4ybNbZnZ8tziJ7D6 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v36-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v34 04/15] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonably complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 2 +- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 8 ++++---- src/test/regress/expected/tablespace.out | 4 ++-- 5 files changed, 20 insertions(+), 19 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index c2ff924d021..6697ab14b14 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -27550,16 +27550,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 1878bac7eb3..1710ebe489b 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -668,7 +668,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 43ee0cdc454..19d062a36dc 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11647,13 +11647,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9858', descr => 'list of files in the pg_logical/snapshots directory', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index d18f401248a..6c64cd6f998 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -438,8 +438,8 @@ select * from pg_ls_replslotdir('') limit 0; (0 rows) select * from pg_ls_tmpdir() limit 0; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select * from pg_ls_waldir() limit 0; @@ -455,8 +455,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out index c6b9074ab02..20a7245d5e1 100644 --- a/src/test/regress/expected/tablespace.out +++ b/src/test/regress/expected/tablespace.out @@ -28,8 +28,8 @@ CREATE TABLESPACE regress_tblspace LOCATION ''; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.1 --smOfPzt+Qjm5bNGJ Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v34-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v35 4/7] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonably complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 2 +- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 8 ++++---- src/test/regress/expected/tablespace.out | 4 ++-- 5 files changed, 20 insertions(+), 19 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index d2a455a3e27..3931411c01f 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -27561,16 +27561,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 75b7bf99849..3775cae225d 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 7307f44f371..142b4b8d37d 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11739,13 +11739,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9858', descr => 'list of files in the pg_logical/snapshots directory', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index e54e38f54ad..77bf661627b 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -461,8 +461,8 @@ select * from pg_ls_replslotdir('') limit 0; (0 rows) select * from pg_ls_tmpdir() limit 0; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select * from pg_ls_waldir() limit 0; @@ -478,8 +478,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out index 8159c9f18f1..ce5d6f73e01 100644 --- a/src/test/regress/expected/tablespace.out +++ b/src/test/regress/expected/tablespace.out @@ -37,8 +37,8 @@ SELECT regexp_replace(pg_tablespace_location(oid), '(pg_tblspc)/(\d+)', '\1/NNN' -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.1 --olLTNZSltDMg5Vbm Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v35-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v32 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonably complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 2 +- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/expected/tablespace.out | 4 ++-- 5 files changed, 18 insertions(+), 17 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 7948e6ec06c..7589ef7ac8a 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -27427,16 +27427,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 0728547ec2b..d6be8cef6e3 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -709,7 +709,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index c99e1633bbf..fa293eca393 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11615,13 +11615,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9858', descr => 'list of files in the pg_logical/snapshots directory', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index b3a9d11b5c0..a2bec47d76c 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -270,8 +270,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out index 21f7f19b67f..7d8398b0ca1 100644 --- a/src/test/regress/expected/tablespace.out +++ b/src/test/regress/expected/tablespace.out @@ -20,8 +20,8 @@ CREATE TABLESPACE regress_tblspace LOCATION :'testtablespace'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --Bne5rrxQd65beI7a Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v32-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v31 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonably complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 2 +- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 18 insertions(+), 17 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index e0099e77df..9ddd23f90e 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -27342,16 +27342,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 0728547ec2..d6be8cef6e 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -709,7 +709,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 39fbaacbc9..3ffdcaf3a7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11615,13 +11615,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9858', descr => 'list of files in the pg_logical/snapshots directory', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index b3a9d11b5c..a2bec47d76 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -270,8 +270,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index c2f6d64c5c..c146a4c129 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --qZVVwWJgpX9Jzs7f Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v31-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v33 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonably complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 2 +- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 8 ++++---- src/test/regress/expected/tablespace.out | 4 ++-- 5 files changed, 20 insertions(+), 19 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 14b8fba3123..e0b0a5ab121 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -27495,16 +27495,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 3921723fda6..4829b34f0f6 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -710,7 +710,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index b58987b713c..95746fc0dcf 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11631,13 +11631,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9858', descr => 'list of files in the pg_logical/snapshots directory', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index a627839bc28..0305f038369 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -295,8 +295,8 @@ select * from pg_ls_replslotdir('') limit 0; (0 rows) select * from pg_ls_tmpdir() limit 0; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select * from pg_ls_waldir() limit 0; @@ -312,8 +312,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out index c6b9074ab02..20a7245d5e1 100644 --- a/src/test/regress/expected/tablespace.out +++ b/src/test/regress/expected/tablespace.out @@ -28,8 +28,8 @@ CREATE TABLESPACE regress_tblspace LOCATION ''; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.1 --9CzcV6dAFIr7O1Ie Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v33-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v30 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonably complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 2 +- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 18 insertions(+), 17 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index dd9b1afde4..2adc8f7d83 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -26936,16 +26936,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 8efb798b2f..243e86a2a3 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -708,7 +708,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index b405aa341c..0a4caf8aed 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11539,13 +11539,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '8450', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index b4112ec298..0d3e88fccf 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -241,8 +241,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index 5f7ce3d09f..4183a77b23 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --ZwgA9U+XZDXt4+m+ Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v30-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v25 05/11] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonably complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 226923a134..4fb4ec9f33 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25829,16 +25829,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 7351615f8a..da7d4a7c11 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -517,7 +517,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if MISSING_OK. * Other unreadable-directory cases throw an error. @@ -706,7 +706,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 8e88c549d9..a0148e6130 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10968,13 +10968,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9979', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index fd7c3c791f..7279c83d89 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -228,8 +228,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --SBikYMzjhZGK9d4p Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v25-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v26 05/11] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonably complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 2 +- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 18 insertions(+), 17 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 79d30a1568..c68c9d548b 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -26491,16 +26491,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 198a99391a..da7d4a7c11 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -706,7 +706,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 17b03d1abd..d2594c2e45 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11258,13 +11258,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9979', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index fd7c3c791f..7279c83d89 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -228,8 +228,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --yKkOmjQZXRsvHRX8 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v26-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v27 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonably complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 2 +- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 18 insertions(+), 17 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 76ae2f2dd8..09a5b5b48a 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -26922,16 +26922,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index f59f853983..a657d8d431 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -709,7 +709,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index b85bee2d1e..10ffab88c5 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11509,13 +11509,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '8450', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index b4112ec298..0d3e88fccf 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -241,8 +241,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index 5f7ce3d09f..4183a77b23 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --19uQFt6ulqmgNgg1 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v27-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v28 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonably complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 2 +- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 18 insertions(+), 17 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 507a6d73f8..940c653807 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -26950,16 +26950,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index f59f853983..a657d8d431 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -709,7 +709,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 550fbf734a..441cbb2cde 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11526,13 +11526,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '8450', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index b4112ec298..0d3e88fccf 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -241,8 +241,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index 5f7ce3d09f..4183a77b23 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --vk/v8fjDPiDepTtA Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v28-0005-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v22 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 34c92903d8..4308a50ab2 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25849,16 +25849,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 054b8d4b1a..454348a5e3 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -517,7 +517,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -706,7 +706,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 02b1436d83..149dfacf4b 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10949,13 +10949,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9147', descr => 'check pages of a relation', proname => 'pg_relation_check_pages', procost => '10000', prorows => '20', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 20bfe71822..0144f611c2 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --d6Gm4EdcadzBjdND Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v22-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v23 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 0a924fdcc3..ee23bb763c 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25840,16 +25840,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 6f003bbf67..ec016149b6 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -517,7 +517,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -706,7 +706,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index add3c7dc62..cffcd03634 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10963,13 +10963,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9979', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 20bfe71822..0144f611c2 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --mhjHhnbe5PrRcwjY Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v23-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v24 05/11] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonably complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 5e267c4f8d..dcd0aa84d2 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25836,16 +25836,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 7351615f8a..da7d4a7c11 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -517,7 +517,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if MISSING_OK. * Other unreadable-directory cases throw an error. @@ -706,7 +706,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index f35a684dcf..e2aea6f48b 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10967,13 +10967,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9979', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index fd7c3c791f..7279c83d89 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -228,8 +228,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --mPTHnM80CEnHQ2WJ Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v24-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v19 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 2931f9d7d5..24dc90a74d 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25813,16 +25813,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return the file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 5b0147ee2d..08c7189470 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10894,13 +10894,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9979', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --S0GG+JvAI2G0KxBG Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v19-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 17 +++++++++-------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 ++++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index fc1b4ac98c..c9d1857182 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25793,16 +25793,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, - <parameter>modification</parameter> <type>timestamp with time zone</type> ) + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> - Returns the name, size, and last modification time (mtime) of each - ordinary file in the temporary file directory for the - specified <parameter>tablespace</parameter>. - If <parameter>tablespace</parameter> is not provided, - the <literal>pg_default</literal> tablespace is examined. Filenames - beginning with a dot, directories, and other special files are - excluded. + For each file in the temporary directory within the given + <parameter>tablespace</parameter>, return each file's name, size, last + modification time (mtime) and a boolean indicating if the file is a directory. + Directories are used for temporary files shared by parallel processes. + If <parameter>tablespace</parameter> is not provided, the + <literal>pg_default</literal> tablespace is examined. + Filenames beginning with a dot and special file types are excluded. </para> <para> This function is restricted to superusers and members of diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. @ 2020-03-09 03:57 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Justin Pryzby @ 2020-03-09 03:57 UTC (permalink / raw) similar to pg_stat_file(). It's worth breaking the function's return type, since core postgres creates "shared filesets" underneath the temp dirs, and it's unreasonable to not show them here, and the alternative query to show them is unreasaonbly complicated. See following commit which also adds these columns to the other pg_ls_* functions. Although I don't think it matters that they're easily UNIONed, it'd still make great sense if they returned the same columns. Need catversion bump --- doc/src/sgml/func.sgml | 22 +++++++++++--------- src/backend/utils/adt/genfile.c | 4 ++-- src/include/catalog/pg_proc.dat | 8 +++---- src/test/regress/expected/misc_functions.out | 4 ++-- src/test/regress/output/tablespace.source | 4 ++-- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9d617f95b9..891b990859 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25351,12 +25351,12 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - List the name, size, and last modification time of files in the - temporary directory for <parameter>tablespace</parameter>. If - <parameter>tablespace</parameter> is not provided, the - <literal>pg_default</literal> tablespace is used. Access is granted - to members of the <literal>pg_monitor</literal> role and may be - granted to other non-superuser roles. + For the temporary directory within <parameter>tablespace</parameter>, + list each file's name, size, last modification time, and a boolean + indicating if it is a directory. If <parameter>tablespace</parameter> + is not provided, the <literal>pg_default</literal> tablespace is used. + Access is granted to members of the <literal>pg_monitor</literal> role + and may be granted to other non-superuser roles. </entry> </row> <row> @@ -25459,14 +25459,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <primary>pg_ls_tmpdir</primary> </indexterm> <para> - <function>pg_ls_tmpdir</function> returns the name, size, and last modified - time (mtime) of each file in the temporary file directory for the specified - <parameter>tablespace</parameter>. If <parameter>tablespace</parameter> is + <function>pg_ls_tmpdir</function> lists each file in the temporary file + directory for the specified <parameter>tablespace</parameter>, along with + its size, last modified time (mtime) and a boolean indicating if the file is a + directory. Directories are used for temporary files shared by parallel + processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. - Filenames beginning with a dot, directories, and other special files are not shown. + Filenames beginning with a dot and special file types are not shown. </para> <indexterm> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 4824a55480..023f1ad605 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -481,7 +481,7 @@ pg_ls_dir_1arg(PG_FUNCTION_ARGS) } /* - * Generic function to return a directory listing of files. + * Generic function to return a directory listing of files (and optionally dirs). * * If the directory isn't there, silently return an empty set if missing_ok. * Other unreadable-directory cases throw an error. @@ -670,7 +670,7 @@ pg_ls_tmpdir(FunctionCallInfo fcinfo, Oid tblspc) TempTablespacePath(path, tblspc); return pg_ls_dir_files(fcinfo, path, - LS_DIR_HISTORIC | LS_DIR_MISSING_OK); + LS_DIR_SKIP_HIDDEN | LS_DIR_SKIP_SPECIAL | LS_DIR_ISDIR | LS_DIR_METADATA | LS_DIR_MISSING_OK); } /* diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9f46cba5ed..dc3b497587 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10896,13 +10896,13 @@ { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz}', proargmodes => '{o,o,o}', - proargnames => '{name,size,modification}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', + proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz}', proargmodes => '{i,o,o,o}', - proargnames => '{tablespace,name,size,modification}', + proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', + proargnames => '{tablespace,name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 7930909f02..38493de732 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index a42714bf40..1c88e914e3 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,8 +17,8 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | modification | isdir +------+------+--------------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --2FkSFaIQeDFoAt0B Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v16-0006-pg_ls_-dir-to-show-directories-and-isdir-column.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* Re: Logical replication timeout @ 2024-12-12 13:50 RECHTÉ Marc <[email protected]> 0 siblings, 1 reply; 62+ messages in thread From: RECHTÉ Marc @ 2024-12-12 13:50 UTC (permalink / raw) To: Shlok Kyal <[email protected]>; +Cc: [email protected] Hi, Thanks for sharing the test case. Unfortunately I donot have a powerful machine which would generate such large number of spill files. But I created a patch as per your suggestion in point(2) in thread [1]. Can you test with this patch on your machine? With this patch instead of calling unlink for every wal segment, we are first reading the directory and filtering the files related to our transaction and then unlinking those files. You can apply the patch on your publisher source code and check. I have created this patch on top of Postgres 15.6. [1]: https://www.postgresql.org/message-id/[email protected] Thanks and Regards, Shlok Kyal Thanks for the parch. I tried it, but it does not compile. Attached another version that I tested on PostgreSQL 17.2. This is much worse: it deletes only 3 files / s ! The problem here, is that for a given xid, there is just one spill file to delete. ReorderBufferRestoreCleanup is called over a million times, so for each call, it has to open the directory and filter the one file to delete. By the way, you don't need a powerful machine to test, as spill files are very small. Marc Attachments: [text/x-patch] scan_dir2.patch (1.7K, ../../[email protected]/2-scan_dir2.patch) download | inline diff: --- a/src/backend/replication/logical/reorderbuffer.c 2024-12-12 10:21:12.962202250 +0100 +++ b/src/backend/replication/logical/reorderbuffer.c 2024-12-12 11:40:24.217634527 +0100 @@ -4567,27 +4567,35 @@ static void ReorderBufferRestoreCleanup(ReorderBuffer *rb, ReorderBufferTXN *txn) { - XLogSegNo first; - XLogSegNo cur; - XLogSegNo last; + DIR *spill_dir; + struct dirent *spill_de; + char path[MAXPGPATH]; + char spill_path[MAXPGPATH]; + char file_filter[MAXPGPATH]; + int len_filter; + + snprintf(spill_path, MAXPGPATH, "pg_replslot/%s", + NameStr(MyReplicationSlot->data.name)); - Assert(txn->first_lsn != InvalidXLogRecPtr); - Assert(txn->final_lsn != InvalidXLogRecPtr); - - XLByteToSeg(txn->first_lsn, first, wal_segment_size); - XLByteToSeg(txn->final_lsn, last, wal_segment_size); - - /* iterate over all possible filenames, and delete them */ - for (cur = first; cur <= last; cur++) - { - char path[MAXPGPATH]; - - ReorderBufferSerializedPath(path, MyReplicationSlot, txn->xid, cur); - if (unlink(path) != 0 && errno != ENOENT) - ereport(ERROR, - (errcode_for_file_access(), - errmsg("could not remove file \"%s\": %m", path))); - } + snprintf(file_filter, sizeof(file_filter), "xid-%u-", txn->xid); + len_filter = strlen(file_filter); + + spill_dir = AllocateDir(spill_path); + while ((spill_de = ReadDirExtended(spill_dir, spill_path, INFO)) != NULL) + { + /* only look at names that can be ours */ + if (strncmp(spill_de->d_name, file_filter, len_filter) == 0) + { + snprintf(path, sizeof(path), "%s/%s", spill_path, + spill_de->d_name); + + if (unlink(path) != 0) + ereport(ERROR, + (errcode_for_file_access(), + errmsg("could not remove file \"%s\": %m", path))); + } + } + FreeDir(spill_dir); } /* ^ permalink raw reply [nested|flat] 62+ messages in thread
* Re: Logical replication timeout @ 2024-12-19 06:27 Shlok Kyal <[email protected]> parent: RECHTÉ Marc <[email protected]> 0 siblings, 1 reply; 62+ messages in thread From: Shlok Kyal @ 2024-12-19 06:27 UTC (permalink / raw) To: RECHTÉ Marc <[email protected]>; +Cc: [email protected] On Thu, 12 Dec 2024 at 19:20, RECHTÉ Marc <[email protected]> wrote: > > Hi, > > Thanks for sharing the test case. > Unfortunately I donot have a powerful machine which would generate > such large number of spill files. But I created a patch as per your > suggestion in point(2) in thread [1]. Can you test with this patch on > your machine? > > With this patch instead of calling unlink for every wal segment, we > are first reading the directory and filtering the files related to our > transaction and then unlinking those files. > You can apply the patch on your publisher source code and check. I > have created this patch on top of Postgres 15.6. > > [1]: https://www.postgresql.org/message-id/[email protected] > > > Thanks and Regards, > Shlok Kyal > > Thanks for the parch. > > I tried it, but it does not compile. > > Attached another version that I tested on PostgreSQL 17.2. > > This is much worse: it deletes only 3 files / s ! > > The problem here, is that for a given xid, there is just one spill file to delete. > ReorderBufferRestoreCleanup is called over a million times, so for each call, > it has to open the directory and filter the one file to delete. > > By the way, you don't need a powerful machine to test, as spill files are very small. > Thanks for sharing the analysis. I tested the patch on my machine as well and it has worse performance for me as well. I came up with an alternate approach. In this approach we keep track of wal segment the transaction is part of. This helps to iterate through only required files during clean up. On my machine, I am running the testcase provided by you in [1]. It is generating ~1.9 million spill files. For me the transaction completed in 56sec. Cleanup (deletion of spill files) took around following time: With HEAD : ~ 5min With latest patch (attached here) : ~2min Can you test if this improves performance for you? The patch applies on HEAD. Thanks and Regards, Shlok Kyal Attachments: [application/octet-stream] track_wal_segments.patch (7.7K, ../../CANhcyEVbT2N-=Dwg-EENq4U=Eqf-rGf6ozfXrtvkc+jw_euYLQ@mail.gmail.com/2-track_wal_segments.patch) download | inline diff: diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c index e3a5c7b660..61943e72d8 100644 --- a/src/backend/replication/logical/reorderbuffer.c +++ b/src/backend/replication/logical/reorderbuffer.c @@ -149,7 +149,7 @@ typedef struct ReorderBufferIterTXNEntry ReorderBufferChange *change; ReorderBufferTXN *txn; TXNEntryFile file; - XLogSegNo segno; + int restore_from; } ReorderBufferIterTXNEntry; typedef struct ReorderBufferIterTXNState @@ -254,7 +254,7 @@ static void ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn); static void ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn, int fd, ReorderBufferChange *change); static Size ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn, - TXNEntryFile *file, XLogSegNo *segno); + TXNEntryFile *file, int *restore_from); static void ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn, char *data); static void ReorderBufferRestoreCleanup(ReorderBuffer *rb, ReorderBufferTXN *txn); @@ -432,6 +432,7 @@ ReorderBufferGetTXN(ReorderBuffer *rb) /* InvalidCommandId is not zero, so set it explicitly */ txn->command_id = InvalidCommandId; txn->output_plugin_private = NULL; + txn->walsgmts = NIL; return txn; } @@ -1305,7 +1306,7 @@ ReorderBufferIterTXNInit(ReorderBuffer *rb, ReorderBufferTXN *txn, for (off = 0; off < state->nr_txns; off++) { state->entries[off].file.vfd = -1; - state->entries[off].segno = 0; + state->entries[off].restore_from = 0; } /* allocate heap */ @@ -1333,7 +1334,7 @@ ReorderBufferIterTXNInit(ReorderBuffer *rb, ReorderBufferTXN *txn, /* serialize remaining changes */ ReorderBufferSerializeTXN(rb, txn); ReorderBufferRestoreChanges(rb, txn, &state->entries[off].file, - &state->entries[off].segno); + &state->entries[off].restore_from); } cur_change = dlist_head_element(ReorderBufferChange, node, @@ -1363,7 +1364,7 @@ ReorderBufferIterTXNInit(ReorderBuffer *rb, ReorderBufferTXN *txn, ReorderBufferSerializeTXN(rb, cur_txn); ReorderBufferRestoreChanges(rb, cur_txn, &state->entries[off].file, - &state->entries[off].segno); + &state->entries[off].restore_from); } cur_change = dlist_head_element(ReorderBufferChange, node, &cur_txn->changes); @@ -1448,7 +1449,7 @@ ReorderBufferIterTXNNext(ReorderBuffer *rb, ReorderBufferIterTXNState *state) */ rb->totalBytes += entry->txn->size; if (ReorderBufferRestoreChanges(rb, entry->txn, &entry->file, - &state->entries[off].segno)) + &state->entries[off].restore_from)) { /* successfully restored changes from disk */ ReorderBufferChange *next_change = @@ -3715,6 +3716,7 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn) XLogSegNo curOpenSegNo = 0; Size spilled = 0; Size size = txn->size; + MemoryContext oldcontext; elog(DEBUG2, "spill %u changes in XID %u to disk", (uint32) txn->nentries_mem, txn->xid); @@ -3758,7 +3760,17 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn) /* open segment, create it if necessary */ fd = OpenTransientFile(path, - O_CREAT | O_WRONLY | O_APPEND | PG_BINARY); + O_CREAT | O_EXCL | O_WRONLY | O_APPEND | PG_BINARY); + + if (fd < 0) + fd = OpenTransientFile(path, + O_CREAT | O_WRONLY | O_APPEND | PG_BINARY); + else + { + oldcontext = MemoryContextSwitchTo(rb->context); + txn->walsgmts = lappend(txn->walsgmts, curOpenSegNo); + MemoryContextSwitchTo(oldcontext); + } if (fd < 0) ereport(ERROR, @@ -4255,16 +4267,12 @@ ReorderBufferChangeSize(ReorderBufferChange *change) */ static Size ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn, - TXNEntryFile *file, XLogSegNo *segno) + TXNEntryFile *file, int *restore_from) { Size restored = 0; - XLogSegNo last_segno; dlist_mutable_iter cleanup_iter; File *fd = &file->vfd; - Assert(txn->first_lsn != InvalidXLogRecPtr); - Assert(txn->final_lsn != InvalidXLogRecPtr); - /* free current entries, so we have memory for more */ dlist_foreach_modify(cleanup_iter, &txn->changes) { @@ -4277,9 +4285,8 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn, txn->nentries_mem = 0; Assert(dlist_is_empty(&txn->changes)); - XLByteToSeg(txn->final_lsn, last_segno, wal_segment_size); - - while (restored < max_changes_in_memory && *segno <= last_segno) + while (restored < max_changes_in_memory && + (*restore_from) < txn->walsgmts->length) { int readBytes; ReorderBufferDiskChange *ondisk; @@ -4289,19 +4296,21 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn, if (*fd == -1) { char path[MAXPGPATH]; + ListCell *lc; + XLogSegNo segno; - /* first time in */ - if (*segno == 0) - XLByteToSeg(txn->first_lsn, *segno, wal_segment_size); + /* next wal segment for the transaction */ + lc = list_nth_cell(txn->walsgmts, *restore_from); + segno = lfirst(lc); - Assert(*segno != 0 || dlist_is_empty(&txn->changes)); + Assert(segno != 0 || dlist_is_empty(&txn->changes)); /* * No need to care about TLIs here, only used during a single run, * so each LSN only maps to a specific WAL record. */ ReorderBufferSerializedPath(path, MyReplicationSlot, txn->xid, - *segno); + segno); *fd = PathNameOpenFile(path, O_RDONLY | PG_BINARY); @@ -4311,7 +4320,7 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn, if (*fd < 0 && errno == ENOENT) { *fd = -1; - (*segno)++; + (*restore_from)++; continue; } else if (*fd < 0) @@ -4336,7 +4345,7 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn, { FileClose(*fd); *fd = -1; - (*segno)++; + (*restore_from)++; continue; } else if (readBytes < 0) @@ -4567,27 +4576,26 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn, static void ReorderBufferRestoreCleanup(ReorderBuffer *rb, ReorderBufferTXN *txn) { - XLogSegNo first; - XLogSegNo cur; - XLogSegNo last; - - Assert(txn->first_lsn != InvalidXLogRecPtr); - Assert(txn->final_lsn != InvalidXLogRecPtr); - - XLByteToSeg(txn->first_lsn, first, wal_segment_size); - XLByteToSeg(txn->final_lsn, last, wal_segment_size); + ListCell *cell; /* iterate over all possible filenames, and delete them */ - for (cur = first; cur <= last; cur++) + foreach(cell, txn->walsgmts) { + XLogSegNo curr_segno = (XLogSegNo) lfirst(cell); char path[MAXPGPATH]; - ReorderBufferSerializedPath(path, MyReplicationSlot, txn->xid, cur); + ReorderBufferSerializedPath(path, MyReplicationSlot, txn->xid, curr_segno); if (unlink(path) != 0 && errno != ENOENT) ereport(ERROR, (errcode_for_file_access(), errmsg("could not remove file \"%s\": %m", path))); } + + if(txn->walsgmts != NIL) + { + pfree(txn->walsgmts); + txn->walsgmts = NIL; + } } /* diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h index 3bc365a7b0..c23fa877dc 100644 --- a/src/include/replication/reorderbuffer.h +++ b/src/include/replication/reorderbuffer.h @@ -426,6 +426,11 @@ typedef struct ReorderBufferTXN * Private data pointer of the output plugin. */ void *output_plugin_private; + + /* + * List of wal segments this txn is part of + */ + List *walsgmts; } ReorderBufferTXN; /* so we can define the callbacks used inside struct ReorderBuffer itself */ ^ permalink raw reply [nested|flat] 62+ messages in thread
* RE: Logical replication timeout @ 2024-12-23 12:29 Hayato Kuroda (Fujitsu) <[email protected]> parent: Shlok Kyal <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Hayato Kuroda (Fujitsu) @ 2024-12-23 12:29 UTC (permalink / raw) To: 'Shlok Kyal' <[email protected]>; +Cc: [email protected] <[email protected]>; RECHTÉ Marc <[email protected]> Dear Shlok, > > Thanks for sharing the analysis. > > I tested the patch on my machine as well and it has worse performance > for me as well. > I came up with an alternate approach. In this approach we keep track > of wal segment the transaction is part of. This helps to iterate > through only required files during clean up. > > On my machine, I am running the testcase provided by you in [1]. It is > generating ~1.9 million spill files. For me the transaction completed > in 56sec. > Cleanup (deletion of spill files) took around following time: > With HEAD : ~ 5min > With latest patch (attached here) : ~2min > > Can you test if this improves performance for you? I'm also not sure the performance, but I can post my comments. I'm not sure your patch can properly handle the list operations. ``` + oldcontext = MemoryContextSwitchTo(rb->context); + txn->walsgmts = lappend(txn->walsgmts, curOpenSegNo); + MemoryContextSwitchTo(oldcontext); + ``` IIUC lappend() accepts a point of a Datum, but here a normal value is passed. Should we define a new struct which represents a node of list and append it after it is palloc()'d? Or your code is enough for some reasons? ``` /* iterate over all possible filenames, and delete them */ - for (cur = first; cur <= last; cur++) + foreach(cell, txn->walsgmts) { + XLogSegNo curr_segno = (XLogSegNo) lfirst(cell); char path[MAXPGPATH]; - ReorderBufferSerializedPath(path, MyReplicationSlot, txn->xid, cur); + ReorderBufferSerializedPath(path, MyReplicationSlot, txn->xid, curr_segno); if (unlink(path) != 0 && errno != ENOENT) ereport(ERROR, (errcode_for_file_access(), errmsg("could not remove file \"%s\": %m", path))); } + + if(txn->walsgmts != NIL) + { + pfree(txn->walsgmts); + txn->walsgmts = NIL; + } ``` If above comment is accepted, I feel you can use foreach_delete_current(). ======= Also, even when we optimize the truncation of files, there is a possibility that replication is timed out. Can you also create a patch which implements [1]? [1]: https://www.postgresql.org/message-id/CAExHW5s2_T9mULDQRKsdV72wpnA%2BNLT63cX51b51QQVEV4sG5g%40mail.g... Best regards, Hayato Kuroda FUJITSU LIMITED ^ permalink raw reply [nested|flat] 62+ messages in thread
end of thread, other threads:[~2024-12-23 12:29 UTC | newest] Thread overview: 62+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2020-03-09 03:57 [PATCH v21 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v26 05/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v35 4/7] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v30 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v28 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v23 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v22 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v18 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v33 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v19 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v25 05/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v20 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v37 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v32 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v27 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v34 04/15] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v31 04/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v36 4/7] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v24 05/11] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v16 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2020-03-09 03:57 [PATCH v17 05/10] pg_ls_tmpdir to show directories and "isdir" argument.. Justin Pryzby <[email protected]> 2024-12-12 13:50 Re: Logical replication timeout RECHTÉ Marc <[email protected]> 2024-12-19 06:27 ` Re: Logical replication timeout Shlok Kyal <[email protected]> 2024-12-23 12:29 ` RE: Logical replication timeout Hayato Kuroda (Fujitsu) <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox