public inbox for [email protected]help / color / mirror / Atom feed
[PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. 2+ messages / 2 participants [nested] [flat]
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. @ 2020-03-09 03:52 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 2+ messages in thread From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw) ..possibly there's a better place to put this, like maybe a doc-only example ? Need catversion bumped ? --- doc/src/sgml/func.sgml | 32 ++++++++++++++++++++ src/backend/catalog/system_views.sql | 1 + src/include/catalog/pg_proc.dat | 6 ++++ src/test/regress/expected/misc_functions.out | 14 +++++++++ src/test/regress/sql/misc_functions.sql | 6 ++++ 5 files changed, 59 insertions(+) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 8e0de7c02d..334b6beb3a 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>pg_ls_dir_recurse</primary> + </indexterm> + <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> ) + <returnvalue>setof record</returnvalue> + ( <parameter>name</parameter> <type>text</type>, + <parameter>size</parameter> <type>bigint</type>, + <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>isdir</parameter> <type>boolean</type> ) + </para> + <para> + Recursively list each file in the specified directory, along with the + files' metadata. + </para> + <para> + Restricted to superusers by default, but other users can be granted + EXECUTE to run the function. + </para></entry> + </row> + + <!--para> + To recursively list temporary directories in all tablespaces: +<programlisting> +SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir +FROM pg_tablespace b, pg_control_system() pcs, +LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir, +LATERAL pg_ls_dir_recurse(dir) AS a; +</programlisting> + </para--> + <row> <entry role="func_table_entry"><para role="func_signature"> <indexterm> diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 1c77430f0c..215fba3b7d 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public; REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public; REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public; REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public; +REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public; -- -- We also set up some things as accessible to standard roles. diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index adfce45d1a..f94f403475 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10916,6 +10916,12 @@ proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', proargnames => '{dirname,name,size,modification,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, +{ oid => '5034', descr => 'list all files in a directory recursively', + proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't', + provolatile => 'v', prorettype => 'record', proargtypes => 'text', + proallargtypes => '{text,text,int8,timestamptz,bool}', + proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}', + prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" }, # hash partitioning constraint function { oid => '5028', descr => 'hash partition CHECK constraint', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 64b1417fb8..4188d684f0 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0; ------+------+--------------+------- (0 rows) +-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix +SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal'; + name | isdir +-----------------------+------- + pg_wal | t + pg_wal/archive_status | t +(2 rows) + +-- Check that expected columns are present +SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0; + name | size | modification | isdir +------+------+--------------+------- +(0 rows) + -- -- Test adding a support function to a subject function -- diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql index 372345720d..6041c4f3dc 100644 --- a/src/test/regress/sql/misc_functions.sql +++ b/src/test/regress/sql/misc_functions.sql @@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; -- -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; +-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix +SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal'; + +-- Check that expected columns are present +SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0; + -- -- Test adding a support function to a subject function -- -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch" ^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: "ERROR: latch already owned" on gharial @ 2024-02-08 12:57 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 2+ messages in thread From: Heikki Linnakangas @ 2024-02-08 12:57 UTC (permalink / raw) To: Soumyadeep Chakraborty <[email protected]>; Alvaro Herrera <[email protected]>; +Cc: Sandeep Thakkar <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; [email protected]; CM Team <[email protected]> On 08/02/2024 04:08, Soumyadeep Chakraborty wrote: > A possible ordering of events: > > (1) DisownLatch() is called by pid Y during ProcKill() and the write for > latch->owner_pid = 0 is NOT yet flushed to shmem. > > (2) The PGPROC object for pid Y is returned to the free list. > > (3) Pid X sees the same PGPROC object on the free list and grabs it. > > (4) Pid X does sanity check inside OwnLatch during InitProcess and > still sees the > old value of latch->owner_pid = Y (and not = 0), and trips the ERROR. > > The above sequence of operations should apply to PG HEAD as well. > > Suggestion: > > Should we do a pg_memory_barrier() at the end of DisownLatch(), like in > ResetLatch(), like the one introduced in [3]? This would ensure that the write > latch->owner_pid = 0; is flushed to shmem. The attached patch does this. Hmm, there is a pair of SpinLockAcquire() and SpinLockRelease() in ProcKill(), before step 3 can happen. Comment in spin.h about SpinLockAcquire/Release: > * Load and store operations in calling code are guaranteed not to be > * reordered with respect to these operations, because they include a > * compiler barrier. (Before PostgreSQL 9.5, callers needed to use a > * volatile qualifier to access data protected by spinlocks.) That talks about a compiler barrier, though, not a memory barrier. But looking at the implementations in s_lock.h, I believe they do act as memory barrier, too. So you might indeed have that problem on 9.4, but AFAICS not on later versions. -- Heikki Linnakangas Neon (https://neon.tech) ^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2024-02-08 12:57 UTC | newest] Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]> 2024-02-08 12:57 Re: "ERROR: latch already owned" on gharial Heikki Linnakangas <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox