Subject: Re: pgindent vs. pgperltidy command-line arguments

From: Tom Lane <[email-masked]>
Date: Thu, 25 May 2023 09:20:32 -0400

Peter Eisentraut <[email-masked]> writes:
> Until PG15, calling pgindent without arguments would process the whole 
> tree.  Now you get
> No files to process at ./src/tools/pgindent/pgindent line 372.
> Is that intentional?

It was intentional, cf b16259b3c and the linked discussion.

> Also, pgperltidy accepts no arguments and always processes the whole 
> tree.  It would be nice if there were a way to process individual files 
> or directories, like pgindent can.

+1, although I wonder if we shouldn't follow pgindent's new lead
and require some argument(s).

> Attached is a patch for this.
> (It seems that it works ok to pass regular files (not directories) to 
> "find", but I'm not sure if it's portable.)

The POSIX spec for find(1) gives an example of applying find to
what they evidently intend to be a plain file:

	if [ -n "$(find file1 -prune -newer file2)" ]; then
	    printf %s\\n "file1 is newer than file2"
	fi

So while I don't see it written in so many words, I think you
can assume it's portable.

			regards, tom lane

---

From: Peter Eisentraut <[email-masked]>
Date: Wed, 14 Jun 2023 09:37:45 +0200

> and require some argument(s).

That makes sense to me.  Here is a small update with this behavior 
change and associated documentation update.
From 44f7bcdcc0849a55459d4c2da27ee1976c704933 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email-masked]>
Date: Wed, 14 Jun 2023 09:33:03 +0200
Subject: [PATCH v2] Allow and require passing files on command line of
 pgperltidy

pgperltidy as well as pgperlcritic and pgperlsyncheck now allow
passing files and directories on the command line, like pgindent does.
(Previously, they would always operate on the whole tree.)

Also, for consistency with pgindent's new behavior (as of b16259b3c1),
passing an argument is now required.  To get the previous default
behavior, use "pgperltidy ." for example.

Discussion: https://www.postgresql.org/message-id/flat/45aacd8a-5265-d9da-8df2-b8e2c0cf6a07%40eisentraut.org
---
 src/tools/perlcheck/find_perl_files | 8 ++++++--
 src/tools/perlcheck/pgperlcritic    | 2 +-
 src/tools/perlcheck/pgperlsyncheck  | 2 +-
 src/tools/pgindent/README           | 2 +-
 src/tools/pgindent/pgperltidy       | 2 +-
 5 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/src/tools/perlcheck/find_perl_files b/src/tools/perlcheck/find_perl_files
index fd99dab83b..20dceb800d 100644
--- a/src/tools/perlcheck/find_perl_files
+++ b/src/tools/perlcheck/find_perl_files
@@ -3,11 +3,15 @@
 # shell function to find all perl files in the source tree
 
 find_perl_files () {
+	if [ $# -eq 0 ]; then
+		echo 'No files to process' 1>&2
+		return
+	fi
     {
 		# take all .pl and .pm files
-		find . -type f -name '*.p[lm]' -print
+		find "$@" -type f -name '*.p[lm]' -print
 		# take executable files that file(1) thinks are perl files
-		find . -type f -perm -100 -exec file {} \; -print |
+		find "$@" -type f -perm -100 -exec file {} \; -print |
 		egrep -i ':.*perl[0-9]*\>' |
 		cut -d: -f1
 	} | sort -u | grep -v '^\./\.git/'
diff --git a/src/tools/perlcheck/pgperlcritic b/src/tools/perlcheck/pgperlcritic
index 1c2f787580..2ec6f20de3 100755
--- a/src/tools/perlcheck/pgperlcritic
+++ b/src/tools/perlcheck/pgperlcritic
@@ -14,7 +14,7 @@ PERLCRITIC=${PERLCRITIC:-perlcritic}
 
 . src/tools/perlcheck/find_perl_files
 
-find_perl_files | xargs $PERLCRITIC \
+find_perl_files "$@" | xargs $PERLCRITIC \
 	  --quiet \
 	  --program-extensions .pl \
 	  --profile=src/tools/perlcheck/perlcriticrc
diff --git a/src/tools/perlcheck/pgperlsyncheck b/src/tools/perlcheck/pgperlsyncheck
index 730f5927cd..da59c9727c 100755
--- a/src/tools/perlcheck/pgperlsyncheck
+++ b/src/tools/perlcheck/pgperlsyncheck
@@ -13,4 +13,4 @@ set -e
 # for zsh
 setopt shwordsplit 2>/dev/null || true
 
-find_perl_files | xargs -L 1 perl $INCLUDES -cw 2>&1 | grep -v OK
+find_perl_files "$@" | xargs -L 1 perl $INCLUDES -cw 2>&1 | grep -v OK
diff --git a/src/tools/pgindent/README b/src/tools/pgindent/README
index b2b134ee6a..f5fdfc5d2f 100644
--- a/src/tools/pgindent/README
+++ b/src/tools/pgindent/README
@@ -45,7 +45,7 @@ DOING THE INDENT RUN:
 
 4) Indent the Perl code using perltidy:
 
-	src/tools/pgindent/pgperltidy
+	src/tools/pgindent/pgperltidy .
 
    If you want to use some perltidy version that's not in your PATH,
    first set the PERLTIDY environment variable to point to it.
diff --git a/src/tools/pgindent/pgperltidy b/src/tools/pgindent/pgperltidy
index 5e704119eb..6af27d21d5 100755
--- a/src/tools/pgindent/pgperltidy
+++ b/src/tools/pgindent/pgperltidy
@@ -9,4 +9,4 @@ PERLTIDY=${PERLTIDY:-perltidy}
 
 . src/tools/perlcheck/find_perl_files
 
-find_perl_files | xargs $PERLTIDY --profile=src/tools/pgindent/perltidyrc
+find_perl_files "$@" | xargs $PERLTIDY --profile=src/tools/pgindent/perltidyrc

base-commit: 0f8cfaf8921fed35f0b92d918ce95eec7b46ff05
-- 
2.41.0

---

From: Andrew Dunstan <[email-masked]>
Date: Tue, 20 Jun 2023 11:38:10 -0400

> change and associated documentation update.

I'm intending to add some of the new pgindent features to pgperltidy. 
Preparatory to that here's a rewrite of pgperltidy in perl - no new 
features yet but it does remove the hardcoded path, and requires you to 
pass in one or more files / directories as arguments.

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

---

From: Peter Eisentraut <[email-masked]>
Date: Wed, 21 Jun 2023 11:09:02 +0200

> pass in one or more files / directories as arguments.

Are you planning to touch pgperlcritic and pgperlsyncheck as well?  If 
not, part of my patch would still be useful.  Maybe I should commit my 
posted patch for PG16, to keep consistency with pgindent, and then your 
work would presumably be considered for PG17.

---

From: Andrew Dunstan <[email-masked]>
Date: Wed, 21 Jun 2023 07:35:15 -0400

On 2023-06-21 We 05:09, Peter Eisentraut wrote:
>> features yet but it does remove the hardcoded path, and requires you 
>> to pass in one or more files / directories as arguments.
>
> Are you planning to touch pgperlcritic and pgperlsyncheck as well? 

Yeah, it would make sense to.

> If not, part of my patch would still be useful.  Maybe I should commit 
> my posted patch for PG16, to keep consistency with pgindent, and then 
> your work would presumably be considered for PG17.

That sounds like a good plan.

cheers

andrew

---

From: Peter Eisentraut <[email-masked]>
Date: Wed, 21 Jun 2023 16:36:06 +0200

> That sounds like a good plan.

done

---

From: Andrew Dunstan <[email-masked]>
Date: Thu, 6 Jul 2023 11:47:33 -0400

On 2023-06-21 We 07:35, Andrew Dunstan wrote:
>>> I'm intending to add some of the new pgindent features to 
>>> pgperltidy. Preparatory to that here's a rewrite of pgperltidy in 
>>> perl - no new features yet but it does remove the hardcoded path, 
>>> and requires you to pass in one or more files / directories as 
>>> arguments.
>>

Here's a patch that turns all these into perl scripts.

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com
diff --git a/src/tools/perlcheck/FindPerlFiles.pm b/src/tools/perlcheck/FindPerlFiles.pm
new file mode 100644
index 0000000000..3a098f6614
--- /dev/null
+++ b/src/tools/perlcheck/FindPerlFiles.pm
@@ -0,0 +1,62 @@
+
+# Copyright (c) 2023, PostgreSQL Global Development Group
+
+=pod
+
+=head1 NAME
+
+FindPerlFiles - module for finding perl files from a list of paths
+
+=head1 SYNOPSIS
+
+  use FindPerlFiles;
+
+  my @files = FindPerlFiles::findperl(path, ...);
+
+
+=head1 DESCRIPTION
+
+FindPerlFiles finds files which either have a perl extension (.pl or .pm) or
+are bot executable and found by the `file` program to be perl scripts.
+
+=cut
+
+
+package FindPerlFiles;
+
+use strict;
+use warnings;
+
+use File::Find;
+use File::stat;
+use Fcntl ':mode';
+
+my @files;
+
+sub _is_perl_exec
+{
+	my $name = shift;
+	my $out = `file $name 2>/dev/null`;
+	return $out =~ /:.*perl[0-9]*\b/i;
+}
+
+sub findperl
+{
+	my @files ;
+	my $wanted = sub
+	{
+		my $name = $File::Find::name;
+		my $st;
+		# check it's a plain file and either it has a perl extension (.p[lm])
+		# or it's executable and `file` thinks it's a perl script.
+		($st = lstat($_))
+		  && -f $st
+		  && (/\.p[lm]$/ || ((($st->mode & S_IXUSR) && _is_perl_exec($_))))
+		  && push(@files, $name);
+	};
+
+	File::Find::find({ wanted => $wanted }, @_);
+	return @files;
+}
+
+1;
diff --git a/src/tools/perlcheck/find_perl_files b/src/tools/perlcheck/find_perl_files
deleted file mode 100644
index 20dceb800d..0000000000
--- a/src/tools/perlcheck/find_perl_files
+++ /dev/null
@@ -1,18 +0,0 @@
-# src/tools/perlcheck/find_perl_files
-
-# shell function to find all perl files in the source tree
-
-find_perl_files () {
-	if [ $# -eq 0 ]; then
-		echo 'No files to process' 1>&2
-		return
-	fi
-    {
-		# take all .pl and .pm files
-		find "$@" -type f -name '*.p[lm]' -print
-		# take executable files that file(1) thinks are perl files
-		find "$@" -type f -perm -100 -exec file {} \; -print |
-		egrep -i ':.*perl[0-9]*\>' |
-		cut -d: -f1
-	} | sort -u | grep -v '^\./\.git/'
-}
diff --git a/src/tools/perlcheck/pgperlcritic b/src/tools/perlcheck/pgperlcritic
index 2ec6f20de3..87d5e92df7 100755
--- a/src/tools/perlcheck/pgperlcritic
+++ b/src/tools/perlcheck/pgperlcritic
@@ -1,20 +1,27 @@
-#!/bin/sh
+#!/usr/bin/perl
+
+# Copyright (c) 2023, PostgreSQL Global Development Group
 
 # src/tools/perlcheck/pgperlcritic
 
-test -f src/tools/perlcheck/perlcriticrc || {
-	echo could not find src/tools/perlcheck/perlcriticrc
-	exit 1
-	}
+use strict;
+use warnings;
 
-set -e
+use FindBin ();
+use lib "$FindBin::Bin";
+use FindPerlFiles;
+
+die "No directories or files specified\n" unless @ARGV;
+
+my $rc_loc = "$FindBin::Bin/perlcriticrc";
+
+die "no $rc_loc\n" unless -f $rc_loc;
 
 # set this to override default perlcritic program:
-PERLCRITIC=${PERLCRITIC:-perlcritic}
+my $perlcritic = $ENV{PERLCRITIC} || 'perlcritic';
 
-. src/tools/perlcheck/find_perl_files
+my @files = FindPerlFiles::findperl(@ARGV);
 
-find_perl_files "$@" | xargs $PERLCRITIC \
-	  --quiet \
-	  --program-extensions .pl \
-	  --profile=src/tools/perlcheck/perlcriticrc
+exit unless @files;
+
+exec $perlcritic, '--program-extensions',  '.pl', "--profile=$rc_loc", @files
diff --git a/src/tools/perlcheck/pgperlsyncheck b/src/tools/perlcheck/pgperlsyncheck
index da59c9727c..70ff248daf 100755
--- a/src/tools/perlcheck/pgperlsyncheck
+++ b/src/tools/perlcheck/pgperlsyncheck
@@ -1,16 +1,48 @@
-#!/bin/sh
+#!/usr/bin/perl
+
+# Copyright (c) 2023, PostgreSQL Global Development Group
 
 # script to detect compile time errors and warnings in all perl files
 
-INCLUDES="-I src/tools/msvc -I src/tools/msvc/dummylib -I src/backend/catalog"
-INCLUDES="-I src/test/perl -I src/backend/utils/mb/Unicode $INCLUDES"
-INCLUDES="-I src/bin/pg_rewind -I src/test/ssl/t $INCLUDES"
+use strict;
+use warnings;
 
-set -e
+use FindBin ();
+use lib "$FindBin::Bin";
 
-. src/tools/perlcheck/find_perl_files
+use Cwd qw(abs_path);
+use FindPerlFiles;
 
-# for zsh
-setopt shwordsplit 2>/dev/null || true
+die "No directories or files specified\n" unless @ARGV;
 
-find_perl_files "$@" | xargs -L 1 perl $INCLUDES -cw 2>&1 | grep -v OK
+my $srcroot = abs_path("$FindBin::Bin/../../..");
+
+my $includes="";
+
+my @incdirs = qw(
+    src/tools/msvc
+    src/tools/msvc/dummylib
+    src/backend/catalog
+    src/test/perl
+    src/backend/utils/mb/Unicode
+    src/bin/pg_rewind src/test/ssl/t
+);
+
+foreach my $inc (@incdirs)
+{
+    $includes .= "-I $srcroot/$inc ";
+}
+
+my @files =  FindPerlFiles::findperl(@ARGV);
+
+my $status = 0;
+
+foreach my $file (@files)
+{
+    my $out = `perl $includes -cw $file 2>&1`;
+    next if $out =~ /OK/;
+    $status = 1;
+    print $out;
+}
+
+exit $status;
diff --git a/src/tools/pgindent/pgperltidy b/src/tools/pgindent/pgperltidy
index 6af27d21d5..be433c1836 100755
--- a/src/tools/pgindent/pgperltidy
+++ b/src/tools/pgindent/pgperltidy
@@ -1,12 +1,22 @@
-#!/bin/sh
+#!/usr/bin/perl
+
+# Copyright (c) 2023, PostgreSQL Global Development Group
 
 # src/tools/pgindent/pgperltidy
 
-set -e
+use strict;
+use warnings;
 
-# set this to override default perltidy program:
-PERLTIDY=${PERLTIDY:-perltidy}
+use FindBin ();
+use lib "$FindBin::Bin/../perlcheck";
+use FindPerlFiles;
 
-. src/tools/perlcheck/find_perl_files
+my $perltidy = $ENV{PERLTIDY} || 'perltidy';
 
-find_perl_files "$@" | xargs $PERLTIDY --profile=src/tools/pgindent/perltidyrc
+die "No directories or files specified\n" unless @ARGV;
+
+my @files = FindPerlFiles::findperl(@ARGV);
+
+exit unless @files;
+
+exec "$perltidy", "--profile=$FindBin::Bin /perltidyrc", @files;