Subject: Clean up find_typedefs and add support for Mac

From: "Tristan Partin" <[email-masked]>
Date: Tue, 12 Dec 2023 15:16:10 -0600

The script was using a few deprecated things according to POSIX:

- -o instead of ||
- egrep
- `` instead of $()

I removed those for their "modern" equivalents. Hopefully no buildfarm 
member complains. I can remove any of those patches though. I did go 
ahead and remove egrep usage from the entire codebase while I was at it. 
There is still a configure check though. I'm thinking that could also be 
removed?

I moved system detection to use uname -s. I hope that isn't a big deal. 
Not sure the best way to identify Mac otherwise.

The big patch here is adding support for Mac. objdump -W doesn't work on 
Mac. So, I used dsymutil and dwarfdump to achieve the same result. I am 
not someone who ever uses awk, so someone should definitely check my 
work there. I can only confirm this works on the latest version of Mac, 
and have no clue how backward compatible it is. I also wrote this 
without having a Mac. I had to ping a coworker with a Mac for help.

My goal with the Mac support is to enable use of find_typedef for 
extension developers, where using a Mac might be more prominent than 
upstream Postgres development, but that is just a guess.

-- 
Tristan Partin
Neon (https://neon.tech)
From e717a14a171c0226921ffed003dedd104bf3cf99 Mon Sep 17 00:00:00 2001
From: Tristan Partin <[email-masked]>
Date: Tue, 12 Dec 2023 10:13:13 -0600
Subject: [PATCH v1 1/6] Replace egrep with grep -E

GNU grep has been warning about grep -E since the 3.8 series. Further
GNU commentary:

> 7th Edition Unix had commands egrep and fgrep that were the counterparts
> of the modern grep -E and grep -F. Although breaking up grep into three
> programs was perhaps useful on the small computers of the 1970s, egrep
> and fgrep were not standardized by POSIX and are no longer needed. In
> the current GNU implementation, egrep and fgrep issue a warning and then
> act like their modern counterparts; eventually, they are planned to be
> removed entirely.

Further man page documentation:

> This grep has been enhanced in an upwards-compatible way to provide the
> exact functionality of the historical egrep and fgrep commands as well.
> It was the clear intention of the standard developers to consolidate the
> three greps into a single command.
---
 src/backend/port/aix/mkldexport.sh  | 4 ++--
 src/tools/find_typedef              | 6 +++---
 src/tools/perlcheck/find_perl_files | 2 +-
 src/tools/pginclude/pgrminclude     | 6 +++---
 4 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/src/backend/port/aix/mkldexport.sh b/src/backend/port/aix/mkldexport.sh
index adf3793e86..41405eba02 100755
--- a/src/backend/port/aix/mkldexport.sh
+++ b/src/backend/port/aix/mkldexport.sh
@@ -53,9 +53,9 @@ else
 	fi
 fi
 $NM -BCg $1 | \
-	egrep ' [TDB] ' | \
+	grep -E ' [TDB] ' | \
 	sed -e 's/.* //' | \
-	egrep -v '\$' | \
+	grep -E -v '\$' | \
 	sed -e 's/^[.]//' | \
 	sort | \
 	uniq
diff --git a/src/tools/find_typedef b/src/tools/find_typedef
index 24e9b76651..fec0520c32 100755
--- a/src/tools/find_typedef
+++ b/src/tools/find_typedef
@@ -36,12 +36,12 @@ do	# if objdump -W is recognized, only one line of error should appear
 	if [ `objdump -W 2>&1 | wc -l` -eq 1 ]
 	then	# Linux
 		objdump -W "$DIR"/* |
-		egrep -A3 '\(DW_TAG_typedef\)' |
+		grep -E -A3 '\(DW_TAG_typedef\)' |
 		awk ' $2 == "DW_AT_name" {print $NF}'
 	elif [ `readelf -w 2>&1 | wc -l` -gt 1 ]
 	then	# FreeBSD, similar output to Linux
 		readelf -w "$DIR"/* |
-		egrep -A3 '\(DW_TAG_typedef\)' |
+		grep -E -A3 '\(DW_TAG_typedef\)' |
 		awk ' $1 == "DW_AT_name" {print $NF}'
 	fi
 done |
@@ -50,4 +50,4 @@ sort |
 uniq |
 # these are used both for typedefs and variable names
 # so do not include them
-egrep -v '^(date|interval|timestamp|ANY)$'
+grep -E -v '^(date|interval|timestamp|ANY)$'
diff --git a/src/tools/perlcheck/find_perl_files b/src/tools/perlcheck/find_perl_files
index 20dceb800d..406ec7f3a0 100644
--- a/src/tools/perlcheck/find_perl_files
+++ b/src/tools/perlcheck/find_perl_files
@@ -12,7 +12,7 @@ find_perl_files () {
 		find "$@" -type f -name '*.p[lm]' -print
 		# take executable files that file(1) thinks are perl files
 		find "$@" -type f -perm -100 -exec file {} \; -print |
-		egrep -i ':.*perl[0-9]*\>' |
+		grep -E -i ':.*perl[0-9]*\>' |
 		cut -d: -f1
 	} | sort -u | grep -v '^\./\.git/'
 }
diff --git a/src/tools/pginclude/pgrminclude b/src/tools/pginclude/pgrminclude
index 7cbd2e7c9c..27c6c5bfb5 100755
--- a/src/tools/pginclude/pgrminclude
+++ b/src/tools/pginclude/pgrminclude
@@ -64,14 +64,14 @@ compile_file() {
 	[ "$INCLUDE" = "pg_config.h" ] && continue
 	[ "$INCLUDE" = "c.h" ] && continue
 	# Stringify macros will expand undefined identifiers, so skip files that use it
-	egrep -q '\<(CppAsString2?|CppConcat)\>' "$FILE" && continue
+	grep -E -q '\<(CppAsString2?|CppConcat)\>' "$FILE" && continue
 
 	# preserve configure-specific includes
 	# these includes are surrounded by #ifdef's
 	grep -B1 '^#include[ 	][ 	]*[<"]'"$INCLUDE"'[>"]' "$FILE" |
-	     egrep -q '^#if|^#else|^#elif' && continue
+	     grep -E -q '^#if|^#else|^#elif' && continue
 	grep -A1 '^#include[ 	][ 	]*[<"]'"$INCLUDE"'[>"]' "$FILE" |
-	     egrep -q '^#else|^#elif|^#endif' && continue
+	     grep -E -q '^#else|^#elif|^#endif' && continue
 
         # Remove all #if and #ifdef blocks because the blocks
 	# might contain code that is not compiled on this platform.
-- 
Tristan Partin
Neon (https://neon.tech)

From 59065b269c86747ece904b4d7a4bd9f494476978 Mon Sep 17 00:00:00 2001
From: Tristan Partin <[email-masked]>
Date: Tue, 12 Dec 2023 10:21:12 -0600
Subject: [PATCH v1 2/6] Remove use of test -o in find_typedef

This has been deprecated for a while. Rationale from POSIX:

> The XSI extensions specifying the -a and -o binary primaries and the '('
> and ')' operators have been marked obsolescent. (Many expressions using
> them are ambiguously defined by the grammar depending on the specific
> expressions being evaluated.) Scripts using these expressions should be
> converted to the forms given below. Even though many implementations
> will continue to support these obsolescent forms, scripts should be
> extremely careful when dealing with user-supplied input that could be
> confused with these and other primaries and operators. Unless the
> application developer knows all the cases that produce input to the
> script, invocations like:
>
>     test "$1" -a "$2"
>
> should be written as:
>
>     test "$1" && test "$2"
---
 src/tools/find_typedef | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/tools/find_typedef b/src/tools/find_typedef
index fec0520c32..ac798a90c1 100755
--- a/src/tools/find_typedef
+++ b/src/tools/find_typedef
@@ -26,7 +26,7 @@
 #    http://www.informatik.uni-frankfurt.de/doc/texi/stabs_toc.html

-if [ "$#" -eq 0 -o ! -d "$1" ]
+if [ "$#" -eq 0 ] || [ ! -d "$1" ]
 then	echo "Usage:  $0 postgres_binary_directory [...]" 1>&2
 	exit 1
 fi
-- 
Tristan Partin
Neon (https://neon.tech)

From 23917026c656af38ef6d781622184d06dcf72ed3 Mon Sep 17 00:00:00 2001
From: Tristan Partin <[email-masked]>
Date: Tue, 12 Dec 2023 10:25:49 -0600
Subject: [PATCH v1 3/6] Remove use of backticks for running processes in
 find_typedef

Backticks are the legacy form of command substitution, required by only
the very oldest of non-POSIX-compatible Bourne shells.

Per POSIX:

> The "$()" form of command substitution solves a problem of inconsistent
> behavior when using backquotes.

Link: https://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xcu_chap02.html#tag_23_02_06_03
---
 src/tools/find_typedef | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/tools/find_typedef b/src/tools/find_typedef
index ac798a90c1..e04c642da8 100755
--- a/src/tools/find_typedef
+++ b/src/tools/find_typedef
@@ -33,12 +33,12 @@ fi
 
 for DIR
 do	# if objdump -W is recognized, only one line of error should appear
-	if [ `objdump -W 2>&1 | wc -l` -eq 1 ]
+	if [ $(objdump -W 2>&1 | wc -l) -eq 1 ]
 	then	# Linux
 		objdump -W "$DIR"/* |
 		grep -E -A3 '\(DW_TAG_typedef\)' |
 		awk ' $2 == "DW_AT_name" {print $NF}'
-	elif [ `readelf -w 2>&1 | wc -l` -gt 1 ]
+	elif [ $(readelf -w 2>&1 | wc -l) -gt 1 ]
 	then	# FreeBSD, similar output to Linux
 		readelf -w "$DIR"/* |
 		grep -E -A3 '\(DW_TAG_typedef\)' |
-- 
Tristan Partin
Neon (https://neon.tech)

From ec28f5a3f7d9c8854dcdd02f57d1f892f51cb7c0 Mon Sep 17 00:00:00 2001
From: Tristan Partin <[email-masked]>
Date: Tue, 12 Dec 2023 13:22:19 -0600
Subject: [PATCH v1 4/6] Use uname -s for determining the system in
 find_typedef

Analyzing error output has proven to be unstable per a patch from Tom
after discovering we were missing some typedefs.

uname(1) is standardized by POSIX, and can be relied upon to be much
more stable.

Link: https://www.postgresql.org/message-id/flat/526703.1652385613%40sss.pgh.pa.us
---
 src/tools/find_typedef | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/src/tools/find_typedef b/src/tools/find_typedef
index e04c642da8..8adbe9ab94 100755
--- a/src/tools/find_typedef
+++ b/src/tools/find_typedef
@@ -31,15 +31,17 @@ then	echo "Usage:  $0 postgres_binary_directory [...]" 1>&2
 	exit 1
 fi
 
+system=$(uname -s)
+
 for DIR
-do	# if objdump -W is recognized, only one line of error should appear
-	if [ $(objdump -W 2>&1 | wc -l) -eq 1 ]
-	then	# Linux
+do
+	if [ "$system" = Linux ]
+	then
 		objdump -W "$DIR"/* |
 		grep -E -A3 '\(DW_TAG_typedef\)' |
 		awk ' $2 == "DW_AT_name" {print $NF}'
-	elif [ $(readelf -w 2>&1 | wc -l) -gt 1 ]
-	then	# FreeBSD, similar output to Linux
+	elif [ "$system" = FreeBSD ]
+	then
 		readelf -w "$DIR"/* |
 		grep -E -A3 '\(DW_TAG_typedef\)' |
 		awk ' $1 == "DW_AT_name" {print $NF}'
-- 
Tristan Partin
Neon (https://neon.tech)

From ed8a89fca03e264ad772f738bc966bbbe0d31b4d Mon Sep 17 00:00:00 2001
From: Tristan Partin <[email-masked]>
Date: Tue, 12 Dec 2023 13:43:32 -0600
Subject: [PATCH v1 5/6] Add Mac support to find_typedef

This was previously unsupported on Mac.
---
 src/tools/find_typedef | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/src/tools/find_typedef b/src/tools/find_typedef
index 8adbe9ab94..ce82897c3f 100755
--- a/src/tools/find_typedef
+++ b/src/tools/find_typedef
@@ -45,6 +45,17 @@ do
 		readelf -w "$DIR"/* |
 		grep -E -A3 '\(DW_TAG_typedef\)' |
 		awk ' $1 == "DW_AT_name" {print $NF}'
+	elif [ "$system" = Darwin ]
+	then
+		# Creates *.dwarf files that we will remove later
+		dsymutil --flat "$DIR"/*
+
+		# Apple's dwarfdump, wraps types: ("example")
+		dwarfdump "$DIR"/*.dwarf |
+		grep -F -A3 'DW_TAG_typedef' |
+		awk ' $1 == "DW_AT_name" {print substr($NF, 3, length($NF) - 4)}'
+
+		rm -f "$DIR"/*.dwarf
 	fi
 done |
 grep -v ' ' | # some typedefs have spaces, remove them
-- 
Tristan Partin
Neon (https://neon.tech)

From b741122e2d4fe3c33cd1ad7c170f54a2c7e2c2f1 Mon Sep 17 00:00:00 2001
From: Tristan Partin <[email-masked]>
Date: Tue, 12 Dec 2023 14:37:32 -0600
Subject: [PATCH v1 6/6] Use grep -F instead of -E in find_typedef

These grep don't need regex support.
---
 src/tools/find_typedef | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/tools/find_typedef b/src/tools/find_typedef
index ce82897c3f..04f6d45021 100755
--- a/src/tools/find_typedef
+++ b/src/tools/find_typedef
@@ -38,12 +38,12 @@ do
 	if [ "$system" = Linux ]
 	then
 		objdump -W "$DIR"/* |
-		grep -E -A3 '\(DW_TAG_typedef\)' |
+		grep -F -A3 DW_TAG_typedef |
 		awk ' $2 == "DW_AT_name" {print $NF}'
 	elif [ "$system" = FreeBSD ]
 	then
 		readelf -w "$DIR"/* |
-		grep -E -A3 '\(DW_TAG_typedef\)' |
+		grep -F -A3 DW_TAG_typedef |
 		awk ' $1 == "DW_AT_name" {print $NF}'
 	elif [ "$system" = Darwin ]
 	then
-- 
Tristan Partin
Neon (https://neon.tech)