Subject: Replace uses of deprecated Python module distutils.sysconfig

From: Peter Eisentraut <[email-masked]>
Date: Thu, 2 Dec 2021 08:20:48 +0100

With Python 3.10, configure spits out warnings about the module 
distutils.sysconfig being deprecated and scheduled for removal in Python 
3.12:

<string>:1: DeprecationWarning: The distutils.sysconfig module is 
deprecated, use sysconfig instead
<string>:1: DeprecationWarning: The distutils package is deprecated and 
slated for removal in Python 3.12. Use setuptools or check PEP 632 for 
potential alternatives

This patch changes the uses in configure to use the module sysconfig 
instead.  The logic stays the same.  (It's basically the same module but 
as its own top-level module.)

Note that sysconfig exists since Python 2.7, so this moves the minimum 
required version up from Python 2.6.

Buildfarm impact:

gaur and prariedog use Python 2.6 and would need to be upgraded.

Possible backpatching:

Backpatching should be considered, since surely someone will otherwise 
complain when Python 3.12 comes around.  But dropping support for Python 
versions in stable branches should be done with some care.

Python 3.10 was released Oct. 4, 2021, so it is quite new.  Python major 
releases are now yearly, so the above-mentioned Python 3.12 can be 
expected in autumn of 2023.

Current PostgreSQL releases support Python versions as follows:

PG10: 2.4+
PG11: 2.4+
PG12: 2.4+  (EOL Nov. 2024)
PG13: 2.6+
PG14: 2.6+

So unfortunately, we won't be able to EOL all versions with Python 2.4 
support before Python 3.12 arrives.

I suggest leaving the backbranches alone for now.  At the moment, we 
don't even know whether additional changes will be required for 3.12 
(and 3.11) support, so the overall impact isn't known yet.  In a few 
months, we will probably know more about this.

In the meantime, the warnings can be silenced using

export PYTHONWARNINGS='ignore::DeprecationWarning'

(It ought to be possible to be more specific, like 
'ignore::DeprecationWarning:distutils.sysconfig', but it doesn't seem to 
work for me.)

(I don't recommend putting that into configure, since then we wouldn't 
be able to learn about issues like this.)

From fcfd4f1702f0cef7d0f6564aff94c4a6258f24e6 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email-masked]>
Date: Thu, 2 Dec 2021 08:08:44 +0100
Subject: [PATCH] Replace uses of deprecated Python module distutils.sysconfig

With Python 3.10, configure spits out warnings about the module
distutils.sysconfig being deprecated and scheduled for removal in
Python 3.12.  Change the uses in configure to use the module sysconfig
instead.  The logic stays the same.

Note that sysconfig exists since Python 2.7, so this moves the minimum
required version up from Python 2.6.
---
 config/python.m4               | 28 ++++++++++++++--------------
 configure                      | 30 +++++++++++++++---------------
 doc/src/sgml/installation.sgml |  4 ++--
 3 files changed, 31 insertions(+), 31 deletions(-)

diff --git a/config/python.m4 b/config/python.m4
index d41aeb2876..8ca1eaa64b 100644
--- a/config/python.m4
+++ b/config/python.m4
@@ -37,28 +37,28 @@ python_majorversion=`echo "$python_fullversion" | sed '[s/^\([0-9]*\).*/\1/]'`
 python_minorversion=`echo "$python_fullversion" | sed '[s/^[0-9]*\.\([0-9]*\).*/\1/]'`
 python_version=`echo "$python_fullversion" | sed '[s/^\([0-9]*\.[0-9]*\).*/\1/]'`
 # Reject unsupported Python versions as soon as practical.
-if test "$python_majorversion" -lt 3 -a "$python_minorversion" -lt 6; then
-  AC_MSG_ERROR([Python version $python_version is too old (version 2.6 or later is required)])
+if test "$python_majorversion" -lt 3 -a "$python_minorversion" -lt 7; then
+  AC_MSG_ERROR([Python version $python_version is too old (version 2.7 or later is required)])
 fi
 
-AC_MSG_CHECKING([for Python distutils module])
-if "${PYTHON}" -c 'import distutils' 2>&AS_MESSAGE_LOG_FD
+AC_MSG_CHECKING([for Python sysconfig module])
+if "${PYTHON}" -c 'import sysconfig' 2>&AS_MESSAGE_LOG_FD
 then
     AC_MSG_RESULT(yes)
 else
     AC_MSG_RESULT(no)
-    AC_MSG_ERROR([distutils module not found])
+    AC_MSG_ERROR([sysconfig module not found])
 fi
 
 AC_MSG_CHECKING([Python configuration directory])
-python_configdir=`${PYTHON} -c "import distutils.sysconfig; print(' '.join(filter(None,distutils.sysconfig.get_config_vars('LIBPL'))))"`
+python_configdir=`${PYTHON} -c "import sysconfig; print(' '.join(filter(None,sysconfig.get_config_vars('LIBPL'))))"`
 AC_MSG_RESULT([$python_configdir])
 
 AC_MSG_CHECKING([Python include directories])
 python_includespec=`${PYTHON} -c "
-import distutils.sysconfig
-a = '-I' + distutils.sysconfig.get_python_inc(False)
-b = '-I' + distutils.sysconfig.get_python_inc(True)
+import sysconfig
+a = '-I' + sysconfig.get_path('include')
+b = '-I' + sysconfig.get_path('platinclude')
 if a == b:
     print(a)
 else:
@@ -96,8 +96,8 @@ AC_DEFUN([PGAC_CHECK_PYTHON_EMBED_SETUP],
 [AC_REQUIRE([_PGAC_CHECK_PYTHON_DIRS])
 AC_MSG_CHECKING([how to link an embedded Python application])
 
-python_libdir=`${PYTHON} -c "import distutils.sysconfig; print(' '.join(filter(None,distutils.sysconfig.get_config_vars('LIBDIR'))))"`
-python_ldlibrary=`${PYTHON} -c "import distutils.sysconfig; print(' '.join(filter(None,distutils.sysconfig.get_config_vars('LDLIBRARY'))))"`
+python_libdir=`${PYTHON} -c "import sysconfig; print(' '.join(filter(None,sysconfig.get_config_vars('LIBDIR'))))"`
+python_ldlibrary=`${PYTHON} -c "import sysconfig; print(' '.join(filter(None,sysconfig.get_config_vars('LDLIBRARY'))))"`
 
 # If LDLIBRARY exists and has a shlib extension, use it verbatim.
 ldlibrary=`echo "${python_ldlibrary}" | sed -e 's/\.so$//' -e 's/\.dll$//' -e 's/\.dylib$//' -e 's/\.sl$//'`
@@ -109,11 +109,11 @@ else
 	# Otherwise, guess the base name of the shlib.
 	# LDVERSION was added in Python 3.2, before that use VERSION,
 	# or failing that, $python_version from _PGAC_CHECK_PYTHON_DIRS.
-	python_ldversion=`${PYTHON} -c "import distutils.sysconfig; print(' '.join(filter(None,distutils.sysconfig.get_config_vars('LDVERSION'))))"`
+	python_ldversion=`${PYTHON} -c "import sysconfig; print(' '.join(filter(None,sysconfig.get_config_vars('LDVERSION'))))"`
 	if test x"${python_ldversion}" != x""; then
 		ldlibrary="python${python_ldversion}"
 	else
-		python_version_var=`${PYTHON} -c "import distutils.sysconfig; print(' '.join(filter(None,distutils.sysconfig.get_config_vars('VERSION'))))"`
+		python_version_var=`${PYTHON} -c "import sysconfig; print(' '.join(filter(None,sysconfig.get_config_vars('VERSION'))))"`
 		if test x"${python_version_var}" != x""; then
 			ldlibrary="python${python_version_var}"
 		else
@@ -173,7 +173,7 @@ PL/Python.])
 fi
 python_libspec="-L${python_libdir} -l${ldlibrary}"
 
-python_additional_libs=`${PYTHON} -c "import distutils.sysconfig; print(' '.join(filter(None,distutils.sysconfig.get_config_vars('LIBS','LIBC','LIBM','BASEMODLIBS'))))"`
+python_additional_libs=`${PYTHON} -c "import sysconfig; print(' '.join(filter(None,sysconfig.get_config_vars('LIBS','LIBC','LIBM','BASEMODLIBS'))))"`
 
 AC_MSG_RESULT([${python_libspec} ${python_additional_libs}])
 
diff --git a/configure b/configure
index 5d72ee3fde..ac61ad8ad0 100755
--- a/configure
+++ b/configure
@@ -10345,34 +10345,34 @@ python_majorversion=`echo "$python_fullversion" | sed 's/^\([0-9]*\).*/\1/'`
 python_minorversion=`echo "$python_fullversion" | sed 's/^[0-9]*\.\([0-9]*\).*/\1/'`
 python_version=`echo "$python_fullversion" | sed 's/^\([0-9]*\.[0-9]*\).*/\1/'`
 # Reject unsupported Python versions as soon as practical.
-if test "$python_majorversion" -lt 3 -a "$python_minorversion" -lt 6; then
-  as_fn_error $? "Python version $python_version is too old (version 2.6 or later is required)" "$LINENO" 5
+if test "$python_majorversion" -lt 3 -a "$python_minorversion" -lt 7; then
+  as_fn_error $? "Python version $python_version is too old (version 2.7 or later is required)" "$LINENO" 5
 fi
 
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for Python distutils module" >&5
-$as_echo_n "checking for Python distutils module... " >&6; }
-if "${PYTHON}" -c 'import distutils' 2>&5
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for Python sysconfig module" >&5
+$as_echo_n "checking for Python sysconfig module... " >&6; }
+if "${PYTHON}" -c 'import sysconfig' 2>&5
 then
     { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
 $as_echo "yes" >&6; }
 else
     { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
 $as_echo "no" >&6; }
-    as_fn_error $? "distutils module not found" "$LINENO" 5
+    as_fn_error $? "sysconfig module not found" "$LINENO" 5
 fi
 
 { $as_echo "$as_me:${as_lineno-$LINENO}: checking Python configuration directory" >&5
 $as_echo_n "checking Python configuration directory... " >&6; }
-python_configdir=`${PYTHON} -c "import distutils.sysconfig; print(' '.join(filter(None,distutils.sysconfig.get_config_vars('LIBPL'))))"`
+python_configdir=`${PYTHON} -c "import sysconfig; print(' '.join(filter(None,sysconfig.get_config_vars('LIBPL'))))"`
 { $as_echo "$as_me:${as_lineno-$LINENO}: result: $python_configdir" >&5
 $as_echo "$python_configdir" >&6; }
 
 { $as_echo "$as_me:${as_lineno-$LINENO}: checking Python include directories" >&5
 $as_echo_n "checking Python include directories... " >&6; }
 python_includespec=`${PYTHON} -c "
-import distutils.sysconfig
-a = '-I' + distutils.sysconfig.get_python_inc(False)
-b = '-I' + distutils.sysconfig.get_python_inc(True)
+import sysconfig
+a = '-I' + sysconfig.get_path('include')
+b = '-I' + sysconfig.get_path('platinclude')
 if a == b:
     print(a)
 else:
@@ -10388,8 +10388,8 @@ $as_echo "$python_includespec" >&6; }
 { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to link an embedded Python application" >&5
 $as_echo_n "checking how to link an embedded Python application... " >&6; }
 
-python_libdir=`${PYTHON} -c "import distutils.sysconfig; print(' '.join(filter(None,distutils.sysconfig.get_config_vars('LIBDIR'))))"`
-python_ldlibrary=`${PYTHON} -c "import distutils.sysconfig; print(' '.join(filter(None,distutils.sysconfig.get_config_vars('LDLIBRARY'))))"`
+python_libdir=`${PYTHON} -c "import sysconfig; print(' '.join(filter(None,sysconfig.get_config_vars('LIBDIR'))))"`
+python_ldlibrary=`${PYTHON} -c "import sysconfig; print(' '.join(filter(None,sysconfig.get_config_vars('LDLIBRARY'))))"`
 
 # If LDLIBRARY exists and has a shlib extension, use it verbatim.
 ldlibrary=`echo "${python_ldlibrary}" | sed -e 's/\.so$//' -e 's/\.dll$//' -e 's/\.dylib$//' -e 's/\.sl$//'`
@@ -10401,11 +10401,11 @@ else
 	# Otherwise, guess the base name of the shlib.
 	# LDVERSION was added in Python 3.2, before that use VERSION,
 	# or failing that, $python_version from _PGAC_CHECK_PYTHON_DIRS.
-	python_ldversion=`${PYTHON} -c "import distutils.sysconfig; print(' '.join(filter(None,distutils.sysconfig.get_config_vars('LDVERSION'))))"`
+	python_ldversion=`${PYTHON} -c "import sysconfig; print(' '.join(filter(None,sysconfig.get_config_vars('LDVERSION'))))"`
 	if test x"${python_ldversion}" != x""; then
 		ldlibrary="python${python_ldversion}"
 	else
-		python_version_var=`${PYTHON} -c "import distutils.sysconfig; print(' '.join(filter(None,distutils.sysconfig.get_config_vars('VERSION'))))"`
+		python_version_var=`${PYTHON} -c "import sysconfig; print(' '.join(filter(None,sysconfig.get_config_vars('VERSION'))))"`
 		if test x"${python_version_var}" != x""; then
 			ldlibrary="python${python_version_var}"
 		else
@@ -10465,7 +10465,7 @@ PL/Python." "$LINENO" 5
 fi
 python_libspec="-L${python_libdir} -l${ldlibrary}"
 
-python_additional_libs=`${PYTHON} -c "import distutils.sysconfig; print(' '.join(filter(None,distutils.sysconfig.get_config_vars('LIBS','LIBC','LIBM','BASEMODLIBS'))))"`
+python_additional_libs=`${PYTHON} -c "import sysconfig; print(' '.join(filter(None,sysconfig.get_config_vars('LIBS','LIBC','LIBM','BASEMODLIBS'))))"`
 
 { $as_echo "$as_me:${as_lineno-$LINENO}: result: ${python_libspec} ${python_additional_libs}" >&5
 $as_echo "${python_libspec} ${python_additional_libs}" >&6; }
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index d38f9bc916..a449719fd3 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -195,8 +195,8 @@ <title>Requirements</title>
       To build the <application>PL/Python</application> server programming
       language, you need a <productname>Python</productname>
       installation with the header files and
-      the <application>distutils</application> module.  The minimum
-      required version is <productname>Python</productname> 2.6.
+      the <application>sysconfig</application> module.  The minimum
+      required version is <productname>Python</productname> 2.7.
       <productname>Python 3</productname> is supported if it's
       version 3.1 or later; but see
       <xref linkend="plpython-python23"/>
-- 
2.34.1

---

From: Tom Lane <[email-masked]>
Date: Thu, 02 Dec 2021 13:22:13 -0500

> distutils.sysconfig being deprecated and scheduled for removal in Python 

Bleah.

> required version up from Python 2.6.

That's surely no problem in HEAD, but as you say, it is an issue for
the older branches.  How difficult would it be to teach configure to
try both ways, or adapt based on its python version check?

> months, we will probably know more about this.

Agreed, this is a moving target so we shouldn't be too concerned
about it yet.

			regards, tom lane

---

From: Peter Eisentraut <[email-masked]>
Date: Fri, 3 Dec 2021 16:52:10 +0100

> try both ways, or adapt based on its python version check?

I think it wouldn't be unreasonable to do that.  I'll look into it.

---

From: Peter Eisentraut <[email-masked]>
Date: Thu, 9 Dec 2021 10:26:01 +0100

> gaur and prariedog use Python 2.6 and would need to be upgraded.

Tom, are you planning to update the Python version on these build farm 
members?  I realize these are very slow machines and this might take 
some time; I'm just wondering if this had registered.

---

From: Tom Lane <[email-masked]>
Date: Thu, 09 Dec 2021 08:31:23 -0500

> some time; I'm just wondering if this had registered.

I can do that when it becomes necessary.  I've got one eye on the meson
conversion discussion, which will kill those two animals altogether;
so it seems possible that updating their Pythons now would just be
wasted effort depending on what lands first.

			regards, tom lane