public inbox for [email protected]
help / color / mirror / Atom feedFrom: Peter Eisentraut <[email protected]>
To: pgsql-hackers <[email protected]>
Subject: Use Python "Limited API" in PL/Python
Date: Mon, 2 Dec 2024 09:51:17 +0100
Message-ID: <[email protected]> (raw)
This patch changes PL/Python to use the Python "limited API". This API
has stronger ABI stability guarantees.[0] This means, you can build
PL/Python against any Python 3.x version and use any other Python 3.x
version at run time.
This is especially useful for binary packages where the operating system
does not come with a fixed suitable version of Python. For example,
Postgres.app (for macOS) would prefer to link against the Python version
supplied by python.org (Python.app). But that has a 3.x version that
changes over time. So instead they bundle a Python version inside
Postgres.app. The Windows installer used to also bundle Python but as of
PG17 you have to get it yourself, but you have to get a very specific
version [1], which is unsatisfactory. This patch fixes that: You can use
any Python version independent of what PL/Python was built against.
(There is a mechanism to say "at least 3.N", but for this patch, we
don't need that, we can stick with the current minimum of 3.2.)
(I have only tested the macOS side of this, not the Windows side. In
fact, the patch currently doesn't build on Windows on CI. I haven't
figured out why.)
For Linux-style packaging, I don't think this would have any benefit for
users right now, since the OS comes with a Python installation and all
the packages are built against that. But it could potentially be helpful
for packagers. For example, on Debian, this could detach the postgresql
packages from python version transitions. But AFAICT, the Python
packaging layout is not prepared for that. (There are only
libpython3.x.so libraries, no libpython3.so that one would have to link
against.)
Finally, I think this patch is part of a path toward making PL/Python
thread-safe. I don't think the patch by itself changes anything, but if
you read through [2], using heap types is part of the things mentioned
there.
[0]: https://docs.python.org/3/c-api/stable.html
[1]:
https://github.com/EnterpriseDB/edb-installers/blob/REL-17/server/resources/installation-notes.html#...
[2]: https://docs.python.org/3/howto/isolating-extensions.html
From 76d4e4b628f9e7913b5c361dbbd4c0ea9d570146 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Mon, 2 Dec 2024 08:53:42 +0100
Subject: [PATCH 1/2] Remove obsolete Python version check
The checked version is already the current minimum supported version
(3.2).
---
src/pl/plpython/plpy_exec.c | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/src/pl/plpython/plpy_exec.c b/src/pl/plpython/plpy_exec.c
index 0e84bb90829..00747bb811b 100644
--- a/src/pl/plpython/plpy_exec.c
+++ b/src/pl/plpython/plpy_exec.c
@@ -1066,13 +1066,7 @@ PLy_procedure_call(PLyProcedure *proc, const char *kargs, PyObject *vargs)
PG_TRY();
{
-#if PY_VERSION_HEX >= 0x03020000
- rv = PyEval_EvalCode(proc->code,
- proc->globals, proc->globals);
-#else
- rv = PyEval_EvalCode((PyCodeObject *) proc->code,
- proc->globals, proc->globals);
-#endif
+ rv = PyEval_EvalCode(proc->code, proc->globals, proc->globals);
/*
* Since plpy will only let you close subtransactions that you
base-commit: 2f696453d2b39fea800d5f7d8e5d3e1a2266de24
--
2.47.1
From a031a128140f3f4c61e6a5468c1a603f547a986b Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Mon, 2 Dec 2024 08:53:42 +0100
Subject: [PATCH 2/2] Use Python "Limited API" in PL/Python
This allows building PL/Python against any Python 3.x version and
using another Python 3.x version at run time.
Implementation details:
- Convert static types to heap types
(https://docs.python.org/3/howto/isolating-extensions.html#heap-types).
- Replace PyRun_String() with component functions.
- Replace PyList_SET_ITEM() with PyList_SetItem().
---
src/pl/plpython/plpy_cursorobject.c | 71 +++++++++++++-------
src/pl/plpython/plpy_planobject.c | 61 +++++++++++------
src/pl/plpython/plpy_procedure.c | 5 +-
src/pl/plpython/plpy_resultobject.c | 98 +++++++++++++++++-----------
src/pl/plpython/plpy_subxactobject.c | 41 +++++++-----
src/pl/plpython/plpy_typeio.c | 6 +-
src/pl/plpython/plpython.h | 2 +
7 files changed, 179 insertions(+), 105 deletions(-)
diff --git a/src/pl/plpython/plpy_cursorobject.c b/src/pl/plpython/plpy_cursorobject.c
index 6108384c9a5..39bcae3f1d9 100644
--- a/src/pl/plpython/plpy_cursorobject.c
+++ b/src/pl/plpython/plpy_cursorobject.c
@@ -20,7 +20,7 @@
#include "utils/memutils.h"
static PyObject *PLy_cursor_query(const char *query);
-static void PLy_cursor_dealloc(PyObject *arg);
+static void PLy_cursor_dealloc(PLyCursorObject *self);
static PyObject *PLy_cursor_iternext(PyObject *self);
static PyObject *PLy_cursor_fetch(PyObject *self, PyObject *args);
static PyObject *PLy_cursor_close(PyObject *self, PyObject *unused);
@@ -33,22 +33,43 @@ static PyMethodDef PLy_cursor_methods[] = {
{NULL, NULL, 0, NULL}
};
-static PyTypeObject PLy_CursorType = {
- PyVarObject_HEAD_INIT(NULL, 0)
- .tp_name = "PLyCursor",
- .tp_basicsize = sizeof(PLyCursorObject),
- .tp_dealloc = PLy_cursor_dealloc,
- .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
- .tp_doc = PLy_cursor_doc,
- .tp_iter = PyObject_SelfIter,
- .tp_iternext = PLy_cursor_iternext,
- .tp_methods = PLy_cursor_methods,
+static PyType_Slot PLyCursor_slots[] =
+{
+ {
+ Py_tp_dealloc, PLy_cursor_dealloc
+ },
+ {
+ Py_tp_doc, (char *) PLy_cursor_doc
+ },
+ {
+ Py_tp_iter, PyObject_SelfIter
+ },
+ {
+ Py_tp_iternext, PLy_cursor_iternext
+ },
+ {
+ Py_tp_methods, PLy_cursor_methods
+ },
+ {
+ 0, NULL
+ }
};
+static PyType_Spec PLyCursor_spec =
+{
+ .name = "PLyCursor",
+ .basicsize = sizeof(PLyCursorObject),
+ .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
+ .slots = PLyCursor_slots,
+};
+
+static PyTypeObject *PLy_CursorType;
+
void
PLy_cursor_init_type(void)
{
- if (PyType_Ready(&PLy_CursorType) < 0)
+ PLy_CursorType = (PyTypeObject *) PyType_FromSpec(&PLyCursor_spec);
+ if (!PLy_CursorType)
elog(ERROR, "could not initialize PLy_CursorType");
}
@@ -80,7 +101,7 @@ PLy_cursor_query(const char *query)
volatile MemoryContext oldcontext;
volatile ResourceOwner oldowner;
- if ((cursor = PyObject_New(PLyCursorObject, &PLy_CursorType)) == NULL)
+ if ((cursor = PyObject_New(PLyCursorObject, PLy_CursorType)) == NULL)
return NULL;
cursor->portalname = NULL;
cursor->closed = false;
@@ -178,7 +199,7 @@ PLy_cursor_plan(PyObject *ob, PyObject *args)
return NULL;
}
- if ((cursor = PyObject_New(PLyCursorObject, &PLy_CursorType)) == NULL)
+ if ((cursor = PyObject_New(PLyCursorObject, PLy_CursorType)) == NULL)
return NULL;
cursor->portalname = NULL;
cursor->closed = false;
@@ -276,30 +297,30 @@ PLy_cursor_plan(PyObject *ob, PyObject *args)
}
static void
-PLy_cursor_dealloc(PyObject *arg)
+PLy_cursor_dealloc(PLyCursorObject *self)
{
- PLyCursorObject *cursor;
+ PyTypeObject *tp = Py_TYPE(self);
Portal portal;
- cursor = (PLyCursorObject *) arg;
-
- if (!cursor->closed)
+ if (!self->closed)
{
- portal = GetPortalByName(cursor->portalname);
+ portal = GetPortalByName(self->portalname);
if (PortalIsValid(portal))
{
UnpinPortal(portal);
SPI_cursor_close(portal);
}
- cursor->closed = true;
+ self->closed = true;
}
- if (cursor->mcxt)
+ if (self->mcxt)
{
- MemoryContextDelete(cursor->mcxt);
- cursor->mcxt = NULL;
+ MemoryContextDelete(self->mcxt);
+ self->mcxt = NULL;
}
- arg->ob_type->tp_free(arg);
+
+ PyObject_Free(self);
+ Py_DECREF(tp);
}
static PyObject *
diff --git a/src/pl/plpython/plpy_planobject.c b/src/pl/plpython/plpy_planobject.c
index bbef889329e..986b7179f14 100644
--- a/src/pl/plpython/plpy_planobject.c
+++ b/src/pl/plpython/plpy_planobject.c
@@ -12,7 +12,7 @@
#include "plpython.h"
#include "utils/memutils.h"
-static void PLy_plan_dealloc(PyObject *arg);
+static void PLy_plan_dealloc(PLyPlanObject *self);
static PyObject *PLy_plan_cursor(PyObject *self, PyObject *args);
static PyObject *PLy_plan_execute(PyObject *self, PyObject *args);
static PyObject *PLy_plan_status(PyObject *self, PyObject *args);
@@ -26,20 +26,37 @@ static PyMethodDef PLy_plan_methods[] = {
{NULL, NULL, 0, NULL}
};
-static PyTypeObject PLy_PlanType = {
- PyVarObject_HEAD_INIT(NULL, 0)
- .tp_name = "PLyPlan",
- .tp_basicsize = sizeof(PLyPlanObject),
- .tp_dealloc = PLy_plan_dealloc,
- .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
- .tp_doc = PLy_plan_doc,
- .tp_methods = PLy_plan_methods,
+static PyType_Slot PLyPlan_slots[] =
+{
+ {
+ Py_tp_dealloc, PLy_plan_dealloc
+ },
+ {
+ Py_tp_doc, (char *) PLy_plan_doc
+ },
+ {
+ Py_tp_methods, PLy_plan_methods
+ },
+ {
+ 0, NULL
+ }
};
+static PyType_Spec PLyPlan_spec =
+{
+ .name = "PLyPlan",
+ .basicsize = sizeof(PLyPlanObject),
+ .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
+ .slots = PLyPlan_slots,
+};
+
+static PyTypeObject *PLy_PlanType;
+
void
PLy_plan_init_type(void)
{
- if (PyType_Ready(&PLy_PlanType) < 0)
+ PLy_PlanType = (PyTypeObject *) PyType_FromSpec(&PLyPlan_spec);
+ if (!PLy_PlanType)
elog(ERROR, "could not initialize PLy_PlanType");
}
@@ -48,7 +65,7 @@ PLy_plan_new(void)
{
PLyPlanObject *ob;
- if ((ob = PyObject_New(PLyPlanObject, &PLy_PlanType)) == NULL)
+ if ((ob = PyObject_New(PLyPlanObject, PLy_PlanType)) == NULL)
return NULL;
ob->plan = NULL;
@@ -64,25 +81,27 @@ PLy_plan_new(void)
bool
is_PLyPlanObject(PyObject *ob)
{
- return ob->ob_type == &PLy_PlanType;
+ return ob->ob_type == PLy_PlanType;
}
static void
-PLy_plan_dealloc(PyObject *arg)
+PLy_plan_dealloc(PLyPlanObject *self)
{
- PLyPlanObject *ob = (PLyPlanObject *) arg;
+ PyTypeObject *tp = Py_TYPE(self);
- if (ob->plan)
+ if (self->plan)
{
- SPI_freeplan(ob->plan);
- ob->plan = NULL;
+ SPI_freeplan(self->plan);
+ self->plan = NULL;
}
- if (ob->mcxt)
+ if (self->mcxt)
{
- MemoryContextDelete(ob->mcxt);
- ob->mcxt = NULL;
+ MemoryContextDelete(self->mcxt);
+ self->mcxt = NULL;
}
- arg->ob_type->tp_free(arg);
+
+ PyObject_Free(self);
+ Py_DECREF(tp);
}
diff --git a/src/pl/plpython/plpy_procedure.c b/src/pl/plpython/plpy_procedure.c
index c35a3b801ab..b494eeb474f 100644
--- a/src/pl/plpython/plpy_procedure.c
+++ b/src/pl/plpython/plpy_procedure.c
@@ -350,6 +350,7 @@ PLy_procedure_compile(PLyProcedure *proc, const char *src)
{
PyObject *crv = NULL;
char *msrc;
+ PyObject *code0;
proc->globals = PyDict_Copy(PLy_interp_globals);
@@ -368,7 +369,9 @@ PLy_procedure_compile(PLyProcedure *proc, const char *src)
msrc = PLy_procedure_munge_source(proc->pyname, src);
/* Save the mangled source for later inclusion in tracebacks */
proc->src = MemoryContextStrdup(proc->mcxt, msrc);
- crv = PyRun_String(msrc, Py_file_input, proc->globals, NULL);
+ code0 = Py_CompileString(msrc, "<string>", Py_file_input);
+ if (code0)
+ crv = PyEval_EvalCode(code0, proc->globals, NULL);
pfree(msrc);
if (crv != NULL)
diff --git a/src/pl/plpython/plpy_resultobject.c b/src/pl/plpython/plpy_resultobject.c
index 95acce65493..f2628205669 100644
--- a/src/pl/plpython/plpy_resultobject.c
+++ b/src/pl/plpython/plpy_resultobject.c
@@ -10,7 +10,7 @@
#include "plpy_resultobject.h"
#include "plpython.h"
-static void PLy_result_dealloc(PyObject *arg);
+static void PLy_result_dealloc(PLyResultObject *self);
static PyObject *PLy_result_colnames(PyObject *self, PyObject *unused);
static PyObject *PLy_result_coltypes(PyObject *self, PyObject *unused);
static PyObject *PLy_result_coltypmods(PyObject *self, PyObject *unused);
@@ -24,17 +24,6 @@ static int PLy_result_ass_subscript(PyObject *arg, PyObject *item, PyObject *val
static char PLy_result_doc[] = "Results of a PostgreSQL query";
-static PySequenceMethods PLy_result_as_sequence = {
- .sq_length = PLy_result_length,
- .sq_item = PLy_result_item,
-};
-
-static PyMappingMethods PLy_result_as_mapping = {
- .mp_length = PLy_result_length,
- .mp_subscript = PLy_result_subscript,
- .mp_ass_subscript = PLy_result_ass_subscript,
-};
-
static PyMethodDef PLy_result_methods[] = {
{"colnames", PLy_result_colnames, METH_NOARGS, NULL},
{"coltypes", PLy_result_coltypes, METH_NOARGS, NULL},
@@ -44,23 +33,55 @@ static PyMethodDef PLy_result_methods[] = {
{NULL, NULL, 0, NULL}
};
-static PyTypeObject PLy_ResultType = {
- PyVarObject_HEAD_INIT(NULL, 0)
- .tp_name = "PLyResult",
- .tp_basicsize = sizeof(PLyResultObject),
- .tp_dealloc = PLy_result_dealloc,
- .tp_as_sequence = &PLy_result_as_sequence,
- .tp_as_mapping = &PLy_result_as_mapping,
- .tp_str = &PLy_result_str,
- .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
- .tp_doc = PLy_result_doc,
- .tp_methods = PLy_result_methods,
+static PyType_Slot PLyResult_slots[] =
+{
+ {
+ Py_tp_dealloc, PLy_result_dealloc
+ },
+ {
+ Py_sq_length, PLy_result_length
+ },
+ {
+ Py_sq_item, PLy_result_item
+ },
+ {
+ Py_mp_length, PLy_result_length
+ },
+ {
+ Py_mp_subscript, PLy_result_subscript
+ },
+ {
+ Py_mp_ass_subscript, PLy_result_ass_subscript
+ },
+ {
+ Py_tp_str, PLy_result_str
+ },
+ {
+ Py_tp_doc, (char *) PLy_result_doc
+ },
+ {
+ Py_tp_methods, PLy_result_methods
+ },
+ {
+ 0, NULL
+ }
};
+static PyType_Spec PLyResult_spec =
+{
+ .name = "PLyResult",
+ .basicsize = sizeof(PLyResultObject),
+ .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
+ .slots = PLyResult_slots,
+};
+
+static PyTypeObject *PLy_ResultType;
+
void
PLy_result_init_type(void)
{
- if (PyType_Ready(&PLy_ResultType) < 0)
+ PLy_ResultType = (PyTypeObject *) PyType_FromSpec(&PLyResult_spec);
+ if (!PLy_ResultType)
elog(ERROR, "could not initialize PLy_ResultType");
}
@@ -69,7 +90,7 @@ PLy_result_new(void)
{
PLyResultObject *ob;
- if ((ob = PyObject_New(PLyResultObject, &PLy_ResultType)) == NULL)
+ if ((ob = PyObject_New(PLyResultObject, PLy_ResultType)) == NULL)
return NULL;
/* ob->tuples = NULL; */
@@ -89,20 +110,21 @@ PLy_result_new(void)
}
static void
-PLy_result_dealloc(PyObject *arg)
+PLy_result_dealloc(PLyResultObject *self)
{
- PLyResultObject *ob = (PLyResultObject *) arg;
+ PyTypeObject *tp = Py_TYPE(self);
- Py_XDECREF(ob->nrows);
- Py_XDECREF(ob->rows);
- Py_XDECREF(ob->status);
- if (ob->tupdesc)
+ Py_XDECREF(self->nrows);
+ Py_XDECREF(self->rows);
+ Py_XDECREF(self->status);
+ if (self->tupdesc)
{
- FreeTupleDesc(ob->tupdesc);
- ob->tupdesc = NULL;
+ FreeTupleDesc(self->tupdesc);
+ self->tupdesc = NULL;
}
- arg->ob_type->tp_free(arg);
+ PyObject_Free(self);
+ Py_DECREF(tp);
}
static PyObject *
@@ -125,7 +147,7 @@ PLy_result_colnames(PyObject *self, PyObject *unused)
{
Form_pg_attribute attr = TupleDescAttr(ob->tupdesc, i);
- PyList_SET_ITEM(list, i, PLyUnicode_FromString(NameStr(attr->attname)));
+ PyList_SetItem(list, i, PLyUnicode_FromString(NameStr(attr->attname)));
}
return list;
@@ -151,7 +173,7 @@ PLy_result_coltypes(PyObject *self, PyObject *unused)
{
Form_pg_attribute attr = TupleDescAttr(ob->tupdesc, i);
- PyList_SET_ITEM(list, i, PyLong_FromLong(attr->atttypid));
+ PyList_SetItem(list, i, PyLong_FromLong(attr->atttypid));
}
return list;
@@ -177,7 +199,7 @@ PLy_result_coltypmods(PyObject *self, PyObject *unused)
{
Form_pg_attribute attr = TupleDescAttr(ob->tupdesc, i);
- PyList_SET_ITEM(list, i, PyLong_FromLong(attr->atttypmod));
+ PyList_SetItem(list, i, PyLong_FromLong(attr->atttypmod));
}
return list;
@@ -227,7 +249,7 @@ PLy_result_str(PyObject *arg)
PLyResultObject *ob = (PLyResultObject *) arg;
return PyUnicode_FromFormat("<%s status=%S nrows=%S rows=%S>",
- Py_TYPE(ob)->tp_name,
+ "PLyResult",
ob->status,
ob->nrows,
ob->rows);
diff --git a/src/pl/plpython/plpy_subxactobject.c b/src/pl/plpython/plpy_subxactobject.c
index 5c92a0e089a..cc7ff3f9df7 100644
--- a/src/pl/plpython/plpy_subxactobject.c
+++ b/src/pl/plpython/plpy_subxactobject.c
@@ -15,7 +15,6 @@
List *explicit_subtransactions = NIL;
-static void PLy_subtransaction_dealloc(PyObject *subxact);
static PyObject *PLy_subtransaction_enter(PyObject *self, PyObject *unused);
static PyObject *PLy_subtransaction_exit(PyObject *self, PyObject *args);
@@ -31,21 +30,35 @@ static PyMethodDef PLy_subtransaction_methods[] = {
{NULL, NULL, 0, NULL}
};
-static PyTypeObject PLy_SubtransactionType = {
- PyVarObject_HEAD_INIT(NULL, 0)
- .tp_name = "PLySubtransaction",
- .tp_basicsize = sizeof(PLySubtransactionObject),
- .tp_dealloc = PLy_subtransaction_dealloc,
- .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
- .tp_doc = PLy_subtransaction_doc,
- .tp_methods = PLy_subtransaction_methods,
+static PyType_Slot PLySubtransaction_slots[] =
+{
+ {
+ Py_tp_doc, (char *) PLy_subtransaction_doc
+ },
+ {
+ Py_tp_methods, PLy_subtransaction_methods
+ },
+ {
+ 0, NULL
+ }
+};
+
+static PyType_Spec PLySubtransaction_spec =
+{
+ .name = "PLySubtransaction",
+ .basicsize = sizeof(PLySubtransactionObject),
+ .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
+ .slots = PLySubtransaction_slots,
};
+static PyTypeObject *PLy_SubtransactionType;
+
void
PLy_subtransaction_init_type(void)
{
- if (PyType_Ready(&PLy_SubtransactionType) < 0)
+ PLy_SubtransactionType = (PyTypeObject *) PyType_FromSpec(&PLySubtransaction_spec);
+ if (!PLy_SubtransactionType)
elog(ERROR, "could not initialize PLy_SubtransactionType");
}
@@ -55,7 +68,7 @@ PLy_subtransaction_new(PyObject *self, PyObject *unused)
{
PLySubtransactionObject *ob;
- ob = PyObject_New(PLySubtransactionObject, &PLy_SubtransactionType);
+ ob = PyObject_New(PLySubtransactionObject, PLy_SubtransactionType);
if (ob == NULL)
return NULL;
@@ -66,12 +79,6 @@ PLy_subtransaction_new(PyObject *self, PyObject *unused)
return (PyObject *) ob;
}
-/* Python requires a dealloc function to be defined */
-static void
-PLy_subtransaction_dealloc(PyObject *subxact)
-{
-}
-
/*
* subxact.__enter__() or subxact.enter()
*
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index db14c5f8dae..27d9b2f1af6 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -723,7 +723,7 @@ PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
sublist = PLyList_FromArray_recurse(elm, dims, ndim, dim + 1,
dataptr_p, bitmap_p, bitmask_p);
- PyList_SET_ITEM(list, i, sublist);
+ PyList_SetItem(list, i, sublist);
}
}
else
@@ -742,14 +742,14 @@ PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
if (bitmap && (*bitmap & bitmask) == 0)
{
Py_INCREF(Py_None);
- PyList_SET_ITEM(list, i, Py_None);
+ PyList_SetItem(list, i, Py_None);
}
else
{
Datum itemvalue;
itemvalue = fetch_att(dataptr, elm->typbyval, elm->typlen);
- PyList_SET_ITEM(list, i, elm->func(elm, itemvalue));
+ PyList_SetItem(list, i, elm->func(elm, itemvalue));
dataptr = att_addlength_pointer(dataptr, elm->typlen, dataptr);
dataptr = (char *) att_align_nominal(dataptr, elm->typalign);
}
diff --git a/src/pl/plpython/plpython.h b/src/pl/plpython/plpython.h
index d929afefc33..fb95b23730f 100644
--- a/src/pl/plpython/plpython.h
+++ b/src/pl/plpython/plpython.h
@@ -19,6 +19,8 @@
#error Python.h must be included via plpython.h
#endif
+#define Py_LIMITED_API 0x03020000
+
/*
* Pull in Python headers via a wrapper header, to control the scope of
* the system_header pragma therein.
--
2.47.1
Attachments:
[text/plain] 0001-Remove-obsolete-Python-version-check.patch (1.1K, ../[email protected]/2-0001-Remove-obsolete-Python-version-check.patch)
download | inline diff:
From 76d4e4b628f9e7913b5c361dbbd4c0ea9d570146 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Mon, 2 Dec 2024 08:53:42 +0100
Subject: [PATCH 1/2] Remove obsolete Python version check
The checked version is already the current minimum supported version
(3.2).
---
src/pl/plpython/plpy_exec.c | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/src/pl/plpython/plpy_exec.c b/src/pl/plpython/plpy_exec.c
index 0e84bb90829..00747bb811b 100644
--- a/src/pl/plpython/plpy_exec.c
+++ b/src/pl/plpython/plpy_exec.c
@@ -1066,13 +1066,7 @@ PLy_procedure_call(PLyProcedure *proc, const char *kargs, PyObject *vargs)
PG_TRY();
{
-#if PY_VERSION_HEX >= 0x03020000
- rv = PyEval_EvalCode(proc->code,
- proc->globals, proc->globals);
-#else
- rv = PyEval_EvalCode((PyCodeObject *) proc->code,
- proc->globals, proc->globals);
-#endif
+ rv = PyEval_EvalCode(proc->code, proc->globals, proc->globals);
/*
* Since plpy will only let you close subtransactions that you
base-commit: 2f696453d2b39fea800d5f7d8e5d3e1a2266de24
--
2.47.1
[text/plain] 0002-Use-Python-Limited-API-in-PL-Python.patch (15.9K, ../[email protected]/3-0002-Use-Python-Limited-API-in-PL-Python.patch)
download | inline diff:
From a031a128140f3f4c61e6a5468c1a603f547a986b Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Mon, 2 Dec 2024 08:53:42 +0100
Subject: [PATCH 2/2] Use Python "Limited API" in PL/Python
This allows building PL/Python against any Python 3.x version and
using another Python 3.x version at run time.
Implementation details:
- Convert static types to heap types
(https://docs.python.org/3/howto/isolating-extensions.html#heap-types).
- Replace PyRun_String() with component functions.
- Replace PyList_SET_ITEM() with PyList_SetItem().
---
src/pl/plpython/plpy_cursorobject.c | 71 +++++++++++++-------
src/pl/plpython/plpy_planobject.c | 61 +++++++++++------
src/pl/plpython/plpy_procedure.c | 5 +-
src/pl/plpython/plpy_resultobject.c | 98 +++++++++++++++++-----------
src/pl/plpython/plpy_subxactobject.c | 41 +++++++-----
src/pl/plpython/plpy_typeio.c | 6 +-
src/pl/plpython/plpython.h | 2 +
7 files changed, 179 insertions(+), 105 deletions(-)
diff --git a/src/pl/plpython/plpy_cursorobject.c b/src/pl/plpython/plpy_cursorobject.c
index 6108384c9a5..39bcae3f1d9 100644
--- a/src/pl/plpython/plpy_cursorobject.c
+++ b/src/pl/plpython/plpy_cursorobject.c
@@ -20,7 +20,7 @@
#include "utils/memutils.h"
static PyObject *PLy_cursor_query(const char *query);
-static void PLy_cursor_dealloc(PyObject *arg);
+static void PLy_cursor_dealloc(PLyCursorObject *self);
static PyObject *PLy_cursor_iternext(PyObject *self);
static PyObject *PLy_cursor_fetch(PyObject *self, PyObject *args);
static PyObject *PLy_cursor_close(PyObject *self, PyObject *unused);
@@ -33,22 +33,43 @@ static PyMethodDef PLy_cursor_methods[] = {
{NULL, NULL, 0, NULL}
};
-static PyTypeObject PLy_CursorType = {
- PyVarObject_HEAD_INIT(NULL, 0)
- .tp_name = "PLyCursor",
- .tp_basicsize = sizeof(PLyCursorObject),
- .tp_dealloc = PLy_cursor_dealloc,
- .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
- .tp_doc = PLy_cursor_doc,
- .tp_iter = PyObject_SelfIter,
- .tp_iternext = PLy_cursor_iternext,
- .tp_methods = PLy_cursor_methods,
+static PyType_Slot PLyCursor_slots[] =
+{
+ {
+ Py_tp_dealloc, PLy_cursor_dealloc
+ },
+ {
+ Py_tp_doc, (char *) PLy_cursor_doc
+ },
+ {
+ Py_tp_iter, PyObject_SelfIter
+ },
+ {
+ Py_tp_iternext, PLy_cursor_iternext
+ },
+ {
+ Py_tp_methods, PLy_cursor_methods
+ },
+ {
+ 0, NULL
+ }
};
+static PyType_Spec PLyCursor_spec =
+{
+ .name = "PLyCursor",
+ .basicsize = sizeof(PLyCursorObject),
+ .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
+ .slots = PLyCursor_slots,
+};
+
+static PyTypeObject *PLy_CursorType;
+
void
PLy_cursor_init_type(void)
{
- if (PyType_Ready(&PLy_CursorType) < 0)
+ PLy_CursorType = (PyTypeObject *) PyType_FromSpec(&PLyCursor_spec);
+ if (!PLy_CursorType)
elog(ERROR, "could not initialize PLy_CursorType");
}
@@ -80,7 +101,7 @@ PLy_cursor_query(const char *query)
volatile MemoryContext oldcontext;
volatile ResourceOwner oldowner;
- if ((cursor = PyObject_New(PLyCursorObject, &PLy_CursorType)) == NULL)
+ if ((cursor = PyObject_New(PLyCursorObject, PLy_CursorType)) == NULL)
return NULL;
cursor->portalname = NULL;
cursor->closed = false;
@@ -178,7 +199,7 @@ PLy_cursor_plan(PyObject *ob, PyObject *args)
return NULL;
}
- if ((cursor = PyObject_New(PLyCursorObject, &PLy_CursorType)) == NULL)
+ if ((cursor = PyObject_New(PLyCursorObject, PLy_CursorType)) == NULL)
return NULL;
cursor->portalname = NULL;
cursor->closed = false;
@@ -276,30 +297,30 @@ PLy_cursor_plan(PyObject *ob, PyObject *args)
}
static void
-PLy_cursor_dealloc(PyObject *arg)
+PLy_cursor_dealloc(PLyCursorObject *self)
{
- PLyCursorObject *cursor;
+ PyTypeObject *tp = Py_TYPE(self);
Portal portal;
- cursor = (PLyCursorObject *) arg;
-
- if (!cursor->closed)
+ if (!self->closed)
{
- portal = GetPortalByName(cursor->portalname);
+ portal = GetPortalByName(self->portalname);
if (PortalIsValid(portal))
{
UnpinPortal(portal);
SPI_cursor_close(portal);
}
- cursor->closed = true;
+ self->closed = true;
}
- if (cursor->mcxt)
+ if (self->mcxt)
{
- MemoryContextDelete(cursor->mcxt);
- cursor->mcxt = NULL;
+ MemoryContextDelete(self->mcxt);
+ self->mcxt = NULL;
}
- arg->ob_type->tp_free(arg);
+
+ PyObject_Free(self);
+ Py_DECREF(tp);
}
static PyObject *
diff --git a/src/pl/plpython/plpy_planobject.c b/src/pl/plpython/plpy_planobject.c
index bbef889329e..986b7179f14 100644
--- a/src/pl/plpython/plpy_planobject.c
+++ b/src/pl/plpython/plpy_planobject.c
@@ -12,7 +12,7 @@
#include "plpython.h"
#include "utils/memutils.h"
-static void PLy_plan_dealloc(PyObject *arg);
+static void PLy_plan_dealloc(PLyPlanObject *self);
static PyObject *PLy_plan_cursor(PyObject *self, PyObject *args);
static PyObject *PLy_plan_execute(PyObject *self, PyObject *args);
static PyObject *PLy_plan_status(PyObject *self, PyObject *args);
@@ -26,20 +26,37 @@ static PyMethodDef PLy_plan_methods[] = {
{NULL, NULL, 0, NULL}
};
-static PyTypeObject PLy_PlanType = {
- PyVarObject_HEAD_INIT(NULL, 0)
- .tp_name = "PLyPlan",
- .tp_basicsize = sizeof(PLyPlanObject),
- .tp_dealloc = PLy_plan_dealloc,
- .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
- .tp_doc = PLy_plan_doc,
- .tp_methods = PLy_plan_methods,
+static PyType_Slot PLyPlan_slots[] =
+{
+ {
+ Py_tp_dealloc, PLy_plan_dealloc
+ },
+ {
+ Py_tp_doc, (char *) PLy_plan_doc
+ },
+ {
+ Py_tp_methods, PLy_plan_methods
+ },
+ {
+ 0, NULL
+ }
};
+static PyType_Spec PLyPlan_spec =
+{
+ .name = "PLyPlan",
+ .basicsize = sizeof(PLyPlanObject),
+ .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
+ .slots = PLyPlan_slots,
+};
+
+static PyTypeObject *PLy_PlanType;
+
void
PLy_plan_init_type(void)
{
- if (PyType_Ready(&PLy_PlanType) < 0)
+ PLy_PlanType = (PyTypeObject *) PyType_FromSpec(&PLyPlan_spec);
+ if (!PLy_PlanType)
elog(ERROR, "could not initialize PLy_PlanType");
}
@@ -48,7 +65,7 @@ PLy_plan_new(void)
{
PLyPlanObject *ob;
- if ((ob = PyObject_New(PLyPlanObject, &PLy_PlanType)) == NULL)
+ if ((ob = PyObject_New(PLyPlanObject, PLy_PlanType)) == NULL)
return NULL;
ob->plan = NULL;
@@ -64,25 +81,27 @@ PLy_plan_new(void)
bool
is_PLyPlanObject(PyObject *ob)
{
- return ob->ob_type == &PLy_PlanType;
+ return ob->ob_type == PLy_PlanType;
}
static void
-PLy_plan_dealloc(PyObject *arg)
+PLy_plan_dealloc(PLyPlanObject *self)
{
- PLyPlanObject *ob = (PLyPlanObject *) arg;
+ PyTypeObject *tp = Py_TYPE(self);
- if (ob->plan)
+ if (self->plan)
{
- SPI_freeplan(ob->plan);
- ob->plan = NULL;
+ SPI_freeplan(self->plan);
+ self->plan = NULL;
}
- if (ob->mcxt)
+ if (self->mcxt)
{
- MemoryContextDelete(ob->mcxt);
- ob->mcxt = NULL;
+ MemoryContextDelete(self->mcxt);
+ self->mcxt = NULL;
}
- arg->ob_type->tp_free(arg);
+
+ PyObject_Free(self);
+ Py_DECREF(tp);
}
diff --git a/src/pl/plpython/plpy_procedure.c b/src/pl/plpython/plpy_procedure.c
index c35a3b801ab..b494eeb474f 100644
--- a/src/pl/plpython/plpy_procedure.c
+++ b/src/pl/plpython/plpy_procedure.c
@@ -350,6 +350,7 @@ PLy_procedure_compile(PLyProcedure *proc, const char *src)
{
PyObject *crv = NULL;
char *msrc;
+ PyObject *code0;
proc->globals = PyDict_Copy(PLy_interp_globals);
@@ -368,7 +369,9 @@ PLy_procedure_compile(PLyProcedure *proc, const char *src)
msrc = PLy_procedure_munge_source(proc->pyname, src);
/* Save the mangled source for later inclusion in tracebacks */
proc->src = MemoryContextStrdup(proc->mcxt, msrc);
- crv = PyRun_String(msrc, Py_file_input, proc->globals, NULL);
+ code0 = Py_CompileString(msrc, "<string>", Py_file_input);
+ if (code0)
+ crv = PyEval_EvalCode(code0, proc->globals, NULL);
pfree(msrc);
if (crv != NULL)
diff --git a/src/pl/plpython/plpy_resultobject.c b/src/pl/plpython/plpy_resultobject.c
index 95acce65493..f2628205669 100644
--- a/src/pl/plpython/plpy_resultobject.c
+++ b/src/pl/plpython/plpy_resultobject.c
@@ -10,7 +10,7 @@
#include "plpy_resultobject.h"
#include "plpython.h"
-static void PLy_result_dealloc(PyObject *arg);
+static void PLy_result_dealloc(PLyResultObject *self);
static PyObject *PLy_result_colnames(PyObject *self, PyObject *unused);
static PyObject *PLy_result_coltypes(PyObject *self, PyObject *unused);
static PyObject *PLy_result_coltypmods(PyObject *self, PyObject *unused);
@@ -24,17 +24,6 @@ static int PLy_result_ass_subscript(PyObject *arg, PyObject *item, PyObject *val
static char PLy_result_doc[] = "Results of a PostgreSQL query";
-static PySequenceMethods PLy_result_as_sequence = {
- .sq_length = PLy_result_length,
- .sq_item = PLy_result_item,
-};
-
-static PyMappingMethods PLy_result_as_mapping = {
- .mp_length = PLy_result_length,
- .mp_subscript = PLy_result_subscript,
- .mp_ass_subscript = PLy_result_ass_subscript,
-};
-
static PyMethodDef PLy_result_methods[] = {
{"colnames", PLy_result_colnames, METH_NOARGS, NULL},
{"coltypes", PLy_result_coltypes, METH_NOARGS, NULL},
@@ -44,23 +33,55 @@ static PyMethodDef PLy_result_methods[] = {
{NULL, NULL, 0, NULL}
};
-static PyTypeObject PLy_ResultType = {
- PyVarObject_HEAD_INIT(NULL, 0)
- .tp_name = "PLyResult",
- .tp_basicsize = sizeof(PLyResultObject),
- .tp_dealloc = PLy_result_dealloc,
- .tp_as_sequence = &PLy_result_as_sequence,
- .tp_as_mapping = &PLy_result_as_mapping,
- .tp_str = &PLy_result_str,
- .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
- .tp_doc = PLy_result_doc,
- .tp_methods = PLy_result_methods,
+static PyType_Slot PLyResult_slots[] =
+{
+ {
+ Py_tp_dealloc, PLy_result_dealloc
+ },
+ {
+ Py_sq_length, PLy_result_length
+ },
+ {
+ Py_sq_item, PLy_result_item
+ },
+ {
+ Py_mp_length, PLy_result_length
+ },
+ {
+ Py_mp_subscript, PLy_result_subscript
+ },
+ {
+ Py_mp_ass_subscript, PLy_result_ass_subscript
+ },
+ {
+ Py_tp_str, PLy_result_str
+ },
+ {
+ Py_tp_doc, (char *) PLy_result_doc
+ },
+ {
+ Py_tp_methods, PLy_result_methods
+ },
+ {
+ 0, NULL
+ }
};
+static PyType_Spec PLyResult_spec =
+{
+ .name = "PLyResult",
+ .basicsize = sizeof(PLyResultObject),
+ .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
+ .slots = PLyResult_slots,
+};
+
+static PyTypeObject *PLy_ResultType;
+
void
PLy_result_init_type(void)
{
- if (PyType_Ready(&PLy_ResultType) < 0)
+ PLy_ResultType = (PyTypeObject *) PyType_FromSpec(&PLyResult_spec);
+ if (!PLy_ResultType)
elog(ERROR, "could not initialize PLy_ResultType");
}
@@ -69,7 +90,7 @@ PLy_result_new(void)
{
PLyResultObject *ob;
- if ((ob = PyObject_New(PLyResultObject, &PLy_ResultType)) == NULL)
+ if ((ob = PyObject_New(PLyResultObject, PLy_ResultType)) == NULL)
return NULL;
/* ob->tuples = NULL; */
@@ -89,20 +110,21 @@ PLy_result_new(void)
}
static void
-PLy_result_dealloc(PyObject *arg)
+PLy_result_dealloc(PLyResultObject *self)
{
- PLyResultObject *ob = (PLyResultObject *) arg;
+ PyTypeObject *tp = Py_TYPE(self);
- Py_XDECREF(ob->nrows);
- Py_XDECREF(ob->rows);
- Py_XDECREF(ob->status);
- if (ob->tupdesc)
+ Py_XDECREF(self->nrows);
+ Py_XDECREF(self->rows);
+ Py_XDECREF(self->status);
+ if (self->tupdesc)
{
- FreeTupleDesc(ob->tupdesc);
- ob->tupdesc = NULL;
+ FreeTupleDesc(self->tupdesc);
+ self->tupdesc = NULL;
}
- arg->ob_type->tp_free(arg);
+ PyObject_Free(self);
+ Py_DECREF(tp);
}
static PyObject *
@@ -125,7 +147,7 @@ PLy_result_colnames(PyObject *self, PyObject *unused)
{
Form_pg_attribute attr = TupleDescAttr(ob->tupdesc, i);
- PyList_SET_ITEM(list, i, PLyUnicode_FromString(NameStr(attr->attname)));
+ PyList_SetItem(list, i, PLyUnicode_FromString(NameStr(attr->attname)));
}
return list;
@@ -151,7 +173,7 @@ PLy_result_coltypes(PyObject *self, PyObject *unused)
{
Form_pg_attribute attr = TupleDescAttr(ob->tupdesc, i);
- PyList_SET_ITEM(list, i, PyLong_FromLong(attr->atttypid));
+ PyList_SetItem(list, i, PyLong_FromLong(attr->atttypid));
}
return list;
@@ -177,7 +199,7 @@ PLy_result_coltypmods(PyObject *self, PyObject *unused)
{
Form_pg_attribute attr = TupleDescAttr(ob->tupdesc, i);
- PyList_SET_ITEM(list, i, PyLong_FromLong(attr->atttypmod));
+ PyList_SetItem(list, i, PyLong_FromLong(attr->atttypmod));
}
return list;
@@ -227,7 +249,7 @@ PLy_result_str(PyObject *arg)
PLyResultObject *ob = (PLyResultObject *) arg;
return PyUnicode_FromFormat("<%s status=%S nrows=%S rows=%S>",
- Py_TYPE(ob)->tp_name,
+ "PLyResult",
ob->status,
ob->nrows,
ob->rows);
diff --git a/src/pl/plpython/plpy_subxactobject.c b/src/pl/plpython/plpy_subxactobject.c
index 5c92a0e089a..cc7ff3f9df7 100644
--- a/src/pl/plpython/plpy_subxactobject.c
+++ b/src/pl/plpython/plpy_subxactobject.c
@@ -15,7 +15,6 @@
List *explicit_subtransactions = NIL;
-static void PLy_subtransaction_dealloc(PyObject *subxact);
static PyObject *PLy_subtransaction_enter(PyObject *self, PyObject *unused);
static PyObject *PLy_subtransaction_exit(PyObject *self, PyObject *args);
@@ -31,21 +30,35 @@ static PyMethodDef PLy_subtransaction_methods[] = {
{NULL, NULL, 0, NULL}
};
-static PyTypeObject PLy_SubtransactionType = {
- PyVarObject_HEAD_INIT(NULL, 0)
- .tp_name = "PLySubtransaction",
- .tp_basicsize = sizeof(PLySubtransactionObject),
- .tp_dealloc = PLy_subtransaction_dealloc,
- .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
- .tp_doc = PLy_subtransaction_doc,
- .tp_methods = PLy_subtransaction_methods,
+static PyType_Slot PLySubtransaction_slots[] =
+{
+ {
+ Py_tp_doc, (char *) PLy_subtransaction_doc
+ },
+ {
+ Py_tp_methods, PLy_subtransaction_methods
+ },
+ {
+ 0, NULL
+ }
+};
+
+static PyType_Spec PLySubtransaction_spec =
+{
+ .name = "PLySubtransaction",
+ .basicsize = sizeof(PLySubtransactionObject),
+ .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
+ .slots = PLySubtransaction_slots,
};
+static PyTypeObject *PLy_SubtransactionType;
+
void
PLy_subtransaction_init_type(void)
{
- if (PyType_Ready(&PLy_SubtransactionType) < 0)
+ PLy_SubtransactionType = (PyTypeObject *) PyType_FromSpec(&PLySubtransaction_spec);
+ if (!PLy_SubtransactionType)
elog(ERROR, "could not initialize PLy_SubtransactionType");
}
@@ -55,7 +68,7 @@ PLy_subtransaction_new(PyObject *self, PyObject *unused)
{
PLySubtransactionObject *ob;
- ob = PyObject_New(PLySubtransactionObject, &PLy_SubtransactionType);
+ ob = PyObject_New(PLySubtransactionObject, PLy_SubtransactionType);
if (ob == NULL)
return NULL;
@@ -66,12 +79,6 @@ PLy_subtransaction_new(PyObject *self, PyObject *unused)
return (PyObject *) ob;
}
-/* Python requires a dealloc function to be defined */
-static void
-PLy_subtransaction_dealloc(PyObject *subxact)
-{
-}
-
/*
* subxact.__enter__() or subxact.enter()
*
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index db14c5f8dae..27d9b2f1af6 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -723,7 +723,7 @@ PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
sublist = PLyList_FromArray_recurse(elm, dims, ndim, dim + 1,
dataptr_p, bitmap_p, bitmask_p);
- PyList_SET_ITEM(list, i, sublist);
+ PyList_SetItem(list, i, sublist);
}
}
else
@@ -742,14 +742,14 @@ PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
if (bitmap && (*bitmap & bitmask) == 0)
{
Py_INCREF(Py_None);
- PyList_SET_ITEM(list, i, Py_None);
+ PyList_SetItem(list, i, Py_None);
}
else
{
Datum itemvalue;
itemvalue = fetch_att(dataptr, elm->typbyval, elm->typlen);
- PyList_SET_ITEM(list, i, elm->func(elm, itemvalue));
+ PyList_SetItem(list, i, elm->func(elm, itemvalue));
dataptr = att_addlength_pointer(dataptr, elm->typlen, dataptr);
dataptr = (char *) att_align_nominal(dataptr, elm->typalign);
}
diff --git a/src/pl/plpython/plpython.h b/src/pl/plpython/plpython.h
index d929afefc33..fb95b23730f 100644
--- a/src/pl/plpython/plpython.h
+++ b/src/pl/plpython/plpython.h
@@ -19,6 +19,8 @@
#error Python.h must be included via plpython.h
#endif
+#define Py_LIMITED_API 0x03020000
+
/*
* Pull in Python headers via a wrapper header, to control the scope of
* the system_header pragma therein.
--
2.47.1
view thread (2+ messages)
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected]
Subject: Re: Use Python "Limited API" in PL/Python
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox