pg.ddx.io pgsql-bugs@postgresql.org mailing list archive
help / color / mirror / Atom feedBUG #19899: PL/Python backend crashes (SIGSEGV) when converting a column value to a Python object fails
7+ messages / 5 participants
[nested] [flat]
* BUG #19899: PL/Python backend crashes (SIGSEGV) when converting a column value to a Python object fails
@ 2026-11-12 23:50 PG Bug reporting form <noreply@postgresql.org>
0 siblings, 1 reply; 7+ messages in thread
From: PG Bug reporting form @ 2026-11-12 23:50 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: marcos.castedo@anachronics.com
The following bug has been logged on the website:
Bug reference: 19899
Logged by: Marcos Castedo
Email address: marcos.castedo@anachronics.com
PostgreSQL version: 19rc2
Operating system: Debian GNU/Linux (x86_64)
Description:
When the conversion of a column value to a Python object fails inside
Python, the backend
terminates with signal 11 and the postmaster restarts all sessions. The
expected result is an ERROR
in the calling session.
The steps below read a row with plpy.execute and make the conversion of one
column fail on demand
by starting the server with a lower memory limit; the limit is only a way to
trigger the failure.
The conversion is done by PLyDict_FromTuple, which also builds row arguments
and
TD["new"]/TD["old"]; only the plpy.execute case was run.
Steps to reproduce:
1. In a new cluster:
CREATE EXTENSION plpython3u;
CREATE TABLE big (id int, b bytea);
INSERT INTO big SELECT 1, convert_to(repeat('a', 300000000), 'UTF8');
2. Restart the server under an address-space limit that leaves room for the
server to detoast the
value but not for Python to copy it. Here a backend with Python loaded uses
about 188 MB, and the
limit was 188 MB + 450 MB:
ulimit -v 653312
pg_ctl -D <datadir> -l <logfile> start
The numbers depend on the build and platform; the limit has to leave room
for the detoasted value
but not for a second copy of it.
3. Then:
CREATE FUNCTION g() RETURNS int AS $$
r = plpy.execute("SELECT b FROM big")
return len(r[0]["b"])
$$ LANGUAGE plpython3u;
SELECT g();
server closed the connection unexpectedly
Server log (19rc2):
LOG: client backend (PID 548414) was terminated by signal 11: Segmentation
fault
LOG: terminating any other active server processes
LOG: all server processes terminated; reinitializing
17.12 shows the same with the same limit:
LOG: server process (PID 546767) was terminated by signal 11: Segmentation
fault
On 19rc2, with a higher limit (188 MB + 1200 MB) SELECT g() returns
300000000, and with a lower
one (188 MB + 100 MB) the server fails first, while detoasting, with an
ordinary out-of-memory
ERROR and the session survives.
Backtrace from the core of the crashing backend (19rc2, same run as the log
above):
#0 Py_INCREF (op=0x0) at ./Include/refcount.h:282
#1 _Py_NewRef (obj=0x0) at ./Include/refcount.h:543
#2 PyDict_SetItem (op=..., key=..., value=0x0) at Objects/dictobject.c:2727
#3 PyDict_SetItemString (v=..., key="b", item=0x0) at
Objects/dictobject.c:5094
#4 PLyDict_FromTuple (...) at src/pl/plpython/plpy_typeio.c
#5 PLy_input_from_tuple (...) at src/pl/plpython/plpy_typeio.c
#6 PLy_spi_execute_fetch_result (tuptable=..., rows=1, status=5) at
src/pl/plpython/plpy_spi.c:412
#7 PLy_spi_execute_query (query="SELECT b FROM big", limit=0) at
src/pl/plpython/plpy_spi.c:324
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #19899: PL/Python backend crashes (SIGSEGV) when converting a column value to a Python object fails
@ 2026-11-16 17:53 Tom Lane <tgl@sss.pgh.pa.us>
parent: PG Bug reporting form <noreply@postgresql.org>
0 siblings, 2 replies; 7+ messages in thread
From: Tom Lane @ 2026-11-16 17:53 UTC (permalink / raw)
To: marcos.castedo@anachronics.com; +Cc: pgsql-bugs@lists.postgresql.org
PG Bug reporting form <noreply@postgresql.org> writes:
> When the conversion of a column value to a Python object fails inside
> Python, the backend
> terminates with signal 11 and the postmaster restarts all sessions. The
> expected result is an ERROR
> in the calling session.
The immediate cause in this example seems to be
(1) PyBytes_FromStringAndSize returns NULL after a malloc failure
(2) We blindly pass that NULL to PyDict_SetItemString
(3) PyDict_SetItemString doesn't cope with that and dumps core.
It looks to me like plpy_typeio.c is just full of bugs like this,
because it doesn't have any coherent strategy for detecting or
dealing with failures. There are a few places that check for
NULL results and throw PLy_elog(ERROR), but not many. I think
we'd first want to decide whether to check immediately after
each call to a Python function, or try to centralize it a bit.
I wonder now about the rest of plpython, too.
I don't care enough about PL/Python to work on this myself,
but perhaps someone else will pick it up.
regards, tom lane
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #19899: PL/Python backend crashes (SIGSEGV) when converting a column value to a Python object fails
@ 2026-11-16 19:10 Andrey Rachitskiy <pl0h0yp1@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 7+ messages in thread
From: Andrey Rachitskiy @ 2026-11-16 19:10 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: marcos.castedo@anachronics.com; pgsql-bugs@lists.postgresql.org
пн, 16 нояб. 2026 г. в 22:53, Tom Lane <tgl@sss.pgh.pa.us>:
> I think
> we'd first want to decide whether to check immediately after
> each call to a Python function, or try to centralize it a bit.
>
>
Hi Tom!
I centralized the problematic spots in plpy_typeio - a small step toward
order.
I'll look at the other places in plpython a bit later.
Thoughts?
--
Regards,
Rachitskiy Andrey
Attachments:
0001-Fix-unchecked-NULL-returns-in-PL-Python-type-convers.patch (text/x-patch, 8.1K) download
From 8e46e9e036166a687c479a0e65756e960c5d7be0 Mon Sep 17 00:00:00 2001
From: Andrey Rachitskiy <pl0h0yp1@gmail.com>
Date: Mon, 16 Nov 2026 23:57:23 +0500
Subject: [PATCH] Fix unchecked NULL returns in PL/Python type conversion
PLyDict_FromTuple() passed a NULL from att->func() to
PyDict_SetItemString(), crashing the backend.
PLyList_FromArray_recurse() passed the same NULL to PyList_SetItem().
PLyObject_AsString() passed a NULL from PyObject_Repr() or PyObject_Str()
to PLyUnicode_Bytes().
Dict and list insertion goes through PLy_dict_setitem_string() and
PLy_list_setitem(). Other NULL object pointers go through PLy_check(),
which reports the Python error with PLy_elog().
A missing mapping key or object attribute is still reported as before.
Any other exception from those lookups is reported with PLy_elog().
BUG #19899
Author: Andrey Rachitskiy <pl0h0yp1@gmail.com>
Reported-by: Marcos Castedo <marcos.castedo@anachronics.com>
Discussion: https://www.postgresql.org/message-id/19899-3e21845d988f0c0f@postgresql.org
---
src/pl/plpython/plpy_typeio.c | 50 +++++++++++++++++------------
src/pl/plpython/plpy_util.c | 60 +++++++++++++++++++++++++++++++++++
src/pl/plpython/plpy_util.h | 6 ++++
3 files changed, 96 insertions(+), 20 deletions(-)
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index e499ed8cbfd..7975836766c 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -104,7 +104,7 @@ PLy_input_convert(PLyDatumToOb *arg, Datum val)
MemoryContextSwitchTo(oldcontext);
- return result;
+ return PLy_check(result);
}
/*
@@ -149,7 +149,7 @@ PLy_input_from_tuple(PLyDatumToOb *arg, HeapTuple tuple, TupleDesc desc, bool in
MemoryContextSwitchTo(oldcontext);
- return dict;
+ return PLy_check(dict);
}
/*
@@ -723,7 +723,7 @@ PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
sublist = PLyList_FromArray_recurse(elm, dims, ndim, dim + 1,
dataptr_p, bitmap_p, bitmask_p);
- PyList_SetItem(list, i, sublist);
+ PLy_list_setitem(list, i, sublist);
}
}
else
@@ -741,16 +741,13 @@ PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
{
/* checking for NULL */
if (bitmap && (*bitmap & bitmask) == 0)
- {
- Py_INCREF(Py_None);
- PyList_SetItem(list, i, Py_None);
- }
+ PLy_list_setitem(list, i, Py_None);
else
{
Datum itemvalue;
itemvalue = fetch_att(dataptr, elm->typbyval, elm->typlen);
- PyList_SetItem(list, i, elm->func(elm, itemvalue));
+ PLy_list_setitem(list, i, elm->func(elm, itemvalue));
dataptr = att_addlength_pointer(dataptr, elm->typlen, dataptr);
dataptr = (char *) att_nominal_alignby(dataptr, typalignby);
}
@@ -835,7 +832,6 @@ PLyDict_FromTuple(PLyDatumToOb *arg, HeapTuple tuple, TupleDesc desc, bool inclu
char *key;
Datum vattr;
bool is_null;
- PyObject *value;
if (attr->attisdropped)
continue;
@@ -854,13 +850,9 @@ PLyDict_FromTuple(PLyDatumToOb *arg, HeapTuple tuple, TupleDesc desc, bool inclu
vattr = heap_getattr(tuple, (i + 1), desc, &is_null);
if (is_null)
- PyDict_SetItemString(dict, key, Py_None);
+ PLy_dict_setitem_string(dict, key, Py_None);
else
- {
- value = att->func(att, vattr);
- PyDict_SetItemString(dict, key, value);
- Py_DECREF(value);
- }
+ PLy_dict_setitem_string(dict, key, att->func(att, vattr));
}
}
PG_CATCH();
@@ -1037,22 +1029,25 @@ PLyObject_AsString(PyObject *plrv)
else if (PyFloat_Check(plrv))
{
/* use repr() for floats, str() is lossy */
- PyObject *s = PyObject_Repr(plrv);
+ PyObject *s = PLy_check(PyObject_Repr(plrv));
plrv_bo = PLyUnicode_Bytes(s);
- Py_XDECREF(s);
+ Py_DECREF(s);
}
else
{
- PyObject *s = PyObject_Str(plrv);
+ PyObject *s = PLy_check(PyObject_Str(plrv));
plrv_bo = PLyUnicode_Bytes(s);
- Py_XDECREF(s);
+ Py_DECREF(s);
}
if (!plrv_bo)
PLy_elog(ERROR, "could not create string representation of Python object");
- plrv_sc = pstrdup(PyBytes_AsString(plrv_bo));
+ plrv_sc = PyBytes_AsString(plrv_bo);
+ if (plrv_sc == NULL)
+ PLy_elog(ERROR, "could not create string representation of Python object");
+ plrv_sc = pstrdup(plrv_sc);
plen = PyBytes_Size(plrv_bo);
slen = strlen(plrv_sc);
@@ -1381,11 +1376,20 @@ PLyMapping_ToComposite(PLyObToDatum *arg, TupleDesc desc, PyObject *mapping)
{
value = PyMapping_GetItemString(mapping, key);
if (!value)
+ {
+ /*
+ * A missing key sets KeyError. Another exception, such as
+ * MemoryError, is a real failure.
+ */
+ if (!PyErr_ExceptionMatches(PyExc_KeyError))
+ PLy_elog(ERROR, NULL);
+ PyErr_Clear();
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_COLUMN),
errmsg("key \"%s\" not found in mapping", key),
errhint("To return null in a column, "
"add the value None to the mapping with the key named after the column.")));
+ }
values[i] = att->func(att, value, &nulls[i], false);
@@ -1535,7 +1539,13 @@ PLyGenericObject_ToComposite(PLyObToDatum *arg, TupleDesc desc, PyObject *object
* it's interpreted as a two-dimensional array, and we try to
* interpret "123" as the composite type. See also similar
* heuristic in PLyObject_ToScalar().
+ *
+ * A missing attribute sets AttributeError. Another exception,
+ * such as MemoryError, is a real failure.
*/
+ if (!PyErr_ExceptionMatches(PyExc_AttributeError))
+ PLy_elog(ERROR, NULL);
+ PyErr_Clear();
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_COLUMN),
errmsg("attribute \"%s\" does not exist in Python object", key),
diff --git a/src/pl/plpython/plpy_util.c b/src/pl/plpython/plpy_util.c
index c2338decba2..bc7eb4515f2 100644
--- a/src/pl/plpython/plpy_util.c
+++ b/src/pl/plpython/plpy_util.c
@@ -117,3 +117,63 @@ PLyUnicode_FromString(const char *s)
{
return PLyUnicode_FromStringAndSize(s, strlen(s));
}
+
+/*
+ * Return obj. NULL means a Python exception is set. Report it with
+ * PLy_elog(ERROR) and do not return.
+ */
+PyObject *
+PLy_check(PyObject *obj)
+{
+ if (obj == NULL)
+ PLy_elog(ERROR, NULL);
+ return obj;
+}
+
+/*
+ * Insert value into dict under key.
+ *
+ * value is a new reference owned by the caller, or NULL with a Python
+ * exception set. A new reference is stolen. Py_None is borrowed, and this
+ * function takes its own reference. On failure the Python exception is
+ * reported with PLy_elog(ERROR).
+ */
+void
+PLy_dict_setitem_string(PyObject *dict, const char *key, PyObject *value)
+{
+ if (value == NULL)
+ PLy_elog(ERROR, NULL);
+
+ if (value == Py_None)
+ Py_INCREF(Py_None);
+
+ if (PyDict_SetItemString(dict, key, value) < 0)
+ {
+ Py_DECREF(value);
+ PLy_elog(ERROR, NULL);
+ }
+ Py_DECREF(value);
+}
+
+/*
+ * Store value at index i.
+ *
+ * value is a new reference owned by the caller, or NULL with a Python
+ * exception set. A new reference is stolen. Py_None is borrowed, and this
+ * function takes its own reference. listobject.h says PyList_SetItem
+ * decrements the new reference even when the item is not inserted, so it
+ * is not released again here. On failure the Python exception is reported
+ * with PLy_elog(ERROR).
+ */
+void
+PLy_list_setitem(PyObject *list, Py_ssize_t i, PyObject *value)
+{
+ if (value == NULL)
+ PLy_elog(ERROR, NULL);
+
+ if (value == Py_None)
+ Py_INCREF(Py_None);
+
+ if (PyList_SetItem(list, i, value) < 0)
+ PLy_elog(ERROR, NULL);
+}
diff --git a/src/pl/plpython/plpy_util.h b/src/pl/plpython/plpy_util.h
index 6f491b0f95b..2c3a7d4c06c 100644
--- a/src/pl/plpython/plpy_util.h
+++ b/src/pl/plpython/plpy_util.h
@@ -14,4 +14,10 @@ extern PGDLLEXPORT char *PLyUnicode_AsString(PyObject *unicode);
extern PGDLLEXPORT PyObject *PLyUnicode_FromString(const char *s);
extern PGDLLEXPORT PyObject *PLyUnicode_FromStringAndSize(const char *s, Py_ssize_t size);
+extern PGDLLEXPORT PyObject *PLy_check(PyObject *obj);
+extern PGDLLEXPORT void PLy_dict_setitem_string(PyObject *dict, const char *key,
+ PyObject *value);
+extern PGDLLEXPORT void PLy_list_setitem(PyObject *list, Py_ssize_t i,
+ PyObject *value);
+
#endif /* PLPY_UTIL_H */
--
2.53.0
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #19899: PL/Python backend crashes (SIGSEGV) when converting a column value to a Python object fails
@ 2026-11-16 19:44 Robert Haas <robertmhaas@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 7+ messages in thread
From: Robert Haas @ 2026-11-16 19:44 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: marcos.castedo@anachronics.com; pgsql-bugs@lists.postgresql.org
On Mon, Nov 16, 2026 at 12:53 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> The immediate cause in this example seems to be
> (1) PyBytes_FromStringAndSize returns NULL after a malloc failure
> (2) We blindly pass that NULL to PyDict_SetItemString
> (3) PyDict_SetItemString doesn't cope with that and dumps core.
>
> It looks to me like plpy_typeio.c is just full of bugs like this,
> because it doesn't have any coherent strategy for detecting or
> dealing with failures. There are a few places that check for
> NULL results and throw PLy_elog(ERROR), but not many. I think
> we'd first want to decide whether to check immediately after
> each call to a Python function, or try to centralize it a bit.
Here's what my LLM says:
The problem here is that PL/Python's input conversion functions follow
the Python convention of returning NULL, with an exception set, when
Python can't build the object, and none of the backend code that
calls them checks. Here the copy of the bytea fails with MemoryError,
and PLyDict_FromTuple stores the NULL straight into the dictionary.
d0aa965c0a (Peter Eisentraut, 2017-10-31) stated the rule: code called
from Python returns NULL, and code called from PostgreSQL raises an
error. The input conversions are only ever called from backend code,
which was supposed to turn the NULL into an error and never does.
Marcos is right that row-type arguments and TD["new"] go through the
same code, and they crash too, as does an array element. A scalar
argument is worse: argument setup treats a NULL result as an SQL NULL,
so the function quietly runs with None and returns a wrong answer.
The attached patch makes the two entry points, PLy_input_convert and
PLy_input_from_tuple, throw an error on a NULL result, as the
numeric-to-Decimal conversion already does, and makes argument setup
use None only for an SQL NULL. Every case then fails with an ordinary
ERROR and the session survives. A plpython3u function that lowers
RLIMIT_AS on its own backend reproduces this without a restart; that's
the attached setup.sql.
--
Robert Haas
Databricks
Attachments:
0001-Fix-PL-Python-crashes-when-a-value-can-t-be-converte.patch (application/octet-stream, 5.6K) download
From 2308dfcda2394df251058d70b618e8da107b23c6 Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Mon, 16 Nov 2026 14:35:47 -0500
Subject: [PATCH] Fix PL/Python crashes when a value can't be converted to
Python
PL/Python's input conversion functions follow the Python convention of
returning NULL, with a Python exception set, when an object can't be
built, as when copying a large value runs out of memory. None of their
callers checked for that. Building the result rows of plpy.execute(),
a row-type argument, or a trigger's TD["new"] or TD["old"] passed the
NULL to PyDict_SetItemString() and crashed the backend, and arrays
stored it as an element. A scalar argument was worse: the NULL was
taken for an SQL null, so the function silently ran with None.
Throw an error in PLy_input_convert() and PLy_input_from_tuple()
instead, and pass None to a function only for an actual SQL null.
Reported-by: Marcos Castedo <marcos.castedo@anachronics.com>
Discussion: https://postgr.es/m/19899-3e21845d988f0c0f@postgresql.org
Backpatch-through: 14
---
src/pl/plpython/plpy_exec.c | 7 ++-----
src/pl/plpython/plpy_typeio.c | 37 ++++++++++++++++++++++++++++++++---
src/pl/plpython/plpy_typeio.h | 5 ++++-
3 files changed, 40 insertions(+), 9 deletions(-)
diff --git a/src/pl/plpython/plpy_exec.c b/src/pl/plpython/plpy_exec.c
index 14e6362507..0b5f215c71 100644
--- a/src/pl/plpython/plpy_exec.c
+++ b/src/pl/plpython/plpy_exec.c
@@ -563,15 +563,12 @@ PLy_function_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc)
PLyDatumToOb *arginfo = &proc->args[i];
if (fcinfo->args[i].isnull)
- arg = NULL;
- else
- arg = PLy_input_convert(arginfo, fcinfo->args[i].value);
-
- if (arg == NULL)
{
Py_INCREF(Py_None);
arg = Py_None;
}
+ else
+ arg = PLy_input_convert(arginfo, fcinfo->args[i].value);
if (PyList_SetItem(args, i, arg) == -1)
PLy_elog(ERROR, "PyList_SetItem() failed, while setting up arguments");
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index e499ed8cbf..2fc997dfea 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -76,6 +76,10 @@ static Datum PLyGenericObject_ToComposite(PLyObToDatum *arg, TupleDesc desc, PyO
*
* This is the outer-level entry point for any input conversion. Internally,
* the conversion functions recurse directly to each other.
+ *
+ * The individual conversion functions follow the Python convention of
+ * returning NULL, with a Python exception set, on failure. Our callers
+ * aren't prepared for that, so we turn it into a PostgreSQL error here.
*/
PyObject *
PLy_input_convert(PLyDatumToOb *arg, Datum val)
@@ -104,6 +108,9 @@ PLy_input_convert(PLyDatumToOb *arg, Datum val)
MemoryContextSwitchTo(oldcontext);
+ if (result == NULL)
+ PLy_elog(ERROR, "could not convert value to a Python object");
+
return result;
}
@@ -149,6 +156,9 @@ PLy_input_from_tuple(PLyDatumToOb *arg, HeapTuple tuple, TupleDesc desc, bool in
MemoryContextSwitchTo(oldcontext);
+ if (dict == NULL)
+ PLy_elog(ERROR, "could not convert row to a Python dictionary");
+
return dict;
}
@@ -723,6 +733,11 @@ PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
sublist = PLyList_FromArray_recurse(elm, dims, ndim, dim + 1,
dataptr_p, bitmap_p, bitmask_p);
+ if (sublist == NULL)
+ {
+ Py_DECREF(list);
+ return NULL;
+ }
PyList_SetItem(list, i, sublist);
}
}
@@ -748,9 +763,16 @@ PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
else
{
Datum itemvalue;
+ PyObject *item;
itemvalue = fetch_att(dataptr, elm->typbyval, elm->typlen);
- PyList_SetItem(list, i, elm->func(elm, itemvalue));
+ item = elm->func(elm, itemvalue);
+ if (item == NULL)
+ {
+ Py_DECREF(list);
+ return NULL;
+ }
+ PyList_SetItem(list, i, item);
dataptr = att_addlength_pointer(dataptr, elm->typlen, dataptr);
dataptr = (char *) att_nominal_alignby(dataptr, typalignby);
}
@@ -854,12 +876,21 @@ PLyDict_FromTuple(PLyDatumToOb *arg, HeapTuple tuple, TupleDesc desc, bool inclu
vattr = heap_getattr(tuple, (i + 1), desc, &is_null);
if (is_null)
- PyDict_SetItemString(dict, key, Py_None);
+ {
+ if (PyDict_SetItemString(dict, key, Py_None) < 0)
+ PLy_elog(ERROR, "could not add column \"%s\" to Python dictionary", key);
+ }
else
{
+ int rc;
+
value = att->func(att, vattr);
- PyDict_SetItemString(dict, key, value);
+ if (value == NULL)
+ PLy_elog(ERROR, "could not convert column \"%s\" to a Python object", key);
+ rc = PyDict_SetItemString(dict, key, value);
Py_DECREF(value);
+ if (rc < 0)
+ PLy_elog(ERROR, "could not add column \"%s\" to Python dictionary", key);
}
}
}
diff --git a/src/pl/plpython/plpy_typeio.h b/src/pl/plpython/plpy_typeio.h
index 29258509b5..d9d4b79875 100644
--- a/src/pl/plpython/plpy_typeio.h
+++ b/src/pl/plpython/plpy_typeio.h
@@ -17,7 +17,10 @@ struct PLyProcedure; /* avoid requiring plpy_procedure.h here */
* "Input" conversion from PostgreSQL Datum to a Python object.
*
* arg is the previously-set-up conversion data, val is the value to convert.
- * val mustn't be NULL.
+ * val mustn't be NULL. The result is a new reference, or NULL with a Python
+ * exception set if Python fails to construct the object. Conversion can also
+ * throw a PostgreSQL error. PLy_input_convert and PLy_input_from_tuple never
+ * return NULL.
*
* Note: the conversion data structs should be regarded as private to
* plpy_typeio.c. We declare them here only so that other modules can
--
2.53.0
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #19899: PL/Python backend crashes (SIGSEGV) when converting a column value to a Python object fails
@ 2026-11-17 00:55 shihao zhong <zhong950419@gmail.com>
parent: Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 7+ messages in thread
From: shihao zhong @ 2026-11-17 00:55 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; marcos.castedo@anachronics.com; pgsql-bugs@lists.postgresql.org
Hi Robert,
Thanks for the patch.
> Every case then fails with an ordinary ERROR and the session survives
I tested your patch and triggers and function calls can still crash.
PLy_trigger_build_args() puts unchecked results into the TD
dictionary. PLy_procedure_call() crashes when building the argument
list fails.
V1-0001 is your patch without change
V2-0002 adds checks for those spots and it fixed ^^^
Thanks,
Shihao
Attachments: download all (2 files, .tar.gz)
v2-0001-Fix-PL-Python-crashes-when-a-value-can-t-be-conve.patch (application/octet-stream, 5.6K) download
From 9811257aad65e17d2115d234fe15e0f10e6eaedf Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Mon, 16 Nov 2026 14:35:47 -0500
Subject: [PATCH v2 1/2] Fix PL/Python crashes when a value can't be converted
to Python
PL/Python's input conversion functions follow the Python convention of
returning NULL, with a Python exception set, when an object can't be
built, as when copying a large value runs out of memory. None of their
callers checked for that. Building the result rows of plpy.execute(),
a row-type argument, or a trigger's TD["new"] or TD["old"] passed the
NULL to PyDict_SetItemString() and crashed the backend, and arrays
stored it as an element. A scalar argument was worse: the NULL was
taken for an SQL null, so the function silently ran with None.
Throw an error in PLy_input_convert() and PLy_input_from_tuple()
instead, and pass None to a function only for an actual SQL null.
Reported-by: Marcos Castedo <marcos.castedo@anachronics.com>
Discussion: https://postgr.es/m/19899-3e21845d988f0c0f@postgresql.org
Backpatch-through: 14
---
src/pl/plpython/plpy_exec.c | 7 ++-----
src/pl/plpython/plpy_typeio.c | 37 ++++++++++++++++++++++++++++++++---
src/pl/plpython/plpy_typeio.h | 5 ++++-
3 files changed, 40 insertions(+), 9 deletions(-)
diff --git a/src/pl/plpython/plpy_exec.c b/src/pl/plpython/plpy_exec.c
index 14e63625070..0b5f215c713 100644
--- a/src/pl/plpython/plpy_exec.c
+++ b/src/pl/plpython/plpy_exec.c
@@ -563,15 +563,12 @@ PLy_function_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc)
PLyDatumToOb *arginfo = &proc->args[i];
if (fcinfo->args[i].isnull)
- arg = NULL;
- else
- arg = PLy_input_convert(arginfo, fcinfo->args[i].value);
-
- if (arg == NULL)
{
Py_INCREF(Py_None);
arg = Py_None;
}
+ else
+ arg = PLy_input_convert(arginfo, fcinfo->args[i].value);
if (PyList_SetItem(args, i, arg) == -1)
PLy_elog(ERROR, "PyList_SetItem() failed, while setting up arguments");
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index e499ed8cbfd..2fc997dfea6 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -76,6 +76,10 @@ static Datum PLyGenericObject_ToComposite(PLyObToDatum *arg, TupleDesc desc, PyO
*
* This is the outer-level entry point for any input conversion. Internally,
* the conversion functions recurse directly to each other.
+ *
+ * The individual conversion functions follow the Python convention of
+ * returning NULL, with a Python exception set, on failure. Our callers
+ * aren't prepared for that, so we turn it into a PostgreSQL error here.
*/
PyObject *
PLy_input_convert(PLyDatumToOb *arg, Datum val)
@@ -104,6 +108,9 @@ PLy_input_convert(PLyDatumToOb *arg, Datum val)
MemoryContextSwitchTo(oldcontext);
+ if (result == NULL)
+ PLy_elog(ERROR, "could not convert value to a Python object");
+
return result;
}
@@ -149,6 +156,9 @@ PLy_input_from_tuple(PLyDatumToOb *arg, HeapTuple tuple, TupleDesc desc, bool in
MemoryContextSwitchTo(oldcontext);
+ if (dict == NULL)
+ PLy_elog(ERROR, "could not convert row to a Python dictionary");
+
return dict;
}
@@ -723,6 +733,11 @@ PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
sublist = PLyList_FromArray_recurse(elm, dims, ndim, dim + 1,
dataptr_p, bitmap_p, bitmask_p);
+ if (sublist == NULL)
+ {
+ Py_DECREF(list);
+ return NULL;
+ }
PyList_SetItem(list, i, sublist);
}
}
@@ -748,9 +763,16 @@ PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
else
{
Datum itemvalue;
+ PyObject *item;
itemvalue = fetch_att(dataptr, elm->typbyval, elm->typlen);
- PyList_SetItem(list, i, elm->func(elm, itemvalue));
+ item = elm->func(elm, itemvalue);
+ if (item == NULL)
+ {
+ Py_DECREF(list);
+ return NULL;
+ }
+ PyList_SetItem(list, i, item);
dataptr = att_addlength_pointer(dataptr, elm->typlen, dataptr);
dataptr = (char *) att_nominal_alignby(dataptr, typalignby);
}
@@ -854,12 +876,21 @@ PLyDict_FromTuple(PLyDatumToOb *arg, HeapTuple tuple, TupleDesc desc, bool inclu
vattr = heap_getattr(tuple, (i + 1), desc, &is_null);
if (is_null)
- PyDict_SetItemString(dict, key, Py_None);
+ {
+ if (PyDict_SetItemString(dict, key, Py_None) < 0)
+ PLy_elog(ERROR, "could not add column \"%s\" to Python dictionary", key);
+ }
else
{
+ int rc;
+
value = att->func(att, vattr);
- PyDict_SetItemString(dict, key, value);
+ if (value == NULL)
+ PLy_elog(ERROR, "could not convert column \"%s\" to a Python object", key);
+ rc = PyDict_SetItemString(dict, key, value);
Py_DECREF(value);
+ if (rc < 0)
+ PLy_elog(ERROR, "could not add column \"%s\" to Python dictionary", key);
}
}
}
diff --git a/src/pl/plpython/plpy_typeio.h b/src/pl/plpython/plpy_typeio.h
index 29258509b5c..d9d4b79875b 100644
--- a/src/pl/plpython/plpy_typeio.h
+++ b/src/pl/plpython/plpy_typeio.h
@@ -17,7 +17,10 @@ struct PLyProcedure; /* avoid requiring plpy_procedure.h here */
* "Input" conversion from PostgreSQL Datum to a Python object.
*
* arg is the previously-set-up conversion data, val is the value to convert.
- * val mustn't be NULL.
+ * val mustn't be NULL. The result is a new reference, or NULL with a Python
+ * exception set if Python fails to construct the object. Conversion can also
+ * throw a PostgreSQL error. PLy_input_convert and PLy_input_from_tuple never
+ * return NULL.
*
* Note: the conversion data structs should be regarded as private to
* plpy_typeio.c. We declare them here only so that other modules can
--
2.37.1 (Apple Git-137.1)
v2-0002-plpython-Check-for-failures-while-setting-up-func.patch (application/octet-stream, 8.5K) download
From 18a1aad7bf2d534768fc3ca92a61754e5f287093 Mon Sep 17 00:00:00 2001
From: shihao zhong <zhong950419@gmail.com>
Date: Mon, 16 Nov 2026 13:24:44 -0500
Subject: [PATCH v2 2/2] plpython: Check for failures while setting up function
arguments.
Building the argument list or the TD dictionary could also fail and
pass NULL along unchecked. Raise an error instead.
Bug: #19899
Reported-by: Marcos Castedo <marcos.castedo@anachronics.com>
Discussion: https://postgr.es/m/19899-3e21845d988f0c0f@postgresql.org
Backpatch-through: 14
---
src/pl/plpython/plpy_exec.c | 88 ++++++++++++++++++++++---------------
1 file changed, 52 insertions(+), 36 deletions(-)
diff --git a/src/pl/plpython/plpy_exec.c b/src/pl/plpython/plpy_exec.c
index 0b5f215c713..01a137ad306 100644
--- a/src/pl/plpython/plpy_exec.c
+++ b/src/pl/plpython/plpy_exec.c
@@ -31,6 +31,7 @@ static PyObject *PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *p
static HeapTuple PLy_modify_tuple(PLyProcedure *proc, PyObject *pltd,
TriggerData *tdata, HeapTuple otup);
static void plpython_trigger_error_callback(void *arg);
+static void PLy_td_set_item(PyObject *td, const char *key, PyObject *value);
static PyObject *PLy_procedure_call(PLyProcedure *proc, const char *kargs, PyObject *vargs);
static void PLy_abort_open_subtransactions(int save_subxact_level);
@@ -519,12 +520,10 @@ PLy_exec_event_trigger(FunctionCallInfo fcinfo, PLyProcedure *proc)
PLy_elog(ERROR, NULL);
pltevent = PLyUnicode_FromString(tdata->event);
- PyDict_SetItemString(pltdata, "event", pltevent);
- Py_DECREF(pltevent);
+ PLy_td_set_item(pltdata, "event", pltevent);
plttag = PLyUnicode_FromString(GetCommandTagName(tdata->tag));
- PyDict_SetItemString(pltdata, "tag", plttag);
- Py_DECREF(plttag);
+ PLy_td_set_item(pltdata, "tag", plttag);
PLy_procedure_call(proc, "TD", pltdata);
@@ -833,26 +832,22 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
PG_TRY();
{
pltname = PLyUnicode_FromString(tdata->tg_trigger->tgname);
- PyDict_SetItemString(pltdata, "name", pltname);
- Py_DECREF(pltname);
+ PLy_td_set_item(pltdata, "name", pltname);
stroid = DatumGetCString(DirectFunctionCall1(oidout,
ObjectIdGetDatum(tdata->tg_relation->rd_id)));
pltrelid = PLyUnicode_FromString(stroid);
- PyDict_SetItemString(pltdata, "relid", pltrelid);
- Py_DECREF(pltrelid);
+ PLy_td_set_item(pltdata, "relid", pltrelid);
pfree(stroid);
stroid = SPI_getrelname(tdata->tg_relation);
plttablename = PLyUnicode_FromString(stroid);
- PyDict_SetItemString(pltdata, "table_name", plttablename);
- Py_DECREF(plttablename);
+ PLy_td_set_item(pltdata, "table_name", plttablename);
pfree(stroid);
stroid = SPI_getnspname(tdata->tg_relation);
plttableschema = PLyUnicode_FromString(stroid);
- PyDict_SetItemString(pltdata, "table_schema", plttableschema);
- Py_DECREF(plttableschema);
+ PLy_td_set_item(pltdata, "table_schema", plttableschema);
pfree(stroid);
if (TRIGGER_FIRED_BEFORE(tdata->tg_event))
@@ -866,14 +861,12 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
elog(ERROR, "unrecognized WHEN tg_event: %u", tdata->tg_event);
pltwhen = NULL; /* keep compiler quiet */
}
- PyDict_SetItemString(pltdata, "when", pltwhen);
- Py_DECREF(pltwhen);
+ PLy_td_set_item(pltdata, "when", pltwhen);
if (TRIGGER_FIRED_FOR_ROW(tdata->tg_event))
{
pltlevel = PLyUnicode_FromString("ROW");
- PyDict_SetItemString(pltdata, "level", pltlevel);
- Py_DECREF(pltlevel);
+ PLy_td_set_item(pltdata, "level", pltlevel);
/*
* Note: In BEFORE trigger, stored generated columns are not
@@ -884,26 +877,26 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
{
pltevent = PLyUnicode_FromString("INSERT");
- PyDict_SetItemString(pltdata, "old", Py_None);
+ Py_INCREF(Py_None);
+ PLy_td_set_item(pltdata, "old", Py_None);
pytnew = PLy_input_from_tuple(&proc->result_in,
tdata->tg_trigtuple,
rel_descr,
!TRIGGER_FIRED_BEFORE(tdata->tg_event));
- PyDict_SetItemString(pltdata, "new", pytnew);
- Py_DECREF(pytnew);
+ PLy_td_set_item(pltdata, "new", pytnew);
*rv = tdata->tg_trigtuple;
}
else if (TRIGGER_FIRED_BY_DELETE(tdata->tg_event))
{
pltevent = PLyUnicode_FromString("DELETE");
- PyDict_SetItemString(pltdata, "new", Py_None);
+ Py_INCREF(Py_None);
+ PLy_td_set_item(pltdata, "new", Py_None);
pytold = PLy_input_from_tuple(&proc->result_in,
tdata->tg_trigtuple,
rel_descr,
true);
- PyDict_SetItemString(pltdata, "old", pytold);
- Py_DECREF(pytold);
+ PLy_td_set_item(pltdata, "old", pytold);
*rv = tdata->tg_trigtuple;
}
else if (TRIGGER_FIRED_BY_UPDATE(tdata->tg_event))
@@ -914,14 +907,12 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
tdata->tg_newtuple,
rel_descr,
!TRIGGER_FIRED_BEFORE(tdata->tg_event));
- PyDict_SetItemString(pltdata, "new", pytnew);
- Py_DECREF(pytnew);
+ PLy_td_set_item(pltdata, "new", pytnew);
pytold = PLy_input_from_tuple(&proc->result_in,
tdata->tg_trigtuple,
rel_descr,
true);
- PyDict_SetItemString(pltdata, "old", pytold);
- Py_DECREF(pytold);
+ PLy_td_set_item(pltdata, "old", pytold);
*rv = tdata->tg_newtuple;
}
else
@@ -930,17 +921,17 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
pltevent = NULL; /* keep compiler quiet */
}
- PyDict_SetItemString(pltdata, "event", pltevent);
- Py_DECREF(pltevent);
+ PLy_td_set_item(pltdata, "event", pltevent);
}
else if (TRIGGER_FIRED_FOR_STATEMENT(tdata->tg_event))
{
pltlevel = PLyUnicode_FromString("STATEMENT");
- PyDict_SetItemString(pltdata, "level", pltlevel);
- Py_DECREF(pltlevel);
+ PLy_td_set_item(pltdata, "level", pltlevel);
- PyDict_SetItemString(pltdata, "old", Py_None);
- PyDict_SetItemString(pltdata, "new", Py_None);
+ Py_INCREF(Py_None);
+ PLy_td_set_item(pltdata, "old", Py_None);
+ Py_INCREF(Py_None);
+ PLy_td_set_item(pltdata, "new", Py_None);
*rv = NULL;
if (TRIGGER_FIRED_BY_INSERT(tdata->tg_event))
@@ -957,8 +948,7 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
pltevent = NULL; /* keep compiler quiet */
}
- PyDict_SetItemString(pltdata, "event", pltevent);
- Py_DECREF(pltevent);
+ PLy_td_set_item(pltdata, "event", pltevent);
}
else
elog(ERROR, "unrecognized LEVEL tg_event: %u", tdata->tg_event);
@@ -977,6 +967,8 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
for (i = 0; i < tdata->tg_trigger->tgnargs; i++)
{
pltarg = PLyUnicode_FromString(tdata->tg_trigger->tgargs[i]);
+ if (pltarg == NULL)
+ PLy_elog(ERROR, "could not build TD dictionary");
/*
* stolen, don't Py_DECREF
@@ -988,7 +980,8 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
{
Assert(pltargs == Py_None);
}
- PyDict_SetItemString(pltdata, "args", pltargs);
+ if (PyDict_SetItemString(pltdata, "args", pltargs) < 0)
+ PLy_elog(ERROR, "could not build TD dictionary");
Py_DECREF(pltargs);
}
PG_CATCH();
@@ -1002,6 +995,26 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
return pltdata;
}
+/*
+ * Store "value" under "key" in a TD dictionary, raising an error on failure.
+ *
+ * This steals the reference to "value", which can be NULL (with a Python
+ * exception set) if creating it failed.
+ */
+static void
+PLy_td_set_item(PyObject *td, const char *key, PyObject *value)
+{
+ int rc = -1;
+
+ if (value != NULL)
+ {
+ rc = PyDict_SetItemString(td, key, value);
+ Py_DECREF(value);
+ }
+ if (rc < 0)
+ PLy_elog(ERROR, "could not build TD dictionary");
+}
+
/*
* Apply changes requested by a MODIFY return from a trigger function.
*/
@@ -1151,7 +1164,10 @@ PLy_procedure_call(PLyProcedure *proc, const char *kargs, PyObject *vargs)
PyObject *rv = NULL;
int volatile save_subxact_level = list_length(explicit_subtransactions);
- PyDict_SetItemString(proc->globals, kargs, vargs);
+ /* vargs is NULL, with a Python exception set, if building it failed */
+ if (vargs == NULL ||
+ PyDict_SetItemString(proc->globals, kargs, vargs) < 0)
+ PLy_elog(ERROR, "could not set up arguments for PL/Python function");
PG_TRY();
{
--
2.37.1 (Apple Git-137.1)
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #19899: PL/Python backend crashes (SIGSEGV) when converting a column value to a Python object fails
@ 2026-11-17 07:39 Andrey Rachitskiy <pl0h0yp1@gmail.com>
parent: shihao zhong <zhong950419@gmail.com>
0 siblings, 1 reply; 7+ messages in thread
From: Andrey Rachitskiy @ 2026-11-17 07:39 UTC (permalink / raw)
To: shihao zhong <zhong950419@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; marcos.castedo@anachronics.com; pgsql-bugs@lists.postgresql.org
вт, 17 нояб. 2026 г. в 05:55, shihao zhong <zhong950419@gmail.com>:
> Hi Robert,
>
> Thanks for the patch.
>
> > Every case then fails with an ordinary ERROR and the session survives
>
> I tested your patch and triggers and function calls can still crash.
>
> PLy_trigger_build_args() puts unchecked results into the TD
> dictionary. PLy_procedure_call() crashes when building the argument
> list fails.
>
> V1-0001 is your patch without change
> V2-0002 adds checks for those spots and it fixed ^^^
>
> Thanks,
> Shihao
>
I also found quite a few more places where fixes can be made and I'm
attaching v3. Reworked the error handling somewhat in 0001, introduced
shared functions, and fixed plpy_typeio. In 0002 I fixed other problematic
spots in pl_python. In my view, the code has become much better.
Attachments: download all (2 files, .tar.gz)
v3-0002-Check-the-remaining-Python-API-failures-in-PL-Pyt.patch (text/x-patch, 12.1K) download
From 83e275f01d51ae5ae3f26d29461389c4587c0993 Mon Sep 17 00:00:00 2001
From: Andrey Rachitskiy <pl0h0yp1@gmail.com>
Date: Tue, 17 Nov 2026 11:01:26 +0500
Subject: [PATCH v4] Check the remaining Python API failures in PL/Python
These are the same class of bug as the object construction fix, in the
places that do not build a row or an argument list.
PLy_result_new() left result->nrows NULL when PyLong_FromLong() failed,
so plpy.execute(...).nrows() did Py_INCREF(NULL). PLy_result_colnames()
and its two siblings stored an unchecked object into the list they
return, which leaves a NULL slot that crashes Python on first access.
PLy_spi_execute_fetch_result() and PLy_cursor_fetch() left the result's
status and nrows NULL the same way. PLy_procedure_compile() passed an
unchecked PyDict_Copy() result to PyDict_SetItemString().
PLyUnicode_Bytes() returned the unchecked result of
PyBytes_FromStringAndSize(), and PLyUnicode_AsString() passed it
straight to PyBytes_AsString(), which starts with PyBytes_Check() and
dereferences it. PLy_elog() reaches that path through PLy_traceback(),
so this is the OOM case where the message itself crashes the backend.
PLyUnicode_Bytes() now reports the failure, like its other two error
paths.
PLy_cursor_query() and PLy_cursor_fetch() returned NULL from PG_CATCH
without releasing the object they had built, so an error inside the
subtransaction leaked it. PLy_cursor_plan() and
PLy_spi_execute_fetch_result() already release theirs.
The remaining inserts whose result was ignored are checked too: GD in
PLy_init_interp(), SD in PLy_procedure_compile() and sqlstate in
PLy_generate_spi_exceptions().
Building the trigger TD dictionary now stores "event" as soon as it is
built. Setting it last leaked the string if the NEW or OLD conversion
threw in between.
BUG #19899
Author: Andrey Rachitskiy <pl0h0yp1@gmail.com>
Discussion: https://www.postgresql.org/message-id/19899-3e21845d988f0c0f@postgresql.org
---
src/pl/plpython/plpy_cursorobject.c | 27 +++++++++++++++++-------
src/pl/plpython/plpy_exec.c | 8 +++-----
src/pl/plpython/plpy_main.c | 3 ++-
src/pl/plpython/plpy_plpymodule.c | 4 ++--
src/pl/plpython/plpy_procedure.c | 5 ++++-
src/pl/plpython/plpy_resultobject.c | 32 ++++++++++++++++++++++-------
src/pl/plpython/plpy_spi.c | 23 +++++++++++++++------
src/pl/plpython/plpy_util.c | 19 ++++++++++++++---
8 files changed, 89 insertions(+), 32 deletions(-)
diff --git a/src/pl/plpython/plpy_cursorobject.c b/src/pl/plpython/plpy_cursorobject.c
index 8d29e4725b2..9639f8e25f9 100644
--- a/src/pl/plpython/plpy_cursorobject.c
+++ b/src/pl/plpython/plpy_cursorobject.c
@@ -151,7 +151,12 @@ PLy_cursor_query(const char *query)
}
PG_CATCH();
{
+ /*
+ * Release the cursor after the abort, so that its deallocator does
+ * not try to close a portal that the abort has already dropped.
+ */
PLy_spi_subtransaction_abort(oldcontext, oldowner);
+ Py_DECREF(cursor);
return NULL;
}
PG_END_TRY();
@@ -444,9 +449,13 @@ PLy_cursor_fetch(PyObject *self, PyObject *args)
Py_DECREF(ret->status);
ret->status = PyLong_FromLong(SPI_OK_FETCH);
+ if (ret->status == NULL)
+ PLy_elog(ERROR, "could not create the fetch status");
Py_DECREF(ret->nrows);
ret->nrows = PyLong_FromUnsignedLongLong(SPI_processed);
+ if (ret->nrows == NULL)
+ PLy_elog(ERROR, "could not create the fetched row count");
if (SPI_processed != 0)
{
@@ -464,12 +473,7 @@ PLy_cursor_fetch(PyObject *self, PyObject *args)
Py_DECREF(ret->rows);
ret->rows = PyList_New(SPI_processed);
- if (!ret->rows)
- {
- Py_DECREF(ret);
- ret = NULL;
- }
- else
+ if (ret->rows)
{
PLy_input_setup_tuple(&cursor->result, SPI_tuptable->tupdesc,
exec_ctx->curr_proc);
@@ -481,7 +485,8 @@ PLy_cursor_fetch(PyObject *self, PyObject *args)
SPI_tuptable->tupdesc,
true);
- PyList_SetItem(ret->rows, i, row);
+ PLyList_SetItem(ret->rows, i, row,
+ "could not store a row in the fetch result");
}
}
}
@@ -492,11 +497,19 @@ PLy_cursor_fetch(PyObject *self, PyObject *args)
}
PG_CATCH();
{
+ Py_DECREF(ret);
PLy_spi_subtransaction_abort(oldcontext, oldowner);
return NULL;
}
PG_END_TRY();
+ /* in case PyList_New() failed above */
+ if (!ret->rows)
+ {
+ Py_DECREF(ret);
+ return NULL;
+ }
+
return (PyObject *) ret;
}
diff --git a/src/pl/plpython/plpy_exec.c b/src/pl/plpython/plpy_exec.c
index d31a072ead6..a6e3ab3912a 100644
--- a/src/pl/plpython/plpy_exec.c
+++ b/src/pl/plpython/plpy_exec.c
@@ -909,6 +909,7 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
if (TRIGGER_FIRED_BY_INSERT(tdata->tg_event))
{
pltevent = PLyUnicode_FromString("INSERT");
+ PLyTD_SetItem(pltdata, "event", pltevent);
PLyTD_SetItemNone(pltdata, "old");
pytnew = PLy_input_from_tuple(&proc->result_in,
@@ -921,6 +922,7 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
else if (TRIGGER_FIRED_BY_DELETE(tdata->tg_event))
{
pltevent = PLyUnicode_FromString("DELETE");
+ PLyTD_SetItem(pltdata, "event", pltevent);
PLyTD_SetItemNone(pltdata, "new");
pytold = PLy_input_from_tuple(&proc->result_in,
@@ -933,6 +935,7 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
else if (TRIGGER_FIRED_BY_UPDATE(tdata->tg_event))
{
pltevent = PLyUnicode_FromString("UPDATE");
+ PLyTD_SetItem(pltdata, "event", pltevent);
pytnew = PLy_input_from_tuple(&proc->result_in,
tdata->tg_newtuple,
@@ -947,12 +950,7 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
*rv = tdata->tg_newtuple;
}
else
- {
elog(ERROR, "unrecognized OP tg_event: %u", tdata->tg_event);
- pltevent = NULL; /* keep compiler quiet */
- }
-
- PLyTD_SetItem(pltdata, "event", pltevent);
}
else if (TRIGGER_FIRED_FOR_STATEMENT(tdata->tg_event))
{
diff --git a/src/pl/plpython/plpy_main.c b/src/pl/plpython/plpy_main.c
index e305f2a848a..d461bdbb562 100644
--- a/src/pl/plpython/plpy_main.c
+++ b/src/pl/plpython/plpy_main.c
@@ -83,7 +83,8 @@ _PG_init(void)
GD = PyDict_New();
if (GD == NULL)
PLy_elog(ERROR, NULL);
- PyDict_SetItemString(main_dict, "GD", GD);
+ if (PyDict_SetItemString(main_dict, "GD", GD) < 0)
+ PLy_elog(ERROR, NULL);
/*
* Import plpy.
diff --git a/src/pl/plpython/plpy_plpymodule.c b/src/pl/plpython/plpy_plpymodule.c
index 72806c17e17..9b4ca35eeed 100644
--- a/src/pl/plpython/plpy_plpymodule.c
+++ b/src/pl/plpython/plpy_plpymodule.c
@@ -227,8 +227,8 @@ PLy_generate_spi_exceptions(PyObject *mod, PyObject *base)
if (sqlstate == NULL)
PLy_elog(ERROR, "could not generate SPI exceptions");
- PyDict_SetItemString(dict, "sqlstate", sqlstate);
- Py_DECREF(sqlstate);
+ PLyDict_SetItemString(dict, "sqlstate", sqlstate,
+ "could not generate SPI exceptions");
exc = PLy_create_exception(exception_map[i].name, base, dict,
exception_map[i].classname, mod);
diff --git a/src/pl/plpython/plpy_procedure.c b/src/pl/plpython/plpy_procedure.c
index 76fee7974a2..8e806cc96bb 100644
--- a/src/pl/plpython/plpy_procedure.c
+++ b/src/pl/plpython/plpy_procedure.c
@@ -355,13 +355,16 @@ PLy_procedure_compile(PLyProcedure *proc, const char *src)
PyObject *code0;
proc->globals = PyDict_Copy(PLy_interp_globals);
+ if (proc->globals == NULL)
+ PLy_elog(ERROR, NULL);
/*
* SD is private preserved data between calls. GD is global data shared by
* all functions. The caller supplies SD.
*/
Assert(proc->statics != NULL);
- PyDict_SetItemString(proc->globals, "SD", proc->statics);
+ if (PyDict_SetItemString(proc->globals, "SD", proc->statics) < 0)
+ PLy_elog(ERROR, NULL);
/*
* insert the function code into the interpreter
diff --git a/src/pl/plpython/plpy_resultobject.c b/src/pl/plpython/plpy_resultobject.c
index d433929b360..9993ebbf850 100644
--- a/src/pl/plpython/plpy_resultobject.c
+++ b/src/pl/plpython/plpy_resultobject.c
@@ -104,7 +104,7 @@ PLy_result_new(void)
ob->nrows = PyLong_FromLong(-1);
ob->rows = PyList_New(0);
ob->tupdesc = NULL;
- if (!ob->rows)
+ if (!ob->nrows || !ob->rows)
{
Py_DECREF(ob);
return NULL;
@@ -155,8 +155,14 @@ PLy_result_colnames(PyObject *self, PyObject *unused)
for (i = 0; i < ob->tupdesc->natts; i++)
{
Form_pg_attribute attr = TupleDescAttr(ob->tupdesc, i);
-
- PyList_SetItem(list, i, PLyUnicode_FromString(NameStr(attr->attname)));
+ PyObject *name = PLyUnicode_FromString(NameStr(attr->attname));
+
+ /* PyList_SetItem consumes name even when the store fails */
+ if (name == NULL || PyList_SetItem(list, i, name) < 0)
+ {
+ Py_DECREF(list);
+ return NULL;
+ }
}
return list;
@@ -181,8 +187,14 @@ PLy_result_coltypes(PyObject *self, PyObject *unused)
for (i = 0; i < ob->tupdesc->natts; i++)
{
Form_pg_attribute attr = TupleDescAttr(ob->tupdesc, i);
-
- PyList_SetItem(list, i, PyLong_FromLong(attr->atttypid));
+ PyObject *typid = PyLong_FromLong(attr->atttypid);
+
+ /* PyList_SetItem consumes typid even when the store fails */
+ if (typid == NULL || PyList_SetItem(list, i, typid) < 0)
+ {
+ Py_DECREF(list);
+ return NULL;
+ }
}
return list;
@@ -207,8 +219,14 @@ PLy_result_coltypmods(PyObject *self, PyObject *unused)
for (i = 0; i < ob->tupdesc->natts; i++)
{
Form_pg_attribute attr = TupleDescAttr(ob->tupdesc, i);
-
- PyList_SetItem(list, i, PyLong_FromLong(attr->atttypmod));
+ PyObject *typmod = PyLong_FromLong(attr->atttypmod);
+
+ /* PyList_SetItem consumes typmod even when the store fails */
+ if (typmod == NULL || PyList_SetItem(list, i, typmod) < 0)
+ {
+ Py_DECREF(list);
+ return NULL;
+ }
}
return list;
diff --git a/src/pl/plpython/plpy_spi.c b/src/pl/plpython/plpy_spi.c
index 419e0a9b9ae..f455b8c8531 100644
--- a/src/pl/plpython/plpy_spi.c
+++ b/src/pl/plpython/plpy_spi.c
@@ -359,20 +359,30 @@ PLy_spi_execute_fetch_result(SPITupleTable *tuptable, uint64 rows, int status)
}
Py_DECREF(result->status);
result->status = PyLong_FromLong(status);
+ if (result->status == NULL)
+ {
+ Py_DECREF(result);
+ SPI_freetuptable(tuptable);
+ return NULL;
+ }
- if (status > 0 && tuptable == NULL)
+ if (status > 0)
{
Py_DECREF(result->nrows);
result->nrows = PyLong_FromUnsignedLongLong(rows);
+ if (result->nrows == NULL)
+ {
+ Py_DECREF(result);
+ SPI_freetuptable(tuptable);
+ return NULL;
+ }
}
- else if (status > 0 && tuptable != NULL)
+
+ if (status > 0 && tuptable != NULL)
{
PLyDatumToOb ininfo;
MemoryContext cxt;
- Py_DECREF(result->nrows);
- result->nrows = PyLong_FromUnsignedLongLong(rows);
-
cxt = AllocSetContextCreate(CurrentMemoryContext,
"PL/Python temp context",
ALLOCSET_DEFAULT_SIZES);
@@ -414,7 +424,8 @@ PLy_spi_execute_fetch_result(SPITupleTable *tuptable, uint64 rows, int status)
tuptable->tupdesc,
true);
- PyList_SetItem(result->rows, i, row);
+ PLyList_SetItem(result->rows, i, row,
+ "could not store a row in the query result");
}
}
}
diff --git a/src/pl/plpython/plpy_util.c b/src/pl/plpython/plpy_util.c
index 1b8a53f2401..ed25dfcfdfc 100644
--- a/src/pl/plpython/plpy_util.c
+++ b/src/pl/plpython/plpy_util.c
@@ -13,7 +13,8 @@
/*
* Convert a Python unicode object to a Python string/bytes object in
* PostgreSQL server encoding. Reference ownership is passed to the
- * caller.
+ * caller. Failure is reported with PLy_elog(ERROR), so the result is
+ * never NULL.
*/
PyObject *
PLyUnicode_Bytes(PyObject *unicode)
@@ -69,6 +70,10 @@ PLyUnicode_Bytes(PyObject *unicode)
pfree(encoded);
Py_DECREF(bytes);
+
+ if (rv == NULL)
+ PLy_elog(ERROR, "could not create bytes representation of Python object");
+
return rv;
}
@@ -81,9 +86,17 @@ char *
PLyUnicode_AsString(PyObject *unicode)
{
PyObject *o = PLyUnicode_Bytes(unicode);
- char *rv = pstrdup(PyBytes_AsString(o));
+ char *s = PyBytes_AsString(o);
+ char *rv;
+
+ if (s == NULL)
+ {
+ Py_DECREF(o);
+ PLy_elog(ERROR, "could not extract bytes from encoded string");
+ }
- Py_XDECREF(o);
+ rv = pstrdup(s);
+ Py_DECREF(o);
return rv;
}
--
2.53.0
v3-0001-Fix-unchecked-NULL-returns-in-PL-Python-object-co.patch (text/x-patch, 27.7K) download
From 4686e4c53ea0e3ffca54c01591c26539a350a37d Mon Sep 17 00:00:00 2001
From: Andrey Rachitskiy <pl0h0yp1@gmail.com>
Date: Tue, 17 Nov 2026 10:31:05 +0500
Subject: [PATCH v4] Fix unchecked NULL returns in PL/Python object
construction
PL/Python input conversion functions return NULL with a Python exception
set when the object cannot be built, most often on out of memory. Most
callers did not check.
PLyDict_FromTuple() passed that NULL to PyDict_SetItemString(), which
increments the reference count of the value and crashed the backend.
PLy_function_build_args() treated it as a SQL NULL and ran the function
with None, discarding the Python error. PLyList_FromArray_recurse()
stored it into the list, and the trigger TD dictionary ignored the result
of every insert.
Route those stores through PLyDict_SetItemString(), PLyDict_SetItemNone()
and PLyList_SetItem(). They consume the caller's reference and report the
Python exception under a message the caller supplies, so the exception
text lands in the detail. PLy_elog_impl_v() is the va_list entry point
they need. PLy_input_convert() and PLy_input_from_tuple() now throw
instead of returning NULL.
PLy_function_build_args() also left arg set after PyList_SetItem() had
consumed the reference, so the PG_CATCH block released it a second time.
PLyMapping_ToComposite() and PLyGenericObject_ToComposite() reported
every failure as a missing key or attribute. Only KeyError and
AttributeError mean that; any other exception is now reported as itself.
BUG #19899
Author: Andrey Rachitskiy <pl0h0yp1@gmail.com>
Reported-by: Marcos Castedo <marcos.castedo@anachronics.com>
Discussion: https://www.postgresql.org/message-id/19899-3e21845d988f0c0f@postgresql.org
---
src/pl/plpython/expected/plpython_record.out | 28 ++++
src/pl/plpython/nls.mk | 6 +
src/pl/plpython/plpy_elog.c | 22 ++-
src/pl/plpython/plpy_elog.h | 2 +
src/pl/plpython/plpy_exec.c | 125 ++++++++-------
src/pl/plpython/plpy_typeio.c | 155 ++++++++++++-------
src/pl/plpython/plpy_typeio.h | 4 +
src/pl/plpython/plpy_util.c | 81 ++++++++++
src/pl/plpython/plpy_util.h | 8 +
src/pl/plpython/sql/plpython_record.sql | 22 +++
10 files changed, 343 insertions(+), 110 deletions(-)
diff --git a/src/pl/plpython/expected/plpython_record.out b/src/pl/plpython/expected/plpython_record.out
index 26eca741f77..2bfc5b370d1 100644
--- a/src/pl/plpython/expected/plpython_record.out
+++ b/src/pl/plpython/expected/plpython_record.out
@@ -399,3 +399,31 @@ ERROR: malformed record literal: "foo"
DETAIL: Missing left parenthesis.
CONTEXT: while creating return value
PL/Python function "test_type_record_error4"
+-- a mapping that fails for a reason other than a missing key
+CREATE FUNCTION test_type_record_error5() RETURNS type_record AS $$
+ class broken_mapping(dict):
+ def __getitem__(self, key):
+ raise TypeError('broken mapping')
+ return broken_mapping(first='first', second=1)
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_record_error5();
+ERROR: could not get value for key "first" from mapping
+DETAIL: TypeError: broken mapping
+CONTEXT: Traceback (most recent call last):
+while creating return value
+PL/Python function "test_type_record_error5"
+-- an object whose attribute lookup fails for a reason other than a
+-- missing attribute
+CREATE FUNCTION test_type_record_error6() RETURNS type_record AS $$
+ class broken_object:
+ first = 'first'
+ def __getattr__(self, name):
+ raise ValueError('broken attribute')
+ return broken_object()
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_record_error6();
+ERROR: could not get attribute "second" of Python object
+DETAIL: ValueError: broken attribute
+CONTEXT: Traceback (most recent call last):
+while creating return value
+PL/Python function "test_type_record_error6"
diff --git a/src/pl/plpython/nls.mk b/src/pl/plpython/nls.mk
index f6aafcf2494..714d24fab7b 100644
--- a/src/pl/plpython/nls.mk
+++ b/src/pl/plpython/nls.mk
@@ -13,10 +13,16 @@ GETTEXT_FILES = plpy_cursorobject.c \
plpy_typeio.c \
plpy_util.c
GETTEXT_TRIGGERS = $(BACKEND_COMMON_GETTEXT_TRIGGERS) \
+ PLyDict_SetItemNone:3 \
+ PLyDict_SetItemString:4 \
+ PLyList_SetItem:4 \
PLy_elog:2 \
PLy_exception_set:2 \
PLy_exception_set_plural:2,3
GETTEXT_FLAGS = $(BACKEND_COMMON_GETTEXT_FLAGS) \
+ PLyDict_SetItemNone:3:c-format \
+ PLyDict_SetItemString:4:c-format \
+ PLyList_SetItem:4:c-format \
PLy_elog:2:c-format \
PLy_exception_set:2:c-format \
PLy_exception_set_plural:2:c-format \
diff --git a/src/pl/plpython/plpy_elog.c b/src/pl/plpython/plpy_elog.c
index 7c5ddb06ea8..99e2c6ed1a8 100644
--- a/src/pl/plpython/plpy_elog.c
+++ b/src/pl/plpython/plpy_elog.c
@@ -42,6 +42,20 @@ static bool set_string_attr(PyObject *obj, char *attrname, char *str);
*/
void
PLy_elog_impl(int elevel, const char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ PLy_elog_impl_v(elevel, fmt, ap);
+ va_end(ap);
+}
+
+/*
+ * va_list version of PLy_elog_impl(), for callers that pass a message
+ * through from their own variadic argument list.
+ */
+void
+PLy_elog_impl_v(int elevel, const char *fmt, va_list ap)
{
int save_errno = errno;
char *volatile xmsg = NULL;
@@ -97,13 +111,13 @@ PLy_elog_impl(int elevel, const char *fmt, ...)
{
for (;;)
{
- va_list ap;
+ va_list ap2;
int needed;
errno = save_errno;
- va_start(ap, fmt);
- needed = appendStringInfoVA(&emsg, dgettext(TEXTDOMAIN, fmt), ap);
- va_end(ap);
+ va_copy(ap2, ap);
+ needed = appendStringInfoVA(&emsg, dgettext(TEXTDOMAIN, fmt), ap2);
+ va_end(ap2);
if (needed == 0)
break;
enlargeStringInfo(&emsg, needed);
diff --git a/src/pl/plpython/plpy_elog.h b/src/pl/plpython/plpy_elog.h
index 887535d5860..ccc07f44f05 100644
--- a/src/pl/plpython/plpy_elog.h
+++ b/src/pl/plpython/plpy_elog.h
@@ -36,6 +36,8 @@ extern PyObject *PLy_exc_spi_error;
extern PGDLLEXPORT void PLy_elog_impl(int elevel, const char *fmt, ...) pg_attribute_printf(2, 3);
+extern void PLy_elog_impl_v(int elevel, const char *fmt, va_list ap) pg_attribute_printf(2, 0);
+
extern PGDLLEXPORT void PLy_exception_set(PyObject *exc, const char *fmt, ...) pg_attribute_printf(2, 3);
extern PGDLLEXPORT void PLy_exception_set_plural(PyObject *exc, const char *fmt_singular, const char *fmt_plural,
diff --git a/src/pl/plpython/plpy_exec.c b/src/pl/plpython/plpy_exec.c
index 14e63625070..d31a072ead6 100644
--- a/src/pl/plpython/plpy_exec.c
+++ b/src/pl/plpython/plpy_exec.c
@@ -28,6 +28,8 @@ static void plpython_return_error_callback(void *arg);
static PyObject *PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc,
HeapTuple *rv);
+static void PLyTD_SetItem(PyObject *td, const char *key, PyObject *value);
+static void PLyTD_SetItemNone(PyObject *td, const char *key);
static HeapTuple PLy_modify_tuple(PLyProcedure *proc, PyObject *pltd,
TriggerData *tdata, HeapTuple otup);
static void plpython_trigger_error_callback(void *arg);
@@ -519,12 +521,10 @@ PLy_exec_event_trigger(FunctionCallInfo fcinfo, PLyProcedure *proc)
PLy_elog(ERROR, NULL);
pltevent = PLyUnicode_FromString(tdata->event);
- PyDict_SetItemString(pltdata, "event", pltevent);
- Py_DECREF(pltevent);
+ PLyTD_SetItem(pltdata, "event", pltevent);
plttag = PLyUnicode_FromString(GetCommandTagName(tdata->tag));
- PyDict_SetItemString(pltdata, "tag", plttag);
- Py_DECREF(plttag);
+ PLyTD_SetItem(pltdata, "tag", plttag);
PLy_procedure_call(proc, "TD", pltdata);
@@ -561,25 +561,28 @@ PLy_function_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc)
for (i = 0; i < proc->nargs; i++)
{
PLyDatumToOb *arginfo = &proc->args[i];
+ PyObject *value;
if (fcinfo->args[i].isnull)
- arg = NULL;
- else
- arg = PLy_input_convert(arginfo, fcinfo->args[i].value);
-
- if (arg == NULL)
{
Py_INCREF(Py_None);
arg = Py_None;
}
+ else
+ arg = PLy_input_convert(arginfo, fcinfo->args[i].value);
- if (PyList_SetItem(args, i, arg) == -1)
- PLy_elog(ERROR, "PyList_SetItem() failed, while setting up arguments");
+ /*
+ * PLyList_SetItem() consumes the reference even when the store
+ * fails, so drop our pointer before the call.
+ */
+ value = arg;
+ arg = NULL;
+ PLyList_SetItem(args, i, value,
+ "PyList_SetItem() failed, while setting up arguments");
if (proc->argnames && proc->argnames[i] &&
- PyDict_SetItemString(proc->globals, proc->argnames[i], arg) == -1)
+ PyDict_SetItemString(proc->globals, proc->argnames[i], value) < 0)
PLy_elog(ERROR, "PyDict_SetItemString() failed, while setting up arguments");
- arg = NULL;
}
}
PG_CATCH();
@@ -791,6 +794,25 @@ plpython_return_error_callback(void *arg)
errcontext("while creating return value");
}
+/*
+ * Insert value into the TD dictionary under key, consuming the caller's
+ * reference.
+ */
+static void
+PLyTD_SetItem(PyObject *td, const char *key, PyObject *value)
+{
+ PLyDict_SetItemString(td, key, value, "could not build TD dictionary");
+}
+
+/*
+ * Insert the borrowed Py_None singleton into the TD dictionary under key.
+ */
+static void
+PLyTD_SetItemNone(PyObject *td, const char *key)
+{
+ PLyDict_SetItemNone(td, key, "could not build TD dictionary");
+}
+
static PyObject *
PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *rv)
{
@@ -803,10 +825,15 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
*pltrelid,
*plttablename,
*plttableschema,
- *pltargs,
*pytnew,
*pytold,
*pltdata;
+
+ /*
+ * pltargs is cleared inside the PG_TRY block once its reference has been
+ * handed to the TD dictionary, so the PG_CATCH block must see that store.
+ */
+ PyObject *volatile pltargs;
char *stroid;
/*
@@ -835,27 +862,25 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
PG_TRY();
{
+ PyObject *args;
+
pltname = PLyUnicode_FromString(tdata->tg_trigger->tgname);
- PyDict_SetItemString(pltdata, "name", pltname);
- Py_DECREF(pltname);
+ PLyTD_SetItem(pltdata, "name", pltname);
stroid = DatumGetCString(DirectFunctionCall1(oidout,
ObjectIdGetDatum(tdata->tg_relation->rd_id)));
pltrelid = PLyUnicode_FromString(stroid);
- PyDict_SetItemString(pltdata, "relid", pltrelid);
- Py_DECREF(pltrelid);
+ PLyTD_SetItem(pltdata, "relid", pltrelid);
pfree(stroid);
stroid = SPI_getrelname(tdata->tg_relation);
plttablename = PLyUnicode_FromString(stroid);
- PyDict_SetItemString(pltdata, "table_name", plttablename);
- Py_DECREF(plttablename);
+ PLyTD_SetItem(pltdata, "table_name", plttablename);
pfree(stroid);
stroid = SPI_getnspname(tdata->tg_relation);
plttableschema = PLyUnicode_FromString(stroid);
- PyDict_SetItemString(pltdata, "table_schema", plttableschema);
- Py_DECREF(plttableschema);
+ PLyTD_SetItem(pltdata, "table_schema", plttableschema);
pfree(stroid);
if (TRIGGER_FIRED_BEFORE(tdata->tg_event))
@@ -869,14 +894,12 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
elog(ERROR, "unrecognized WHEN tg_event: %u", tdata->tg_event);
pltwhen = NULL; /* keep compiler quiet */
}
- PyDict_SetItemString(pltdata, "when", pltwhen);
- Py_DECREF(pltwhen);
+ PLyTD_SetItem(pltdata, "when", pltwhen);
if (TRIGGER_FIRED_FOR_ROW(tdata->tg_event))
{
pltlevel = PLyUnicode_FromString("ROW");
- PyDict_SetItemString(pltdata, "level", pltlevel);
- Py_DECREF(pltlevel);
+ PLyTD_SetItem(pltdata, "level", pltlevel);
/*
* Note: In BEFORE trigger, stored generated columns are not
@@ -887,26 +910,24 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
{
pltevent = PLyUnicode_FromString("INSERT");
- PyDict_SetItemString(pltdata, "old", Py_None);
+ PLyTD_SetItemNone(pltdata, "old");
pytnew = PLy_input_from_tuple(&proc->result_in,
tdata->tg_trigtuple,
rel_descr,
!TRIGGER_FIRED_BEFORE(tdata->tg_event));
- PyDict_SetItemString(pltdata, "new", pytnew);
- Py_DECREF(pytnew);
+ PLyTD_SetItem(pltdata, "new", pytnew);
*rv = tdata->tg_trigtuple;
}
else if (TRIGGER_FIRED_BY_DELETE(tdata->tg_event))
{
pltevent = PLyUnicode_FromString("DELETE");
- PyDict_SetItemString(pltdata, "new", Py_None);
+ PLyTD_SetItemNone(pltdata, "new");
pytold = PLy_input_from_tuple(&proc->result_in,
tdata->tg_trigtuple,
rel_descr,
true);
- PyDict_SetItemString(pltdata, "old", pytold);
- Py_DECREF(pytold);
+ PLyTD_SetItem(pltdata, "old", pytold);
*rv = tdata->tg_trigtuple;
}
else if (TRIGGER_FIRED_BY_UPDATE(tdata->tg_event))
@@ -917,14 +938,12 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
tdata->tg_newtuple,
rel_descr,
!TRIGGER_FIRED_BEFORE(tdata->tg_event));
- PyDict_SetItemString(pltdata, "new", pytnew);
- Py_DECREF(pytnew);
+ PLyTD_SetItem(pltdata, "new", pytnew);
pytold = PLy_input_from_tuple(&proc->result_in,
tdata->tg_trigtuple,
rel_descr,
true);
- PyDict_SetItemString(pltdata, "old", pytold);
- Py_DECREF(pytold);
+ PLyTD_SetItem(pltdata, "old", pytold);
*rv = tdata->tg_newtuple;
}
else
@@ -933,17 +952,15 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
pltevent = NULL; /* keep compiler quiet */
}
- PyDict_SetItemString(pltdata, "event", pltevent);
- Py_DECREF(pltevent);
+ PLyTD_SetItem(pltdata, "event", pltevent);
}
else if (TRIGGER_FIRED_FOR_STATEMENT(tdata->tg_event))
{
pltlevel = PLyUnicode_FromString("STATEMENT");
- PyDict_SetItemString(pltdata, "level", pltlevel);
- Py_DECREF(pltlevel);
+ PLyTD_SetItem(pltdata, "level", pltlevel);
- PyDict_SetItemString(pltdata, "old", Py_None);
- PyDict_SetItemString(pltdata, "new", Py_None);
+ PLyTD_SetItemNone(pltdata, "old");
+ PLyTD_SetItemNone(pltdata, "new");
*rv = NULL;
if (TRIGGER_FIRED_BY_INSERT(tdata->tg_event))
@@ -960,8 +977,7 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
pltevent = NULL; /* keep compiler quiet */
}
- PyDict_SetItemString(pltdata, "event", pltevent);
- Py_DECREF(pltevent);
+ PLyTD_SetItem(pltdata, "event", pltevent);
}
else
elog(ERROR, "unrecognized LEVEL tg_event: %u", tdata->tg_event);
@@ -980,19 +996,18 @@ PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *proc, HeapTuple *r
for (i = 0; i < tdata->tg_trigger->tgnargs; i++)
{
pltarg = PLyUnicode_FromString(tdata->tg_trigger->tgargs[i]);
-
- /*
- * stolen, don't Py_DECREF
- */
- PyList_SetItem(pltargs, i, pltarg);
+ PLyList_SetItem(pltargs, i, pltarg,
+ "could not build TD dictionary");
}
}
else
{
Assert(pltargs == Py_None);
}
- PyDict_SetItemString(pltdata, "args", pltargs);
- Py_DECREF(pltargs);
+ /* The store consumes the reference, so give up our pointer. */
+ args = pltargs;
+ pltargs = NULL;
+ PLyTD_SetItem(pltdata, "args", args);
}
PG_CATCH();
{
@@ -1154,7 +1169,13 @@ PLy_procedure_call(PLyProcedure *proc, const char *kargs, PyObject *vargs)
PyObject *rv = NULL;
int volatile save_subxact_level = list_length(explicit_subtransactions);
- PyDict_SetItemString(proc->globals, kargs, vargs);
+ /*
+ * vargs is NULL, with a Python exception set, if building it failed. The
+ * caller still owns the reference, so do not steal it.
+ */
+ if (vargs == NULL ||
+ PyDict_SetItemString(proc->globals, kargs, vargs) < 0)
+ PLy_elog(ERROR, "could not set up arguments for PL/Python function");
PG_TRY();
{
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index e499ed8cbfd..a6490d99871 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -104,6 +104,9 @@ PLy_input_convert(PLyDatumToOb *arg, Datum val)
MemoryContextSwitchTo(oldcontext);
+ if (result == NULL)
+ PLy_elog(ERROR, "could not convert value to a Python object");
+
return result;
}
@@ -149,6 +152,9 @@ PLy_input_from_tuple(PLyDatumToOb *arg, HeapTuple tuple, TupleDesc desc, bool in
MemoryContextSwitchTo(oldcontext);
+ if (dict == NULL)
+ PLy_elog(ERROR, "could not convert row to a Python dictionary");
+
return dict;
}
@@ -714,63 +720,79 @@ PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
if (!list)
return NULL;
- if (dim < ndim - 1)
+ /*
+ * Filling the list can now throw, and nothing else refers to it yet, so
+ * release it here rather than leaking it.
+ */
+ PG_TRY();
{
- /* Outer dimension. Recurse for each inner slice. */
- for (i = 0; i < dims[dim]; i++)
+ if (dim < ndim - 1)
{
- PyObject *sublist;
+ /* Outer dimension. Recurse for each inner slice. */
+ for (i = 0; i < dims[dim]; i++)
+ {
+ PyObject *sublist;
- sublist = PLyList_FromArray_recurse(elm, dims, ndim, dim + 1,
- dataptr_p, bitmap_p, bitmask_p);
- PyList_SetItem(list, i, sublist);
+ sublist = PLyList_FromArray_recurse(elm, dims, ndim, dim + 1,
+ dataptr_p, bitmap_p, bitmask_p);
+ PLyList_SetItem(list, i, sublist,
+ "could not convert array dimension to a Python list");
+ }
}
- }
- else
- {
- /*
- * Innermost dimension. Fill the list with the values from the array
- * for this slice.
- */
- char *dataptr = *dataptr_p;
- uint8 *bitmap = *bitmap_p;
- int bitmask = *bitmask_p;
- uint8 typalignby = typalign_to_alignby(elm->typalign);
-
- for (i = 0; i < dims[dim]; i++)
+ else
{
- /* checking for NULL */
- if (bitmap && (*bitmap & bitmask) == 0)
- {
- Py_INCREF(Py_None);
- PyList_SetItem(list, i, Py_None);
- }
- else
+ /*
+ * Innermost dimension. Fill the list with the values from the
+ * array for this slice.
+ */
+ char *dataptr = *dataptr_p;
+ uint8 *bitmap = *bitmap_p;
+ int bitmask = *bitmask_p;
+ uint8 typalignby = typalign_to_alignby(elm->typalign);
+
+ for (i = 0; i < dims[dim]; i++)
{
- Datum itemvalue;
+ /* checking for NULL */
+ if (bitmap && (*bitmap & bitmask) == 0)
+ {
+ Py_INCREF(Py_None);
+ PLyList_SetItem(list, i, Py_None,
+ "could not add array element to a Python list");
+ }
+ else
+ {
+ Datum itemvalue;
- itemvalue = fetch_att(dataptr, elm->typbyval, elm->typlen);
- PyList_SetItem(list, i, elm->func(elm, itemvalue));
- dataptr = att_addlength_pointer(dataptr, elm->typlen, dataptr);
- dataptr = (char *) att_nominal_alignby(dataptr, typalignby);
- }
+ itemvalue = fetch_att(dataptr, elm->typbyval, elm->typlen);
+ PLyList_SetItem(list, i, elm->func(elm, itemvalue),
+ "could not convert array element to a Python object");
+ dataptr = att_addlength_pointer(dataptr, elm->typlen, dataptr);
+ dataptr = (char *) att_nominal_alignby(dataptr, typalignby);
+ }
- /* advance bitmap pointer if any */
- if (bitmap)
- {
- bitmask <<= 1;
- if (bitmask == 0x100 /* (1<<8) */ )
+ /* advance bitmap pointer if any */
+ if (bitmap)
{
- bitmap++;
- bitmask = 1;
+ bitmask <<= 1;
+ if (bitmask == 0x100 /* (1<<8) */ )
+ {
+ bitmap++;
+ bitmask = 1;
+ }
}
}
- }
- *dataptr_p = dataptr;
- *bitmap_p = bitmap;
- *bitmask_p = bitmask;
+ *dataptr_p = dataptr;
+ *bitmap_p = bitmap;
+ *bitmask_p = bitmask;
+ }
+ }
+ PG_CATCH();
+ {
+ Py_DECREF(list);
+ PG_RE_THROW();
}
+ PG_END_TRY();
return list;
}
@@ -835,7 +857,6 @@ PLyDict_FromTuple(PLyDatumToOb *arg, HeapTuple tuple, TupleDesc desc, bool inclu
char *key;
Datum vattr;
bool is_null;
- PyObject *value;
if (attr->attisdropped)
continue;
@@ -854,13 +875,13 @@ PLyDict_FromTuple(PLyDatumToOb *arg, HeapTuple tuple, TupleDesc desc, bool inclu
vattr = heap_getattr(tuple, (i + 1), desc, &is_null);
if (is_null)
- PyDict_SetItemString(dict, key, Py_None);
+ PLyDict_SetItemNone(dict, key,
+ "could not add column \"%s\" to Python dictionary",
+ key);
else
- {
- value = att->func(att, vattr);
- PyDict_SetItemString(dict, key, value);
- Py_DECREF(value);
- }
+ PLyDict_SetItemString(dict, key, att->func(att, vattr),
+ "could not convert column \"%s\" to a Python object",
+ key);
}
}
PG_CATCH();
@@ -1039,20 +1060,27 @@ PLyObject_AsString(PyObject *plrv)
/* use repr() for floats, str() is lossy */
PyObject *s = PyObject_Repr(plrv);
+ if (s == NULL)
+ PLy_elog(ERROR, "could not create string representation of Python object");
plrv_bo = PLyUnicode_Bytes(s);
- Py_XDECREF(s);
+ Py_DECREF(s);
}
else
{
PyObject *s = PyObject_Str(plrv);
+ if (s == NULL)
+ PLy_elog(ERROR, "could not create string representation of Python object");
plrv_bo = PLyUnicode_Bytes(s);
- Py_XDECREF(s);
+ Py_DECREF(s);
}
if (!plrv_bo)
PLy_elog(ERROR, "could not create string representation of Python object");
- plrv_sc = pstrdup(PyBytes_AsString(plrv_bo));
+ plrv_sc = PyBytes_AsString(plrv_bo);
+ if (plrv_sc == NULL)
+ PLy_elog(ERROR, "could not create string representation of Python object");
+ plrv_sc = pstrdup(plrv_sc);
plen = PyBytes_Size(plrv_bo);
slen = strlen(plrv_sc);
@@ -1381,11 +1409,21 @@ PLyMapping_ToComposite(PLyObToDatum *arg, TupleDesc desc, PyObject *mapping)
{
value = PyMapping_GetItemString(mapping, key);
if (!value)
+ {
+ /*
+ * A missing key sets KeyError. Another exception, such as
+ * MemoryError, is a real failure.
+ */
+ if (!PyErr_ExceptionMatches(PyExc_KeyError))
+ PLy_elog(ERROR, "could not get value for key \"%s\" from mapping",
+ key);
+ PyErr_Clear();
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_COLUMN),
errmsg("key \"%s\" not found in mapping", key),
errhint("To return null in a column, "
"add the value None to the mapping with the key named after the column.")));
+ }
values[i] = att->func(att, value, &nulls[i], false);
@@ -1536,6 +1574,15 @@ PLyGenericObject_ToComposite(PLyObToDatum *arg, TupleDesc desc, PyObject *object
* interpret "123" as the composite type. See also similar
* heuristic in PLyObject_ToScalar().
*/
+
+ /*
+ * A missing attribute sets AttributeError. Another exception,
+ * such as MemoryError, is a real failure.
+ */
+ if (!PyErr_ExceptionMatches(PyExc_AttributeError))
+ PLy_elog(ERROR, "could not get attribute \"%s\" of Python object",
+ key);
+ PyErr_Clear();
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_COLUMN),
errmsg("attribute \"%s\" does not exist in Python object", key),
diff --git a/src/pl/plpython/plpy_typeio.h b/src/pl/plpython/plpy_typeio.h
index 29258509b5c..82541f2b89d 100644
--- a/src/pl/plpython/plpy_typeio.h
+++ b/src/pl/plpython/plpy_typeio.h
@@ -147,6 +147,10 @@ struct PLyObToDatum
};
+/*
+ * PLy_input_convert and PLy_input_from_tuple throw an error rather than
+ * returning NULL when the Python object cannot be built.
+ */
extern PGDLLEXPORT PyObject *PLy_input_convert(PLyDatumToOb *arg, Datum val);
extern PGDLLEXPORT Datum PLy_output_convert(PLyObToDatum *arg, PyObject *val,
bool *isnull);
diff --git a/src/pl/plpython/plpy_util.c b/src/pl/plpython/plpy_util.c
index c2338decba2..1b8a53f2401 100644
--- a/src/pl/plpython/plpy_util.c
+++ b/src/pl/plpython/plpy_util.c
@@ -117,3 +117,84 @@ PLyUnicode_FromString(const char *s)
{
return PLyUnicode_FromStringAndSize(s, strlen(s));
}
+
+/*
+ * Common part of PLyDict_SetItemString() and PLyDict_SetItemNone().
+ */
+static void
+PLyDict_SetItem_v(PyObject *dict, const char *key, PyObject *value,
+ const char *fmt, va_list ap)
+{
+ if (value != NULL)
+ {
+ int rc = PyDict_SetItemString(dict, key, value);
+
+ /* PyDict_SetItemString does not steal the reference */
+ Py_DECREF(value);
+
+ if (rc == 0)
+ return;
+ }
+
+ PLy_elog_impl_v(ERROR, fmt, ap);
+ pg_unreachable();
+}
+
+/*
+ * Insert value into dict under key, consuming the caller's reference.
+ *
+ * value is a new reference owned by the caller, or NULL with a Python
+ * exception set. Py_None is not special here: the caller must own a
+ * reference to it, or use PLyDict_SetItemNone(). A failed conversion and a
+ * failed insert are both reported under fmt.
+ */
+void
+PLyDict_SetItemString(PyObject *dict, const char *key, PyObject *value,
+ const char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ PLyDict_SetItem_v(dict, key, value, fmt, ap);
+ va_end(ap);
+}
+
+/*
+ * Insert the borrowed Py_None singleton into dict under key.
+ */
+void
+PLyDict_SetItemNone(PyObject *dict, const char *key, const char *fmt, ...)
+{
+ va_list ap;
+
+ Py_INCREF(Py_None);
+
+ va_start(ap, fmt);
+ PLyDict_SetItem_v(dict, key, Py_None, fmt, ap);
+ va_end(ap);
+}
+
+/*
+ * Store value at index i of list, consuming the caller's reference.
+ *
+ * value is a new reference owned by the caller, or NULL with a Python
+ * exception set. Py_None is not special here: the caller must own a
+ * reference to it. listobject.h says PyList_SetItem decrements the new
+ * reference even when the item is not inserted, so a failed store must not
+ * release it again. A failed conversion and a failed store are both
+ * reported under fmt.
+ */
+void
+PLyList_SetItem(PyObject *list, Py_ssize_t i, PyObject *value,
+ const char *fmt, ...)
+{
+ va_list ap;
+
+ if (value != NULL && PyList_SetItem(list, i, value) == 0)
+ return;
+
+ va_start(ap, fmt);
+ PLy_elog_impl_v(ERROR, fmt, ap);
+ va_end(ap);
+ pg_unreachable();
+}
diff --git a/src/pl/plpython/plpy_util.h b/src/pl/plpython/plpy_util.h
index 6f491b0f95b..4665b123eae 100644
--- a/src/pl/plpython/plpy_util.h
+++ b/src/pl/plpython/plpy_util.h
@@ -14,4 +14,12 @@ extern PGDLLEXPORT char *PLyUnicode_AsString(PyObject *unicode);
extern PGDLLEXPORT PyObject *PLyUnicode_FromString(const char *s);
extern PGDLLEXPORT PyObject *PLyUnicode_FromStringAndSize(const char *s, Py_ssize_t size);
+extern void PLyDict_SetItemString(PyObject *dict, const char *key,
+ PyObject *value,
+ const char *fmt, ...) pg_attribute_printf(4, 5);
+extern void PLyDict_SetItemNone(PyObject *dict, const char *key,
+ const char *fmt, ...) pg_attribute_printf(3, 4);
+extern void PLyList_SetItem(PyObject *list, Py_ssize_t i, PyObject *value,
+ const char *fmt, ...) pg_attribute_printf(4, 5);
+
#endif /* PLPY_UTIL_H */
diff --git a/src/pl/plpython/sql/plpython_record.sql b/src/pl/plpython/sql/plpython_record.sql
index bc1e7d54c8a..a07b4264c41 100644
--- a/src/pl/plpython/sql/plpython_record.sql
+++ b/src/pl/plpython/sql/plpython_record.sql
@@ -178,3 +178,25 @@ CREATE FUNCTION test_type_record_error4() RETURNS type_record AS $$
$$ LANGUAGE plpython3u;
SELECT * FROM test_type_record_error4();
+
+-- a mapping that fails for a reason other than a missing key
+CREATE FUNCTION test_type_record_error5() RETURNS type_record AS $$
+ class broken_mapping(dict):
+ def __getitem__(self, key):
+ raise TypeError('broken mapping')
+ return broken_mapping(first='first', second=1)
+$$ LANGUAGE plpython3u;
+
+SELECT * FROM test_type_record_error5();
+
+-- an object whose attribute lookup fails for a reason other than a
+-- missing attribute
+CREATE FUNCTION test_type_record_error6() RETURNS type_record AS $$
+ class broken_object:
+ first = 'first'
+ def __getattr__(self, name):
+ raise ValueError('broken attribute')
+ return broken_object()
+$$ LANGUAGE plpython3u;
+
+SELECT * FROM test_type_record_error6();
--
2.53.0
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #19899: PL/Python backend crashes (SIGSEGV) when converting a column value to a Python object fails
@ 2026-11-19 05:09 shihao zhong <zhong950419@gmail.com>
parent: Andrey Rachitskiy <pl0h0yp1@gmail.com>
0 siblings, 0 replies; 7+ messages in thread
From: shihao zhong @ 2026-11-19 05:09 UTC (permalink / raw)
To: Andrey Rachitskiy <pl0h0yp1@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; marcos.castedo@anachronics.com; pgsql-bugs@lists.postgresql.org
Hi Andrey,
Thanks for your patches.
> In 0002 I fixed other problematic spots in pl_python.
Thanks for the v3. I ran the same allocation failure tests and it
doesn't crash. It also fixes a crash in PLy_procedure_compile() that my
v2 missed.
Thanks,
Shihao
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2026-11-19 05:09 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-11-12 23:50 BUG #19899: PL/Python backend crashes (SIGSEGV) when converting a column value to a Python object fails PG Bug reporting form <noreply@postgresql.org>
2026-11-16 17:53 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-11-16 19:10 ` Andrey Rachitskiy <pl0h0yp1@gmail.com>
2026-11-16 19:44 ` Robert Haas <robertmhaas@gmail.com>
2026-11-17 00:55 ` shihao zhong <zhong950419@gmail.com>
2026-11-17 07:39 ` Andrey Rachitskiy <pl0h0yp1@gmail.com>
2026-11-19 05:09 ` shihao zhong <zhong950419@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox