pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: Fix memory leak in plpgsql's CALL processing.
7+ messages / 2 participants
[nested] [flat]
Commits from this thread:
2020-12-28 5f2e09bccc 63d78d106d d37965965a 7966b41ded Further fix thinko in plpgsql memory leak fix.
2020-12-28 2e15f48d97 fd1347f9d8 ea80d8d943 0ea25ed108 Fix thinko in plpgsql memory leak fix.
* pgsql: Fix memory leak in plpgsql's CALL processing.
@ 2020-09-29 15:18 Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 7+ messages in thread
From: Tom Lane @ 2020-09-29 15:18 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Fix memory leak in plpgsql's CALL processing.
When executing a CALL or DO in a non-atomic context (i.e., not inside
a function or query), plpgsql creates a new plan each time through,
as a rather hacky solution to some resource management issues. But
it failed to free this plan until exit of the current procedure or DO
block, resulting in serious memory bloat in procedures that called
other procedures many times. Fix by remembering to free the plan,
and by being more honest about restoring the previous state (otherwise,
recursive procedure calls have a problem).
There was also a smaller leak associated with recalculation of the
"target" list of output variables. Fix that by using the statement-
lifespan context to hold non-permanent values.
Back-patch to v11 where procedures were introduced.
Pavel Stehule and Tom Lane
Discussion: https://postgr.es/m/CAFj8pRDiiU1dqym+_P4_GuTWm76knJu7z9opWayBJTC0nQGUUA@mail.gmail.com
Branch
------
REL_13_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/f0e4ec74e452f55922b52f50da4ba4834771a268
Modified Files
--------------
src/pl/plpgsql/src/pl_exec.c | 91 ++++++++++++++++++++++++++++++++++----------
1 file changed, 70 insertions(+), 21 deletions(-)
^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix memory leak in plpgsql's CALL processing.
@ 2020-09-29 15:18 Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 7+ messages in thread
From: Tom Lane @ 2020-09-29 15:18 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Fix memory leak in plpgsql's CALL processing.
When executing a CALL or DO in a non-atomic context (i.e., not inside
a function or query), plpgsql creates a new plan each time through,
as a rather hacky solution to some resource management issues. But
it failed to free this plan until exit of the current procedure or DO
block, resulting in serious memory bloat in procedures that called
other procedures many times. Fix by remembering to free the plan,
and by being more honest about restoring the previous state (otherwise,
recursive procedure calls have a problem).
There was also a smaller leak associated with recalculation of the
"target" list of output variables. Fix that by using the statement-
lifespan context to hold non-permanent values.
Back-patch to v11 where procedures were introduced.
Pavel Stehule and Tom Lane
Discussion: https://postgr.es/m/CAFj8pRDiiU1dqym+_P4_GuTWm76knJu7z9opWayBJTC0nQGUUA@mail.gmail.com
Branch
------
REL_12_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/c1e044bb30f26edf54b7d5f668d18b69a0bb3ea6
Modified Files
--------------
src/pl/plpgsql/src/pl_exec.c | 91 ++++++++++++++++++++++++++++++++++----------
1 file changed, 70 insertions(+), 21 deletions(-)
^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix memory leak in plpgsql's CALL processing.
@ 2020-09-29 15:18 Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 7+ messages in thread
From: Tom Lane @ 2020-09-29 15:18 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Fix memory leak in plpgsql's CALL processing.
When executing a CALL or DO in a non-atomic context (i.e., not inside
a function or query), plpgsql creates a new plan each time through,
as a rather hacky solution to some resource management issues. But
it failed to free this plan until exit of the current procedure or DO
block, resulting in serious memory bloat in procedures that called
other procedures many times. Fix by remembering to free the plan,
and by being more honest about restoring the previous state (otherwise,
recursive procedure calls have a problem).
There was also a smaller leak associated with recalculation of the
"target" list of output variables. Fix that by using the statement-
lifespan context to hold non-permanent values.
Back-patch to v11 where procedures were introduced.
Pavel Stehule and Tom Lane
Discussion: https://postgr.es/m/CAFj8pRDiiU1dqym+_P4_GuTWm76knJu7z9opWayBJTC0nQGUUA@mail.gmail.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/a6b1f5365d58356b5d42829e9cd89a6c725d7a0a
Modified Files
--------------
src/pl/plpgsql/src/pl_exec.c | 91 ++++++++++++++++++++++++++++++++++----------
1 file changed, 70 insertions(+), 21 deletions(-)
^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix memory leak in plpgsql's CALL processing.
@ 2020-09-29 15:18 Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 7+ messages in thread
From: Tom Lane @ 2020-09-29 15:18 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Fix memory leak in plpgsql's CALL processing.
When executing a CALL or DO in a non-atomic context (i.e., not inside
a function or query), plpgsql creates a new plan each time through,
as a rather hacky solution to some resource management issues. But
it failed to free this plan until exit of the current procedure or DO
block, resulting in serious memory bloat in procedures that called
other procedures many times. Fix by remembering to free the plan,
and by being more honest about restoring the previous state (otherwise,
recursive procedure calls have a problem).
There was also a smaller leak associated with recalculation of the
"target" list of output variables. Fix that by using the statement-
lifespan context to hold non-permanent values.
Back-patch to v11 where procedures were introduced.
Pavel Stehule and Tom Lane
Discussion: https://postgr.es/m/CAFj8pRDiiU1dqym+_P4_GuTWm76knJu7z9opWayBJTC0nQGUUA@mail.gmail.com
Branch
------
REL_11_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/13a1901bad47f194f062fe2e699da16a2d6adbd6
Modified Files
--------------
src/pl/plpgsql/src/pl_exec.c | 91 ++++++++++++++++++++++++++++++++++----------
1 file changed, 70 insertions(+), 21 deletions(-)
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: pgsql: Fix memory leak in plpgsql's CALL processing.
@ 2020-12-28 09:47 Peter Eisentraut <peter.eisentraut@enterprisedb.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 7+ messages in thread
From: Peter Eisentraut @ 2020-12-28 09:47 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers <pgsql-hackers@postgresql.org>
This change has broken the following test case (reduced from actual
production code):
CREATE OR REPLACE PROCEDURE prc_inout_test_inner(INOUT x integer)
LANGUAGE plpgsql
AS
$procedure$
BEGIN
COMMIT;
x := 2;
END;
$procedure$;
CREATE OR REPLACE PROCEDURE prc_inout_test_main()
LANGUAGE plpgsql
AS
$procedure$
DECLARE
y INTEGER := 1;
BEGIN
CALL prc_inout_test_inner(y);
END;
$procedure$;
CALL prc_inout_test_main();
With the patch, it fails like this:
ERROR: XX000: unsupported target type: 0
CONTEXT: PL/pgSQL function prc_inout_test_main() line 5 at CALL
LOCATION: exec_move_row_from_fields, pl_exec.c:7196
On 2020-09-29 17:18, Tom Lane wrote:
> Fix memory leak in plpgsql's CALL processing.
>
> When executing a CALL or DO in a non-atomic context (i.e., not inside
> a function or query), plpgsql creates a new plan each time through,
> as a rather hacky solution to some resource management issues. But
> it failed to free this plan until exit of the current procedure or DO
> block, resulting in serious memory bloat in procedures that called
> other procedures many times. Fix by remembering to free the plan,
> and by being more honest about restoring the previous state (otherwise,
> recursive procedure calls have a problem).
>
> There was also a smaller leak associated with recalculation of the
> "target" list of output variables. Fix that by using the statement-
> lifespan context to hold non-permanent values.
>
> Back-patch to v11 where procedures were introduced.
>
> Pavel Stehule and Tom Lane
>
> Discussion: https://postgr.es/m/CAFj8pRDiiU1dqym+_P4_GuTWm76knJu7z9opWayBJTC0nQGUUA@mail.gmail.com
>
> Branch
> ------
> master
>
> Details
> -------
> https://git.postgresql.org/pg/commitdiff/a6b1f5365d58356b5d42829e9cd89a6c725d7a0a
>
> Modified Files
> --------------
> src/pl/plpgsql/src/pl_exec.c | 91 ++++++++++++++++++++++++++++++++++----------
> 1 file changed, 70 insertions(+), 21 deletions(-)
>
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: pgsql: Fix memory leak in plpgsql's CALL processing.
@ 2020-12-28 15:11 Tom Lane <tgl@sss.pgh.pa.us>
parent: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
0 siblings, 1 reply; 7+ messages in thread
From: Tom Lane @ 2020-12-28 15:11 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; +Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
> This change has broken the following test case (reduced from actual
> production code):
Thanks for the report. What I see here is
ERROR: unsupported target type: 2139062143
so it looks like a memory clobber --- probably some data structure is
being built in the wrong memory context. Will look closer shortly.
regards, tom lane
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: pgsql: Fix memory leak in plpgsql's CALL processing.
@ 2020-12-28 16:42 Tom Lane <tgl@sss.pgh.pa.us>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 7+ messages in thread
From: Tom Lane @ 2020-12-28 16:42 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; +Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
I wrote:
> so it looks like a memory clobber --- probably some data structure is
> being built in the wrong memory context. Will look closer shortly.
Brain-dead typo ... where's my brown paper bag?
regards, tom lane
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2020-12-28 16:42 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-09-29 15:18 pgsql: Fix memory leak in plpgsql's CALL processing. Tom Lane <tgl@sss.pgh.pa.us>
2020-12-28 09:47 ` Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2020-12-28 15:11 ` Tom Lane <tgl@sss.pgh.pa.us>
2020-12-28 16:42 ` Tom Lane <tgl@sss.pgh.pa.us>
2020-09-29 15:18 pgsql: Fix memory leak in plpgsql's CALL processing. Tom Lane <tgl@sss.pgh.pa.us>
2020-09-29 15:18 pgsql: Fix memory leak in plpgsql's CALL processing. Tom Lane <tgl@sss.pgh.pa.us>
2020-09-29 15:18 pgsql: Fix memory leak in plpgsql's CALL processing. Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox