pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Use per-tuple context in ExecGetAllUpdatedCols 5+ messages / 1 participants [nested] [flat]
* pgsql: Use per-tuple context in ExecGetAllUpdatedCols @ 2023-06-07 16:55 Tomas Vondra <tomas.vondra@postgresql.org> 0 siblings, 0 replies; 5+ messages in thread From: Tomas Vondra @ 2023-06-07 16:55 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Use per-tuple context in ExecGetAllUpdatedCols Commit fc22b6623b (generated columns) replaced ExecGetUpdatedCols() with ExecGetAllUpdatedCols() in a couple places handling UPDATE (triggers and lock mode). However, ExecGetUpdatedCols() did exec_rt_fetch() while ExecGetAllUpdatedCols() also allocates memory through bms_union() without paying attention to the memory context and happened to use the long-lived ExecutorState, leaking the memory until the end of the query. The amount of leaked memory is proportional to the number of (updated) attributes, types of UPDATE triggers, and the number of processed rows (which for UPDATE ... FROM ... may be much higher than updated rows). Fixed by switching to the per-tuple context in GetAllUpdatedColumns(). This is fine for all in-core callers, but external callers may need to copy the result. But we're not aware of any such callers. Note the issue was introduced by fc22b6623b, but the macros were later renamed by f50e888990. Backpatch to 12, where the issue was introduced. Reported-by: Tomas Vondra Reviewed-by: Andres Freund, Tom Lane, Jakub Wartak Backpatch-through: 12 Discussion: https://postgr.es/m/222a3442-7f7d-246c-ed9b-a76209d19239@enterprisedb.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/f24523672de9c37423800a74cffa7aec27146047 Modified Files -------------- src/backend/executor/execUtils.c | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Use per-tuple context in ExecGetAllUpdatedCols @ 2023-06-07 16:56 Tomas Vondra <tomas.vondra@postgresql.org> 0 siblings, 0 replies; 5+ messages in thread From: Tomas Vondra @ 2023-06-07 16:56 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Use per-tuple context in ExecGetAllUpdatedCols Commit fc22b6623b (generated columns) replaced ExecGetUpdatedCols() with ExecGetAllUpdatedCols() in a couple places handling UPDATE (triggers and lock mode). However, ExecGetUpdatedCols() did exec_rt_fetch() while ExecGetAllUpdatedCols() also allocates memory through bms_union() without paying attention to the memory context and happened to use the long-lived ExecutorState, leaking the memory until the end of the query. The amount of leaked memory is proportional to the number of (updated) attributes, types of UPDATE triggers, and the number of processed rows (which for UPDATE ... FROM ... may be much higher than updated rows). Fixed by switching to the per-tuple context in GetAllUpdatedColumns(). This is fine for all in-core callers, but external callers may need to copy the result. But we're not aware of any such callers. Note the issue was introduced by fc22b6623b, but the macros were later renamed by f50e888990. Backpatch to 12, where the issue was introduced. Reported-by: Tomas Vondra Reviewed-by: Andres Freund, Tom Lane, Jakub Wartak Backpatch-through: 12 Discussion: https://postgr.es/m/222a3442-7f7d-246c-ed9b-a76209d19239@enterprisedb.com Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/ee87f8b63aee39673d014e4ee9a1293b0b45d113 Modified Files -------------- src/backend/executor/execUtils.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Use per-tuple context in ExecGetAllUpdatedCols @ 2023-06-07 16:56 Tomas Vondra <tomas.vondra@postgresql.org> 0 siblings, 0 replies; 5+ messages in thread From: Tomas Vondra @ 2023-06-07 16:56 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Use per-tuple context in ExecGetAllUpdatedCols Commit fc22b6623b (generated columns) replaced ExecGetUpdatedCols() with ExecGetAllUpdatedCols() in a couple places handling UPDATE (triggers and lock mode). However, ExecGetUpdatedCols() did exec_rt_fetch() while ExecGetAllUpdatedCols() also allocates memory through bms_union() without paying attention to the memory context and happened to use the long-lived ExecutorState, leaking the memory until the end of the query. The amount of leaked memory is proportional to the number of (updated) attributes, types of UPDATE triggers, and the number of processed rows (which for UPDATE ... FROM ... may be much higher than updated rows). Fixed by switching to the per-tuple context in GetAllUpdatedColumns(). This is fine for all in-core callers, but external callers may need to copy the result. But we're not aware of any such callers. Note the issue was introduced by fc22b6623b, but the macros were later renamed by f50e888990. Backpatch to 12, where the issue was introduced. Reported-by: Tomas Vondra Reviewed-by: Andres Freund, Tom Lane, Jakub Wartak Backpatch-through: 12 Discussion: https://postgr.es/m/222a3442-7f7d-246c-ed9b-a76209d19239@enterprisedb.com Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/7f528e96c501704202b86edaf0ae115fe809ac5b Modified Files -------------- src/backend/executor/execUtils.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Use per-tuple context in ExecGetAllUpdatedCols @ 2023-06-07 16:56 Tomas Vondra <tomas.vondra@postgresql.org> 0 siblings, 0 replies; 5+ messages in thread From: Tomas Vondra @ 2023-06-07 16:56 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Use per-tuple context in ExecGetAllUpdatedCols Commit fc22b6623b (generated columns) replaced ExecGetUpdatedCols() with ExecGetAllUpdatedCols() in a couple places handling UPDATE (triggers and lock mode). However, ExecGetUpdatedCols() did exec_rt_fetch() while ExecGetAllUpdatedCols() also allocates memory through bms_union() without paying attention to the memory context and happened to use the long-lived ExecutorState, leaking the memory until the end of the query. The amount of leaked memory is proportional to the number of (updated) attributes, types of UPDATE triggers, and the number of processed rows (which for UPDATE ... FROM ... may be much higher than updated rows). Fixed by switching to the per-tuple context in GetAllUpdatedColumns(). This is fine for all in-core callers, but external callers may need to copy the result. But we're not aware of any such callers. Note the issue was introduced by fc22b6623b, but the macros were later renamed by f50e888990. Backpatch to 12, where the issue was introduced. Reported-by: Tomas Vondra Reviewed-by: Andres Freund, Tom Lane, Jakub Wartak Backpatch-through: 12 Discussion: https://postgr.es/m/222a3442-7f7d-246c-ed9b-a76209d19239@enterprisedb.com Branch ------ REL_13_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/c504aa85707151392b0e4a6673f200dbd6fb33a3 Modified Files -------------- src/backend/executor/execUtils.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Use per-tuple context in ExecGetAllUpdatedCols @ 2023-06-07 16:56 Tomas Vondra <tomas.vondra@postgresql.org> 0 siblings, 0 replies; 5+ messages in thread From: Tomas Vondra @ 2023-06-07 16:56 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Use per-tuple context in ExecGetAllUpdatedCols Commit fc22b6623b (generated columns) replaced ExecGetUpdatedCols() with ExecGetAllUpdatedCols() in a couple places handling UPDATE (triggers and lock mode). However, ExecGetUpdatedCols() did exec_rt_fetch() while ExecGetAllUpdatedCols() also allocates memory through bms_union() without paying attention to the memory context and happened to use the long-lived ExecutorState, leaking the memory until the end of the query. The amount of leaked memory is proportional to the number of (updated) attributes, types of UPDATE triggers, and the number of processed rows (which for UPDATE ... FROM ... may be much higher than updated rows). Fixed by switching to the per-tuple context in GetAllUpdatedColumns(). This is fine for all in-core callers, but external callers may need to copy the result. But we're not aware of any such callers. Note the issue was introduced by fc22b6623b, but the macros were later renamed by f50e888990. Backpatch to 12, where the issue was introduced. Reported-by: Tomas Vondra Reviewed-by: Andres Freund, Tom Lane, Jakub Wartak Backpatch-through: 12 Discussion: https://postgr.es/m/222a3442-7f7d-246c-ed9b-a76209d19239@enterprisedb.com Branch ------ REL_12_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/54e1b85872340f448a9c41dc2a3ba88a1d649158 Modified Files -------------- src/backend/executor/execUtils.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2023-06-07 16:56 UTC | newest] Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2023-06-07 16:55 pgsql: Use per-tuple context in ExecGetAllUpdatedCols Tomas Vondra <tomas.vondra@postgresql.org> 2023-06-07 16:56 pgsql: Use per-tuple context in ExecGetAllUpdatedCols Tomas Vondra <tomas.vondra@postgresql.org> 2023-06-07 16:56 pgsql: Use per-tuple context in ExecGetAllUpdatedCols Tomas Vondra <tomas.vondra@postgresql.org> 2023-06-07 16:56 pgsql: Use per-tuple context in ExecGetAllUpdatedCols Tomas Vondra <tomas.vondra@postgresql.org> 2023-06-07 16:56 pgsql: Use per-tuple context in ExecGetAllUpdatedCols Tomas Vondra <tomas.vondra@postgresql.org>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox