pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Repair incorrect handling of AfterTriggerSharedData.ats_modified 6+ messages / 1 participants [nested] [flat]
* pgsql: Repair incorrect handling of AfterTriggerSharedData.ats_modified @ 2025-01-22 16:58 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2025-01-22 16:58 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Repair incorrect handling of AfterTriggerSharedData.ats_modifiedcols. This patch fixes two distinct errors that both ultimately trace to commit 71d60e2aa, which added the ats_modifiedcols field. The more severe error is that ats_modifiedcols wasn't accounted for in afterTriggerAddEvent's scanning loop that looks for a pre-existing duplicate AfterTriggerSharedData. Thus, a new event could be incorrectly matched to an AfterTriggerSharedData that has a different value of ats_modifiedcols, resulting in the wrong tg_updatedcols bitmap getting passed to the trigger whenever it finally gets fired. We'd not noticed because (a) few triggers consult tg_updatedcols, and (b) we had no tests exercising a case where such a trigger was called as an AFTER trigger. In the test case added by this commit, contrib/lo's trigger fails to remove a large object when expected because (without this fix) it thinks the LO OID column hasn't changed. The other problem was introduced by commit ce5aaea8c, which copied the modified-columns bitmap into trigger-related storage. It made a copy for every trigger event, whereas what we really want is to make a new copy only when we make a new AfterTriggerSharedData entry. (We could imagine adding extra logic to reduce the number of bitmap copies still more, but it doesn't look worthwhile at the moment.) In a simple test of an UPDATE of 10000000 rows with a single AFTER trigger, this thinko roughly tripled the amount of memory consumed by the pending-triggers data structures, from 160446744 to 480443440 bytes. Fixing the first problem requires introducing a bms_equal() call into afterTriggerAddEvent's scanning loop, which is slightly annoying from a speed perspective. However, getting rid of the excessive bms_copy() calls from the second problem balances that out; overall speed of trigger operations is the same or slightly better, in my tests. Discussion: https://postgr.es/m/3496294.1737501591@sss.pgh.pa.us Backpatch-through: 13 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/ea68ea6320ff84f55cf30dff1af662fc0bf5dafa Modified Files -------------- contrib/lo/expected/lo.out | 69 ++++++++++++++++++++++++++++++++++++++++++ contrib/lo/sql/lo.sql | 41 +++++++++++++++++++++++++ src/backend/commands/trigger.c | 18 +++++------ 3 files changed, 118 insertions(+), 10 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Repair incorrect handling of AfterTriggerSharedData.ats_modified @ 2025-01-22 16:58 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2025-01-22 16:58 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Repair incorrect handling of AfterTriggerSharedData.ats_modifiedcols. This patch fixes two distinct errors that both ultimately trace to commit 71d60e2aa, which added the ats_modifiedcols field. The more severe error is that ats_modifiedcols wasn't accounted for in afterTriggerAddEvent's scanning loop that looks for a pre-existing duplicate AfterTriggerSharedData. Thus, a new event could be incorrectly matched to an AfterTriggerSharedData that has a different value of ats_modifiedcols, resulting in the wrong tg_updatedcols bitmap getting passed to the trigger whenever it finally gets fired. We'd not noticed because (a) few triggers consult tg_updatedcols, and (b) we had no tests exercising a case where such a trigger was called as an AFTER trigger. In the test case added by this commit, contrib/lo's trigger fails to remove a large object when expected because (without this fix) it thinks the LO OID column hasn't changed. The other problem was introduced by commit ce5aaea8c, which copied the modified-columns bitmap into trigger-related storage. It made a copy for every trigger event, whereas what we really want is to make a new copy only when we make a new AfterTriggerSharedData entry. (We could imagine adding extra logic to reduce the number of bitmap copies still more, but it doesn't look worthwhile at the moment.) In a simple test of an UPDATE of 10000000 rows with a single AFTER trigger, this thinko roughly tripled the amount of memory consumed by the pending-triggers data structures, from 160446744 to 480443440 bytes. Fixing the first problem requires introducing a bms_equal() call into afterTriggerAddEvent's scanning loop, which is slightly annoying from a speed perspective. However, getting rid of the excessive bms_copy() calls from the second problem balances that out; overall speed of trigger operations is the same or slightly better, in my tests. Discussion: https://postgr.es/m/3496294.1737501591@sss.pgh.pa.us Backpatch-through: 13 Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/2b72fed2dd741914064422ea510088563e413c2f Modified Files -------------- contrib/lo/expected/lo.out | 69 ++++++++++++++++++++++++++++++++++++++++++ contrib/lo/sql/lo.sql | 40 ++++++++++++++++++++++++ src/backend/commands/trigger.c | 18 +++++------ 3 files changed, 117 insertions(+), 10 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Repair incorrect handling of AfterTriggerSharedData.ats_modified @ 2025-01-22 16:58 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2025-01-22 16:58 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Repair incorrect handling of AfterTriggerSharedData.ats_modifiedcols. This patch fixes two distinct errors that both ultimately trace to commit 71d60e2aa, which added the ats_modifiedcols field. The more severe error is that ats_modifiedcols wasn't accounted for in afterTriggerAddEvent's scanning loop that looks for a pre-existing duplicate AfterTriggerSharedData. Thus, a new event could be incorrectly matched to an AfterTriggerSharedData that has a different value of ats_modifiedcols, resulting in the wrong tg_updatedcols bitmap getting passed to the trigger whenever it finally gets fired. We'd not noticed because (a) few triggers consult tg_updatedcols, and (b) we had no tests exercising a case where such a trigger was called as an AFTER trigger. In the test case added by this commit, contrib/lo's trigger fails to remove a large object when expected because (without this fix) it thinks the LO OID column hasn't changed. The other problem was introduced by commit ce5aaea8c, which copied the modified-columns bitmap into trigger-related storage. It made a copy for every trigger event, whereas what we really want is to make a new copy only when we make a new AfterTriggerSharedData entry. (We could imagine adding extra logic to reduce the number of bitmap copies still more, but it doesn't look worthwhile at the moment.) In a simple test of an UPDATE of 10000000 rows with a single AFTER trigger, this thinko roughly tripled the amount of memory consumed by the pending-triggers data structures, from 160446744 to 480443440 bytes. Fixing the first problem requires introducing a bms_equal() call into afterTriggerAddEvent's scanning loop, which is slightly annoying from a speed perspective. However, getting rid of the excessive bms_copy() calls from the second problem balances that out; overall speed of trigger operations is the same or slightly better, in my tests. Discussion: https://postgr.es/m/3496294.1737501591@sss.pgh.pa.us Backpatch-through: 13 Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/8c57f548534b71b5977ccc7182521acc2dbfddf1 Modified Files -------------- contrib/lo/expected/lo.out | 69 ++++++++++++++++++++++++++++++++++++++++++ contrib/lo/sql/lo.sql | 40 ++++++++++++++++++++++++ src/backend/commands/trigger.c | 18 +++++------ 3 files changed, 117 insertions(+), 10 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Repair incorrect handling of AfterTriggerSharedData.ats_modified @ 2025-01-22 16:58 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2025-01-22 16:58 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Repair incorrect handling of AfterTriggerSharedData.ats_modifiedcols. This patch fixes two distinct errors that both ultimately trace to commit 71d60e2aa, which added the ats_modifiedcols field. The more severe error is that ats_modifiedcols wasn't accounted for in afterTriggerAddEvent's scanning loop that looks for a pre-existing duplicate AfterTriggerSharedData. Thus, a new event could be incorrectly matched to an AfterTriggerSharedData that has a different value of ats_modifiedcols, resulting in the wrong tg_updatedcols bitmap getting passed to the trigger whenever it finally gets fired. We'd not noticed because (a) few triggers consult tg_updatedcols, and (b) we had no tests exercising a case where such a trigger was called as an AFTER trigger. In the test case added by this commit, contrib/lo's trigger fails to remove a large object when expected because (without this fix) it thinks the LO OID column hasn't changed. The other problem was introduced by commit ce5aaea8c, which copied the modified-columns bitmap into trigger-related storage. It made a copy for every trigger event, whereas what we really want is to make a new copy only when we make a new AfterTriggerSharedData entry. (We could imagine adding extra logic to reduce the number of bitmap copies still more, but it doesn't look worthwhile at the moment.) In a simple test of an UPDATE of 10000000 rows with a single AFTER trigger, this thinko roughly tripled the amount of memory consumed by the pending-triggers data structures, from 160446744 to 480443440 bytes. Fixing the first problem requires introducing a bms_equal() call into afterTriggerAddEvent's scanning loop, which is slightly annoying from a speed perspective. However, getting rid of the excessive bms_copy() calls from the second problem balances that out; overall speed of trigger operations is the same or slightly better, in my tests. Discussion: https://postgr.es/m/3496294.1737501591@sss.pgh.pa.us Backpatch-through: 13 Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/cdeed4de7729f455908e571a69dfd60bd6b52a82 Modified Files -------------- contrib/lo/expected/lo.out | 69 ++++++++++++++++++++++++++++++++++++++++++ contrib/lo/sql/lo.sql | 40 ++++++++++++++++++++++++ src/backend/commands/trigger.c | 18 +++++------ 3 files changed, 117 insertions(+), 10 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Repair incorrect handling of AfterTriggerSharedData.ats_modified @ 2025-01-22 16:58 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2025-01-22 16:58 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Repair incorrect handling of AfterTriggerSharedData.ats_modifiedcols. This patch fixes two distinct errors that both ultimately trace to commit 71d60e2aa, which added the ats_modifiedcols field. The more severe error is that ats_modifiedcols wasn't accounted for in afterTriggerAddEvent's scanning loop that looks for a pre-existing duplicate AfterTriggerSharedData. Thus, a new event could be incorrectly matched to an AfterTriggerSharedData that has a different value of ats_modifiedcols, resulting in the wrong tg_updatedcols bitmap getting passed to the trigger whenever it finally gets fired. We'd not noticed because (a) few triggers consult tg_updatedcols, and (b) we had no tests exercising a case where such a trigger was called as an AFTER trigger. In the test case added by this commit, contrib/lo's trigger fails to remove a large object when expected because (without this fix) it thinks the LO OID column hasn't changed. The other problem was introduced by commit ce5aaea8c, which copied the modified-columns bitmap into trigger-related storage. It made a copy for every trigger event, whereas what we really want is to make a new copy only when we make a new AfterTriggerSharedData entry. (We could imagine adding extra logic to reduce the number of bitmap copies still more, but it doesn't look worthwhile at the moment.) In a simple test of an UPDATE of 10000000 rows with a single AFTER trigger, this thinko roughly tripled the amount of memory consumed by the pending-triggers data structures, from 160446744 to 480443440 bytes. Fixing the first problem requires introducing a bms_equal() call into afterTriggerAddEvent's scanning loop, which is slightly annoying from a speed perspective. However, getting rid of the excessive bms_copy() calls from the second problem balances that out; overall speed of trigger operations is the same or slightly better, in my tests. Discussion: https://postgr.es/m/3496294.1737501591@sss.pgh.pa.us Backpatch-through: 13 Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/30859930528b50e55f1a3ea66b6a9a3704c8df62 Modified Files -------------- contrib/lo/expected/lo.out | 69 ++++++++++++++++++++++++++++++++++++++++++ contrib/lo/sql/lo.sql | 40 ++++++++++++++++++++++++ src/backend/commands/trigger.c | 18 +++++------ 3 files changed, 117 insertions(+), 10 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Repair incorrect handling of AfterTriggerSharedData.ats_modified @ 2025-01-22 16:58 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2025-01-22 16:58 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Repair incorrect handling of AfterTriggerSharedData.ats_modifiedcols. This patch fixes two distinct errors that both ultimately trace to commit 71d60e2aa, which added the ats_modifiedcols field. The more severe error is that ats_modifiedcols wasn't accounted for in afterTriggerAddEvent's scanning loop that looks for a pre-existing duplicate AfterTriggerSharedData. Thus, a new event could be incorrectly matched to an AfterTriggerSharedData that has a different value of ats_modifiedcols, resulting in the wrong tg_updatedcols bitmap getting passed to the trigger whenever it finally gets fired. We'd not noticed because (a) few triggers consult tg_updatedcols, and (b) we had no tests exercising a case where such a trigger was called as an AFTER trigger. In the test case added by this commit, contrib/lo's trigger fails to remove a large object when expected because (without this fix) it thinks the LO OID column hasn't changed. The other problem was introduced by commit ce5aaea8c, which copied the modified-columns bitmap into trigger-related storage. It made a copy for every trigger event, whereas what we really want is to make a new copy only when we make a new AfterTriggerSharedData entry. (We could imagine adding extra logic to reduce the number of bitmap copies still more, but it doesn't look worthwhile at the moment.) In a simple test of an UPDATE of 10000000 rows with a single AFTER trigger, this thinko roughly tripled the amount of memory consumed by the pending-triggers data structures, from 160446744 to 480443440 bytes. Fixing the first problem requires introducing a bms_equal() call into afterTriggerAddEvent's scanning loop, which is slightly annoying from a speed perspective. However, getting rid of the excessive bms_copy() calls from the second problem balances that out; overall speed of trigger operations is the same or slightly better, in my tests. Discussion: https://postgr.es/m/3496294.1737501591@sss.pgh.pa.us Backpatch-through: 13 Branch ------ REL_13_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/001c09d8b72ee67ea670a1e9a534fa96948829dd Modified Files -------------- contrib/lo/expected/lo.out | 69 ++++++++++++++++++++++++++++++++++++++++++ contrib/lo/sql/lo.sql | 40 ++++++++++++++++++++++++ src/backend/commands/trigger.c | 18 +++++------ 3 files changed, 117 insertions(+), 10 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2025-01-22 16:58 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2025-01-22 16:58 pgsql: Repair incorrect handling of AfterTriggerSharedData.ats_modified Tom Lane <tgl@sss.pgh.pa.us> 2025-01-22 16:58 pgsql: Repair incorrect handling of AfterTriggerSharedData.ats_modified Tom Lane <tgl@sss.pgh.pa.us> 2025-01-22 16:58 pgsql: Repair incorrect handling of AfterTriggerSharedData.ats_modified Tom Lane <tgl@sss.pgh.pa.us> 2025-01-22 16:58 pgsql: Repair incorrect handling of AfterTriggerSharedData.ats_modified Tom Lane <tgl@sss.pgh.pa.us> 2025-01-22 16:58 pgsql: Repair incorrect handling of AfterTriggerSharedData.ats_modified Tom Lane <tgl@sss.pgh.pa.us> 2025-01-22 16:58 pgsql: Repair incorrect handling of AfterTriggerSharedData.ats_modified Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox