postgres.git / summary / log / commit / refs

commit    0fcb8e2e0154dedea5c3c7da6dd2cffb731aac06
Author:   Tom Lane <tgl@sss.pgh.pa.us>
Date:     Mon May 10 15:02:30 2021 +0000
First released in: 14.0, 9.6.22, 10.17, 11.12, 12.7, 13.3

    Fix mishandling of resjunk columns in ON CONFLICT ... UPDATE tlists.
    
    It's unusual to have any resjunk columns in an ON CONFLICT ... UPDATE
    list, but it can happen when MULTIEXPR_SUBLINK SubPlans are present.
    If it happens, the ON CONFLICT UPDATE code path would end up storing
    tuples that include the values of the extra resjunk columns.  That's
    fairly harmless in the short run, but if new columns are added to
    the table then the values would become accessible, possibly leading
    to malfunctions if they don't match the datatypes of the new columns.
    
    This had escaped notice through a confluence of missing sanity checks,
    including
    
    * There's no cross-check that a tuple presented to heap_insert or
    heap_update matches the table rowtype.  While it's difficult to
    check that fully at reasonable cost, we can easily add assertions
    that there aren't too many columns.
    
    * The output-column-assignment cases in execExprInterp.c lacked
    any sanity checks on the output column numbers, which seems like
    an oversight considering there are plenty of assertion checks on
    input column numbers.  Add assertions there too.
    
    * We failed to apply nodeModifyTable's ExecCheckPlanOutput() to
    the ON CONFLICT UPDATE tlist.  That wouldn't have caught this
    specific error, since that function is chartered to ignore resjunk
    columns; but it sure seems like a bad omission now that we've seen
    this bug.
    
    In HEAD, the right way to fix this is to make the processing of
    ON CONFLICT UPDATE tlists work the same as regular UPDATE tlists
    now do, that is don't add "SET x = x" entries, and use
    ExecBuildUpdateProjection to evaluate the tlist and combine it with
    old values of the not-set columns.  This adds a little complication
    to ExecBuildUpdateProjection, but allows removal of a comparable
    amount of now-dead code from the planner.
    
    In the back branches, the most expedient solution seems to be to
    (a) use an output slot for the ON CONFLICT UPDATE projection that
    actually matches the target table, and then (b) invent a variant of
    ExecBuildProjectionInfo that can be told to not store values resulting
    from resjunk columns, so it doesn't try to store into nonexistent
    columns of the output slot.  (We can't simply ignore the resjunk columns
    altogether; they have to be evaluated for MULTIEXPR_SUBLINK to work.)
    This works back to v10.  In 9.6, projections work much differently and
    we can't cheaply give them such an option.  The 9.6 version of this
    patch works by inserting a JunkFilter when it's necessary to get rid
    of resjunk columns.
    
    In addition, v11 and up have the reverse problem when trying to
    perform ON CONFLICT UPDATE on a partitioned table.  Through a
    further oversight, adjust_partition_tlist() discarded resjunk columns
    when re-ordering the ON CONFLICT UPDATE tlist to match a partition.
    This accidentally prevented the storing-bogus-tuples problem, but
    at the cost that MULTIEXPR_SUBLINK cases didn't work, typically
    crashing if more than one row has to be updated.  Fix by preserving
    resjunk columns in that routine.  (I failed to resist the temptation
    to add more assertions there too, and to do some minor code
    beautification.)
    
    Per report from Andres Freund.  Back-patch to all supported branches.
    
    Security: CVE-2021-32028


src/backend/access/heap/heapam.c | 8 +++++ src/backend/executor/nodeModifyTable.c | 58 ++++++++++++++++++++++++++++------ src/include/nodes/execnodes.h | 1 + src/test/regress/expected/update.out | 12 ++++--- src/test/regress/sql/update.sql | 6 ++-- 5 files changed, 68 insertions(+), 17 deletions(-) diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index ff3bef8abfb..79dfd1224b0 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -2389,6 +2389,10 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid, Buffer vmbuffer = InvalidBuffer; bool all_visible_cleared = false; + /* Cheap, simplistic check that the tuple matches the rel's rowtype. */ + Assert(HeapTupleHeaderGetNatts(tup->t_data) <= + RelationGetNumberOfAttributes(relation)); + /* * Fill in tuple header fields, assign an OID, and toast the tuple if * necessary. @@ -3490,6 +3494,10 @@ heap_update(Relation relation, ItemPointer otid, HeapTuple newtup, Assert(ItemPointerIsValid(otid)); + /* Cheap, simplistic check that the tuple matches the rel's rowtype. */ + Assert(HeapTupleHeaderGetNatts(newtup->t_data) <= + RelationGetNumberOfAttributes(relation)); + /* * Forbid this during a parallel operation, lest it allocate a combocid. * Other workers might need that combocid for visibility checks, and we diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 91198924a0d..f9c4bd25199 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -1233,6 +1233,10 @@ ExecOnConflictUpdate(ModifyTableState *mtstate, /* Project the new tuple version */ ExecProject(resultRelInfo->ri_onConflictSetProj, NULL); + if (mtstate->mt_confljunk) + (void) ExecFilterJunk(mtstate->mt_confljunk, + resultRelInfo->ri_onConflictSetProj->pi_slot); + /* * Note that it is possible that the target tuple has been modified in * this session, after the above heap_lock_tuple. We choose to not error @@ -1744,7 +1748,6 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) { ExprContext *econtext; ExprState *setexpr; - TupleDesc tupDesc; /* insert may only have one plan, inheritance is not expanded */ Assert(nplans == 1); @@ -1764,17 +1767,54 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) mtstate->mt_excludedtlist = node->exclRelTlist; /* create target slot for UPDATE SET projection */ - tupDesc = ExecTypeFromTL((List *) node->onConflictSet, - resultRelInfo->ri_RelationDesc->rd_rel->relhasoids); mtstate->mt_conflproj = ExecInitExtraTupleSlot(mtstate->ps.state); - ExecSetSlotDescriptor(mtstate->mt_conflproj, tupDesc); + ExecSetSlotDescriptor(mtstate->mt_conflproj, + resultRelInfo->ri_RelationDesc->rd_att); - /* build UPDATE SET expression and projection state */ + /* initialize UPDATE SET tlist expressions */ setexpr = ExecInitExpr((Expr *) node->onConflictSet, &mtstate->ps); - resultRelInfo->ri_onConflictSetProj = - ExecBuildProjectionInfo((List *) setexpr, econtext, - mtstate->mt_conflproj, - resultRelInfo->ri_RelationDesc->rd_att); + + /* + * The onConflictSet tlist should already have been adjusted to emit + * the table's exact column list. + */ + ExecCheckPlanOutput(resultRelInfo->ri_RelationDesc, + node->onConflictSet); + + /* + * However, it might also contain resjunk columns, in which case we'll + * need a junkfilter to get rid of those. + */ + if (ExecCleanTargetListLength(node->onConflictSet) == + list_length(node->onConflictSet)) + { + /* No junk columns, so we'll just project into mt_conflproj. */ + resultRelInfo->ri_onConflictSetProj = + ExecBuildProjectionInfo((List *) setexpr, econtext, + mtstate->mt_conflproj, + resultRelInfo->ri_RelationDesc->rd_att); + mtstate->mt_confljunk = NULL; + } + else + { + /* + * Project into a slot matching the tlist's output rowtype, then + * apply a junkfilter. + */ + TupleDesc tupDesc = ExecTypeFromTL(node->onConflictSet, false); + TupleTableSlot *ocsSlot; + + ocsSlot = ExecInitExtraTupleSlot(mtstate->ps.state); + ExecSetSlotDescriptor(ocsSlot, tupDesc); + resultRelInfo->ri_onConflictSetProj = + ExecBuildProjectionInfo((List *) setexpr, econtext, + ocsSlot, + resultRelInfo->ri_RelationDesc->rd_att); + mtstate->mt_confljunk = + ExecInitJunkFilter(node->onConflictSet, + resultRelInfo->ri_RelationDesc->rd_att->tdhasoid, + mtstate->mt_conflproj); + } /* build DO UPDATE WHERE clause expression */ if (node->onConflictWhere) diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index d7a376e8357..d8f0f90b1a6 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -1141,6 +1141,7 @@ typedef struct ModifyTableState * tlist */ TupleTableSlot *mt_conflproj; /* CONFLICT ... SET ... projection * target */ + JunkFilter *mt_confljunk; /* CONFLICT ... SET might need a junkfilter */ } ModifyTableState; /* ---------------- diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out index cb2743fe079..8130fe10725 100644 --- a/src/test/regress/expected/update.out +++ b/src/test/regress/expected/update.out @@ -201,7 +201,7 @@ SELECT a, b, char_length(c) FROM update_test; (4 rows) -- Test ON CONFLICT DO UPDATE -INSERT INTO upsert_test VALUES(1, 'Boo'); +INSERT INTO upsert_test VALUES(1, 'Boo'), (3, 'Zoo'); -- uncorrelated sub-select: WITH aaa AS (SELECT 1 AS a, 'Foo' AS b) INSERT INTO upsert_test VALUES (1, 'Bar') ON CONFLICT(a) @@ -212,22 +212,24 @@ WITH aaa AS (SELECT 1 AS a, 'Foo' AS b) INSERT INTO upsert_test (1 row) -- correlated sub-select: -INSERT INTO upsert_test VALUES (1, 'Baz') ON CONFLICT(a) +INSERT INTO upsert_test VALUES (1, 'Baz'), (3, 'Zaz') ON CONFLICT(a) DO UPDATE SET (b, a) = (SELECT b || ', Correlated', a from upsert_test i WHERE i.a = upsert_test.a) RETURNING *; a | b ---+----------------- 1 | Foo, Correlated -(1 row) + 3 | Zoo, Correlated +(2 rows) -- correlated sub-select (EXCLUDED.* alias): -INSERT INTO upsert_test VALUES (1, 'Bat') ON CONFLICT(a) +INSERT INTO upsert_test VALUES (1, 'Bat'), (3, 'Zot') ON CONFLICT(a) DO UPDATE SET (b, a) = (SELECT b || ', Excluded', a from upsert_test i WHERE i.a = excluded.a) RETURNING *; a | b ---+--------------------------- 1 | Foo, Correlated, Excluded -(1 row) + 3 | Zoo, Correlated, Excluded +(2 rows) DROP TABLE update_test; DROP TABLE upsert_test; diff --git a/src/test/regress/sql/update.sql b/src/test/regress/sql/update.sql index adfe2da4dae..ee57703ea1a 100644 --- a/src/test/regress/sql/update.sql +++ b/src/test/regress/sql/update.sql @@ -99,17 +99,17 @@ UPDATE update_test t SELECT a, b, char_length(c) FROM update_test; -- Test ON CONFLICT DO UPDATE -INSERT INTO upsert_test VALUES(1, 'Boo'); +INSERT INTO upsert_test VALUES(1, 'Boo'), (3, 'Zoo'); -- uncorrelated sub-select: WITH aaa AS (SELECT 1 AS a, 'Foo' AS b) INSERT INTO upsert_test VALUES (1, 'Bar') ON CONFLICT(a) DO UPDATE SET (b, a) = (SELECT b, a FROM aaa) RETURNING *; -- correlated sub-select: -INSERT INTO upsert_test VALUES (1, 'Baz') ON CONFLICT(a) +INSERT INTO upsert_test VALUES (1, 'Baz'), (3, 'Zaz') ON CONFLICT(a) DO UPDATE SET (b, a) = (SELECT b || ', Correlated', a from upsert_test i WHERE i.a = upsert_test.a) RETURNING *; -- correlated sub-select (EXCLUDED.* alias): -INSERT INTO upsert_test VALUES (1, 'Bat') ON CONFLICT(a) +INSERT INTO upsert_test VALUES (1, 'Bat'), (3, 'Zot') ON CONFLICT(a) DO UPDATE SET (b, a) = (SELECT b || ', Excluded', a from upsert_test i WHERE i.a = excluded.a) RETURNING *; [parent: 0c1caa48d3cc]