public inbox for [email protected]
help / color / mirror / Atom feedRe: Expanding HOT updates for expression and partial indexes
37+ messages / 5 participants
[nested] [flat]
* Re: Expanding HOT updates for expression and partial indexes
@ 2025-03-05 23:39 Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Matthias van de Meent @ 2025-03-05 23:39 UTC (permalink / raw)
To: Burd, Greg <[email protected]>; +Cc: pgsql-hackers
On Wed, 5 Mar 2025 at 18:21, Burd, Greg <[email protected]> wrote:
>
> Hello,
>
> I've rebased and updated the patch a bit. The biggest change is that the performance penalty measured with v1 of this patch is essentially gone in v10. The overhead was due to re-creating IndexInfo information unnecessarily, which I found existed in the estate. I've added a few fields in IndexInfo that are not populated by default but necessary when checking expression indexes, those fields are populated on demand and only once limiting their overhead.
This review is based on a light reading of patch v10. I have not read
all 90kB, and am unlikely to finish a full review soon:
> * assumes estate->es_result_relations[0] is the ResultRelInfo being updated
I'm not sure that's a valid assumption. I suspect it might be false in
cases of nested updates, like
$ UPDATE table1 SET value = other.value FROM (UPDATE table2 SET value
= 2 ) other WHERE other.id = table1.id;
If this table1 or table2 has expression indexes I suspect it may
result in this assertion failing (but I haven't spun up a server with
the patch).
Alternatively, please also check that it doesn't break if any of these
two tables is partitioned with multiple partitions (and/or has
expression indexes, etc.).
> * uses ri_IndexRelationInfo[] from within estate rather than re-creating it
As I mentioned above, I think it's safer to pass the known-correct RRI
(known by callers of table_tuple_update) down the stack.
> * augments IndexInfo only when needed for testing expressions and only once
ExecExpressionIndexesUpdated seems to always loop over all indexes,
always calling AttributeIndexInfo which always updates the fields in
the IndexInfo when the index has only !byval attributes (e.g. text,
json, or other such varlena types). You say it happens only once, have
I missed something?
I'm also somewhat concerned about the use of typecache lookups on
index->rd_opcintype[i], rather than using
TupleDescCompactAttr(index->rd_att, i); the latter of which I think
should be faster, especially when multiple wide indexes are scanned
with various column types. In hot loops of single-tuple update
statements I think this may make a few 0.1%pt difference - not a lot,
but worth considering.
> * only creates a local old/new TupleTableSlot when not present in estate
I'm not sure it's safe for us to touch that RRI's tupleslots.
> * retains existing summarized index HOT update logic
Great, thanks!
Kind regards,
Matthias van de Meent
Neon (https://neon.tech)
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
@ 2025-03-06 12:40 ` Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Burd, Greg @ 2025-03-06 12:40 UTC (permalink / raw)
To: Matthias van de Meent <[email protected]>; +Cc: pgsql-hackers
> On Mar 5, 2025, at 6:39 PM, Matthias van de Meent <[email protected]> wrote:
>
> On Wed, 5 Mar 2025 at 18:21, Burd, Greg <[email protected]> wrote:
>>
>> Hello,
>>
>> I've rebased and updated the patch a bit. The biggest change is that the performance penalty measured with v1 of this patch is essentially gone in v10. The overhead was due to re-creating IndexInfo information unnecessarily, which I found existed in the estate. I've added a few fields in IndexInfo that are not populated by default but necessary when checking expression indexes, those fields are populated on demand and only once limiting their overhead.
>
> This review is based on a light reading of patch v10. I have not read
> all 90kB, and am unlikely to finish a full review soon:
>
>> * assumes estate->es_result_relations[0] is the ResultRelInfo being updated
>
> I'm not sure that's a valid assumption. I suspect it might be false in
> cases of nested updates, like
>
> $ UPDATE table1 SET value = other.value FROM (UPDATE table2 SET value
> = 2 ) other WHERE other.id = table1.id;
>
> If this table1 or table2 has expression indexes I suspect it may
> result in this assertion failing (but I haven't spun up a server with
> the patch).
> Alternatively, please also check that it doesn't break if any of these
> two tables is partitioned with multiple partitions (and/or has
> expression indexes, etc.).
Valid, and possible. I'll check and find a way to pass along the known-correct RRI index into that array.
>> * uses ri_IndexRelationInfo[] from within estate rather than re-creating it
>
> As I mentioned above, I think it's safer to pass the known-correct RRI
> (known by callers of table_tuple_update) down the stack.
I think passing the known-correct RRI index is the way to go as I need information from both ri_IndexRelationInfo/Desc[] arrays.
>> * augments IndexInfo only when needed for testing expressions and only once
>
> ExecExpressionIndexesUpdated seems to always loop over all indexes,
> always calling AttributeIndexInfo which always updates the fields in
> the IndexInfo when the index has only !byval attributes (e.g. text,
> json, or other such varlena types). You say it happens only once, have
> I missed something?
There's a test that avoids doing it more than once, but I'm going to rename this as BuildExpressionIndexInfo() and call it from ExecOpenIndices() if there are expressions on the index. I think that's cleaner and there's precedent for it in the form of BuildSpeculativeIndexInfo().
> I'm also somewhat concerned about the use of typecache lookups on
> index->rd_opcintype[i], rather than using
> TupleDescCompactAttr(index->rd_att, i); the latter of which I think
> should be faster, especially when multiple wide indexes are scanned
> with various column types. In hot loops of single-tuple update
> statements I think this may make a few 0.1%pt difference - not a lot,
> but worth considering.
I was just working on that. Good idea.
>> * only creates a local old/new TupleTableSlot when not present in estate
>
> I'm not sure it's safe for us to touch that RRI's tupleslots.
Me neither, that's why I mentioned it. It was my attempt to avoid the work to create/destroy temp slots over and over that led to that idea. It's working, but needs more thought.
>> * retains existing summarized index HOT update logic
>
> Great, thanks!
>
> Kind regards,
>
> Matthias van de Meent
> Neon (https://neon.tech)
I might widen this patch a bit to include support for testing equality of index tuples using custom operators when they exist for the index. In the use case I'm solving for we use a custom operator for equality that is not the same as a memcmp(). Do you have thoughts on that? It may be hard to accomplish this as the notion of an equality operator is specific to the index access method and not well-defined outside that AFAICT. If that's the case I'd have to augment the definition of an index access method to provide that information.
-greg
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
@ 2025-03-07 22:47 ` Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Matthias van de Meent @ 2025-03-07 22:47 UTC (permalink / raw)
To: Burd, Greg <[email protected]>; +Cc: pgsql-hackers
On Thu, 6 Mar 2025 at 13:40, Burd, Greg <[email protected]> wrote:
>
> > On Mar 5, 2025, at 6:39 PM, Matthias van de Meent <[email protected]> wrote:
> >
> > On Wed, 5 Mar 2025 at 18:21, Burd, Greg <[email protected]> wrote:
> >> * augments IndexInfo only when needed for testing expressions and only once
> >
> > ExecExpressionIndexesUpdated seems to always loop over all indexes,
> > always calling AttributeIndexInfo which always updates the fields in
> > the IndexInfo when the index has only !byval attributes (e.g. text,
> > json, or other such varlena types). You say it happens only once, have
> > I missed something?
>
> There's a test that avoids doing it more than once, [...]
Is this that one?
+ if (indexInfo->ii_IndexAttrByVal)
+ return indexInfo;
I think that test doesn't work consistently: a bitmapset * is NULL
when no bits are set; and for some indexes no attribute will be byval,
thus failing this early-exit even after processing.
Another small issue with this approach is that it always calls and
tests in EEIU(), while it's quite likely we would do better if we
pre-processed _all_ indexes at once, so that we can have a path that
doesn't repeatedly get into EEIU only to exit immediately after. It'll
probably be hot enough to not matter much, but it's still cycles spent
on something that we can optimize for in code.
> >> * retains existing summarized index HOT update logic
> >
> > Great, thanks!
> >
> > Kind regards,
> >
> > Matthias van de Meent
> > Neon (https://neon.tech)
>
> I might widen this patch a bit to include support for testing equality of index tuples using custom operators when they exist for the index. In the use case I'm solving for we use a custom operator for equality that is not the same as a memcmp(). Do you have thoughts on that?
I don't think that's a very great idea. From a certain point of view,
you can see HOT as "deduplicating multiple tuple versions behind a
single TID". Btree doesn't support deduplication for types that can
have more than one representation of the same value so that e.g.
'0.0'::numeric and '0'::numeric are both displayed correctly, even
when they compare as equal according to certain equality operators.
So, I don't think that's worth investing time into right now. Maybe in
the future if there are new discoveries about what we can and cannot
deduplicate, but I don't think it should be part of an MVP or 1.0.
Kind regards,
Matthias van de Meent
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
@ 2025-03-25 11:47 ` Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Burd, Greg @ 2025-03-25 11:47 UTC (permalink / raw)
To: Matthias van de Meent <[email protected]>; +Cc: pgsql-hackers
Matthias,
Rebased patch attached.
Changes in v14:
* UpdateContext now the location I've stored estate, resultRelInfo, etc.
* Reuse the result from the predicate on the partial index.
-greg
> On Mar 7, 2025, at 5:47 PM, Matthias van de Meent <[email protected]> wrote:
>
> On Thu, 6 Mar 2025 at 13:40, Burd, Greg <[email protected]> wrote:
>
>>
>>
>>> On Mar 5, 2025, at 6:39 PM, Matthias van de Meent <[email protected]> wrote:
>>>
>>> On Wed, 5 Mar 2025 at 18:21, Burd, Greg <[email protected]> wrote:
>>>
>>>> * augments IndexInfo only when needed for testing expressions and only once
>>>
>>>
>>> ExecExpressionIndexesUpdated seems to always loop over all indexes,
>>> always calling AttributeIndexInfo which always updates the fields in
>>> the IndexInfo when the index has only !byval attributes (e.g. text,
>>> json, or other such varlena types). You say it happens only once, have
>>> I missed something?
>>
>>
>> There's a test that avoids doing it more than once, [...]
>
>
> Is this that one?
>
> + if (indexInfo->ii_IndexAttrByVal)
> + return indexInfo;
>
> I think that test doesn't work consistently: a bitmapset * is NULL
> when no bits are set; and for some indexes no attribute will be byval,
> thus failing this early-exit even after processing.
>
> Another small issue with this approach is that it always calls and
> tests in EEIU(), while it's quite likely we would do better if we
> pre-processed _all_ indexes at once, so that we can have a path that
> doesn't repeatedly get into EEIU only to exit immediately after. It'll
> probably be hot enough to not matter much, but it's still cycles spent
> on something that we can optimize for in code.
I've changed this a bit, now in ExecOpenIndices() when there are expressions or predicates I augment the IndexInfo with information necessary to perform the tests in EEIU(). I've debated adding another bool to ExecOpenIndices() to indicate that we're opening indexes for the purpose of an update to avoid building that information in cases where we are not. Similar to the bool `speculative` on ExecOpenIndices() today. Thoughts?
>> I might widen this patch a bit to include support for testing equality of index tuples using custom operators when they exist for the index. In the use case I'm solving for we use a custom operator for equality that is not the same as a memcmp(). Do you have thoughts on that?
>
>
> I don't think that's a very great idea. From a certain point of view,
> you can see HOT as "deduplicating multiple tuple versions behind a
> single TID". Btree doesn't support deduplication for types that can
> have more than one representation of the same value so that e.g.
> '0.0'::numeric and '0'::numeric are both displayed correctly, even
> when they compare as equal according to certain equality operators.
Interesting, good point. Seems like it would require a new index AM function:
bool indexed_tuple_would_change()
I'll drop this for now, it seems out of scope for this patch set.
Attachments:
[application/octet-stream] v14-0001-Expand-HOT-update-path-to-include-expression-and.patch (98.7K, 2-v14-0001-Expand-HOT-update-path-to-include-expression-and.patch)
download | inline diff:
From 05e2a84dc3541885392ebe85bd266018ec29e896 Mon Sep 17 00:00:00 2001
From: Gregory Burd <[email protected]>
Date: Mon, 27 Jan 2025 13:28:59 -0500
Subject: [PATCH v14] Expand HOT update path to include expression and partial
indexes.
This patch extends the cases where HOT updates are possible in the heapam by
examining expression indexes and determining if indexed values where mutated
or not. Previously, any expression index on a column would disqualify it
from the HOT update path. Also examines partial indexes to see if the
values are within the predicate or not.
This is a modified application of a patch proposed on the pgsql-hackers list:
https://www.postgresql.org/message-id/flat/4d9928ee-a9e6-15f9-9c82-5981f13ffca6%40postgrespro.ru
applied: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=c203d6cf8
reverted: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=05f84605dbeb9cf8279a157234b24bbb706c5256
Signed-off-by: Greg Burd <[email protected]>
---
doc/src/sgml/ref/create_table.sgml | 18 +
doc/src/sgml/storage.sgml | 21 +
src/backend/access/common/reloptions.c | 12 +-
src/backend/access/heap/README.HOT | 45 +-
src/backend/access/heap/heapam.c | 43 +-
src/backend/access/heap/heapam_handler.c | 15 +-
src/backend/access/table/tableam.c | 5 +-
src/backend/catalog/index.c | 47 ++
src/backend/catalog/indexing.c | 16 +-
src/backend/executor/execIndexing.c | 192 ++++-
src/backend/executor/execReplication.c | 10 +-
src/backend/executor/nodeModifyTable.c | 27 +-
src/backend/utils/cache/relcache.c | 70 +-
src/bin/psql/tab-complete.in.c | 2 +-
src/include/access/heapam.h | 6 +-
src/include/access/tableam.h | 33 +-
src/include/catalog/index.h | 1 +
src/include/executor/executor.h | 5 +
src/include/nodes/execnodes.h | 13 +
src/include/utils/rel.h | 10 +
src/include/utils/relcache.h | 1 +
.../regress/expected/heap_hot_updates.out | 665 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
src/test/regress/sql/heap_hot_updates.sql | 481 +++++++++++++
src/tools/pgindent/typedefs.list | 2 +-
25 files changed, 1640 insertions(+), 105 deletions(-)
create mode 100644 src/test/regress/expected/heap_hot_updates.out
create mode 100644 src/test/regress/sql/heap_hot_updates.sql
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index e5c034d724e..569b2c3ee45 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -1980,6 +1980,24 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry id="reloption-expression-checks" xreflabel="expression_checks">
+ <term><literal>expression_checks</literal> (<type>boolean</type>)
+ <indexterm>
+ <primary><varname>expression_checks</varname> storage parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Enables or disables evaulation of predicate expressions on partial
+ indexes or expressions used to define indexes during updates.
+ If <literal>true</literal>, then these expressions are evaluated during
+ updates to data within the heap relation against the old and new values
+ and then compared to determine if <acronym>HOT</acronym> updates are
+ allowable or not. The default value is <literal>true</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect2>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index 61250799ec0..64a6264d8fd 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -1138,6 +1138,27 @@ data. Empty in ordinary tables.</entry>
</itemizedlist>
</para>
+ <para>
+ <acronym>HOT</acronym> updates can occur when the expression used to define
+ an index shows no changes to the indexed value. To determine this requires
+ that the expression be evaulated for the old and new values to be stored in
+ the index and then compared. This allows for <acronym>HOT</acronym> updates
+ when data indexed within JSONB columns is unchanged. To disable this
+ behavior and avoid the overhead of evaluating the expression during updates
+ set the <literal>expression_checks</literal> option to false for the table.
+ </para>
+
+ <para>
+ <acronym>HOT</acronym> updates can also occur when updated values are not
+ within the predicate of a partial index. However, <acronym>HOT</acronym>
+ updates are not possible when the updated value and the current value differ
+ with regards to the predicate. To determin this requires that the predicate
+ expression be evaluated for the old and new values to be stored in the index
+ and then compared. To disable this behavior and avoid the overhead of
+ evaluating the expression during updates set
+ the <literal>expression_checks</literal> option to false for the table.
+ </para>
+
<para>
You can increase the likelihood of sufficient page space for
<acronym>HOT</acronym> updates by decreasing a table's <link
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 645b5c00467..41c727eafcb 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -166,6 +166,15 @@ static relopt_bool boolRelOpts[] =
},
true
},
+ {
+ {
+ "expression_checks",
+ "When disabled prevents checking expressions on indexes and predicates on partial indexes for changes that might influence heap-only tuple (HOT) updates.",
+ RELOPT_KIND_HEAP,
+ ShareUpdateExclusiveLock
+ },
+ true
+ },
/* list terminator */
{{NULL}}
};
@@ -1914,7 +1923,8 @@ default_reloptions(Datum reloptions, bool validate, relopt_kind kind)
{"vacuum_truncate", RELOPT_TYPE_BOOL,
offsetof(StdRdOptions, vacuum_truncate), offsetof(StdRdOptions, vacuum_truncate_set)},
{"vacuum_max_eager_freeze_failure_rate", RELOPT_TYPE_REAL,
- offsetof(StdRdOptions, vacuum_max_eager_freeze_failure_rate)}
+ offsetof(StdRdOptions, vacuum_max_eager_freeze_failure_rate)},
+ {"expression_checks", RELOPT_TYPE_BOOL, offsetof(StdRdOptions, expression_checks)}
};
return (bytea *) build_reloptions(reloptions, validate, kind,
diff --git a/src/backend/access/heap/README.HOT b/src/backend/access/heap/README.HOT
index 74e407f375a..2340d82053f 100644
--- a/src/backend/access/heap/README.HOT
+++ b/src/backend/access/heap/README.HOT
@@ -36,7 +36,7 @@ HOT solves this problem for two restricted but useful special cases:
First, where a tuple is repeatedly updated in ways that do not change
its indexed columns. (Here, "indexed column" means any column referenced
at all in an index definition, including for example columns that are
-tested in a partial-index predicate but are not stored in the index.)
+tested in a partial-index predicate, that has materially changed.)
Second, where the modified columns are only used in indexes that do not
contain tuple IDs, but maintain summaries of the indexed data by block.
@@ -133,28 +133,34 @@ Note: we can use a "dead" line pointer for any DELETEd tuple,
whether it was part of a HOT chain or not. This allows space reclamation
in advance of running VACUUM for plain DELETEs as well as HOT updates.
-The requirement for doing a HOT update is that indexes which point to
-the root line pointer (and thus need to be cleaned up by VACUUM when the
-tuple is dead) do not reference columns which are updated in that HOT
-chain. Summarizing indexes (such as BRIN) are assumed to have no
-references to individual tuples and thus are ignored when checking HOT
-applicability. The updated columns are checked at execution time by
-comparing the binary representation of the old and new values. We insist
-on bitwise equality rather than using datatype-specific equality routines.
-The main reason to avoid the latter is that there might be multiple
-notions of equality for a datatype, and we don't know exactly which one
-is relevant for the indexes at hand. We assume that bitwise equality
-guarantees equality for all purposes.
+The requirement for doing a HOT update is that indexes which point to the root
+line pointer (and thus need to be cleaned up by VACUUM when the tuple is dead)
+do not reference columns which are updated in that HOT chain.
+
+Summarizing indexes (such as BRIN) are assumed to have no references to
+individual tuples and thus are ignored when checking HOT applicability.
+
+Expressions on indexes are evaluated and the results used when checking for
+changes. This allows for the JSONB datatype to have HOT updates when the
+indexed portion of the document are not modified.
+
+Partial index expressions are evaluated, HOT updates are allowed when the
+updated index values do not satisfy the predicate.
+
+The updated columns are checked at execution time by comparing the binary
+representation of the old and new values. We insist on bitwise equality rather
+than using datatype-specific equality routines. The main reason to avoid the
+latter is that there might be multiple notions of equality for a datatype, and
+we don't know exactly which one is relevant for the indexes at hand. We assume
+that bitwise equality guarantees equality for all purposes.
If any columns that are included by non-summarizing indexes are updated,
the HOT optimization is not applied, and the new tuple is inserted into
all indexes of the table. If none of the updated columns are included in
the table's indexes, the HOT optimization is applied and no indexes are
updated. If instead the updated columns are only indexed by summarizing
-indexes, the HOT optimization is applied, but the update is propagated to
-all summarizing indexes. (Realistically, we only need to propagate the
-update to the indexes that contain the updated values, but that is yet to
-be implemented.)
+indexes, the HOT optimization is applied and the update is propagated to
+all of the summarizing indexes.
Abort Cases
-----------
@@ -477,8 +483,9 @@ Heap-only tuple
HOT-safe
A proposed tuple update is said to be HOT-safe if it changes
- none of the tuple's indexed columns. It will only become an
- actual HOT update if we can find room on the same page for
+ none of the tuple's indexed columns or if the changes remain
+ outside of a partial index's predicate. It will only become
+ an actual HOT update if we can find room on the same page for
the new tuple version.
HOT update
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index b12b583c4d9..8b523d9cb82 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -3244,13 +3244,14 @@ simple_heap_delete(Relation relation, ItemPointer tid)
TM_Result
heap_update(Relation relation, ItemPointer otid, HeapTuple newtup,
CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ TM_FailureData *tmfd, UpdateContext *updateCxt)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
+ LockTupleMode *lockmode = &updateCxt->lockmode;
Bitmapset *hot_attrs;
Bitmapset *sum_attrs;
+ Bitmapset *exp_attrs;
Bitmapset *key_attrs;
Bitmapset *id_attrs;
Bitmapset *interesting_attrs;
@@ -3327,6 +3328,8 @@ heap_update(Relation relation, ItemPointer otid, HeapTuple newtup,
INDEX_ATTR_BITMAP_HOT_BLOCKING);
sum_attrs = RelationGetIndexAttrBitmap(relation,
INDEX_ATTR_BITMAP_SUMMARIZED);
+ exp_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_EXPRESSION);
key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
id_attrs = RelationGetIndexAttrBitmap(relation,
INDEX_ATTR_BITMAP_IDENTITY_KEY);
@@ -3388,10 +3391,11 @@ heap_update(Relation relation, ItemPointer otid, HeapTuple newtup,
tmfd->ctid = *otid;
tmfd->xmax = InvalidTransactionId;
tmfd->cmax = InvalidCommandId;
- *update_indexes = TU_None;
+ updateCxt->updateIndexes = TU_None;
bms_free(hot_attrs);
bms_free(sum_attrs);
+ bms_free(exp_attrs);
bms_free(key_attrs);
bms_free(id_attrs);
/* modified_attrs not yet initialized */
@@ -3689,10 +3693,11 @@ l2:
UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
if (vmbuffer != InvalidBuffer)
ReleaseBuffer(vmbuffer);
- *update_indexes = TU_None;
+ updateCxt->updateIndexes = TU_None;
bms_free(hot_attrs);
bms_free(sum_attrs);
+ bms_free(exp_attrs);
bms_free(key_attrs);
bms_free(id_attrs);
bms_free(modified_attrs);
@@ -4009,12 +4014,24 @@ l2:
if (newbuf == buffer)
{
+ ResultRelInfo *resultRelInfo = updateCxt->rri;
+ EState *estate = updateCxt->estate;
+ bool expression_checks = RelationGetExpressionChecks(relation);
+
/*
* Since the new tuple is going into the same page, we might be able
- * to do a HOT update. Check if any of the index columns have been
- * changed.
+ * to do a HOT update. As a reminder, hot_attrs includes attributes
+ * used by indexes including within expressions and predicates, but
+ * not attributes only used by summarizing indexes.
*/
- if (!bms_overlap(modified_attrs, hot_attrs))
+ if (!bms_overlap(modified_attrs, hot_attrs) ||
+ (estate && resultRelInfo && expression_checks &&
+ bms_overlap(modified_attrs, exp_attrs) &&
+ !ExecExpressionIndexesUpdated(resultRelInfo,
+ modified_attrs,
+ estate,
+ resultRelInfo->ri_oldTupleSlot,
+ resultRelInfo->ri_newTupleSlot)))
{
use_hot_update = true;
@@ -4196,18 +4213,19 @@ l2:
if (use_hot_update)
{
if (summarized_update)
- *update_indexes = TU_Summarizing;
+ updateCxt->updateIndexes = TU_Summarizing;
else
- *update_indexes = TU_None;
+ updateCxt->updateIndexes = TU_None;
}
else
- *update_indexes = TU_All;
+ updateCxt->updateIndexes = TU_All;
if (old_key_tuple != NULL && old_key_copied)
heap_freetuple(old_key_tuple);
bms_free(hot_attrs);
bms_free(sum_attrs);
+ bms_free(exp_attrs);
bms_free(key_attrs);
bms_free(id_attrs);
bms_free(modified_attrs);
@@ -4485,16 +4503,15 @@ HeapDetermineColumnsInfo(Relation relation,
*/
void
simple_heap_update(Relation relation, ItemPointer otid, HeapTuple tup,
- TU_UpdateIndexes *update_indexes)
+ UpdateContext *updateCxt)
{
TM_Result result;
TM_FailureData tmfd;
- LockTupleMode lockmode;
result = heap_update(relation, otid, tup,
GetCurrentCommandId(true), InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, updateCxt);
switch (result)
{
case TM_SelfModified:
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 24d3765aa20..54f627c5b27 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -316,8 +316,7 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd,
- LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
+ bool wait, TM_FailureData *tmfd, UpdateContext *updateCxt)
{
bool shouldFree = true;
HeapTuple tuple = ExecFetchSlotHeapTuple(slot, true, &shouldFree);
@@ -328,7 +327,7 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
tuple->t_tableOid = slot->tts_tableOid;
result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
- tmfd, lockmode, update_indexes);
+ tmfd, updateCxt);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
@@ -343,14 +342,14 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
*/
if (result != TM_Ok)
{
- Assert(*update_indexes == TU_None);
- *update_indexes = TU_None;
+ Assert(updateCxt->updateIndexes == TU_None);
+ updateCxt->updateIndexes = TU_None;
}
else if (!HeapTupleIsHeapOnly(tuple))
- Assert(*update_indexes == TU_All);
+ Assert(updateCxt->updateIndexes == TU_All);
else
- Assert((*update_indexes == TU_Summarizing) ||
- (*update_indexes == TU_None));
+ Assert((updateCxt->updateIndexes == TU_Summarizing) ||
+ (updateCxt->updateIndexes == TU_None));
if (shouldFree)
pfree(tuple);
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index a56c5eceb14..1eeeebadc09 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -336,17 +336,16 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
- TU_UpdateIndexes *update_indexes)
+ UpdateContext *updateCxt)
{
TM_Result result;
TM_FailureData tmfd;
- LockTupleMode lockmode;
result = table_tuple_update(rel, otid, slot,
GetCurrentCommandId(true),
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, updateCxt);
switch (result)
{
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 739a92bdcc1..b8c9ff89b3b 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2707,6 +2707,53 @@ BuildSpeculativeIndexInfo(Relation index, IndexInfo *ii)
}
}
+/* ----------------
+ * BuildExpressionIndexInfo
+ * Add extra state to IndexInfo record
+ *
+ * For expression indexes updates may not change the indexed value allowing
+ * for a HOT update. Add information to the IndexInfo to allow for checking
+ * if the indexed value has changed.
+ *
+ * Do this processing here rather than in BuildIndexInfo() to not incur the
+ * overhead in the common non-expression cases.
+ * ----------------
+ */
+void
+BuildExpressionIndexInfo(Relation index, IndexInfo *ii)
+{
+ int i;
+ int indnkeyatts;
+ Form_pg_index indexStruct = index->rd_index;
+
+ indnkeyatts = IndexRelationGetNumberOfKeyAttributes(index);
+
+ /*
+ * Collect attributes used by the index, their len and if they are by
+ * value.
+ */
+ for (i = 0; i < indnkeyatts; i++)
+ {
+ CompactAttribute *attr = TupleDescCompactAttr(RelationGetDescr(index), i);
+
+ ii->ii_IndexAttrs =
+ bms_add_member(ii->ii_IndexAttrs,
+ ii->ii_IndexAttrNumbers[i] - FirstLowInvalidHeapAttributeNumber);
+
+ ii->ii_IndexAttrLen[i] = attr->attlen;
+ if (attr->attbyval)
+ ii->ii_IndexAttrByVal = bms_add_member(ii->ii_IndexAttrByVal, i);
+ }
+
+ /* collect attributes used in the expression */
+ if (ii->ii_Expressions)
+ pull_varattnos((Node *) ii->ii_Expressions, 1, &ii->ii_ExpressionAttrs);
+
+ /* collect attributes used in the predicate */
+ if (ii->ii_Predicate)
+ pull_varattnos((Node *) ii->ii_Predicate, 1, &ii->ii_PredicateAttrs);
+}
+
/* ----------------
* FormIndexDatum
* Construct values[] and isnull[] arrays for a new index tuple.
diff --git a/src/backend/catalog/indexing.c b/src/backend/catalog/indexing.c
index 25c4b6bdc87..91a6b455b14 100644
--- a/src/backend/catalog/indexing.c
+++ b/src/backend/catalog/indexing.c
@@ -313,15 +313,17 @@ void
CatalogTupleUpdate(Relation heapRel, ItemPointer otid, HeapTuple tup)
{
CatalogIndexState indstate;
- TU_UpdateIndexes updateIndexes = TU_All;
+ UpdateContext updateCxt = {0};
+
+ updateCxt.updateIndexes = TU_All;
CatalogTupleCheckConstraints(heapRel, tup);
indstate = CatalogOpenIndexes(heapRel);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ simple_heap_update(heapRel, otid, tup, &updateCxt);
- CatalogIndexInsert(indstate, tup, updateIndexes);
+ CatalogIndexInsert(indstate, tup, updateCxt.updateIndexes);
CatalogCloseIndexes(indstate);
}
@@ -337,13 +339,15 @@ void
CatalogTupleUpdateWithInfo(Relation heapRel, ItemPointer otid, HeapTuple tup,
CatalogIndexState indstate)
{
- TU_UpdateIndexes updateIndexes = TU_All;
+ UpdateContext updateCxt = {0};
+
+ updateCxt.updateIndexes = TU_All;
CatalogTupleCheckConstraints(heapRel, tup);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ simple_heap_update(heapRel, otid, tup, &updateCxt);
- CatalogIndexInsert(indstate, tup, updateIndexes);
+ CatalogIndexInsert(indstate, tup, updateCxt.updateIndexes);
}
/*
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index e3fe9b78bb5..429e08f759c 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -113,10 +113,13 @@
#include "catalog/index.h"
#include "executor/executor.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/optimizer.h"
#include "storage/lmgr.h"
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
+#include "utils/datum.h"
+#include "utils/lsyscache.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
typedef enum
@@ -221,6 +224,13 @@ ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative)
if (speculative && ii->ii_Unique && !indexDesc->rd_index->indisexclusion)
BuildSpeculativeIndexInfo(indexDesc, ii);
+ /*
+ * If the index uses expressions then let's populate the additional
+ * information nessaary to evaluate them for changes during updates.
+ */
+ if (ii->ii_Expressions || ii->ii_Predicate)
+ BuildExpressionIndexInfo(indexDesc, ii);
+
relationDescs[i] = indexDesc;
indexInfoArray[i] = ii;
i++;
@@ -383,19 +393,33 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
ExprState *predicate;
/*
- * If predicate state not set up yet, create it (in the estate's
- * per-query context)
+ * It is possible that we've already checked the predicate, if so
+ * then avoid the duplicate work.
*/
- predicate = indexInfo->ii_PredicateState;
- if (predicate == NULL)
+ if (indexInfo->ii_CheckedPredicate)
{
- predicate = ExecPrepareQual(indexInfo->ii_Predicate, estate);
- indexInfo->ii_PredicateState = predicate;
+ /* Skip this index-update if the predicate isn't satisfied */
+ if (!indexInfo->ii_PredicateSatisfied)
+ continue;
}
+ else
+ {
- /* Skip this index-update if the predicate isn't satisfied */
- if (!ExecQual(predicate, econtext))
- continue;
+ /*
+ * If predicate state not set up yet, create it (in the
+ * estate's per-query context)
+ */
+ predicate = indexInfo->ii_PredicateState;
+ if (predicate == NULL)
+ {
+ predicate = ExecPrepareQual(indexInfo->ii_Predicate, estate);
+ indexInfo->ii_PredicateState = predicate;
+ }
+
+ /* Skip this index-update if the predicate isn't satisfied */
+ if (!ExecQual(predicate, econtext))
+ continue;
+ }
}
/*
@@ -1096,6 +1120,9 @@ index_unchanged_by_update(ResultRelInfo *resultRelInfo, EState *estate,
if (hasexpression)
{
+ if (indexInfo->ii_IndexUnchanged)
+ return true;
+
indexInfo->ii_IndexUnchanged = false;
return false;
}
@@ -1173,3 +1200,150 @@ ExecWithoutOverlapsNotEmpty(Relation rel, NameData attname, Datum attval, char t
errmsg("empty WITHOUT OVERLAPS value found in column \"%s\" in relation \"%s\"",
NameStr(attname), RelationGetRelationName(rel))));
}
+
+/*
+ * Determine if indexes that have expressions or predicates require updates
+ * for the purposes of allowing HOT updates. Returns false iff all indexed
+ * values are unchanged.
+ */
+bool
+ExecExpressionIndexesUpdated(ResultRelInfo *resultRelInfo,
+ Bitmapset *modifiedAttrs,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts)
+{
+ bool result = false;
+ IndexInfo *indexInfo;
+ ExprContext *econtext = NULL;
+
+ if (old_tts == NULL || new_tts == NULL)
+ return true;
+
+ econtext = GetPerTupleExprContext(estate);
+
+ /*
+ * Examine each index on this relation relative to the changes between old
+ * and new tuples.
+ */
+ for (int i = 0; i < resultRelInfo->ri_NumIndices; i++)
+ {
+ indexInfo = resultRelInfo->ri_IndexRelationInfo[i];
+
+ /*
+ * If this is a partial index it has a predicate, evaluate the
+ * expression to determine if we need to include it or not.
+ */
+ if (bms_overlap(indexInfo->ii_PredicateAttrs, modifiedAttrs))
+ {
+ ExprState *pstate;
+ bool old_tuple_qualifies,
+ new_tuple_qualifies;
+
+ pstate = ExecPrepareQual(indexInfo->ii_Predicate, estate);
+
+ /*
+ * Here the term "qualifies" means "satisfies the predicate
+ * condition of the partial index".
+ */
+ econtext->ecxt_scantuple = old_tts;
+ old_tuple_qualifies = ExecQual(pstate, econtext);
+
+ econtext->ecxt_scantuple = new_tts;
+ new_tuple_qualifies = ExecQual(pstate, econtext);
+ indexInfo->ii_CheckedPredicate = true;
+ indexInfo->ii_PredicateSatisfied = new_tuple_qualifies;
+
+ /*
+ * If neither the old nor the new tuples satisfy the predicate we
+ * can be sure that this index doesn't need updating, continue to
+ * the next index.
+ */
+ if ((new_tuple_qualifies == false) && (old_tuple_qualifies == false))
+ continue;
+
+ /*
+ * If there is a transition between indexed and not indexed,
+ * that's enough to require that this index is updated. If any
+ * single non-summarizing index requires updates then they all
+ * should be updated.
+ */
+ if (new_tuple_qualifies != old_tuple_qualifies)
+ {
+ result = true;
+ break;
+ }
+
+ /*
+ * Otherwise the old and new values exist in the index, but did
+ * they get updated? We don't yet know, so proceed with the next
+ * statement in the loop to find out.
+ */
+ }
+
+ /*
+ * Indexes with expressions may or may not have changed, it is
+ * impossible to know without exercising their expression and
+ * reviewing index tuple state for changes. This is a lot of work,
+ * but because all indexes on JSONB columns fall into this category it
+ * can be worth it to avoid index updates and remain on the HOT update
+ * path when possible.
+ */
+ if (bms_overlap(indexInfo->ii_ExpressionAttrs, modifiedAttrs))
+ {
+ TupleTableSlot *save_scantuple;
+ Datum old_values[INDEX_MAX_KEYS];
+ bool old_isnull[INDEX_MAX_KEYS];
+ Datum new_values[INDEX_MAX_KEYS];
+ bool new_isnull[INDEX_MAX_KEYS];
+ bool changed = false;
+
+ save_scantuple = econtext->ecxt_scantuple;
+ econtext->ecxt_scantuple = old_tts;
+ FormIndexDatum(indexInfo,
+ old_tts,
+ estate,
+ old_values,
+ old_isnull);
+ econtext->ecxt_scantuple = new_tts;
+ FormIndexDatum(indexInfo,
+ new_tts,
+ estate,
+ new_values,
+ new_isnull);
+ econtext->ecxt_scantuple = save_scantuple;
+
+ for (int j = 0; j < indexInfo->ii_NumIndexKeyAttrs; j++)
+ {
+ if (old_isnull[j] != new_isnull[j])
+ {
+ changed = true;
+ break;
+ }
+ else if (!old_isnull[j])
+ {
+ int16 elmlen = indexInfo->ii_IndexAttrLen[j];
+ bool elmbyval = bms_is_member(j, indexInfo->ii_IndexAttrByVal);
+
+ if (!datum_image_eq(old_values[j], new_values[j],
+ elmbyval, elmlen))
+ {
+ changed = true;
+ break;
+ }
+ }
+ }
+
+ indexInfo->ii_CheckedUnchanged = true;
+ indexInfo->ii_IndexUnchanged = !changed;
+
+ if (changed)
+ {
+ result = true;
+ break;
+ }
+ }
+ }
+
+ return result;
+}
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index ede89ea3cf9..c13fb6befd5 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -647,7 +647,7 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (!skip_tuple)
{
List *recheckIndexes = NIL;
- TU_UpdateIndexes update_indexes;
+ UpdateContext updateCxt = {0};
List *conflictindexes;
bool conflict = false;
@@ -663,17 +663,19 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ updateCxt.estate = estate;
+ updateCxt.rri = resultRelInfo;
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
- &update_indexes);
+ &updateCxt);
conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
- if (resultRelInfo->ri_NumIndices > 0 && (update_indexes != TU_None))
+ if (resultRelInfo->ri_NumIndices > 0 && (updateCxt.updateIndexes != TU_None))
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
slot, estate, true,
conflictindexes ? true : false,
&conflict, conflictindexes,
- (update_indexes == TU_Summarizing));
+ (updateCxt.updateIndexes == TU_Summarizing));
/*
* Refer to the comments above the call to CheckAndReportConflict() in
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 87c820276a8..8650c2be096 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -115,21 +115,6 @@ typedef struct ModifyTableContext
TupleTableSlot *cpUpdateReturningSlot;
} ModifyTableContext;
-/*
- * Context struct containing output data specific to UPDATE operations.
- */
-typedef struct UpdateContext
-{
- bool crossPartUpdate; /* was it a cross-partition update? */
- TU_UpdateIndexes updateIndexes; /* Which index updates are required? */
-
- /*
- * Lock mode to acquire on the latest tuple version before performing
- * EvalPlanQual on it
- */
- LockTupleMode lockmode;
-} UpdateContext;
-
static void ExecBatchInsert(ModifyTableState *mtstate,
ResultRelInfo *resultRelInfo,
@@ -2282,8 +2267,7 @@ lreplace:
estate->es_snapshot,
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
- &context->tmfd, &updateCxt->lockmode,
- &updateCxt->updateIndexes);
+ &context->tmfd, updateCxt);
return result;
}
@@ -2301,6 +2285,7 @@ ExecUpdateEpilogue(ModifyTableContext *context, UpdateContext *updateCxt,
{
ModifyTableState *mtstate = context->mtstate;
List *recheckIndexes = NIL;
+ bool onlySummarizing = updateCxt->updateIndexes == TU_Summarizing;
/* insert index entries for tuple if necessary */
if (resultRelInfo->ri_NumIndices > 0 && (updateCxt->updateIndexes != TU_None))
@@ -2308,7 +2293,7 @@ ExecUpdateEpilogue(ModifyTableContext *context, UpdateContext *updateCxt,
slot, context->estate,
true, false,
NULL, NIL,
- (updateCxt->updateIndexes == TU_Summarizing));
+ onlySummarizing);
/* AFTER ROW UPDATE Triggers */
ExecARUpdateTriggers(context->estate, resultRelInfo,
@@ -2444,6 +2429,9 @@ ExecUpdate(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
UpdateContext updateCxt = {0};
TM_Result result;
+ updateCxt.estate = estate;
+ updateCxt.rri = resultRelInfo;
+
/*
* abort the operation if not running transactions
*/
@@ -3132,6 +3120,9 @@ lmerge_matched:
TM_Result result;
UpdateContext updateCxt = {0};
+ updateCxt.rri = resultRelInfo;
+ updateCxt.estate = estate;
+
/*
* Test condition, if any.
*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 9f54a9e72b7..29cef048a87 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -64,6 +64,7 @@
#include "catalog/pg_type.h"
#include "catalog/schemapg.h"
#include "catalog/storage.h"
+#include "catalog/index.h"
#include "commands/policy.h"
#include "commands/publicationcmds.h"
#include "commands/trigger.h"
@@ -5211,6 +5212,7 @@ RelationGetIndexPredicate(Relation relation)
* index (empty if FULL)
* INDEX_ATTR_BITMAP_HOT_BLOCKING Columns that block updates from being HOT
* INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
+ * INDEX_ATTR_BITMAP_EXPRESSION Columns included in expresion indexes
*
* Attribute numbers are offset by FirstLowInvalidHeapAttributeNumber so that
* we can include system attributes (e.g., OID) in the bitmap representation.
@@ -5234,7 +5236,11 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
Bitmapset *pkindexattrs; /* columns in the primary index */
Bitmapset *idindexattrs; /* columns in the replica identity */
Bitmapset *hotblockingattrs; /* columns with HOT blocking indexes */
+ Bitmapset *hotblockingexprattrs; /* as above, but only those in
+ * expressions */
Bitmapset *summarizedattrs; /* columns with summarizing indexes */
+ Bitmapset *summarizedexprattrs; /* as above, but only those in
+ * expressions */
List *indexoidlist;
List *newindexoidlist;
Oid relpkindex;
@@ -5257,6 +5263,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
return bms_copy(relation->rd_hotblockingattr);
case INDEX_ATTR_BITMAP_SUMMARIZED:
return bms_copy(relation->rd_summarizedattr);
+ case INDEX_ATTR_BITMAP_EXPRESSION:
+ return bms_copy(relation->rd_expressionattr);
default:
elog(ERROR, "unknown attrKind %u", attrKind);
}
@@ -5300,7 +5308,9 @@ restart:
pkindexattrs = NULL;
idindexattrs = NULL;
hotblockingattrs = NULL;
+ hotblockingexprattrs = NULL;
summarizedattrs = NULL;
+ summarizedexprattrs = NULL;
foreach(l, indexoidlist)
{
Oid indexOid = lfirst_oid(l);
@@ -5314,6 +5324,7 @@ restart:
bool isPK; /* primary key */
bool isIDKey; /* replica identity index */
Bitmapset **attrs;
+ Bitmapset **exprattrs;
indexDesc = index_open(indexOid, AccessShareLock);
@@ -5357,14 +5368,20 @@ restart:
* decide which bitmap we'll update in the following loop.
*/
if (indexDesc->rd_indam->amsummarizing)
+ {
attrs = &summarizedattrs;
+ exprattrs = &summarizedexprattrs;
+ }
else
+ {
attrs = &hotblockingattrs;
+ exprattrs = &hotblockingexprattrs;
+ }
/* Collect simple attribute references */
for (i = 0; i < indexDesc->rd_index->indnatts; i++)
{
- int attrnum = indexDesc->rd_index->indkey.values[i];
+ int attridx = indexDesc->rd_index->indkey.values[i];
/*
* Since we have covering indexes with non-key columns, we must
@@ -5380,30 +5397,28 @@ restart:
* key or identity key. Hence we do not include them into
* uindexattrs, pkindexattrs and idindexattrs bitmaps.
*/
- if (attrnum != 0)
+ if (attridx != 0)
{
- *attrs = bms_add_member(*attrs,
- attrnum - FirstLowInvalidHeapAttributeNumber);
+ AttrNumber attrnum = attridx - FirstLowInvalidHeapAttributeNumber;
+
+ *attrs = bms_add_member(*attrs, attrnum);
if (isKey && i < indexDesc->rd_index->indnkeyatts)
- uindexattrs = bms_add_member(uindexattrs,
- attrnum - FirstLowInvalidHeapAttributeNumber);
+ uindexattrs = bms_add_member(uindexattrs, attrnum);
if (isPK && i < indexDesc->rd_index->indnkeyatts)
- pkindexattrs = bms_add_member(pkindexattrs,
- attrnum - FirstLowInvalidHeapAttributeNumber);
+ pkindexattrs = bms_add_member(pkindexattrs, attrnum);
if (isIDKey && i < indexDesc->rd_index->indnkeyatts)
- idindexattrs = bms_add_member(idindexattrs,
- attrnum - FirstLowInvalidHeapAttributeNumber);
+ idindexattrs = bms_add_member(idindexattrs, attrnum);
}
}
/* Collect all attributes used in expressions, too */
- pull_varattnos(indexExpressions, 1, attrs);
+ pull_varattnos(indexExpressions, 1, exprattrs);
/* Collect all attributes in the index predicate, too */
- pull_varattnos(indexPredicate, 1, attrs);
+ pull_varattnos(indexPredicate, 1, exprattrs);
index_close(indexDesc, AccessShareLock);
}
@@ -5432,11 +5447,37 @@ restart:
bms_free(pkindexattrs);
bms_free(idindexattrs);
bms_free(hotblockingattrs);
+ bms_free(hotblockingexprattrs);
bms_free(summarizedattrs);
+ bms_free(summarizedexprattrs);
goto restart;
}
+ /*
+ * HOT-blocking attributes should include all columns that are part of the
+ * index except attributes only referenced in expressions, including
+ * expressions used to form partial indexes. So, we need to remove the
+ * expression-only columns from the HOT-blocking columns bitmap as those
+ * will be checked separately. This is true for both summarizing and
+ * non-summarizing indexes which we've separated above, so we have to do
+ * this for both bitmaps.
+ */
+
+ /* {expression-only columns} = {expression columns} - {direct columns} */
+ hotblockingexprattrs = bms_del_members(hotblockingexprattrs,
+ hotblockingattrs);
+ /* {hot-blocking columns} = {direct columns} + {expression-only columns} */
+ hotblockingattrs = bms_add_members(hotblockingattrs,
+ hotblockingexprattrs);
+
+ /* {summarized-only columns} = {all summarized columns} - {direct columns} */
+ summarizedexprattrs = bms_del_members(summarizedexprattrs,
+ summarizedattrs);
+ /* {summarized columns} = {all direct columns} + {summarized-only columns} */
+ summarizedattrs = bms_add_members(summarizedattrs,
+ summarizedexprattrs);
+
/* Don't leak the old values of these bitmaps, if any */
relation->rd_attrsvalid = false;
bms_free(relation->rd_keyattr);
@@ -5449,6 +5490,8 @@ restart:
relation->rd_hotblockingattr = NULL;
bms_free(relation->rd_summarizedattr);
relation->rd_summarizedattr = NULL;
+ bms_free(relation->rd_expressionattr);
+ relation->rd_expressionattr = NULL;
/*
* Now save copies of the bitmaps in the relcache entry. We intentionally
@@ -5463,6 +5506,7 @@ restart:
relation->rd_idattr = bms_copy(idindexattrs);
relation->rd_hotblockingattr = bms_copy(hotblockingattrs);
relation->rd_summarizedattr = bms_copy(summarizedattrs);
+ relation->rd_expressionattr = bms_copy(hotblockingexprattrs);
relation->rd_attrsvalid = true;
MemoryContextSwitchTo(oldcxt);
@@ -5479,6 +5523,8 @@ restart:
return hotblockingattrs;
case INDEX_ATTR_BITMAP_SUMMARIZED:
return summarizedattrs;
+ case INDEX_ATTR_BITMAP_EXPRESSION:
+ return hotblockingexprattrs;
default:
elog(ERROR, "unknown attrKind %u", attrKind);
return NULL;
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index 98951aef82c..3d4d50a95e9 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -2993,7 +2993,7 @@ match_previous_words(int pattern_id,
COMPLETE_WITH("(");
/* ALTER TABLESPACE <foo> SET|RESET ( */
else if (Matches("ALTER", "TABLESPACE", MatchAny, "SET|RESET", "("))
- COMPLETE_WITH("seq_page_cost", "random_page_cost",
+ COMPLETE_WITH("seq_page_cost", "random_page_cost", "expression_checks",
"effective_io_concurrency", "maintenance_io_concurrency");
/* ALTER TEXT SEARCH */
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 1640d9c32f7..a7eb7bd542d 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -21,6 +21,7 @@
#include "access/skey.h"
#include "access/table.h" /* for backward compatibility */
#include "access/tableam.h"
+#include "executor/executor.h"
#include "nodes/lockoptions.h"
#include "nodes/primnodes.h"
#include "storage/bufpage.h"
@@ -338,8 +339,7 @@ extern void heap_abort_speculative(Relation relation, ItemPointer tid);
extern TM_Result heap_update(Relation relation, ItemPointer otid,
HeapTuple newtup,
CommandId cid, Snapshot crosscheck, bool wait,
- struct TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+ struct TM_FailureData *tmfd, UpdateContext *updateCxt);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -374,7 +374,7 @@ extern bool heap_tuple_needs_eventual_freeze(HeapTupleHeader tuple);
extern void simple_heap_insert(Relation relation, HeapTuple tup);
extern void simple_heap_delete(Relation relation, ItemPointer tid);
extern void simple_heap_update(Relation relation, ItemPointer otid,
- HeapTuple tup, TU_UpdateIndexes *update_indexes);
+ HeapTuple tup, UpdateContext *updateCxt);
extern TransactionId heap_index_delete_tuples(Relation rel,
TM_IndexDeleteOp *delstate);
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index b8cb1e744ad..13a0bf1e114 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -125,6 +125,28 @@ typedef enum TU_UpdateIndexes
TU_Summarizing,
} TU_UpdateIndexes;
+/*
+ * Data specific to processing UPDATE operations.
+ *
+ * When table_tuple_update is called some storage managers, notably heapam,
+ * can at times avoid index updates. In the heapam this is known as a HOT
+ * update. This struct is used to provide the state required to test for
+ * HOT updates and to communicate that decision on to the index AMs.
+ */
+typedef struct UpdateContext
+{
+ TU_UpdateIndexes updateIndexes; /* Which index updates are required? */
+ struct ResultRelInfo *rri; /* ResultRelInfo for the updated table. */
+ struct EState *estate; /* EState used within the update. */
+ bool crossPartUpdate; /* Was it a cross-partition update? */
+
+ /*
+ * Lock mode to acquire on the latest tuple version before performing
+ * EvalPlanQual on it
+ */
+ LockTupleMode lockmode;
+} UpdateContext;
+
/*
* When table_tuple_update, table_tuple_delete, or table_tuple_lock fail
* because the target tuple is already outdated, they fill in this struct to
@@ -549,8 +571,7 @@ typedef struct TableAmRoutine
Snapshot crosscheck,
bool wait,
TM_FailureData *tmfd,
- LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+ UpdateContext *updateCxt);
/* see table_tuple_lock() for reference about parameters */
TM_Result (*tuple_lock) (Relation rel,
@@ -1504,13 +1525,11 @@ table_tuple_delete(Relation rel, ItemPointer tid, CommandId cid,
static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ bool wait, TM_FailureData *tmfd, UpdateContext *updateCxt)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
- wait, tmfd,
- lockmode, update_indexes);
+ wait, tmfd, updateCxt);
}
/*
@@ -2011,7 +2030,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
- TU_UpdateIndexes *update_indexes);
+ UpdateContext *updateCxt);
/* ----------------------------------------------------------------------------
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 4daa8bef5ee..fdbf47f607b 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -132,6 +132,7 @@ extern bool CompareIndexInfo(const IndexInfo *info1, const IndexInfo *info2,
const AttrMap *attmap);
extern void BuildSpeculativeIndexInfo(Relation index, IndexInfo *ii);
+extern void BuildExpressionIndexInfo(Relation index, IndexInfo *indexInfo);
extern void FormIndexDatum(IndexInfo *indexInfo,
TupleTableSlot *slot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6a1fec88928..507e142a10e 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -773,6 +773,11 @@ extern void check_exclusion_constraint(Relation heap, Relation index,
ItemPointer tupleid,
const Datum *values, const bool *isnull,
EState *estate, bool newIndex);
+extern bool ExecExpressionIndexesUpdated(ResultRelInfo *resultRelInfo,
+ Bitmapset *modifiedAttrs,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
/*
* prototypes from functions in execReplication.c
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index e42f9f9f957..f60b20671b9 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -160,12 +160,15 @@ typedef struct ExprState
*
* NumIndexAttrs total number of columns in this index
* NumIndexKeyAttrs number of key columns in index
+ * IndexAttrs bitmap of index attributes
* IndexAttrNumbers underlying-rel attribute numbers used as keys
* (zeroes indicate expressions). It also contains
* info about included columns.
* Expressions expr trees for expression entries, or NIL if none
+ * ExpressionAttrs bitmap of attributes used within the expression
* ExpressionsState exec state for expressions, or NIL if none
* Predicate partial-index predicate, or NIL if none
+ * PredicateAttrs bitmap of attributes used within the predicate
* PredicateState exec state for predicate, or NIL if none
* ExclusionOps Per-column exclusion operators, or NULL if none
* ExclusionProcs Underlying function OIDs for ExclusionOps
@@ -184,6 +187,7 @@ typedef struct ExprState
* ParallelWorkers # of workers requested (excludes leader)
* Am Oid of index AM
* AmCache private cache area for index AM
+ * OpClassDataTypes operator class data types
* Context memory context holding this IndexInfo
*
* ii_Concurrent, ii_BrokenHotChain, and ii_ParallelWorkers are used only
@@ -195,10 +199,17 @@ typedef struct IndexInfo
NodeTag type;
int ii_NumIndexAttrs; /* total number of columns in index */
int ii_NumIndexKeyAttrs; /* number of key columns in index */
+ Bitmapset *ii_IndexAttrs;
AttrNumber ii_IndexAttrNumbers[INDEX_MAX_KEYS];
+ uint16 ii_IndexAttrLen[INDEX_MAX_KEYS];
+ Bitmapset *ii_IndexAttrByVal;
+ FmgrInfo *ii_EqualityProc[INDEX_MAX_KEYS];
+ Oid ii_Collation[INDEX_MAX_KEYS];
List *ii_Expressions; /* list of Expr */
+ Bitmapset *ii_ExpressionAttrs;
List *ii_ExpressionsState; /* list of ExprState */
List *ii_Predicate; /* list of Expr */
+ Bitmapset *ii_PredicateAttrs;
ExprState *ii_PredicateState;
Oid *ii_ExclusionOps; /* array with one entry per column */
Oid *ii_ExclusionProcs; /* array with one entry per column */
@@ -211,6 +222,8 @@ typedef struct IndexInfo
bool ii_ReadyForInserts;
bool ii_CheckedUnchanged;
bool ii_IndexUnchanged;
+ bool ii_CheckedPredicate;
+ bool ii_PredicateSatisfied;
bool ii_Concurrent;
bool ii_BrokenHotChain;
bool ii_Summarizing;
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index d94fddd7cef..e44d9966778 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -164,6 +164,7 @@ typedef struct RelationData
Bitmapset *rd_idattr; /* included in replica identity index */
Bitmapset *rd_hotblockingattr; /* cols blocking HOT update */
Bitmapset *rd_summarizedattr; /* cols indexed by summarizing indexes */
+ Bitmapset *rd_expressionattr; /* indexed cols referenced by expressions */
PublicationDesc *rd_pubdesc; /* publication descriptor, or NULL */
@@ -345,6 +346,7 @@ typedef struct StdRdOptions
StdRdOptIndexCleanup vacuum_index_cleanup; /* controls index vacuuming */
bool vacuum_truncate; /* enables vacuum to truncate a relation */
bool vacuum_truncate_set; /* whether vacuum_truncate is set */
+ bool expression_checks; /* use expression to checks for changes */
/*
* Fraction of pages in a relation that vacuum can eagerly scan and fail
@@ -406,6 +408,14 @@ typedef struct StdRdOptions
((relation)->rd_options ? \
((StdRdOptions *) (relation)->rd_options)->parallel_workers : (defaultpw))
+/*
+ * RelationGetExpressionChecks
+ * Returns the relation's expression_checks reloption setting.
+ */
+#define RelationGetExpressionChecks(relation) \
+ ((relation)->rd_options ? \
+ ((StdRdOptions *) (relation)->rd_options)->expression_checks : true)
+
/* ViewOptions->check_option values */
typedef enum ViewOptCheckOption
{
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index a7c55db339e..0cc28cb97e2 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -63,6 +63,7 @@ typedef enum IndexAttrBitmapKind
INDEX_ATTR_BITMAP_IDENTITY_KEY,
INDEX_ATTR_BITMAP_HOT_BLOCKING,
INDEX_ATTR_BITMAP_SUMMARIZED,
+ INDEX_ATTR_BITMAP_EXPRESSION,
} IndexAttrBitmapKind;
extern Bitmapset *RelationGetIndexAttrBitmap(Relation relation,
diff --git a/src/test/regress/expected/heap_hot_updates.out b/src/test/regress/expected/heap_hot_updates.out
new file mode 100644
index 00000000000..027c8760d69
--- /dev/null
+++ b/src/test/regress/expected/heap_hot_updates.out
@@ -0,0 +1,665 @@
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table will have two columns and two indexes, one on the primary key
+-- id and one on the expression (info->>'name'). That means that the indexed
+-- attributes are 'id' and 'info'.
+create table keyvalue(id integer primary key, info jsonb);
+create index nameindex on keyvalue((info->>'name'));
+insert into keyvalue values (1, '{"name": "john", "data": "some data"}');
+-- Disable expression checks.
+ALTER TABLE keyvalue SET (expression_checks = false);
+SELECT reloptions FROM pg_class WHERE relname = 'keyvalue';
+ reloptions
+---------------------------
+ {expression_checks=false}
+(1 row)
+
+-- While the indexed attribute "name" is unchanged we've disabled expression
+-- checks so this update should not go HOT as the system can't determine if
+-- the indexed attribute has changed without evaluating the expression.
+update keyvalue set info='{"name": "john", "data": "something else"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 0 row
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 0
+(1 row)
+
+-- Re-enable expression checks.
+ALTER TABLE keyvalue SET (expression_checks = true);
+SELECT reloptions FROM pg_class WHERE relname = 'keyvalue';
+ reloptions
+--------------------------
+ {expression_checks=true}
+(1 row)
+
+-- The indexed attribute "name" with value "john" is unchanged, expect a HOT update.
+update keyvalue set info='{"name": "john", "data": "some other data"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 1 row
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- The following update changes the indexed attribute "name", this should not be a HOT update.
+update keyvalue set info='{"name": "smith", "data": "some other data"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Now, this update does not change the indexed attribute "name" from "smith", this should be HOT.
+update keyvalue set info='{"name": "smith", "data": "some more data"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 2 rows now
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 2
+(1 row)
+
+drop table keyvalue;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table is the same as the previous one but it has a third index. The
+-- index 'colindex' isn't an expression index, it indexes the entire value
+-- in the info column. There are still only two indexed attributes for this
+-- relation, the same two as before. The presence of an index on the entire
+-- value of the info column should prevent HOT updates for any updates to any
+-- portion of JSONB content in that column.
+create table keyvalue(id integer primary key, info jsonb);
+create index nameindex on keyvalue((info->>'name'));
+create index colindex on keyvalue(info);
+insert into keyvalue values (1, '{"name": "john", "data": "some data"}');
+-- This update doesn't change the value of the expression index, but it does
+-- change the content of the info column and so should not be HOT because the
+-- indexed value changed as a result of the update.
+update keyvalue set info='{"name": "john", "data": "some other data"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 0 rows
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 0
+(1 row)
+
+drop table keyvalue;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- The table has one column docs and two indexes. They are both expression
+-- indexes referencing the same column attribute (docs) but one is a partial
+-- index.
+CREATE TABLE ex (docs JSONB) WITH (fillfactor = 60);
+INSERT INTO ex (docs) VALUES ('{"a": 0, "b": 0}');
+INSERT INTO ex (docs) SELECT jsonb_build_object('b', n) FROM generate_series(100, 10000) as n;
+CREATE INDEX idx_ex_a ON ex ((docs->>'a'));
+CREATE INDEX idx_ex_b ON ex ((docs->>'b')) WHERE (docs->>'b')::numeric > 9;
+-- We're using BTREE indexes and for this test we want to make sure that they remain
+-- in sync with changes to our relation. Force the choice of index scans below so
+-- that we know we're checking the index's understanding of what values should be
+-- in the index or not.
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+-- Leave 'a' unchanged but modify 'b' to a value outside of the index predicate.
+-- This should be a HOT update because neither index is changed.
+UPDATE ex SET docs = jsonb_build_object('a', 0, 'b', 1) WHERE (docs->>'a')::numeric = 0;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 row
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Let's check to make sure that the index does not contain a value for 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ QUERY PLAN
+--------------------------------------------------------------
+ Index Scan using idx_ex_b on ex
+ Filter: (((docs ->> 'b'::text))::numeric < '100'::numeric)
+(2 rows)
+
+SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ docs
+------
+(0 rows)
+
+-- Leave 'a' unchanged but modify 'b' to a value within the index predicate.
+-- This represents a change for field 'b' from unindexed to indexed and so
+-- this should not take the HOT path.
+UPDATE ex SET docs = jsonb_build_object('a', 0, 'b', 10) WHERE (docs->>'a')::numeric = 0;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Let's check to make sure that the index contains the new value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ QUERY PLAN
+--------------------------------------------------------------
+ Index Scan using idx_ex_b on ex
+ Filter: (((docs ->> 'b'::text))::numeric < '100'::numeric)
+(2 rows)
+
+SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ docs
+-------------------
+ {"a": 0, "b": 10}
+(1 row)
+
+-- This update modifies the value of 'a', an indexed field, so it also cannot
+-- be a HOT update.
+UPDATE ex SET docs = jsonb_build_object('a', 1, 'b', 10) WHERE (docs->>'b')::numeric = 10;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- This update changes both 'a' and 'b' to new values that require index updates,
+-- this cannot use the HOT path.
+UPDATE ex SET docs = jsonb_build_object('a', 2, 'b', 12) WHERE (docs->>'b')::numeric = 10;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Let's check to make sure that the index contains the new value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ QUERY PLAN
+--------------------------------------------------------------
+ Index Scan using idx_ex_b on ex
+ Filter: (((docs ->> 'b'::text))::numeric < '100'::numeric)
+(2 rows)
+
+SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ docs
+-------------------
+ {"a": 2, "b": 12}
+(1 row)
+
+-- This update changes 'b' to a value outside its predicate requiring that
+-- we remove it from the index. That's a transition that can't be done
+-- during a HOT update.
+UPDATE ex SET docs = jsonb_build_object('a', 2, 'b', 1) WHERE (docs->>'b')::numeric = 12;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Let's check to make sure that the index no longer contains the value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ QUERY PLAN
+--------------------------------------------------------------
+ Index Scan using idx_ex_b on ex
+ Filter: (((docs ->> 'b'::text))::numeric < '100'::numeric)
+(2 rows)
+
+SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ docs
+------
+(0 rows)
+
+-- Disable expression checks.
+ALTER TABLE ex SET (expression_checks = false);
+SELECT reloptions FROM pg_class WHERE relname = 'ex';
+ reloptions
+-----------------------------------------
+ {fillfactor=60,expression_checks=false}
+(1 row)
+
+-- This update changes 'b' to a value within its predicate just like it
+-- previous value, which would allow for a HOT update but with expression
+-- checks disabled we can't determine that so this should not be a HOT update.
+UPDATE ex SET docs = jsonb_build_object('a', 2, 'b', 2) WHERE (docs->>'b')::numeric = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Let's make sure we're recording HOT updates for our 'ex' relation properly in the system
+-- table pg_stat_user_tables. Note that statistics are stored within a transaction context
+-- first (xact) and then later into the global statistics for a relation, so first we need
+-- to ensure pending stats are flushed.
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT
+ c.relname AS table_name,
+ -- Transaction statistics
+ pg_stat_get_xact_tuples_updated(c.oid) AS xact_updates,
+ pg_stat_get_xact_tuples_hot_updated(c.oid) AS xact_hot_updates,
+ ROUND((
+ pg_stat_get_xact_tuples_hot_updated(c.oid)::float /
+ NULLIF(pg_stat_get_xact_tuples_updated(c.oid), 0) * 100
+ )::numeric, 2) AS xact_hot_update_percentage,
+ -- Cumulative statistics
+ s.n_tup_upd AS total_updates,
+ s.n_tup_hot_upd AS hot_updates,
+ ROUND((
+ s.n_tup_hot_upd::float /
+ NULLIF(s.n_tup_upd, 0) * 100
+ )::numeric, 2) AS total_hot_update_percentage
+FROM pg_class c
+LEFT JOIN pg_stat_user_tables s ON c.relname = s.relname
+WHERE c.relname = 'ex'
+AND c.relnamespace = 'public'::regnamespace;
+ table_name | xact_updates | xact_hot_updates | xact_hot_update_percentage | total_updates | hot_updates | total_hot_update_percentage
+------------+--------------+------------------+----------------------------+---------------+-------------+-----------------------------
+ ex | 0 | 0 | | 6 | 1 | 16.67
+(1 row)
+
+-- expect: 5 xact updates with 1 xact hot update and no cumulative updates as yet
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+DROP TABLE ex;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table has a single column 'email' and a unique constraint on it that
+-- should preclude HOT updates.
+CREATE TABLE users (
+ user_id serial primary key,
+ name VARCHAR(255) NOT NULL,
+ email VARCHAR(255) NOT NULL,
+ EXCLUDE USING btree (lower(email) WITH =)
+);
+-- Add some data to the table and then update it in ways that should and should
+-- not be HOT updates.
+INSERT INTO users (name, email) VALUES
+('user1', '[email protected]'),
+('user2', '[email protected]'),
+('taken', '[email protected]'),
+('you', '[email protected]'),
+('taken', '[email protected]');
+-- Should fail because of the unique constraint on the email column.
+UPDATE users SET email = '[email protected]' WHERE email = '[email protected]';
+ERROR: conflicting key value violates exclusion constraint "users_lower_excl"
+DETAIL: Key (lower(email::text))=([email protected]) conflicts with existing key (lower(email::text))=([email protected]).
+SELECT pg_stat_get_xact_tuples_hot_updated('users'::regclass); -- expect: 0 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 0
+(1 row)
+
+-- Should succeed because the email column is not being updated and should go HOT.
+UPDATE users SET name = 'foo' WHERE email = '[email protected]';
+SELECT pg_stat_get_xact_tuples_hot_updated('users'::regclass); -- expect: 1 a single new HOT update
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Create a partial index on the email column, updates
+CREATE INDEX idx_users_email_no_example ON users (lower(email)) WHERE lower(email) LIKE '%@example.com%';
+-- An update that changes the email column but not the indexed portion of it and falls outside the constraint.
+-- Shouldn't be a HOT update because of the exclusion constraint.
+UPDATE users SET email = '[email protected]' WHERE name = 'you';
+SELECT pg_stat_get_xact_tuples_hot_updated('users'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- An update that changes the email column but not the indexed portion of it and falls within the constraint.
+-- Again, should fail constraint and fail to be a HOT update.
+UPDATE users SET email = '[email protected]' WHERE name = 'you';
+ERROR: conflicting key value violates exclusion constraint "users_lower_excl"
+DETAIL: Key (lower(email::text))=([email protected]) conflicts with existing key (lower(email::text))=([email protected]).
+SELECT pg_stat_get_xact_tuples_hot_updated('users'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+DROP TABLE users;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- Another test of constraints spoiling HOT updates, this time with a range.
+CREATE TABLE events (
+ id serial primary key,
+ name VARCHAR(255) NOT NULL,
+ event_time tstzrange,
+ constraint no_screening_time_overlap exclude using gist (
+ event_time WITH &&
+ )
+);
+-- Add two non-overlapping events.
+INSERT INTO events (id, event_time, name)
+VALUES
+ (1, '["2023-01-01 19:00:00", "2023-01-01 20:45:00"]', 'event1'),
+ (2, '["2023-01-01 21:00:00", "2023-01-01 21:45:00"]', 'event2');
+-- Update the first event to overlap with the second, should fail the constraint and not be HOT.
+UPDATE events SET event_time = '["2023-01-01 20:00:00", "2023-01-01 21:45:00"]' WHERE id = 1;
+ERROR: conflicting key value violates exclusion constraint "no_screening_time_overlap"
+DETAIL: Key (event_time)=(["Sun Jan 01 20:00:00 2023 PST","Sun Jan 01 21:45:00 2023 PST"]) conflicts with existing key (event_time)=(["Sun Jan 01 21:00:00 2023 PST","Sun Jan 01 21:45:00 2023 PST"]).
+SELECT pg_stat_get_xact_tuples_hot_updated('events'::regclass); -- expect: 0 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 0
+(1 row)
+
+-- Update the first event to not overlap with the second, again not HOT due to the constraint.
+UPDATE events SET event_time = '["2023-01-01 22:00:00", "2023-01-01 22:45:00"]' WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('events'::regclass); -- expect: 0 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 0
+(1 row)
+
+-- Update the first event to not overlap with the second, this time we're HOT because we don't overlap with the constraint.
+UPDATE events SET name = 'new name here' WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('events'::regclass); -- expect: 1 one new HOT update
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+DROP TABLE events;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- A test to ensure that only modified summarizing indexes are updated, not
+-- all of them.
+CREATE TABLE ex (id SERIAL primary key, att1 JSONB, att2 text, att3 text, att4 text) WITH (fillfactor = 60);
+CREATE INDEX ex_expr1_idx ON ex USING btree((att1->'data'));
+CREATE INDEX ex_sumr1_idx ON ex USING BRIN(att2);
+CREATE INDEX ex_expr2_idx ON ex USING btree((att1->'a'));
+CREATE INDEX ex_expr3_idx ON ex USING btree((att1->'b'));
+CREATE INDEX ex_expr4_idx ON ex USING btree((att1->'c'));
+CREATE INDEX ex_sumr2_idx ON ex USING BRIN(att3);
+CREATE INDEX ex_sumr3_idx ON ex USING BRIN(att4);
+CREATE INDEX ex_expr5_idx ON ex USING btree((att1->'d'));
+INSERT INTO ex (att1, att2) VALUES ('{"data": []}'::json, 'nothing special');
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+--------------+-----------------+------+------
+ 1 | {"data": []} | nothing special | |
+(1 row)
+
+-- Update att2 and att4 both are BRIN/summarizing indexes, this should be a HOT update and
+-- only update two of the three summarizing indexes.
+UPDATE ex SET att2 = 'special indeed', att4 = 'whatever';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 one new HOT update
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+--------------+----------------+------+----------
+ 1 | {"data": []} | special indeed | | whatever
+(1 row)
+
+-- Update att1 and att2, only one is BRIN/summarizing, this should NOT be a HOT update.
+UPDATE ex SET att1 = att1 || '{"data": "howdy"}', att2 = 'special, so special';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+-------------------+---------------------+------+----------
+ 1 | {"data": "howdy"} | special, so special | | whatever
+(1 row)
+
+-- Update att2, att3, and att4 all are BRIN/summarizing indexes, this should be a HOT update
+-- and yet still update all three summarizing indexes.
+UPDATE ex SET att2 = 'a', att3 = 'b', att4 = 'c';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 2 with one new HOT update
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 2
+(1 row)
+
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+-------------------+------+------+------
+ 1 | {"data": "howdy"} | a | b | c
+(1 row)
+
+-- Update att1, att2, and att3 all modified values are BRIN/summarizing indexes, this should be a HOT update
+-- and yet still update all three summarizing indexes.
+UPDATE ex SET att1 = '{"data": "howdy"}', att2 = 'd', att3 = 'e';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 3 with one new HOT update
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 3
+(1 row)
+
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+-------------------+------+------+------
+ 1 | {"data": "howdy"} | d | e | c
+(1 row)
+
+DROP TABLE ex;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- A test to ensure that summarizing indexes are not updated when they don't
+-- change, but are updated when they do while not prefent HOT updates.
+CREATE TABLE ex (att1 JSONB, att2 text) WITH (fillfactor = 60);
+CREATE INDEX ex_expr1_idx ON ex USING btree((att1->'data'));
+CREATE INDEX ex_sumr1_idx ON ex USING BRIN(att2);
+INSERT INTO ex VALUES ('{"data": []}', 'nothing special');
+-- Update the unindexed value of att1, this should be a HOT update and and should
+-- update the summarizing index.
+UPDATE ex SET att1 = att1 || '{"status": "stalemate"}';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 one new HOT update
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Update the indexed value of att2, a summarized value, this is a summarized
+-- only update and should use the HOT path while still triggering an update to
+-- the summarizing BRIN index.
+UPDATE ex SET att2 = 'special indeed';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 2 one new HOT update
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 2
+(1 row)
+
+-- Update to att1 doesn't change the indexed value while the update to att2 does,
+-- this again is a summarized only update and should use the HOT path as well as
+-- trigger an update to the BRIN index.
+UPDATE ex SET att1 = att1 || '{"status": "checkmate"}', att2 = 'special, so special';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 3 one new HOT update
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 3
+(1 row)
+
+-- This updates both indexes, the expression index on att1 and the summarizing
+-- index on att2. This should not be a HOT update because there are modified
+-- indexes and only some are summarized, not all. This should force all
+-- indexes to be updated.
+UPDATE ex SET att1 = att1 || '{"data": [1,2,3]}', att2 = 'do you want to play a game?';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 4 both indexes updated
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 3
+(1 row)
+
+DROP TABLE ex;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This test is for a table with a custom type and a custom operators on
+-- the BTREE index. The question is, when comparing values for equality
+-- to determine if there are changes on the index or not... shouldn't we
+-- be using the custom operators?
+-- Create a type
+CREATE TYPE my_custom_type AS (val int);
+-- Comparison functions (returns boolean)
+CREATE FUNCTION my_custom_lt(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val < b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_le(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val <= b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_eq(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val = b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_ge(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val >= b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_gt(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val > b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_ne(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val != b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+-- Comparison function (returns -1, 0, 1)
+CREATE FUNCTION my_custom_cmp(a my_custom_type, b my_custom_type) RETURNS int AS $$
+BEGIN
+ IF a.val < b.val THEN
+ RETURN -1;
+ ELSIF a.val > b.val THEN
+ RETURN 1;
+ ELSE
+ RETURN 0;
+ END IF;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+-- Create the operators
+CREATE OPERATOR < (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_lt,
+ COMMUTATOR = >,
+ NEGATOR = >=
+);
+CREATE OPERATOR <= (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_le,
+ COMMUTATOR = >=,
+ NEGATOR = >
+);
+CREATE OPERATOR = (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_eq,
+ COMMUTATOR = =,
+ NEGATOR = <>
+);
+CREATE OPERATOR >= (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_ge,
+ COMMUTATOR = <=,
+ NEGATOR = <
+);
+CREATE OPERATOR > (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_gt,
+ COMMUTATOR = <,
+ NEGATOR = <=
+);
+CREATE OPERATOR <> (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_ne,
+ COMMUTATOR = <>,
+ NEGATOR = =
+);
+-- Create the operator class (including the support function)
+CREATE OPERATOR CLASS my_custom_ops
+ DEFAULT FOR TYPE my_custom_type USING btree AS
+ OPERATOR 1 <,
+ OPERATOR 2 <=,
+ OPERATOR 3 =,
+ OPERATOR 4 >=,
+ OPERATOR 5 >,
+ FUNCTION 1 my_custom_cmp(my_custom_type, my_custom_type);
+-- Create the table
+CREATE TABLE my_table (
+ id int,
+ custom_val my_custom_type
+);
+-- Insert some data
+INSERT INTO my_table (id, custom_val) VALUES
+(1, ROW(3)::my_custom_type),
+(2, ROW(1)::my_custom_type),
+(3, ROW(4)::my_custom_type),
+(4, ROW(2)::my_custom_type);
+-- Create a function to use when indexing
+CREATE OR REPLACE FUNCTION abs_val(val my_custom_type) RETURNS int AS $$
+BEGIN
+ RETURN abs(val.val);
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+-- Create the index
+CREATE INDEX idx_custom_val_abs ON my_table (abs_val(custom_val));
+-- Update 1
+UPDATE my_table SET custom_val = ROW(5)::my_custom_type WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 0
+(1 row)
+
+-- Update 2
+UPDATE my_table SET custom_val = ROW(0)::my_custom_type WHERE custom_val < ROW(3)::my_custom_type;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 0
+(1 row)
+
+-- Update 3
+UPDATE my_table SET custom_val = ROW(6)::my_custom_type WHERE id = 3;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 0
+(1 row)
+
+-- Update 4
+UPDATE my_table SET id = 5 WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Query using the index
+EXPLAIN (COSTS OFF) SELECT * FROM my_table WHERE abs_val(custom_val) = 6;
+ QUERY PLAN
+-------------------------------------
+ Seq Scan on my_table
+ Filter: (abs_val(custom_val) = 6)
+(2 rows)
+
+SELECT * FROM my_table WHERE abs_val(custom_val) = 6;
+ id | custom_val
+----+------------
+ 3 | (6)
+(1 row)
+
+-- Clean up
+DROP TABLE my_table CASCADE;
+DROP OPERATOR CLASS my_custom_ops USING btree CASCADE;
+DROP OPERATOR < (my_custom_type, my_custom_type);
+DROP OPERATOR <= (my_custom_type, my_custom_type);
+DROP OPERATOR = (my_custom_type, my_custom_type);
+DROP OPERATOR >= (my_custom_type, my_custom_type);
+DROP OPERATOR > (my_custom_type, my_custom_type);
+DROP OPERATOR <> (my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_lt(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_le(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_eq(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_ge(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_gt(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_ne(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_cmp(my_custom_type, my_custom_type);
+DROP FUNCTION abs_val(my_custom_type);
+DROP TYPE my_custom_type CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 0a35f2f8f6a..e2fd5fb74a5 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -68,6 +68,11 @@ test: select_into select_distinct select_distinct_on select_implicit select_havi
# ----------
test: brin gin gist spgist privileges init_privs security_label collate matview lock replica_identity rowsecurity object_address tablesample groupingsets drop_operator password identity generated_stored join_hash
+# ----------
+# Another group of parallel tests
+# ----------
+test: heap_hot_updates
+
# ----------
# Additional BRIN tests
# ----------
diff --git a/src/test/regress/sql/heap_hot_updates.sql b/src/test/regress/sql/heap_hot_updates.sql
new file mode 100644
index 00000000000..56ec4f2f96c
--- /dev/null
+++ b/src/test/regress/sql/heap_hot_updates.sql
@@ -0,0 +1,481 @@
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table will have two columns and two indexes, one on the primary key
+-- id and one on the expression (info->>'name'). That means that the indexed
+-- attributes are 'id' and 'info'.
+create table keyvalue(id integer primary key, info jsonb);
+create index nameindex on keyvalue((info->>'name'));
+insert into keyvalue values (1, '{"name": "john", "data": "some data"}');
+
+-- Disable expression checks.
+ALTER TABLE keyvalue SET (expression_checks = false);
+SELECT reloptions FROM pg_class WHERE relname = 'keyvalue';
+
+-- While the indexed attribute "name" is unchanged we've disabled expression
+-- checks so this update should not go HOT as the system can't determine if
+-- the indexed attribute has changed without evaluating the expression.
+update keyvalue set info='{"name": "john", "data": "something else"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 0 row
+
+-- Re-enable expression checks.
+ALTER TABLE keyvalue SET (expression_checks = true);
+SELECT reloptions FROM pg_class WHERE relname = 'keyvalue';
+
+-- The indexed attribute "name" with value "john" is unchanged, expect a HOT update.
+update keyvalue set info='{"name": "john", "data": "some other data"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 1 row
+
+-- The following update changes the indexed attribute "name", this should not be a HOT update.
+update keyvalue set info='{"name": "smith", "data": "some other data"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 1 no new HOT updates
+
+-- Now, this update does not change the indexed attribute "name" from "smith", this should be HOT.
+update keyvalue set info='{"name": "smith", "data": "some more data"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 2 rows now
+drop table keyvalue;
+
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table is the same as the previous one but it has a third index. The
+-- index 'colindex' isn't an expression index, it indexes the entire value
+-- in the info column. There are still only two indexed attributes for this
+-- relation, the same two as before. The presence of an index on the entire
+-- value of the info column should prevent HOT updates for any updates to any
+-- portion of JSONB content in that column.
+create table keyvalue(id integer primary key, info jsonb);
+create index nameindex on keyvalue((info->>'name'));
+create index colindex on keyvalue(info);
+insert into keyvalue values (1, '{"name": "john", "data": "some data"}');
+
+-- This update doesn't change the value of the expression index, but it does
+-- change the content of the info column and so should not be HOT because the
+-- indexed value changed as a result of the update.
+update keyvalue set info='{"name": "john", "data": "some other data"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 0 rows
+drop table keyvalue;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- The table has one column docs and two indexes. They are both expression
+-- indexes referencing the same column attribute (docs) but one is a partial
+-- index.
+CREATE TABLE ex (docs JSONB) WITH (fillfactor = 60);
+INSERT INTO ex (docs) VALUES ('{"a": 0, "b": 0}');
+INSERT INTO ex (docs) SELECT jsonb_build_object('b', n) FROM generate_series(100, 10000) as n;
+CREATE INDEX idx_ex_a ON ex ((docs->>'a'));
+CREATE INDEX idx_ex_b ON ex ((docs->>'b')) WHERE (docs->>'b')::numeric > 9;
+
+-- We're using BTREE indexes and for this test we want to make sure that they remain
+-- in sync with changes to our relation. Force the choice of index scans below so
+-- that we know we're checking the index's understanding of what values should be
+-- in the index or not.
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+
+-- Leave 'a' unchanged but modify 'b' to a value outside of the index predicate.
+-- This should be a HOT update because neither index is changed.
+UPDATE ex SET docs = jsonb_build_object('a', 0, 'b', 1) WHERE (docs->>'a')::numeric = 0;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 row
+-- Let's check to make sure that the index does not contain a value for 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+
+-- Leave 'a' unchanged but modify 'b' to a value within the index predicate.
+-- This represents a change for field 'b' from unindexed to indexed and so
+-- this should not take the HOT path.
+UPDATE ex SET docs = jsonb_build_object('a', 0, 'b', 10) WHERE (docs->>'a')::numeric = 0;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+-- Let's check to make sure that the index contains the new value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+
+-- This update modifies the value of 'a', an indexed field, so it also cannot
+-- be a HOT update.
+UPDATE ex SET docs = jsonb_build_object('a', 1, 'b', 10) WHERE (docs->>'b')::numeric = 10;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+
+-- This update changes both 'a' and 'b' to new values that require index updates,
+-- this cannot use the HOT path.
+UPDATE ex SET docs = jsonb_build_object('a', 2, 'b', 12) WHERE (docs->>'b')::numeric = 10;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+-- Let's check to make sure that the index contains the new value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+
+-- This update changes 'b' to a value outside its predicate requiring that
+-- we remove it from the index. That's a transition that can't be done
+-- during a HOT update.
+UPDATE ex SET docs = jsonb_build_object('a', 2, 'b', 1) WHERE (docs->>'b')::numeric = 12;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+-- Let's check to make sure that the index no longer contains the value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+
+-- Disable expression checks.
+ALTER TABLE ex SET (expression_checks = false);
+SELECT reloptions FROM pg_class WHERE relname = 'ex';
+
+-- This update changes 'b' to a value within its predicate just like it
+-- previous value, which would allow for a HOT update but with expression
+-- checks disabled we can't determine that so this should not be a HOT update.
+UPDATE ex SET docs = jsonb_build_object('a', 2, 'b', 2) WHERE (docs->>'b')::numeric = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+
+
+-- Let's make sure we're recording HOT updates for our 'ex' relation properly in the system
+-- table pg_stat_user_tables. Note that statistics are stored within a transaction context
+-- first (xact) and then later into the global statistics for a relation, so first we need
+-- to ensure pending stats are flushed.
+SELECT pg_stat_force_next_flush();
+SELECT
+ c.relname AS table_name,
+ -- Transaction statistics
+ pg_stat_get_xact_tuples_updated(c.oid) AS xact_updates,
+ pg_stat_get_xact_tuples_hot_updated(c.oid) AS xact_hot_updates,
+ ROUND((
+ pg_stat_get_xact_tuples_hot_updated(c.oid)::float /
+ NULLIF(pg_stat_get_xact_tuples_updated(c.oid), 0) * 100
+ )::numeric, 2) AS xact_hot_update_percentage,
+ -- Cumulative statistics
+ s.n_tup_upd AS total_updates,
+ s.n_tup_hot_upd AS hot_updates,
+ ROUND((
+ s.n_tup_hot_upd::float /
+ NULLIF(s.n_tup_upd, 0) * 100
+ )::numeric, 2) AS total_hot_update_percentage
+FROM pg_class c
+LEFT JOIN pg_stat_user_tables s ON c.relname = s.relname
+WHERE c.relname = 'ex'
+AND c.relnamespace = 'public'::regnamespace;
+-- expect: 5 xact updates with 1 xact hot update and no cumulative updates as yet
+
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+
+DROP TABLE ex;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table has a single column 'email' and a unique constraint on it that
+-- should preclude HOT updates.
+CREATE TABLE users (
+ user_id serial primary key,
+ name VARCHAR(255) NOT NULL,
+ email VARCHAR(255) NOT NULL,
+ EXCLUDE USING btree (lower(email) WITH =)
+);
+
+-- Add some data to the table and then update it in ways that should and should
+-- not be HOT updates.
+INSERT INTO users (name, email) VALUES
+('user1', '[email protected]'),
+('user2', '[email protected]'),
+('taken', '[email protected]'),
+('you', '[email protected]'),
+('taken', '[email protected]');
+
+-- Should fail because of the unique constraint on the email column.
+UPDATE users SET email = '[email protected]' WHERE email = '[email protected]';
+SELECT pg_stat_get_xact_tuples_hot_updated('users'::regclass); -- expect: 0 no new HOT updates
+
+-- Should succeed because the email column is not being updated and should go HOT.
+UPDATE users SET name = 'foo' WHERE email = '[email protected]';
+SELECT pg_stat_get_xact_tuples_hot_updated('users'::regclass); -- expect: 1 a single new HOT update
+
+-- Create a partial index on the email column, updates
+CREATE INDEX idx_users_email_no_example ON users (lower(email)) WHERE lower(email) LIKE '%@example.com%';
+
+-- An update that changes the email column but not the indexed portion of it and falls outside the constraint.
+-- Shouldn't be a HOT update because of the exclusion constraint.
+UPDATE users SET email = '[email protected]' WHERE name = 'you';
+SELECT pg_stat_get_xact_tuples_hot_updated('users'::regclass); -- expect: 1 no new HOT updates
+
+-- An update that changes the email column but not the indexed portion of it and falls within the constraint.
+-- Again, should fail constraint and fail to be a HOT update.
+UPDATE users SET email = '[email protected]' WHERE name = 'you';
+SELECT pg_stat_get_xact_tuples_hot_updated('users'::regclass); -- expect: 1 no new HOT updates
+
+DROP TABLE users;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- Another test of constraints spoiling HOT updates, this time with a range.
+CREATE TABLE events (
+ id serial primary key,
+ name VARCHAR(255) NOT NULL,
+ event_time tstzrange,
+ constraint no_screening_time_overlap exclude using gist (
+ event_time WITH &&
+ )
+);
+
+-- Add two non-overlapping events.
+INSERT INTO events (id, event_time, name)
+VALUES
+ (1, '["2023-01-01 19:00:00", "2023-01-01 20:45:00"]', 'event1'),
+ (2, '["2023-01-01 21:00:00", "2023-01-01 21:45:00"]', 'event2');
+
+-- Update the first event to overlap with the second, should fail the constraint and not be HOT.
+UPDATE events SET event_time = '["2023-01-01 20:00:00", "2023-01-01 21:45:00"]' WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('events'::regclass); -- expect: 0 no new HOT updates
+
+-- Update the first event to not overlap with the second, again not HOT due to the constraint.
+UPDATE events SET event_time = '["2023-01-01 22:00:00", "2023-01-01 22:45:00"]' WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('events'::regclass); -- expect: 0 no new HOT updates
+
+-- Update the first event to not overlap with the second, this time we're HOT because we don't overlap with the constraint.
+UPDATE events SET name = 'new name here' WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('events'::regclass); -- expect: 1 one new HOT update
+
+DROP TABLE events;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- A test to ensure that only modified summarizing indexes are updated, not
+-- all of them.
+CREATE TABLE ex (id SERIAL primary key, att1 JSONB, att2 text, att3 text, att4 text) WITH (fillfactor = 60);
+CREATE INDEX ex_expr1_idx ON ex USING btree((att1->'data'));
+CREATE INDEX ex_sumr1_idx ON ex USING BRIN(att2);
+CREATE INDEX ex_expr2_idx ON ex USING btree((att1->'a'));
+CREATE INDEX ex_expr3_idx ON ex USING btree((att1->'b'));
+CREATE INDEX ex_expr4_idx ON ex USING btree((att1->'c'));
+CREATE INDEX ex_sumr2_idx ON ex USING BRIN(att3);
+CREATE INDEX ex_sumr3_idx ON ex USING BRIN(att4);
+CREATE INDEX ex_expr5_idx ON ex USING btree((att1->'d'));
+INSERT INTO ex (att1, att2) VALUES ('{"data": []}'::json, 'nothing special');
+
+SELECT * FROM ex;
+
+-- Update att2 and att4 both are BRIN/summarizing indexes, this should be a HOT update and
+-- only update two of the three summarizing indexes.
+UPDATE ex SET att2 = 'special indeed', att4 = 'whatever';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 one new HOT update
+SELECT * FROM ex;
+
+-- Update att1 and att2, only one is BRIN/summarizing, this should NOT be a HOT update.
+UPDATE ex SET att1 = att1 || '{"data": "howdy"}', att2 = 'special, so special';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+SELECT * FROM ex;
+
+-- Update att2, att3, and att4 all are BRIN/summarizing indexes, this should be a HOT update
+-- and yet still update all three summarizing indexes.
+UPDATE ex SET att2 = 'a', att3 = 'b', att4 = 'c';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 2 with one new HOT update
+SELECT * FROM ex;
+
+-- Update att1, att2, and att3 all modified values are BRIN/summarizing indexes, this should be a HOT update
+-- and yet still update all three summarizing indexes.
+UPDATE ex SET att1 = '{"data": "howdy"}', att2 = 'd', att3 = 'e';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 3 with one new HOT update
+SELECT * FROM ex;
+
+DROP TABLE ex;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- A test to ensure that summarizing indexes are not updated when they don't
+-- change, but are updated when they do while not prefent HOT updates.
+CREATE TABLE ex (att1 JSONB, att2 text) WITH (fillfactor = 60);
+CREATE INDEX ex_expr1_idx ON ex USING btree((att1->'data'));
+CREATE INDEX ex_sumr1_idx ON ex USING BRIN(att2);
+INSERT INTO ex VALUES ('{"data": []}', 'nothing special');
+
+-- Update the unindexed value of att1, this should be a HOT update and and should
+-- update the summarizing index.
+UPDATE ex SET att1 = att1 || '{"status": "stalemate"}';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 one new HOT update
+
+-- Update the indexed value of att2, a summarized value, this is a summarized
+-- only update and should use the HOT path while still triggering an update to
+-- the summarizing BRIN index.
+UPDATE ex SET att2 = 'special indeed';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 2 one new HOT update
+
+-- Update to att1 doesn't change the indexed value while the update to att2 does,
+-- this again is a summarized only update and should use the HOT path as well as
+-- trigger an update to the BRIN index.
+UPDATE ex SET att1 = att1 || '{"status": "checkmate"}', att2 = 'special, so special';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 3 one new HOT update
+
+-- This updates both indexes, the expression index on att1 and the summarizing
+-- index on att2. This should not be a HOT update because there are modified
+-- indexes and only some are summarized, not all. This should force all
+-- indexes to be updated.
+UPDATE ex SET att1 = att1 || '{"data": [1,2,3]}', att2 = 'do you want to play a game?';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 4 both indexes updated
+
+DROP TABLE ex;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This test is for a table with a custom type and a custom operators on
+-- the BTREE index. The question is, when comparing values for equality
+-- to determine if there are changes on the index or not... shouldn't we
+-- be using the custom operators?
+
+-- Create a type
+CREATE TYPE my_custom_type AS (val int);
+
+-- Comparison functions (returns boolean)
+CREATE FUNCTION my_custom_lt(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val < b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_le(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val <= b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_eq(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val = b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_ge(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val >= b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_gt(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val > b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_ne(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val != b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+-- Comparison function (returns -1, 0, 1)
+CREATE FUNCTION my_custom_cmp(a my_custom_type, b my_custom_type) RETURNS int AS $$
+BEGIN
+ IF a.val < b.val THEN
+ RETURN -1;
+ ELSIF a.val > b.val THEN
+ RETURN 1;
+ ELSE
+ RETURN 0;
+ END IF;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+-- Create the operators
+CREATE OPERATOR < (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_lt,
+ COMMUTATOR = >,
+ NEGATOR = >=
+);
+
+CREATE OPERATOR <= (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_le,
+ COMMUTATOR = >=,
+ NEGATOR = >
+);
+
+CREATE OPERATOR = (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_eq,
+ COMMUTATOR = =,
+ NEGATOR = <>
+);
+
+CREATE OPERATOR >= (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_ge,
+ COMMUTATOR = <=,
+ NEGATOR = <
+);
+
+CREATE OPERATOR > (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_gt,
+ COMMUTATOR = <,
+ NEGATOR = <=
+);
+
+CREATE OPERATOR <> (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_ne,
+ COMMUTATOR = <>,
+ NEGATOR = =
+);
+
+-- Create the operator class (including the support function)
+CREATE OPERATOR CLASS my_custom_ops
+ DEFAULT FOR TYPE my_custom_type USING btree AS
+ OPERATOR 1 <,
+ OPERATOR 2 <=,
+ OPERATOR 3 =,
+ OPERATOR 4 >=,
+ OPERATOR 5 >,
+ FUNCTION 1 my_custom_cmp(my_custom_type, my_custom_type);
+
+-- Create the table
+CREATE TABLE my_table (
+ id int,
+ custom_val my_custom_type
+);
+
+-- Insert some data
+INSERT INTO my_table (id, custom_val) VALUES
+(1, ROW(3)::my_custom_type),
+(2, ROW(1)::my_custom_type),
+(3, ROW(4)::my_custom_type),
+(4, ROW(2)::my_custom_type);
+
+-- Create a function to use when indexing
+CREATE OR REPLACE FUNCTION abs_val(val my_custom_type) RETURNS int AS $$
+BEGIN
+ RETURN abs(val.val);
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+-- Create the index
+CREATE INDEX idx_custom_val_abs ON my_table (abs_val(custom_val));
+
+-- Update 1
+UPDATE my_table SET custom_val = ROW(5)::my_custom_type WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+
+-- Update 2
+UPDATE my_table SET custom_val = ROW(0)::my_custom_type WHERE custom_val < ROW(3)::my_custom_type;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+
+-- Update 3
+UPDATE my_table SET custom_val = ROW(6)::my_custom_type WHERE id = 3;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+
+-- Update 4
+UPDATE my_table SET id = 5 WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+
+-- Query using the index
+EXPLAIN (COSTS OFF) SELECT * FROM my_table WHERE abs_val(custom_val) = 6;
+SELECT * FROM my_table WHERE abs_val(custom_val) = 6;
+
+-- Clean up
+DROP TABLE my_table CASCADE;
+DROP OPERATOR CLASS my_custom_ops USING btree CASCADE;
+DROP OPERATOR < (my_custom_type, my_custom_type);
+DROP OPERATOR <= (my_custom_type, my_custom_type);
+DROP OPERATOR = (my_custom_type, my_custom_type);
+DROP OPERATOR >= (my_custom_type, my_custom_type);
+DROP OPERATOR > (my_custom_type, my_custom_type);
+DROP OPERATOR <> (my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_lt(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_le(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_eq(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_ge(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_gt(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_ne(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_cmp(my_custom_type, my_custom_type);
+DROP FUNCTION abs_val(my_custom_type);
+DROP TYPE my_custom_type CASCADE;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 3fbf5a4c212..0b6587dbd56 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2934,6 +2934,7 @@ TSVectorStat
TState
TStatus
TStoreState
+UpdateContext
TU_UpdateIndexes
TXNEntryFile
TYPCATEGORY
@@ -3118,7 +3119,6 @@ UniqueState
UnlistenStmt
UnresolvedTup
UnresolvedTupData
-UpdateContext
UpdateStmt
UpgradeTask
UpgradeTaskProcessCB
--
2.42.0
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
@ 2025-03-25 14:21 ` Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Burd, Greg @ 2025-03-25 14:21 UTC (permalink / raw)
To: Matthias van de Meent <[email protected]>; +Cc: pgsql-hackers
Apologies for the noise, I overlooked a compiler warning.
fixed.
-greg
> On Mar 25, 2025, at 7:47 AM, Burd, Greg <[email protected]> wrote:
>
> Matthias,
>
> Rebased patch attached.
>
> Changes in v14:
> * UpdateContext now the location I've stored estate, resultRelInfo, etc.
> * Reuse the result from the predicate on the partial index.
>
> -greg
>
>
>
>> On Mar 7, 2025, at 5:47 PM, Matthias van de Meent <[email protected]> wrote:
>>
>> On Thu, 6 Mar 2025 at 13:40, Burd, Greg <[email protected]> wrote:
>>
>>
>>>
>>>
>>>
>>>> On Mar 5, 2025, at 6:39 PM, Matthias van de Meent <[email protected]> wrote:
>>>>
>>>> On Wed, 5 Mar 2025 at 18:21, Burd, Greg <[email protected]> wrote:
>>>>
>>>>
>>>>> * augments IndexInfo only when needed for testing expressions and only once
>>>>
>>>>
>>>>
>>>> ExecExpressionIndexesUpdated seems to always loop over all indexes,
>>>> always calling AttributeIndexInfo which always updates the fields in
>>>> the IndexInfo when the index has only !byval attributes (e.g. text,
>>>> json, or other such varlena types). You say it happens only once, have
>>>> I missed something?
>>>
>>>
>>>
>>> There's a test that avoids doing it more than once, [...]
>>
>>
>>
>> Is this that one?
>>
>> + if (indexInfo->ii_IndexAttrByVal)
>> + return indexInfo;
>>
>> I think that test doesn't work consistently: a bitmapset * is NULL
>> when no bits are set; and for some indexes no attribute will be byval,
>> thus failing this early-exit even after processing.
>>
>> Another small issue with this approach is that it always calls and
>> tests in EEIU(), while it's quite likely we would do better if we
>> pre-processed _all_ indexes at once, so that we can have a path that
>> doesn't repeatedly get into EEIU only to exit immediately after. It'll
>> probably be hot enough to not matter much, but it's still cycles spent
>> on something that we can optimize for in code.
>
>
> I've changed this a bit, now in ExecOpenIndices() when there are expressions or predicates I augment the IndexInfo with information necessary to perform the tests in EEIU(). I've debated adding another bool to ExecOpenIndices() to indicate that we're opening indexes for the purpose of an update to avoid building that information in cases where we are not. Similar to the bool `speculative` on ExecOpenIndices() today. Thoughts?
>
>
>>> I might widen this patch a bit to include support for testing equality of index tuples using custom operators when they exist for the index. In the use case I'm solving for we use a custom operator for equality that is not the same as a memcmp(). Do you have thoughts on that?
>>
>>
>>
>> I don't think that's a very great idea. From a certain point of view,
>> you can see HOT as "deduplicating multiple tuple versions behind a
>> single TID". Btree doesn't support deduplication for types that can
>> have more than one representation of the same value so that e.g.
>> '0.0'::numeric and '0'::numeric are both displayed correctly, even
>> when they compare as equal according to certain equality operators.
>
>
> Interesting, good point. Seems like it would require a new index AM function:
> bool indexed_tuple_would_change()
>
> I'll drop this for now, it seems out of scope for this patch set.
>
>
>
> <v14-0001-Expand-HOT-update-path-to-include-expression-and.patch>
Attachments:
[application/octet-stream] v15-0001-Expand-HOT-update-path-to-include-expression-and.patch (98.7K, 2-v15-0001-Expand-HOT-update-path-to-include-expression-and.patch)
download | inline diff:
From 989601d40a8a693631ae5194dfe59a95c98eb1b9 Mon Sep 17 00:00:00 2001
From: Gregory Burd <[email protected]>
Date: Mon, 27 Jan 2025 13:28:59 -0500
Subject: [PATCH v15] Expand HOT update path to include expression and partial
indexes.
This patch extends the cases where HOT updates are possible in the heapam by
examining expression indexes and determining if indexed values where mutated
or not. Previously, any expression index on a column would disqualify it
from the HOT update path. Also examines partial indexes to see if the
values are within the predicate or not.
This is a modified application of a patch proposed on the pgsql-hackers list:
https://www.postgresql.org/message-id/flat/4d9928ee-a9e6-15f9-9c82-5981f13ffca6%40postgrespro.ru
applied: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=c203d6cf8
reverted: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=05f84605dbeb9cf8279a157234b24bbb706c5256
Signed-off-by: Greg Burd <[email protected]>
---
doc/src/sgml/ref/create_table.sgml | 18 +
doc/src/sgml/storage.sgml | 21 +
src/backend/access/common/reloptions.c | 12 +-
src/backend/access/heap/README.HOT | 45 +-
src/backend/access/heap/heapam.c | 43 +-
src/backend/access/heap/heapam_handler.c | 15 +-
src/backend/access/table/tableam.c | 5 +-
src/backend/catalog/index.c | 46 ++
src/backend/catalog/indexing.c | 16 +-
src/backend/executor/execIndexing.c | 192 ++++-
src/backend/executor/execReplication.c | 10 +-
src/backend/executor/nodeModifyTable.c | 27 +-
src/backend/utils/cache/relcache.c | 70 +-
src/bin/psql/tab-complete.in.c | 2 +-
src/include/access/heapam.h | 6 +-
src/include/access/tableam.h | 33 +-
src/include/catalog/index.h | 1 +
src/include/executor/executor.h | 5 +
src/include/nodes/execnodes.h | 13 +
src/include/utils/rel.h | 10 +
src/include/utils/relcache.h | 1 +
.../regress/expected/heap_hot_updates.out | 665 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
src/test/regress/sql/heap_hot_updates.sql | 481 +++++++++++++
src/tools/pgindent/typedefs.list | 2 +-
25 files changed, 1639 insertions(+), 105 deletions(-)
create mode 100644 src/test/regress/expected/heap_hot_updates.out
create mode 100644 src/test/regress/sql/heap_hot_updates.sql
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index e5c034d724e..569b2c3ee45 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -1980,6 +1980,24 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry id="reloption-expression-checks" xreflabel="expression_checks">
+ <term><literal>expression_checks</literal> (<type>boolean</type>)
+ <indexterm>
+ <primary><varname>expression_checks</varname> storage parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Enables or disables evaulation of predicate expressions on partial
+ indexes or expressions used to define indexes during updates.
+ If <literal>true</literal>, then these expressions are evaluated during
+ updates to data within the heap relation against the old and new values
+ and then compared to determine if <acronym>HOT</acronym> updates are
+ allowable or not. The default value is <literal>true</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect2>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index 61250799ec0..64a6264d8fd 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -1138,6 +1138,27 @@ data. Empty in ordinary tables.</entry>
</itemizedlist>
</para>
+ <para>
+ <acronym>HOT</acronym> updates can occur when the expression used to define
+ an index shows no changes to the indexed value. To determine this requires
+ that the expression be evaulated for the old and new values to be stored in
+ the index and then compared. This allows for <acronym>HOT</acronym> updates
+ when data indexed within JSONB columns is unchanged. To disable this
+ behavior and avoid the overhead of evaluating the expression during updates
+ set the <literal>expression_checks</literal> option to false for the table.
+ </para>
+
+ <para>
+ <acronym>HOT</acronym> updates can also occur when updated values are not
+ within the predicate of a partial index. However, <acronym>HOT</acronym>
+ updates are not possible when the updated value and the current value differ
+ with regards to the predicate. To determin this requires that the predicate
+ expression be evaluated for the old and new values to be stored in the index
+ and then compared. To disable this behavior and avoid the overhead of
+ evaluating the expression during updates set
+ the <literal>expression_checks</literal> option to false for the table.
+ </para>
+
<para>
You can increase the likelihood of sufficient page space for
<acronym>HOT</acronym> updates by decreasing a table's <link
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 645b5c00467..41c727eafcb 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -166,6 +166,15 @@ static relopt_bool boolRelOpts[] =
},
true
},
+ {
+ {
+ "expression_checks",
+ "When disabled prevents checking expressions on indexes and predicates on partial indexes for changes that might influence heap-only tuple (HOT) updates.",
+ RELOPT_KIND_HEAP,
+ ShareUpdateExclusiveLock
+ },
+ true
+ },
/* list terminator */
{{NULL}}
};
@@ -1914,7 +1923,8 @@ default_reloptions(Datum reloptions, bool validate, relopt_kind kind)
{"vacuum_truncate", RELOPT_TYPE_BOOL,
offsetof(StdRdOptions, vacuum_truncate), offsetof(StdRdOptions, vacuum_truncate_set)},
{"vacuum_max_eager_freeze_failure_rate", RELOPT_TYPE_REAL,
- offsetof(StdRdOptions, vacuum_max_eager_freeze_failure_rate)}
+ offsetof(StdRdOptions, vacuum_max_eager_freeze_failure_rate)},
+ {"expression_checks", RELOPT_TYPE_BOOL, offsetof(StdRdOptions, expression_checks)}
};
return (bytea *) build_reloptions(reloptions, validate, kind,
diff --git a/src/backend/access/heap/README.HOT b/src/backend/access/heap/README.HOT
index 74e407f375a..2340d82053f 100644
--- a/src/backend/access/heap/README.HOT
+++ b/src/backend/access/heap/README.HOT
@@ -36,7 +36,7 @@ HOT solves this problem for two restricted but useful special cases:
First, where a tuple is repeatedly updated in ways that do not change
its indexed columns. (Here, "indexed column" means any column referenced
at all in an index definition, including for example columns that are
-tested in a partial-index predicate but are not stored in the index.)
+tested in a partial-index predicate, that has materially changed.)
Second, where the modified columns are only used in indexes that do not
contain tuple IDs, but maintain summaries of the indexed data by block.
@@ -133,28 +133,34 @@ Note: we can use a "dead" line pointer for any DELETEd tuple,
whether it was part of a HOT chain or not. This allows space reclamation
in advance of running VACUUM for plain DELETEs as well as HOT updates.
-The requirement for doing a HOT update is that indexes which point to
-the root line pointer (and thus need to be cleaned up by VACUUM when the
-tuple is dead) do not reference columns which are updated in that HOT
-chain. Summarizing indexes (such as BRIN) are assumed to have no
-references to individual tuples and thus are ignored when checking HOT
-applicability. The updated columns are checked at execution time by
-comparing the binary representation of the old and new values. We insist
-on bitwise equality rather than using datatype-specific equality routines.
-The main reason to avoid the latter is that there might be multiple
-notions of equality for a datatype, and we don't know exactly which one
-is relevant for the indexes at hand. We assume that bitwise equality
-guarantees equality for all purposes.
+The requirement for doing a HOT update is that indexes which point to the root
+line pointer (and thus need to be cleaned up by VACUUM when the tuple is dead)
+do not reference columns which are updated in that HOT chain.
+
+Summarizing indexes (such as BRIN) are assumed to have no references to
+individual tuples and thus are ignored when checking HOT applicability.
+
+Expressions on indexes are evaluated and the results used when checking for
+changes. This allows for the JSONB datatype to have HOT updates when the
+indexed portion of the document are not modified.
+
+Partial index expressions are evaluated, HOT updates are allowed when the
+updated index values do not satisfy the predicate.
+
+The updated columns are checked at execution time by comparing the binary
+representation of the old and new values. We insist on bitwise equality rather
+than using datatype-specific equality routines. The main reason to avoid the
+latter is that there might be multiple notions of equality for a datatype, and
+we don't know exactly which one is relevant for the indexes at hand. We assume
+that bitwise equality guarantees equality for all purposes.
If any columns that are included by non-summarizing indexes are updated,
the HOT optimization is not applied, and the new tuple is inserted into
all indexes of the table. If none of the updated columns are included in
the table's indexes, the HOT optimization is applied and no indexes are
updated. If instead the updated columns are only indexed by summarizing
-indexes, the HOT optimization is applied, but the update is propagated to
-all summarizing indexes. (Realistically, we only need to propagate the
-update to the indexes that contain the updated values, but that is yet to
-be implemented.)
+indexes, the HOT optimization is applied and the update is propagated to
+all of the summarizing indexes.
Abort Cases
-----------
@@ -477,8 +483,9 @@ Heap-only tuple
HOT-safe
A proposed tuple update is said to be HOT-safe if it changes
- none of the tuple's indexed columns. It will only become an
- actual HOT update if we can find room on the same page for
+ none of the tuple's indexed columns or if the changes remain
+ outside of a partial index's predicate. It will only become
+ an actual HOT update if we can find room on the same page for
the new tuple version.
HOT update
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index b12b583c4d9..8b523d9cb82 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -3244,13 +3244,14 @@ simple_heap_delete(Relation relation, ItemPointer tid)
TM_Result
heap_update(Relation relation, ItemPointer otid, HeapTuple newtup,
CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ TM_FailureData *tmfd, UpdateContext *updateCxt)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
+ LockTupleMode *lockmode = &updateCxt->lockmode;
Bitmapset *hot_attrs;
Bitmapset *sum_attrs;
+ Bitmapset *exp_attrs;
Bitmapset *key_attrs;
Bitmapset *id_attrs;
Bitmapset *interesting_attrs;
@@ -3327,6 +3328,8 @@ heap_update(Relation relation, ItemPointer otid, HeapTuple newtup,
INDEX_ATTR_BITMAP_HOT_BLOCKING);
sum_attrs = RelationGetIndexAttrBitmap(relation,
INDEX_ATTR_BITMAP_SUMMARIZED);
+ exp_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_EXPRESSION);
key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
id_attrs = RelationGetIndexAttrBitmap(relation,
INDEX_ATTR_BITMAP_IDENTITY_KEY);
@@ -3388,10 +3391,11 @@ heap_update(Relation relation, ItemPointer otid, HeapTuple newtup,
tmfd->ctid = *otid;
tmfd->xmax = InvalidTransactionId;
tmfd->cmax = InvalidCommandId;
- *update_indexes = TU_None;
+ updateCxt->updateIndexes = TU_None;
bms_free(hot_attrs);
bms_free(sum_attrs);
+ bms_free(exp_attrs);
bms_free(key_attrs);
bms_free(id_attrs);
/* modified_attrs not yet initialized */
@@ -3689,10 +3693,11 @@ l2:
UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
if (vmbuffer != InvalidBuffer)
ReleaseBuffer(vmbuffer);
- *update_indexes = TU_None;
+ updateCxt->updateIndexes = TU_None;
bms_free(hot_attrs);
bms_free(sum_attrs);
+ bms_free(exp_attrs);
bms_free(key_attrs);
bms_free(id_attrs);
bms_free(modified_attrs);
@@ -4009,12 +4014,24 @@ l2:
if (newbuf == buffer)
{
+ ResultRelInfo *resultRelInfo = updateCxt->rri;
+ EState *estate = updateCxt->estate;
+ bool expression_checks = RelationGetExpressionChecks(relation);
+
/*
* Since the new tuple is going into the same page, we might be able
- * to do a HOT update. Check if any of the index columns have been
- * changed.
+ * to do a HOT update. As a reminder, hot_attrs includes attributes
+ * used by indexes including within expressions and predicates, but
+ * not attributes only used by summarizing indexes.
*/
- if (!bms_overlap(modified_attrs, hot_attrs))
+ if (!bms_overlap(modified_attrs, hot_attrs) ||
+ (estate && resultRelInfo && expression_checks &&
+ bms_overlap(modified_attrs, exp_attrs) &&
+ !ExecExpressionIndexesUpdated(resultRelInfo,
+ modified_attrs,
+ estate,
+ resultRelInfo->ri_oldTupleSlot,
+ resultRelInfo->ri_newTupleSlot)))
{
use_hot_update = true;
@@ -4196,18 +4213,19 @@ l2:
if (use_hot_update)
{
if (summarized_update)
- *update_indexes = TU_Summarizing;
+ updateCxt->updateIndexes = TU_Summarizing;
else
- *update_indexes = TU_None;
+ updateCxt->updateIndexes = TU_None;
}
else
- *update_indexes = TU_All;
+ updateCxt->updateIndexes = TU_All;
if (old_key_tuple != NULL && old_key_copied)
heap_freetuple(old_key_tuple);
bms_free(hot_attrs);
bms_free(sum_attrs);
+ bms_free(exp_attrs);
bms_free(key_attrs);
bms_free(id_attrs);
bms_free(modified_attrs);
@@ -4485,16 +4503,15 @@ HeapDetermineColumnsInfo(Relation relation,
*/
void
simple_heap_update(Relation relation, ItemPointer otid, HeapTuple tup,
- TU_UpdateIndexes *update_indexes)
+ UpdateContext *updateCxt)
{
TM_Result result;
TM_FailureData tmfd;
- LockTupleMode lockmode;
result = heap_update(relation, otid, tup,
GetCurrentCommandId(true), InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, updateCxt);
switch (result)
{
case TM_SelfModified:
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 24d3765aa20..54f627c5b27 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -316,8 +316,7 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd,
- LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
+ bool wait, TM_FailureData *tmfd, UpdateContext *updateCxt)
{
bool shouldFree = true;
HeapTuple tuple = ExecFetchSlotHeapTuple(slot, true, &shouldFree);
@@ -328,7 +327,7 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
tuple->t_tableOid = slot->tts_tableOid;
result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
- tmfd, lockmode, update_indexes);
+ tmfd, updateCxt);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
@@ -343,14 +342,14 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
*/
if (result != TM_Ok)
{
- Assert(*update_indexes == TU_None);
- *update_indexes = TU_None;
+ Assert(updateCxt->updateIndexes == TU_None);
+ updateCxt->updateIndexes = TU_None;
}
else if (!HeapTupleIsHeapOnly(tuple))
- Assert(*update_indexes == TU_All);
+ Assert(updateCxt->updateIndexes == TU_All);
else
- Assert((*update_indexes == TU_Summarizing) ||
- (*update_indexes == TU_None));
+ Assert((updateCxt->updateIndexes == TU_Summarizing) ||
+ (updateCxt->updateIndexes == TU_None));
if (shouldFree)
pfree(tuple);
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index a56c5eceb14..1eeeebadc09 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -336,17 +336,16 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
- TU_UpdateIndexes *update_indexes)
+ UpdateContext *updateCxt)
{
TM_Result result;
TM_FailureData tmfd;
- LockTupleMode lockmode;
result = table_tuple_update(rel, otid, slot,
GetCurrentCommandId(true),
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, updateCxt);
switch (result)
{
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 739a92bdcc1..0d6a0235e79 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2707,6 +2707,52 @@ BuildSpeculativeIndexInfo(Relation index, IndexInfo *ii)
}
}
+/* ----------------
+ * BuildExpressionIndexInfo
+ * Add extra state to IndexInfo record
+ *
+ * For expression indexes updates may not change the indexed value allowing
+ * for a HOT update. Add information to the IndexInfo to allow for checking
+ * if the indexed value has changed.
+ *
+ * Do this processing here rather than in BuildIndexInfo() to not incur the
+ * overhead in the common non-expression cases.
+ * ----------------
+ */
+void
+BuildExpressionIndexInfo(Relation index, IndexInfo *ii)
+{
+ int i;
+ int indnkeyatts;
+
+ indnkeyatts = IndexRelationGetNumberOfKeyAttributes(index);
+
+ /*
+ * Collect attributes used by the index, their len and if they are by
+ * value.
+ */
+ for (i = 0; i < indnkeyatts; i++)
+ {
+ CompactAttribute *attr = TupleDescCompactAttr(RelationGetDescr(index), i);
+
+ ii->ii_IndexAttrs =
+ bms_add_member(ii->ii_IndexAttrs,
+ ii->ii_IndexAttrNumbers[i] - FirstLowInvalidHeapAttributeNumber);
+
+ ii->ii_IndexAttrLen[i] = attr->attlen;
+ if (attr->attbyval)
+ ii->ii_IndexAttrByVal = bms_add_member(ii->ii_IndexAttrByVal, i);
+ }
+
+ /* collect attributes used in the expression */
+ if (ii->ii_Expressions)
+ pull_varattnos((Node *) ii->ii_Expressions, 1, &ii->ii_ExpressionAttrs);
+
+ /* collect attributes used in the predicate */
+ if (ii->ii_Predicate)
+ pull_varattnos((Node *) ii->ii_Predicate, 1, &ii->ii_PredicateAttrs);
+}
+
/* ----------------
* FormIndexDatum
* Construct values[] and isnull[] arrays for a new index tuple.
diff --git a/src/backend/catalog/indexing.c b/src/backend/catalog/indexing.c
index 25c4b6bdc87..91a6b455b14 100644
--- a/src/backend/catalog/indexing.c
+++ b/src/backend/catalog/indexing.c
@@ -313,15 +313,17 @@ void
CatalogTupleUpdate(Relation heapRel, ItemPointer otid, HeapTuple tup)
{
CatalogIndexState indstate;
- TU_UpdateIndexes updateIndexes = TU_All;
+ UpdateContext updateCxt = {0};
+
+ updateCxt.updateIndexes = TU_All;
CatalogTupleCheckConstraints(heapRel, tup);
indstate = CatalogOpenIndexes(heapRel);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ simple_heap_update(heapRel, otid, tup, &updateCxt);
- CatalogIndexInsert(indstate, tup, updateIndexes);
+ CatalogIndexInsert(indstate, tup, updateCxt.updateIndexes);
CatalogCloseIndexes(indstate);
}
@@ -337,13 +339,15 @@ void
CatalogTupleUpdateWithInfo(Relation heapRel, ItemPointer otid, HeapTuple tup,
CatalogIndexState indstate)
{
- TU_UpdateIndexes updateIndexes = TU_All;
+ UpdateContext updateCxt = {0};
+
+ updateCxt.updateIndexes = TU_All;
CatalogTupleCheckConstraints(heapRel, tup);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ simple_heap_update(heapRel, otid, tup, &updateCxt);
- CatalogIndexInsert(indstate, tup, updateIndexes);
+ CatalogIndexInsert(indstate, tup, updateCxt.updateIndexes);
}
/*
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index e3fe9b78bb5..429e08f759c 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -113,10 +113,13 @@
#include "catalog/index.h"
#include "executor/executor.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/optimizer.h"
#include "storage/lmgr.h"
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
+#include "utils/datum.h"
+#include "utils/lsyscache.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
typedef enum
@@ -221,6 +224,13 @@ ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative)
if (speculative && ii->ii_Unique && !indexDesc->rd_index->indisexclusion)
BuildSpeculativeIndexInfo(indexDesc, ii);
+ /*
+ * If the index uses expressions then let's populate the additional
+ * information nessaary to evaluate them for changes during updates.
+ */
+ if (ii->ii_Expressions || ii->ii_Predicate)
+ BuildExpressionIndexInfo(indexDesc, ii);
+
relationDescs[i] = indexDesc;
indexInfoArray[i] = ii;
i++;
@@ -383,19 +393,33 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
ExprState *predicate;
/*
- * If predicate state not set up yet, create it (in the estate's
- * per-query context)
+ * It is possible that we've already checked the predicate, if so
+ * then avoid the duplicate work.
*/
- predicate = indexInfo->ii_PredicateState;
- if (predicate == NULL)
+ if (indexInfo->ii_CheckedPredicate)
{
- predicate = ExecPrepareQual(indexInfo->ii_Predicate, estate);
- indexInfo->ii_PredicateState = predicate;
+ /* Skip this index-update if the predicate isn't satisfied */
+ if (!indexInfo->ii_PredicateSatisfied)
+ continue;
}
+ else
+ {
- /* Skip this index-update if the predicate isn't satisfied */
- if (!ExecQual(predicate, econtext))
- continue;
+ /*
+ * If predicate state not set up yet, create it (in the
+ * estate's per-query context)
+ */
+ predicate = indexInfo->ii_PredicateState;
+ if (predicate == NULL)
+ {
+ predicate = ExecPrepareQual(indexInfo->ii_Predicate, estate);
+ indexInfo->ii_PredicateState = predicate;
+ }
+
+ /* Skip this index-update if the predicate isn't satisfied */
+ if (!ExecQual(predicate, econtext))
+ continue;
+ }
}
/*
@@ -1096,6 +1120,9 @@ index_unchanged_by_update(ResultRelInfo *resultRelInfo, EState *estate,
if (hasexpression)
{
+ if (indexInfo->ii_IndexUnchanged)
+ return true;
+
indexInfo->ii_IndexUnchanged = false;
return false;
}
@@ -1173,3 +1200,150 @@ ExecWithoutOverlapsNotEmpty(Relation rel, NameData attname, Datum attval, char t
errmsg("empty WITHOUT OVERLAPS value found in column \"%s\" in relation \"%s\"",
NameStr(attname), RelationGetRelationName(rel))));
}
+
+/*
+ * Determine if indexes that have expressions or predicates require updates
+ * for the purposes of allowing HOT updates. Returns false iff all indexed
+ * values are unchanged.
+ */
+bool
+ExecExpressionIndexesUpdated(ResultRelInfo *resultRelInfo,
+ Bitmapset *modifiedAttrs,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts)
+{
+ bool result = false;
+ IndexInfo *indexInfo;
+ ExprContext *econtext = NULL;
+
+ if (old_tts == NULL || new_tts == NULL)
+ return true;
+
+ econtext = GetPerTupleExprContext(estate);
+
+ /*
+ * Examine each index on this relation relative to the changes between old
+ * and new tuples.
+ */
+ for (int i = 0; i < resultRelInfo->ri_NumIndices; i++)
+ {
+ indexInfo = resultRelInfo->ri_IndexRelationInfo[i];
+
+ /*
+ * If this is a partial index it has a predicate, evaluate the
+ * expression to determine if we need to include it or not.
+ */
+ if (bms_overlap(indexInfo->ii_PredicateAttrs, modifiedAttrs))
+ {
+ ExprState *pstate;
+ bool old_tuple_qualifies,
+ new_tuple_qualifies;
+
+ pstate = ExecPrepareQual(indexInfo->ii_Predicate, estate);
+
+ /*
+ * Here the term "qualifies" means "satisfies the predicate
+ * condition of the partial index".
+ */
+ econtext->ecxt_scantuple = old_tts;
+ old_tuple_qualifies = ExecQual(pstate, econtext);
+
+ econtext->ecxt_scantuple = new_tts;
+ new_tuple_qualifies = ExecQual(pstate, econtext);
+ indexInfo->ii_CheckedPredicate = true;
+ indexInfo->ii_PredicateSatisfied = new_tuple_qualifies;
+
+ /*
+ * If neither the old nor the new tuples satisfy the predicate we
+ * can be sure that this index doesn't need updating, continue to
+ * the next index.
+ */
+ if ((new_tuple_qualifies == false) && (old_tuple_qualifies == false))
+ continue;
+
+ /*
+ * If there is a transition between indexed and not indexed,
+ * that's enough to require that this index is updated. If any
+ * single non-summarizing index requires updates then they all
+ * should be updated.
+ */
+ if (new_tuple_qualifies != old_tuple_qualifies)
+ {
+ result = true;
+ break;
+ }
+
+ /*
+ * Otherwise the old and new values exist in the index, but did
+ * they get updated? We don't yet know, so proceed with the next
+ * statement in the loop to find out.
+ */
+ }
+
+ /*
+ * Indexes with expressions may or may not have changed, it is
+ * impossible to know without exercising their expression and
+ * reviewing index tuple state for changes. This is a lot of work,
+ * but because all indexes on JSONB columns fall into this category it
+ * can be worth it to avoid index updates and remain on the HOT update
+ * path when possible.
+ */
+ if (bms_overlap(indexInfo->ii_ExpressionAttrs, modifiedAttrs))
+ {
+ TupleTableSlot *save_scantuple;
+ Datum old_values[INDEX_MAX_KEYS];
+ bool old_isnull[INDEX_MAX_KEYS];
+ Datum new_values[INDEX_MAX_KEYS];
+ bool new_isnull[INDEX_MAX_KEYS];
+ bool changed = false;
+
+ save_scantuple = econtext->ecxt_scantuple;
+ econtext->ecxt_scantuple = old_tts;
+ FormIndexDatum(indexInfo,
+ old_tts,
+ estate,
+ old_values,
+ old_isnull);
+ econtext->ecxt_scantuple = new_tts;
+ FormIndexDatum(indexInfo,
+ new_tts,
+ estate,
+ new_values,
+ new_isnull);
+ econtext->ecxt_scantuple = save_scantuple;
+
+ for (int j = 0; j < indexInfo->ii_NumIndexKeyAttrs; j++)
+ {
+ if (old_isnull[j] != new_isnull[j])
+ {
+ changed = true;
+ break;
+ }
+ else if (!old_isnull[j])
+ {
+ int16 elmlen = indexInfo->ii_IndexAttrLen[j];
+ bool elmbyval = bms_is_member(j, indexInfo->ii_IndexAttrByVal);
+
+ if (!datum_image_eq(old_values[j], new_values[j],
+ elmbyval, elmlen))
+ {
+ changed = true;
+ break;
+ }
+ }
+ }
+
+ indexInfo->ii_CheckedUnchanged = true;
+ indexInfo->ii_IndexUnchanged = !changed;
+
+ if (changed)
+ {
+ result = true;
+ break;
+ }
+ }
+ }
+
+ return result;
+}
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index ede89ea3cf9..c13fb6befd5 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -647,7 +647,7 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (!skip_tuple)
{
List *recheckIndexes = NIL;
- TU_UpdateIndexes update_indexes;
+ UpdateContext updateCxt = {0};
List *conflictindexes;
bool conflict = false;
@@ -663,17 +663,19 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ updateCxt.estate = estate;
+ updateCxt.rri = resultRelInfo;
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
- &update_indexes);
+ &updateCxt);
conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
- if (resultRelInfo->ri_NumIndices > 0 && (update_indexes != TU_None))
+ if (resultRelInfo->ri_NumIndices > 0 && (updateCxt.updateIndexes != TU_None))
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
slot, estate, true,
conflictindexes ? true : false,
&conflict, conflictindexes,
- (update_indexes == TU_Summarizing));
+ (updateCxt.updateIndexes == TU_Summarizing));
/*
* Refer to the comments above the call to CheckAndReportConflict() in
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 87c820276a8..8650c2be096 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -115,21 +115,6 @@ typedef struct ModifyTableContext
TupleTableSlot *cpUpdateReturningSlot;
} ModifyTableContext;
-/*
- * Context struct containing output data specific to UPDATE operations.
- */
-typedef struct UpdateContext
-{
- bool crossPartUpdate; /* was it a cross-partition update? */
- TU_UpdateIndexes updateIndexes; /* Which index updates are required? */
-
- /*
- * Lock mode to acquire on the latest tuple version before performing
- * EvalPlanQual on it
- */
- LockTupleMode lockmode;
-} UpdateContext;
-
static void ExecBatchInsert(ModifyTableState *mtstate,
ResultRelInfo *resultRelInfo,
@@ -2282,8 +2267,7 @@ lreplace:
estate->es_snapshot,
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
- &context->tmfd, &updateCxt->lockmode,
- &updateCxt->updateIndexes);
+ &context->tmfd, updateCxt);
return result;
}
@@ -2301,6 +2285,7 @@ ExecUpdateEpilogue(ModifyTableContext *context, UpdateContext *updateCxt,
{
ModifyTableState *mtstate = context->mtstate;
List *recheckIndexes = NIL;
+ bool onlySummarizing = updateCxt->updateIndexes == TU_Summarizing;
/* insert index entries for tuple if necessary */
if (resultRelInfo->ri_NumIndices > 0 && (updateCxt->updateIndexes != TU_None))
@@ -2308,7 +2293,7 @@ ExecUpdateEpilogue(ModifyTableContext *context, UpdateContext *updateCxt,
slot, context->estate,
true, false,
NULL, NIL,
- (updateCxt->updateIndexes == TU_Summarizing));
+ onlySummarizing);
/* AFTER ROW UPDATE Triggers */
ExecARUpdateTriggers(context->estate, resultRelInfo,
@@ -2444,6 +2429,9 @@ ExecUpdate(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
UpdateContext updateCxt = {0};
TM_Result result;
+ updateCxt.estate = estate;
+ updateCxt.rri = resultRelInfo;
+
/*
* abort the operation if not running transactions
*/
@@ -3132,6 +3120,9 @@ lmerge_matched:
TM_Result result;
UpdateContext updateCxt = {0};
+ updateCxt.rri = resultRelInfo;
+ updateCxt.estate = estate;
+
/*
* Test condition, if any.
*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 9f54a9e72b7..29cef048a87 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -64,6 +64,7 @@
#include "catalog/pg_type.h"
#include "catalog/schemapg.h"
#include "catalog/storage.h"
+#include "catalog/index.h"
#include "commands/policy.h"
#include "commands/publicationcmds.h"
#include "commands/trigger.h"
@@ -5211,6 +5212,7 @@ RelationGetIndexPredicate(Relation relation)
* index (empty if FULL)
* INDEX_ATTR_BITMAP_HOT_BLOCKING Columns that block updates from being HOT
* INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
+ * INDEX_ATTR_BITMAP_EXPRESSION Columns included in expresion indexes
*
* Attribute numbers are offset by FirstLowInvalidHeapAttributeNumber so that
* we can include system attributes (e.g., OID) in the bitmap representation.
@@ -5234,7 +5236,11 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
Bitmapset *pkindexattrs; /* columns in the primary index */
Bitmapset *idindexattrs; /* columns in the replica identity */
Bitmapset *hotblockingattrs; /* columns with HOT blocking indexes */
+ Bitmapset *hotblockingexprattrs; /* as above, but only those in
+ * expressions */
Bitmapset *summarizedattrs; /* columns with summarizing indexes */
+ Bitmapset *summarizedexprattrs; /* as above, but only those in
+ * expressions */
List *indexoidlist;
List *newindexoidlist;
Oid relpkindex;
@@ -5257,6 +5263,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
return bms_copy(relation->rd_hotblockingattr);
case INDEX_ATTR_BITMAP_SUMMARIZED:
return bms_copy(relation->rd_summarizedattr);
+ case INDEX_ATTR_BITMAP_EXPRESSION:
+ return bms_copy(relation->rd_expressionattr);
default:
elog(ERROR, "unknown attrKind %u", attrKind);
}
@@ -5300,7 +5308,9 @@ restart:
pkindexattrs = NULL;
idindexattrs = NULL;
hotblockingattrs = NULL;
+ hotblockingexprattrs = NULL;
summarizedattrs = NULL;
+ summarizedexprattrs = NULL;
foreach(l, indexoidlist)
{
Oid indexOid = lfirst_oid(l);
@@ -5314,6 +5324,7 @@ restart:
bool isPK; /* primary key */
bool isIDKey; /* replica identity index */
Bitmapset **attrs;
+ Bitmapset **exprattrs;
indexDesc = index_open(indexOid, AccessShareLock);
@@ -5357,14 +5368,20 @@ restart:
* decide which bitmap we'll update in the following loop.
*/
if (indexDesc->rd_indam->amsummarizing)
+ {
attrs = &summarizedattrs;
+ exprattrs = &summarizedexprattrs;
+ }
else
+ {
attrs = &hotblockingattrs;
+ exprattrs = &hotblockingexprattrs;
+ }
/* Collect simple attribute references */
for (i = 0; i < indexDesc->rd_index->indnatts; i++)
{
- int attrnum = indexDesc->rd_index->indkey.values[i];
+ int attridx = indexDesc->rd_index->indkey.values[i];
/*
* Since we have covering indexes with non-key columns, we must
@@ -5380,30 +5397,28 @@ restart:
* key or identity key. Hence we do not include them into
* uindexattrs, pkindexattrs and idindexattrs bitmaps.
*/
- if (attrnum != 0)
+ if (attridx != 0)
{
- *attrs = bms_add_member(*attrs,
- attrnum - FirstLowInvalidHeapAttributeNumber);
+ AttrNumber attrnum = attridx - FirstLowInvalidHeapAttributeNumber;
+
+ *attrs = bms_add_member(*attrs, attrnum);
if (isKey && i < indexDesc->rd_index->indnkeyatts)
- uindexattrs = bms_add_member(uindexattrs,
- attrnum - FirstLowInvalidHeapAttributeNumber);
+ uindexattrs = bms_add_member(uindexattrs, attrnum);
if (isPK && i < indexDesc->rd_index->indnkeyatts)
- pkindexattrs = bms_add_member(pkindexattrs,
- attrnum - FirstLowInvalidHeapAttributeNumber);
+ pkindexattrs = bms_add_member(pkindexattrs, attrnum);
if (isIDKey && i < indexDesc->rd_index->indnkeyatts)
- idindexattrs = bms_add_member(idindexattrs,
- attrnum - FirstLowInvalidHeapAttributeNumber);
+ idindexattrs = bms_add_member(idindexattrs, attrnum);
}
}
/* Collect all attributes used in expressions, too */
- pull_varattnos(indexExpressions, 1, attrs);
+ pull_varattnos(indexExpressions, 1, exprattrs);
/* Collect all attributes in the index predicate, too */
- pull_varattnos(indexPredicate, 1, attrs);
+ pull_varattnos(indexPredicate, 1, exprattrs);
index_close(indexDesc, AccessShareLock);
}
@@ -5432,11 +5447,37 @@ restart:
bms_free(pkindexattrs);
bms_free(idindexattrs);
bms_free(hotblockingattrs);
+ bms_free(hotblockingexprattrs);
bms_free(summarizedattrs);
+ bms_free(summarizedexprattrs);
goto restart;
}
+ /*
+ * HOT-blocking attributes should include all columns that are part of the
+ * index except attributes only referenced in expressions, including
+ * expressions used to form partial indexes. So, we need to remove the
+ * expression-only columns from the HOT-blocking columns bitmap as those
+ * will be checked separately. This is true for both summarizing and
+ * non-summarizing indexes which we've separated above, so we have to do
+ * this for both bitmaps.
+ */
+
+ /* {expression-only columns} = {expression columns} - {direct columns} */
+ hotblockingexprattrs = bms_del_members(hotblockingexprattrs,
+ hotblockingattrs);
+ /* {hot-blocking columns} = {direct columns} + {expression-only columns} */
+ hotblockingattrs = bms_add_members(hotblockingattrs,
+ hotblockingexprattrs);
+
+ /* {summarized-only columns} = {all summarized columns} - {direct columns} */
+ summarizedexprattrs = bms_del_members(summarizedexprattrs,
+ summarizedattrs);
+ /* {summarized columns} = {all direct columns} + {summarized-only columns} */
+ summarizedattrs = bms_add_members(summarizedattrs,
+ summarizedexprattrs);
+
/* Don't leak the old values of these bitmaps, if any */
relation->rd_attrsvalid = false;
bms_free(relation->rd_keyattr);
@@ -5449,6 +5490,8 @@ restart:
relation->rd_hotblockingattr = NULL;
bms_free(relation->rd_summarizedattr);
relation->rd_summarizedattr = NULL;
+ bms_free(relation->rd_expressionattr);
+ relation->rd_expressionattr = NULL;
/*
* Now save copies of the bitmaps in the relcache entry. We intentionally
@@ -5463,6 +5506,7 @@ restart:
relation->rd_idattr = bms_copy(idindexattrs);
relation->rd_hotblockingattr = bms_copy(hotblockingattrs);
relation->rd_summarizedattr = bms_copy(summarizedattrs);
+ relation->rd_expressionattr = bms_copy(hotblockingexprattrs);
relation->rd_attrsvalid = true;
MemoryContextSwitchTo(oldcxt);
@@ -5479,6 +5523,8 @@ restart:
return hotblockingattrs;
case INDEX_ATTR_BITMAP_SUMMARIZED:
return summarizedattrs;
+ case INDEX_ATTR_BITMAP_EXPRESSION:
+ return hotblockingexprattrs;
default:
elog(ERROR, "unknown attrKind %u", attrKind);
return NULL;
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index 98951aef82c..3d4d50a95e9 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -2993,7 +2993,7 @@ match_previous_words(int pattern_id,
COMPLETE_WITH("(");
/* ALTER TABLESPACE <foo> SET|RESET ( */
else if (Matches("ALTER", "TABLESPACE", MatchAny, "SET|RESET", "("))
- COMPLETE_WITH("seq_page_cost", "random_page_cost",
+ COMPLETE_WITH("seq_page_cost", "random_page_cost", "expression_checks",
"effective_io_concurrency", "maintenance_io_concurrency");
/* ALTER TEXT SEARCH */
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 1640d9c32f7..a7eb7bd542d 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -21,6 +21,7 @@
#include "access/skey.h"
#include "access/table.h" /* for backward compatibility */
#include "access/tableam.h"
+#include "executor/executor.h"
#include "nodes/lockoptions.h"
#include "nodes/primnodes.h"
#include "storage/bufpage.h"
@@ -338,8 +339,7 @@ extern void heap_abort_speculative(Relation relation, ItemPointer tid);
extern TM_Result heap_update(Relation relation, ItemPointer otid,
HeapTuple newtup,
CommandId cid, Snapshot crosscheck, bool wait,
- struct TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+ struct TM_FailureData *tmfd, UpdateContext *updateCxt);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -374,7 +374,7 @@ extern bool heap_tuple_needs_eventual_freeze(HeapTupleHeader tuple);
extern void simple_heap_insert(Relation relation, HeapTuple tup);
extern void simple_heap_delete(Relation relation, ItemPointer tid);
extern void simple_heap_update(Relation relation, ItemPointer otid,
- HeapTuple tup, TU_UpdateIndexes *update_indexes);
+ HeapTuple tup, UpdateContext *updateCxt);
extern TransactionId heap_index_delete_tuples(Relation rel,
TM_IndexDeleteOp *delstate);
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index b8cb1e744ad..13a0bf1e114 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -125,6 +125,28 @@ typedef enum TU_UpdateIndexes
TU_Summarizing,
} TU_UpdateIndexes;
+/*
+ * Data specific to processing UPDATE operations.
+ *
+ * When table_tuple_update is called some storage managers, notably heapam,
+ * can at times avoid index updates. In the heapam this is known as a HOT
+ * update. This struct is used to provide the state required to test for
+ * HOT updates and to communicate that decision on to the index AMs.
+ */
+typedef struct UpdateContext
+{
+ TU_UpdateIndexes updateIndexes; /* Which index updates are required? */
+ struct ResultRelInfo *rri; /* ResultRelInfo for the updated table. */
+ struct EState *estate; /* EState used within the update. */
+ bool crossPartUpdate; /* Was it a cross-partition update? */
+
+ /*
+ * Lock mode to acquire on the latest tuple version before performing
+ * EvalPlanQual on it
+ */
+ LockTupleMode lockmode;
+} UpdateContext;
+
/*
* When table_tuple_update, table_tuple_delete, or table_tuple_lock fail
* because the target tuple is already outdated, they fill in this struct to
@@ -549,8 +571,7 @@ typedef struct TableAmRoutine
Snapshot crosscheck,
bool wait,
TM_FailureData *tmfd,
- LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+ UpdateContext *updateCxt);
/* see table_tuple_lock() for reference about parameters */
TM_Result (*tuple_lock) (Relation rel,
@@ -1504,13 +1525,11 @@ table_tuple_delete(Relation rel, ItemPointer tid, CommandId cid,
static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ bool wait, TM_FailureData *tmfd, UpdateContext *updateCxt)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
- wait, tmfd,
- lockmode, update_indexes);
+ wait, tmfd, updateCxt);
}
/*
@@ -2011,7 +2030,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
- TU_UpdateIndexes *update_indexes);
+ UpdateContext *updateCxt);
/* ----------------------------------------------------------------------------
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 4daa8bef5ee..fdbf47f607b 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -132,6 +132,7 @@ extern bool CompareIndexInfo(const IndexInfo *info1, const IndexInfo *info2,
const AttrMap *attmap);
extern void BuildSpeculativeIndexInfo(Relation index, IndexInfo *ii);
+extern void BuildExpressionIndexInfo(Relation index, IndexInfo *indexInfo);
extern void FormIndexDatum(IndexInfo *indexInfo,
TupleTableSlot *slot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6a1fec88928..507e142a10e 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -773,6 +773,11 @@ extern void check_exclusion_constraint(Relation heap, Relation index,
ItemPointer tupleid,
const Datum *values, const bool *isnull,
EState *estate, bool newIndex);
+extern bool ExecExpressionIndexesUpdated(ResultRelInfo *resultRelInfo,
+ Bitmapset *modifiedAttrs,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
/*
* prototypes from functions in execReplication.c
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index e42f9f9f957..f60b20671b9 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -160,12 +160,15 @@ typedef struct ExprState
*
* NumIndexAttrs total number of columns in this index
* NumIndexKeyAttrs number of key columns in index
+ * IndexAttrs bitmap of index attributes
* IndexAttrNumbers underlying-rel attribute numbers used as keys
* (zeroes indicate expressions). It also contains
* info about included columns.
* Expressions expr trees for expression entries, or NIL if none
+ * ExpressionAttrs bitmap of attributes used within the expression
* ExpressionsState exec state for expressions, or NIL if none
* Predicate partial-index predicate, or NIL if none
+ * PredicateAttrs bitmap of attributes used within the predicate
* PredicateState exec state for predicate, or NIL if none
* ExclusionOps Per-column exclusion operators, or NULL if none
* ExclusionProcs Underlying function OIDs for ExclusionOps
@@ -184,6 +187,7 @@ typedef struct ExprState
* ParallelWorkers # of workers requested (excludes leader)
* Am Oid of index AM
* AmCache private cache area for index AM
+ * OpClassDataTypes operator class data types
* Context memory context holding this IndexInfo
*
* ii_Concurrent, ii_BrokenHotChain, and ii_ParallelWorkers are used only
@@ -195,10 +199,17 @@ typedef struct IndexInfo
NodeTag type;
int ii_NumIndexAttrs; /* total number of columns in index */
int ii_NumIndexKeyAttrs; /* number of key columns in index */
+ Bitmapset *ii_IndexAttrs;
AttrNumber ii_IndexAttrNumbers[INDEX_MAX_KEYS];
+ uint16 ii_IndexAttrLen[INDEX_MAX_KEYS];
+ Bitmapset *ii_IndexAttrByVal;
+ FmgrInfo *ii_EqualityProc[INDEX_MAX_KEYS];
+ Oid ii_Collation[INDEX_MAX_KEYS];
List *ii_Expressions; /* list of Expr */
+ Bitmapset *ii_ExpressionAttrs;
List *ii_ExpressionsState; /* list of ExprState */
List *ii_Predicate; /* list of Expr */
+ Bitmapset *ii_PredicateAttrs;
ExprState *ii_PredicateState;
Oid *ii_ExclusionOps; /* array with one entry per column */
Oid *ii_ExclusionProcs; /* array with one entry per column */
@@ -211,6 +222,8 @@ typedef struct IndexInfo
bool ii_ReadyForInserts;
bool ii_CheckedUnchanged;
bool ii_IndexUnchanged;
+ bool ii_CheckedPredicate;
+ bool ii_PredicateSatisfied;
bool ii_Concurrent;
bool ii_BrokenHotChain;
bool ii_Summarizing;
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index d94fddd7cef..e44d9966778 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -164,6 +164,7 @@ typedef struct RelationData
Bitmapset *rd_idattr; /* included in replica identity index */
Bitmapset *rd_hotblockingattr; /* cols blocking HOT update */
Bitmapset *rd_summarizedattr; /* cols indexed by summarizing indexes */
+ Bitmapset *rd_expressionattr; /* indexed cols referenced by expressions */
PublicationDesc *rd_pubdesc; /* publication descriptor, or NULL */
@@ -345,6 +346,7 @@ typedef struct StdRdOptions
StdRdOptIndexCleanup vacuum_index_cleanup; /* controls index vacuuming */
bool vacuum_truncate; /* enables vacuum to truncate a relation */
bool vacuum_truncate_set; /* whether vacuum_truncate is set */
+ bool expression_checks; /* use expression to checks for changes */
/*
* Fraction of pages in a relation that vacuum can eagerly scan and fail
@@ -406,6 +408,14 @@ typedef struct StdRdOptions
((relation)->rd_options ? \
((StdRdOptions *) (relation)->rd_options)->parallel_workers : (defaultpw))
+/*
+ * RelationGetExpressionChecks
+ * Returns the relation's expression_checks reloption setting.
+ */
+#define RelationGetExpressionChecks(relation) \
+ ((relation)->rd_options ? \
+ ((StdRdOptions *) (relation)->rd_options)->expression_checks : true)
+
/* ViewOptions->check_option values */
typedef enum ViewOptCheckOption
{
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index a7c55db339e..0cc28cb97e2 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -63,6 +63,7 @@ typedef enum IndexAttrBitmapKind
INDEX_ATTR_BITMAP_IDENTITY_KEY,
INDEX_ATTR_BITMAP_HOT_BLOCKING,
INDEX_ATTR_BITMAP_SUMMARIZED,
+ INDEX_ATTR_BITMAP_EXPRESSION,
} IndexAttrBitmapKind;
extern Bitmapset *RelationGetIndexAttrBitmap(Relation relation,
diff --git a/src/test/regress/expected/heap_hot_updates.out b/src/test/regress/expected/heap_hot_updates.out
new file mode 100644
index 00000000000..027c8760d69
--- /dev/null
+++ b/src/test/regress/expected/heap_hot_updates.out
@@ -0,0 +1,665 @@
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table will have two columns and two indexes, one on the primary key
+-- id and one on the expression (info->>'name'). That means that the indexed
+-- attributes are 'id' and 'info'.
+create table keyvalue(id integer primary key, info jsonb);
+create index nameindex on keyvalue((info->>'name'));
+insert into keyvalue values (1, '{"name": "john", "data": "some data"}');
+-- Disable expression checks.
+ALTER TABLE keyvalue SET (expression_checks = false);
+SELECT reloptions FROM pg_class WHERE relname = 'keyvalue';
+ reloptions
+---------------------------
+ {expression_checks=false}
+(1 row)
+
+-- While the indexed attribute "name" is unchanged we've disabled expression
+-- checks so this update should not go HOT as the system can't determine if
+-- the indexed attribute has changed without evaluating the expression.
+update keyvalue set info='{"name": "john", "data": "something else"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 0 row
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 0
+(1 row)
+
+-- Re-enable expression checks.
+ALTER TABLE keyvalue SET (expression_checks = true);
+SELECT reloptions FROM pg_class WHERE relname = 'keyvalue';
+ reloptions
+--------------------------
+ {expression_checks=true}
+(1 row)
+
+-- The indexed attribute "name" with value "john" is unchanged, expect a HOT update.
+update keyvalue set info='{"name": "john", "data": "some other data"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 1 row
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- The following update changes the indexed attribute "name", this should not be a HOT update.
+update keyvalue set info='{"name": "smith", "data": "some other data"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Now, this update does not change the indexed attribute "name" from "smith", this should be HOT.
+update keyvalue set info='{"name": "smith", "data": "some more data"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 2 rows now
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 2
+(1 row)
+
+drop table keyvalue;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table is the same as the previous one but it has a third index. The
+-- index 'colindex' isn't an expression index, it indexes the entire value
+-- in the info column. There are still only two indexed attributes for this
+-- relation, the same two as before. The presence of an index on the entire
+-- value of the info column should prevent HOT updates for any updates to any
+-- portion of JSONB content in that column.
+create table keyvalue(id integer primary key, info jsonb);
+create index nameindex on keyvalue((info->>'name'));
+create index colindex on keyvalue(info);
+insert into keyvalue values (1, '{"name": "john", "data": "some data"}');
+-- This update doesn't change the value of the expression index, but it does
+-- change the content of the info column and so should not be HOT because the
+-- indexed value changed as a result of the update.
+update keyvalue set info='{"name": "john", "data": "some other data"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 0 rows
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 0
+(1 row)
+
+drop table keyvalue;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- The table has one column docs and two indexes. They are both expression
+-- indexes referencing the same column attribute (docs) but one is a partial
+-- index.
+CREATE TABLE ex (docs JSONB) WITH (fillfactor = 60);
+INSERT INTO ex (docs) VALUES ('{"a": 0, "b": 0}');
+INSERT INTO ex (docs) SELECT jsonb_build_object('b', n) FROM generate_series(100, 10000) as n;
+CREATE INDEX idx_ex_a ON ex ((docs->>'a'));
+CREATE INDEX idx_ex_b ON ex ((docs->>'b')) WHERE (docs->>'b')::numeric > 9;
+-- We're using BTREE indexes and for this test we want to make sure that they remain
+-- in sync with changes to our relation. Force the choice of index scans below so
+-- that we know we're checking the index's understanding of what values should be
+-- in the index or not.
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+-- Leave 'a' unchanged but modify 'b' to a value outside of the index predicate.
+-- This should be a HOT update because neither index is changed.
+UPDATE ex SET docs = jsonb_build_object('a', 0, 'b', 1) WHERE (docs->>'a')::numeric = 0;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 row
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Let's check to make sure that the index does not contain a value for 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ QUERY PLAN
+--------------------------------------------------------------
+ Index Scan using idx_ex_b on ex
+ Filter: (((docs ->> 'b'::text))::numeric < '100'::numeric)
+(2 rows)
+
+SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ docs
+------
+(0 rows)
+
+-- Leave 'a' unchanged but modify 'b' to a value within the index predicate.
+-- This represents a change for field 'b' from unindexed to indexed and so
+-- this should not take the HOT path.
+UPDATE ex SET docs = jsonb_build_object('a', 0, 'b', 10) WHERE (docs->>'a')::numeric = 0;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Let's check to make sure that the index contains the new value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ QUERY PLAN
+--------------------------------------------------------------
+ Index Scan using idx_ex_b on ex
+ Filter: (((docs ->> 'b'::text))::numeric < '100'::numeric)
+(2 rows)
+
+SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ docs
+-------------------
+ {"a": 0, "b": 10}
+(1 row)
+
+-- This update modifies the value of 'a', an indexed field, so it also cannot
+-- be a HOT update.
+UPDATE ex SET docs = jsonb_build_object('a', 1, 'b', 10) WHERE (docs->>'b')::numeric = 10;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- This update changes both 'a' and 'b' to new values that require index updates,
+-- this cannot use the HOT path.
+UPDATE ex SET docs = jsonb_build_object('a', 2, 'b', 12) WHERE (docs->>'b')::numeric = 10;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Let's check to make sure that the index contains the new value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ QUERY PLAN
+--------------------------------------------------------------
+ Index Scan using idx_ex_b on ex
+ Filter: (((docs ->> 'b'::text))::numeric < '100'::numeric)
+(2 rows)
+
+SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ docs
+-------------------
+ {"a": 2, "b": 12}
+(1 row)
+
+-- This update changes 'b' to a value outside its predicate requiring that
+-- we remove it from the index. That's a transition that can't be done
+-- during a HOT update.
+UPDATE ex SET docs = jsonb_build_object('a', 2, 'b', 1) WHERE (docs->>'b')::numeric = 12;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Let's check to make sure that the index no longer contains the value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ QUERY PLAN
+--------------------------------------------------------------
+ Index Scan using idx_ex_b on ex
+ Filter: (((docs ->> 'b'::text))::numeric < '100'::numeric)
+(2 rows)
+
+SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ docs
+------
+(0 rows)
+
+-- Disable expression checks.
+ALTER TABLE ex SET (expression_checks = false);
+SELECT reloptions FROM pg_class WHERE relname = 'ex';
+ reloptions
+-----------------------------------------
+ {fillfactor=60,expression_checks=false}
+(1 row)
+
+-- This update changes 'b' to a value within its predicate just like it
+-- previous value, which would allow for a HOT update but with expression
+-- checks disabled we can't determine that so this should not be a HOT update.
+UPDATE ex SET docs = jsonb_build_object('a', 2, 'b', 2) WHERE (docs->>'b')::numeric = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Let's make sure we're recording HOT updates for our 'ex' relation properly in the system
+-- table pg_stat_user_tables. Note that statistics are stored within a transaction context
+-- first (xact) and then later into the global statistics for a relation, so first we need
+-- to ensure pending stats are flushed.
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT
+ c.relname AS table_name,
+ -- Transaction statistics
+ pg_stat_get_xact_tuples_updated(c.oid) AS xact_updates,
+ pg_stat_get_xact_tuples_hot_updated(c.oid) AS xact_hot_updates,
+ ROUND((
+ pg_stat_get_xact_tuples_hot_updated(c.oid)::float /
+ NULLIF(pg_stat_get_xact_tuples_updated(c.oid), 0) * 100
+ )::numeric, 2) AS xact_hot_update_percentage,
+ -- Cumulative statistics
+ s.n_tup_upd AS total_updates,
+ s.n_tup_hot_upd AS hot_updates,
+ ROUND((
+ s.n_tup_hot_upd::float /
+ NULLIF(s.n_tup_upd, 0) * 100
+ )::numeric, 2) AS total_hot_update_percentage
+FROM pg_class c
+LEFT JOIN pg_stat_user_tables s ON c.relname = s.relname
+WHERE c.relname = 'ex'
+AND c.relnamespace = 'public'::regnamespace;
+ table_name | xact_updates | xact_hot_updates | xact_hot_update_percentage | total_updates | hot_updates | total_hot_update_percentage
+------------+--------------+------------------+----------------------------+---------------+-------------+-----------------------------
+ ex | 0 | 0 | | 6 | 1 | 16.67
+(1 row)
+
+-- expect: 5 xact updates with 1 xact hot update and no cumulative updates as yet
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+DROP TABLE ex;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table has a single column 'email' and a unique constraint on it that
+-- should preclude HOT updates.
+CREATE TABLE users (
+ user_id serial primary key,
+ name VARCHAR(255) NOT NULL,
+ email VARCHAR(255) NOT NULL,
+ EXCLUDE USING btree (lower(email) WITH =)
+);
+-- Add some data to the table and then update it in ways that should and should
+-- not be HOT updates.
+INSERT INTO users (name, email) VALUES
+('user1', '[email protected]'),
+('user2', '[email protected]'),
+('taken', '[email protected]'),
+('you', '[email protected]'),
+('taken', '[email protected]');
+-- Should fail because of the unique constraint on the email column.
+UPDATE users SET email = '[email protected]' WHERE email = '[email protected]';
+ERROR: conflicting key value violates exclusion constraint "users_lower_excl"
+DETAIL: Key (lower(email::text))=([email protected]) conflicts with existing key (lower(email::text))=([email protected]).
+SELECT pg_stat_get_xact_tuples_hot_updated('users'::regclass); -- expect: 0 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 0
+(1 row)
+
+-- Should succeed because the email column is not being updated and should go HOT.
+UPDATE users SET name = 'foo' WHERE email = '[email protected]';
+SELECT pg_stat_get_xact_tuples_hot_updated('users'::regclass); -- expect: 1 a single new HOT update
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Create a partial index on the email column, updates
+CREATE INDEX idx_users_email_no_example ON users (lower(email)) WHERE lower(email) LIKE '%@example.com%';
+-- An update that changes the email column but not the indexed portion of it and falls outside the constraint.
+-- Shouldn't be a HOT update because of the exclusion constraint.
+UPDATE users SET email = '[email protected]' WHERE name = 'you';
+SELECT pg_stat_get_xact_tuples_hot_updated('users'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- An update that changes the email column but not the indexed portion of it and falls within the constraint.
+-- Again, should fail constraint and fail to be a HOT update.
+UPDATE users SET email = '[email protected]' WHERE name = 'you';
+ERROR: conflicting key value violates exclusion constraint "users_lower_excl"
+DETAIL: Key (lower(email::text))=([email protected]) conflicts with existing key (lower(email::text))=([email protected]).
+SELECT pg_stat_get_xact_tuples_hot_updated('users'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+DROP TABLE users;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- Another test of constraints spoiling HOT updates, this time with a range.
+CREATE TABLE events (
+ id serial primary key,
+ name VARCHAR(255) NOT NULL,
+ event_time tstzrange,
+ constraint no_screening_time_overlap exclude using gist (
+ event_time WITH &&
+ )
+);
+-- Add two non-overlapping events.
+INSERT INTO events (id, event_time, name)
+VALUES
+ (1, '["2023-01-01 19:00:00", "2023-01-01 20:45:00"]', 'event1'),
+ (2, '["2023-01-01 21:00:00", "2023-01-01 21:45:00"]', 'event2');
+-- Update the first event to overlap with the second, should fail the constraint and not be HOT.
+UPDATE events SET event_time = '["2023-01-01 20:00:00", "2023-01-01 21:45:00"]' WHERE id = 1;
+ERROR: conflicting key value violates exclusion constraint "no_screening_time_overlap"
+DETAIL: Key (event_time)=(["Sun Jan 01 20:00:00 2023 PST","Sun Jan 01 21:45:00 2023 PST"]) conflicts with existing key (event_time)=(["Sun Jan 01 21:00:00 2023 PST","Sun Jan 01 21:45:00 2023 PST"]).
+SELECT pg_stat_get_xact_tuples_hot_updated('events'::regclass); -- expect: 0 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 0
+(1 row)
+
+-- Update the first event to not overlap with the second, again not HOT due to the constraint.
+UPDATE events SET event_time = '["2023-01-01 22:00:00", "2023-01-01 22:45:00"]' WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('events'::regclass); -- expect: 0 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 0
+(1 row)
+
+-- Update the first event to not overlap with the second, this time we're HOT because we don't overlap with the constraint.
+UPDATE events SET name = 'new name here' WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('events'::regclass); -- expect: 1 one new HOT update
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+DROP TABLE events;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- A test to ensure that only modified summarizing indexes are updated, not
+-- all of them.
+CREATE TABLE ex (id SERIAL primary key, att1 JSONB, att2 text, att3 text, att4 text) WITH (fillfactor = 60);
+CREATE INDEX ex_expr1_idx ON ex USING btree((att1->'data'));
+CREATE INDEX ex_sumr1_idx ON ex USING BRIN(att2);
+CREATE INDEX ex_expr2_idx ON ex USING btree((att1->'a'));
+CREATE INDEX ex_expr3_idx ON ex USING btree((att1->'b'));
+CREATE INDEX ex_expr4_idx ON ex USING btree((att1->'c'));
+CREATE INDEX ex_sumr2_idx ON ex USING BRIN(att3);
+CREATE INDEX ex_sumr3_idx ON ex USING BRIN(att4);
+CREATE INDEX ex_expr5_idx ON ex USING btree((att1->'d'));
+INSERT INTO ex (att1, att2) VALUES ('{"data": []}'::json, 'nothing special');
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+--------------+-----------------+------+------
+ 1 | {"data": []} | nothing special | |
+(1 row)
+
+-- Update att2 and att4 both are BRIN/summarizing indexes, this should be a HOT update and
+-- only update two of the three summarizing indexes.
+UPDATE ex SET att2 = 'special indeed', att4 = 'whatever';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 one new HOT update
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+--------------+----------------+------+----------
+ 1 | {"data": []} | special indeed | | whatever
+(1 row)
+
+-- Update att1 and att2, only one is BRIN/summarizing, this should NOT be a HOT update.
+UPDATE ex SET att1 = att1 || '{"data": "howdy"}', att2 = 'special, so special';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+-------------------+---------------------+------+----------
+ 1 | {"data": "howdy"} | special, so special | | whatever
+(1 row)
+
+-- Update att2, att3, and att4 all are BRIN/summarizing indexes, this should be a HOT update
+-- and yet still update all three summarizing indexes.
+UPDATE ex SET att2 = 'a', att3 = 'b', att4 = 'c';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 2 with one new HOT update
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 2
+(1 row)
+
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+-------------------+------+------+------
+ 1 | {"data": "howdy"} | a | b | c
+(1 row)
+
+-- Update att1, att2, and att3 all modified values are BRIN/summarizing indexes, this should be a HOT update
+-- and yet still update all three summarizing indexes.
+UPDATE ex SET att1 = '{"data": "howdy"}', att2 = 'd', att3 = 'e';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 3 with one new HOT update
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 3
+(1 row)
+
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+-------------------+------+------+------
+ 1 | {"data": "howdy"} | d | e | c
+(1 row)
+
+DROP TABLE ex;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- A test to ensure that summarizing indexes are not updated when they don't
+-- change, but are updated when they do while not prefent HOT updates.
+CREATE TABLE ex (att1 JSONB, att2 text) WITH (fillfactor = 60);
+CREATE INDEX ex_expr1_idx ON ex USING btree((att1->'data'));
+CREATE INDEX ex_sumr1_idx ON ex USING BRIN(att2);
+INSERT INTO ex VALUES ('{"data": []}', 'nothing special');
+-- Update the unindexed value of att1, this should be a HOT update and and should
+-- update the summarizing index.
+UPDATE ex SET att1 = att1 || '{"status": "stalemate"}';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 one new HOT update
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Update the indexed value of att2, a summarized value, this is a summarized
+-- only update and should use the HOT path while still triggering an update to
+-- the summarizing BRIN index.
+UPDATE ex SET att2 = 'special indeed';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 2 one new HOT update
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 2
+(1 row)
+
+-- Update to att1 doesn't change the indexed value while the update to att2 does,
+-- this again is a summarized only update and should use the HOT path as well as
+-- trigger an update to the BRIN index.
+UPDATE ex SET att1 = att1 || '{"status": "checkmate"}', att2 = 'special, so special';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 3 one new HOT update
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 3
+(1 row)
+
+-- This updates both indexes, the expression index on att1 and the summarizing
+-- index on att2. This should not be a HOT update because there are modified
+-- indexes and only some are summarized, not all. This should force all
+-- indexes to be updated.
+UPDATE ex SET att1 = att1 || '{"data": [1,2,3]}', att2 = 'do you want to play a game?';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 4 both indexes updated
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 3
+(1 row)
+
+DROP TABLE ex;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This test is for a table with a custom type and a custom operators on
+-- the BTREE index. The question is, when comparing values for equality
+-- to determine if there are changes on the index or not... shouldn't we
+-- be using the custom operators?
+-- Create a type
+CREATE TYPE my_custom_type AS (val int);
+-- Comparison functions (returns boolean)
+CREATE FUNCTION my_custom_lt(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val < b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_le(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val <= b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_eq(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val = b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_ge(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val >= b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_gt(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val > b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_ne(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val != b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+-- Comparison function (returns -1, 0, 1)
+CREATE FUNCTION my_custom_cmp(a my_custom_type, b my_custom_type) RETURNS int AS $$
+BEGIN
+ IF a.val < b.val THEN
+ RETURN -1;
+ ELSIF a.val > b.val THEN
+ RETURN 1;
+ ELSE
+ RETURN 0;
+ END IF;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+-- Create the operators
+CREATE OPERATOR < (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_lt,
+ COMMUTATOR = >,
+ NEGATOR = >=
+);
+CREATE OPERATOR <= (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_le,
+ COMMUTATOR = >=,
+ NEGATOR = >
+);
+CREATE OPERATOR = (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_eq,
+ COMMUTATOR = =,
+ NEGATOR = <>
+);
+CREATE OPERATOR >= (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_ge,
+ COMMUTATOR = <=,
+ NEGATOR = <
+);
+CREATE OPERATOR > (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_gt,
+ COMMUTATOR = <,
+ NEGATOR = <=
+);
+CREATE OPERATOR <> (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_ne,
+ COMMUTATOR = <>,
+ NEGATOR = =
+);
+-- Create the operator class (including the support function)
+CREATE OPERATOR CLASS my_custom_ops
+ DEFAULT FOR TYPE my_custom_type USING btree AS
+ OPERATOR 1 <,
+ OPERATOR 2 <=,
+ OPERATOR 3 =,
+ OPERATOR 4 >=,
+ OPERATOR 5 >,
+ FUNCTION 1 my_custom_cmp(my_custom_type, my_custom_type);
+-- Create the table
+CREATE TABLE my_table (
+ id int,
+ custom_val my_custom_type
+);
+-- Insert some data
+INSERT INTO my_table (id, custom_val) VALUES
+(1, ROW(3)::my_custom_type),
+(2, ROW(1)::my_custom_type),
+(3, ROW(4)::my_custom_type),
+(4, ROW(2)::my_custom_type);
+-- Create a function to use when indexing
+CREATE OR REPLACE FUNCTION abs_val(val my_custom_type) RETURNS int AS $$
+BEGIN
+ RETURN abs(val.val);
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+-- Create the index
+CREATE INDEX idx_custom_val_abs ON my_table (abs_val(custom_val));
+-- Update 1
+UPDATE my_table SET custom_val = ROW(5)::my_custom_type WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 0
+(1 row)
+
+-- Update 2
+UPDATE my_table SET custom_val = ROW(0)::my_custom_type WHERE custom_val < ROW(3)::my_custom_type;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 0
+(1 row)
+
+-- Update 3
+UPDATE my_table SET custom_val = ROW(6)::my_custom_type WHERE id = 3;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 0
+(1 row)
+
+-- Update 4
+UPDATE my_table SET id = 5 WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+-- Query using the index
+EXPLAIN (COSTS OFF) SELECT * FROM my_table WHERE abs_val(custom_val) = 6;
+ QUERY PLAN
+-------------------------------------
+ Seq Scan on my_table
+ Filter: (abs_val(custom_val) = 6)
+(2 rows)
+
+SELECT * FROM my_table WHERE abs_val(custom_val) = 6;
+ id | custom_val
+----+------------
+ 3 | (6)
+(1 row)
+
+-- Clean up
+DROP TABLE my_table CASCADE;
+DROP OPERATOR CLASS my_custom_ops USING btree CASCADE;
+DROP OPERATOR < (my_custom_type, my_custom_type);
+DROP OPERATOR <= (my_custom_type, my_custom_type);
+DROP OPERATOR = (my_custom_type, my_custom_type);
+DROP OPERATOR >= (my_custom_type, my_custom_type);
+DROP OPERATOR > (my_custom_type, my_custom_type);
+DROP OPERATOR <> (my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_lt(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_le(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_eq(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_ge(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_gt(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_ne(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_cmp(my_custom_type, my_custom_type);
+DROP FUNCTION abs_val(my_custom_type);
+DROP TYPE my_custom_type CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 0a35f2f8f6a..e2fd5fb74a5 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -68,6 +68,11 @@ test: select_into select_distinct select_distinct_on select_implicit select_havi
# ----------
test: brin gin gist spgist privileges init_privs security_label collate matview lock replica_identity rowsecurity object_address tablesample groupingsets drop_operator password identity generated_stored join_hash
+# ----------
+# Another group of parallel tests
+# ----------
+test: heap_hot_updates
+
# ----------
# Additional BRIN tests
# ----------
diff --git a/src/test/regress/sql/heap_hot_updates.sql b/src/test/regress/sql/heap_hot_updates.sql
new file mode 100644
index 00000000000..56ec4f2f96c
--- /dev/null
+++ b/src/test/regress/sql/heap_hot_updates.sql
@@ -0,0 +1,481 @@
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table will have two columns and two indexes, one on the primary key
+-- id and one on the expression (info->>'name'). That means that the indexed
+-- attributes are 'id' and 'info'.
+create table keyvalue(id integer primary key, info jsonb);
+create index nameindex on keyvalue((info->>'name'));
+insert into keyvalue values (1, '{"name": "john", "data": "some data"}');
+
+-- Disable expression checks.
+ALTER TABLE keyvalue SET (expression_checks = false);
+SELECT reloptions FROM pg_class WHERE relname = 'keyvalue';
+
+-- While the indexed attribute "name" is unchanged we've disabled expression
+-- checks so this update should not go HOT as the system can't determine if
+-- the indexed attribute has changed without evaluating the expression.
+update keyvalue set info='{"name": "john", "data": "something else"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 0 row
+
+-- Re-enable expression checks.
+ALTER TABLE keyvalue SET (expression_checks = true);
+SELECT reloptions FROM pg_class WHERE relname = 'keyvalue';
+
+-- The indexed attribute "name" with value "john" is unchanged, expect a HOT update.
+update keyvalue set info='{"name": "john", "data": "some other data"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 1 row
+
+-- The following update changes the indexed attribute "name", this should not be a HOT update.
+update keyvalue set info='{"name": "smith", "data": "some other data"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 1 no new HOT updates
+
+-- Now, this update does not change the indexed attribute "name" from "smith", this should be HOT.
+update keyvalue set info='{"name": "smith", "data": "some more data"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 2 rows now
+drop table keyvalue;
+
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table is the same as the previous one but it has a third index. The
+-- index 'colindex' isn't an expression index, it indexes the entire value
+-- in the info column. There are still only two indexed attributes for this
+-- relation, the same two as before. The presence of an index on the entire
+-- value of the info column should prevent HOT updates for any updates to any
+-- portion of JSONB content in that column.
+create table keyvalue(id integer primary key, info jsonb);
+create index nameindex on keyvalue((info->>'name'));
+create index colindex on keyvalue(info);
+insert into keyvalue values (1, '{"name": "john", "data": "some data"}');
+
+-- This update doesn't change the value of the expression index, but it does
+-- change the content of the info column and so should not be HOT because the
+-- indexed value changed as a result of the update.
+update keyvalue set info='{"name": "john", "data": "some other data"}' where id=1;
+select pg_stat_get_xact_tuples_hot_updated('keyvalue'::regclass); -- expect: 0 rows
+drop table keyvalue;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- The table has one column docs and two indexes. They are both expression
+-- indexes referencing the same column attribute (docs) but one is a partial
+-- index.
+CREATE TABLE ex (docs JSONB) WITH (fillfactor = 60);
+INSERT INTO ex (docs) VALUES ('{"a": 0, "b": 0}');
+INSERT INTO ex (docs) SELECT jsonb_build_object('b', n) FROM generate_series(100, 10000) as n;
+CREATE INDEX idx_ex_a ON ex ((docs->>'a'));
+CREATE INDEX idx_ex_b ON ex ((docs->>'b')) WHERE (docs->>'b')::numeric > 9;
+
+-- We're using BTREE indexes and for this test we want to make sure that they remain
+-- in sync with changes to our relation. Force the choice of index scans below so
+-- that we know we're checking the index's understanding of what values should be
+-- in the index or not.
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+
+-- Leave 'a' unchanged but modify 'b' to a value outside of the index predicate.
+-- This should be a HOT update because neither index is changed.
+UPDATE ex SET docs = jsonb_build_object('a', 0, 'b', 1) WHERE (docs->>'a')::numeric = 0;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 row
+-- Let's check to make sure that the index does not contain a value for 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+
+-- Leave 'a' unchanged but modify 'b' to a value within the index predicate.
+-- This represents a change for field 'b' from unindexed to indexed and so
+-- this should not take the HOT path.
+UPDATE ex SET docs = jsonb_build_object('a', 0, 'b', 10) WHERE (docs->>'a')::numeric = 0;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+-- Let's check to make sure that the index contains the new value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+
+-- This update modifies the value of 'a', an indexed field, so it also cannot
+-- be a HOT update.
+UPDATE ex SET docs = jsonb_build_object('a', 1, 'b', 10) WHERE (docs->>'b')::numeric = 10;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+
+-- This update changes both 'a' and 'b' to new values that require index updates,
+-- this cannot use the HOT path.
+UPDATE ex SET docs = jsonb_build_object('a', 2, 'b', 12) WHERE (docs->>'b')::numeric = 10;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+-- Let's check to make sure that the index contains the new value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+
+-- This update changes 'b' to a value outside its predicate requiring that
+-- we remove it from the index. That's a transition that can't be done
+-- during a HOT update.
+UPDATE ex SET docs = jsonb_build_object('a', 2, 'b', 1) WHERE (docs->>'b')::numeric = 12;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+-- Let's check to make sure that the index no longer contains the value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+SELECT * FROM ex WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+
+-- Disable expression checks.
+ALTER TABLE ex SET (expression_checks = false);
+SELECT reloptions FROM pg_class WHERE relname = 'ex';
+
+-- This update changes 'b' to a value within its predicate just like it
+-- previous value, which would allow for a HOT update but with expression
+-- checks disabled we can't determine that so this should not be a HOT update.
+UPDATE ex SET docs = jsonb_build_object('a', 2, 'b', 2) WHERE (docs->>'b')::numeric = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+
+
+-- Let's make sure we're recording HOT updates for our 'ex' relation properly in the system
+-- table pg_stat_user_tables. Note that statistics are stored within a transaction context
+-- first (xact) and then later into the global statistics for a relation, so first we need
+-- to ensure pending stats are flushed.
+SELECT pg_stat_force_next_flush();
+SELECT
+ c.relname AS table_name,
+ -- Transaction statistics
+ pg_stat_get_xact_tuples_updated(c.oid) AS xact_updates,
+ pg_stat_get_xact_tuples_hot_updated(c.oid) AS xact_hot_updates,
+ ROUND((
+ pg_stat_get_xact_tuples_hot_updated(c.oid)::float /
+ NULLIF(pg_stat_get_xact_tuples_updated(c.oid), 0) * 100
+ )::numeric, 2) AS xact_hot_update_percentage,
+ -- Cumulative statistics
+ s.n_tup_upd AS total_updates,
+ s.n_tup_hot_upd AS hot_updates,
+ ROUND((
+ s.n_tup_hot_upd::float /
+ NULLIF(s.n_tup_upd, 0) * 100
+ )::numeric, 2) AS total_hot_update_percentage
+FROM pg_class c
+LEFT JOIN pg_stat_user_tables s ON c.relname = s.relname
+WHERE c.relname = 'ex'
+AND c.relnamespace = 'public'::regnamespace;
+-- expect: 5 xact updates with 1 xact hot update and no cumulative updates as yet
+
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+
+DROP TABLE ex;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table has a single column 'email' and a unique constraint on it that
+-- should preclude HOT updates.
+CREATE TABLE users (
+ user_id serial primary key,
+ name VARCHAR(255) NOT NULL,
+ email VARCHAR(255) NOT NULL,
+ EXCLUDE USING btree (lower(email) WITH =)
+);
+
+-- Add some data to the table and then update it in ways that should and should
+-- not be HOT updates.
+INSERT INTO users (name, email) VALUES
+('user1', '[email protected]'),
+('user2', '[email protected]'),
+('taken', '[email protected]'),
+('you', '[email protected]'),
+('taken', '[email protected]');
+
+-- Should fail because of the unique constraint on the email column.
+UPDATE users SET email = '[email protected]' WHERE email = '[email protected]';
+SELECT pg_stat_get_xact_tuples_hot_updated('users'::regclass); -- expect: 0 no new HOT updates
+
+-- Should succeed because the email column is not being updated and should go HOT.
+UPDATE users SET name = 'foo' WHERE email = '[email protected]';
+SELECT pg_stat_get_xact_tuples_hot_updated('users'::regclass); -- expect: 1 a single new HOT update
+
+-- Create a partial index on the email column, updates
+CREATE INDEX idx_users_email_no_example ON users (lower(email)) WHERE lower(email) LIKE '%@example.com%';
+
+-- An update that changes the email column but not the indexed portion of it and falls outside the constraint.
+-- Shouldn't be a HOT update because of the exclusion constraint.
+UPDATE users SET email = '[email protected]' WHERE name = 'you';
+SELECT pg_stat_get_xact_tuples_hot_updated('users'::regclass); -- expect: 1 no new HOT updates
+
+-- An update that changes the email column but not the indexed portion of it and falls within the constraint.
+-- Again, should fail constraint and fail to be a HOT update.
+UPDATE users SET email = '[email protected]' WHERE name = 'you';
+SELECT pg_stat_get_xact_tuples_hot_updated('users'::regclass); -- expect: 1 no new HOT updates
+
+DROP TABLE users;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- Another test of constraints spoiling HOT updates, this time with a range.
+CREATE TABLE events (
+ id serial primary key,
+ name VARCHAR(255) NOT NULL,
+ event_time tstzrange,
+ constraint no_screening_time_overlap exclude using gist (
+ event_time WITH &&
+ )
+);
+
+-- Add two non-overlapping events.
+INSERT INTO events (id, event_time, name)
+VALUES
+ (1, '["2023-01-01 19:00:00", "2023-01-01 20:45:00"]', 'event1'),
+ (2, '["2023-01-01 21:00:00", "2023-01-01 21:45:00"]', 'event2');
+
+-- Update the first event to overlap with the second, should fail the constraint and not be HOT.
+UPDATE events SET event_time = '["2023-01-01 20:00:00", "2023-01-01 21:45:00"]' WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('events'::regclass); -- expect: 0 no new HOT updates
+
+-- Update the first event to not overlap with the second, again not HOT due to the constraint.
+UPDATE events SET event_time = '["2023-01-01 22:00:00", "2023-01-01 22:45:00"]' WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('events'::regclass); -- expect: 0 no new HOT updates
+
+-- Update the first event to not overlap with the second, this time we're HOT because we don't overlap with the constraint.
+UPDATE events SET name = 'new name here' WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('events'::regclass); -- expect: 1 one new HOT update
+
+DROP TABLE events;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- A test to ensure that only modified summarizing indexes are updated, not
+-- all of them.
+CREATE TABLE ex (id SERIAL primary key, att1 JSONB, att2 text, att3 text, att4 text) WITH (fillfactor = 60);
+CREATE INDEX ex_expr1_idx ON ex USING btree((att1->'data'));
+CREATE INDEX ex_sumr1_idx ON ex USING BRIN(att2);
+CREATE INDEX ex_expr2_idx ON ex USING btree((att1->'a'));
+CREATE INDEX ex_expr3_idx ON ex USING btree((att1->'b'));
+CREATE INDEX ex_expr4_idx ON ex USING btree((att1->'c'));
+CREATE INDEX ex_sumr2_idx ON ex USING BRIN(att3);
+CREATE INDEX ex_sumr3_idx ON ex USING BRIN(att4);
+CREATE INDEX ex_expr5_idx ON ex USING btree((att1->'d'));
+INSERT INTO ex (att1, att2) VALUES ('{"data": []}'::json, 'nothing special');
+
+SELECT * FROM ex;
+
+-- Update att2 and att4 both are BRIN/summarizing indexes, this should be a HOT update and
+-- only update two of the three summarizing indexes.
+UPDATE ex SET att2 = 'special indeed', att4 = 'whatever';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 one new HOT update
+SELECT * FROM ex;
+
+-- Update att1 and att2, only one is BRIN/summarizing, this should NOT be a HOT update.
+UPDATE ex SET att1 = att1 || '{"data": "howdy"}', att2 = 'special, so special';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 no new HOT updates
+SELECT * FROM ex;
+
+-- Update att2, att3, and att4 all are BRIN/summarizing indexes, this should be a HOT update
+-- and yet still update all three summarizing indexes.
+UPDATE ex SET att2 = 'a', att3 = 'b', att4 = 'c';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 2 with one new HOT update
+SELECT * FROM ex;
+
+-- Update att1, att2, and att3 all modified values are BRIN/summarizing indexes, this should be a HOT update
+-- and yet still update all three summarizing indexes.
+UPDATE ex SET att1 = '{"data": "howdy"}', att2 = 'd', att3 = 'e';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 3 with one new HOT update
+SELECT * FROM ex;
+
+DROP TABLE ex;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- A test to ensure that summarizing indexes are not updated when they don't
+-- change, but are updated when they do while not prefent HOT updates.
+CREATE TABLE ex (att1 JSONB, att2 text) WITH (fillfactor = 60);
+CREATE INDEX ex_expr1_idx ON ex USING btree((att1->'data'));
+CREATE INDEX ex_sumr1_idx ON ex USING BRIN(att2);
+INSERT INTO ex VALUES ('{"data": []}', 'nothing special');
+
+-- Update the unindexed value of att1, this should be a HOT update and and should
+-- update the summarizing index.
+UPDATE ex SET att1 = att1 || '{"status": "stalemate"}';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 1 one new HOT update
+
+-- Update the indexed value of att2, a summarized value, this is a summarized
+-- only update and should use the HOT path while still triggering an update to
+-- the summarizing BRIN index.
+UPDATE ex SET att2 = 'special indeed';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 2 one new HOT update
+
+-- Update to att1 doesn't change the indexed value while the update to att2 does,
+-- this again is a summarized only update and should use the HOT path as well as
+-- trigger an update to the BRIN index.
+UPDATE ex SET att1 = att1 || '{"status": "checkmate"}', att2 = 'special, so special';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 3 one new HOT update
+
+-- This updates both indexes, the expression index on att1 and the summarizing
+-- index on att2. This should not be a HOT update because there are modified
+-- indexes and only some are summarized, not all. This should force all
+-- indexes to be updated.
+UPDATE ex SET att1 = att1 || '{"data": [1,2,3]}', att2 = 'do you want to play a game?';
+SELECT pg_stat_get_xact_tuples_hot_updated('ex'::regclass); -- expect: 4 both indexes updated
+
+DROP TABLE ex;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This test is for a table with a custom type and a custom operators on
+-- the BTREE index. The question is, when comparing values for equality
+-- to determine if there are changes on the index or not... shouldn't we
+-- be using the custom operators?
+
+-- Create a type
+CREATE TYPE my_custom_type AS (val int);
+
+-- Comparison functions (returns boolean)
+CREATE FUNCTION my_custom_lt(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val < b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_le(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val <= b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_eq(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val = b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_ge(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val >= b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_gt(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val > b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_ne(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val != b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+-- Comparison function (returns -1, 0, 1)
+CREATE FUNCTION my_custom_cmp(a my_custom_type, b my_custom_type) RETURNS int AS $$
+BEGIN
+ IF a.val < b.val THEN
+ RETURN -1;
+ ELSIF a.val > b.val THEN
+ RETURN 1;
+ ELSE
+ RETURN 0;
+ END IF;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+-- Create the operators
+CREATE OPERATOR < (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_lt,
+ COMMUTATOR = >,
+ NEGATOR = >=
+);
+
+CREATE OPERATOR <= (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_le,
+ COMMUTATOR = >=,
+ NEGATOR = >
+);
+
+CREATE OPERATOR = (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_eq,
+ COMMUTATOR = =,
+ NEGATOR = <>
+);
+
+CREATE OPERATOR >= (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_ge,
+ COMMUTATOR = <=,
+ NEGATOR = <
+);
+
+CREATE OPERATOR > (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_gt,
+ COMMUTATOR = <,
+ NEGATOR = <=
+);
+
+CREATE OPERATOR <> (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_ne,
+ COMMUTATOR = <>,
+ NEGATOR = =
+);
+
+-- Create the operator class (including the support function)
+CREATE OPERATOR CLASS my_custom_ops
+ DEFAULT FOR TYPE my_custom_type USING btree AS
+ OPERATOR 1 <,
+ OPERATOR 2 <=,
+ OPERATOR 3 =,
+ OPERATOR 4 >=,
+ OPERATOR 5 >,
+ FUNCTION 1 my_custom_cmp(my_custom_type, my_custom_type);
+
+-- Create the table
+CREATE TABLE my_table (
+ id int,
+ custom_val my_custom_type
+);
+
+-- Insert some data
+INSERT INTO my_table (id, custom_val) VALUES
+(1, ROW(3)::my_custom_type),
+(2, ROW(1)::my_custom_type),
+(3, ROW(4)::my_custom_type),
+(4, ROW(2)::my_custom_type);
+
+-- Create a function to use when indexing
+CREATE OR REPLACE FUNCTION abs_val(val my_custom_type) RETURNS int AS $$
+BEGIN
+ RETURN abs(val.val);
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+-- Create the index
+CREATE INDEX idx_custom_val_abs ON my_table (abs_val(custom_val));
+
+-- Update 1
+UPDATE my_table SET custom_val = ROW(5)::my_custom_type WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+
+-- Update 2
+UPDATE my_table SET custom_val = ROW(0)::my_custom_type WHERE custom_val < ROW(3)::my_custom_type;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+
+-- Update 3
+UPDATE my_table SET custom_val = ROW(6)::my_custom_type WHERE id = 3;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+
+-- Update 4
+UPDATE my_table SET id = 5 WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+
+-- Query using the index
+EXPLAIN (COSTS OFF) SELECT * FROM my_table WHERE abs_val(custom_val) = 6;
+SELECT * FROM my_table WHERE abs_val(custom_val) = 6;
+
+-- Clean up
+DROP TABLE my_table CASCADE;
+DROP OPERATOR CLASS my_custom_ops USING btree CASCADE;
+DROP OPERATOR < (my_custom_type, my_custom_type);
+DROP OPERATOR <= (my_custom_type, my_custom_type);
+DROP OPERATOR = (my_custom_type, my_custom_type);
+DROP OPERATOR >= (my_custom_type, my_custom_type);
+DROP OPERATOR > (my_custom_type, my_custom_type);
+DROP OPERATOR <> (my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_lt(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_le(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_eq(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_ge(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_gt(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_ne(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_cmp(my_custom_type, my_custom_type);
+DROP FUNCTION abs_val(my_custom_type);
+DROP TYPE my_custom_type CASCADE;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 3fbf5a4c212..0b6587dbd56 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2934,6 +2934,7 @@ TSVectorStat
TState
TStatus
TStoreState
+UpdateContext
TU_UpdateIndexes
TXNEntryFile
TYPCATEGORY
@@ -3118,7 +3119,6 @@ UniqueState
UnlistenStmt
UnresolvedTup
UnresolvedTupData
-UpdateContext
UpdateStmt
UpgradeTask
UpgradeTaskProcessCB
--
2.42.0
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
@ 2025-07-02 18:10 ` Greg Burd <[email protected]>
2025-10-07 21:36 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Greg Burd @ 2025-07-02 18:10 UTC (permalink / raw)
To: Burd, Greg <[email protected]>; Matthias van de Meent <[email protected]>; +Cc: pgsql-hackers
I'm working again on expanding the conditions under which HOT updates are allowable, it is still a work-in-progress at this point. It has been a while since my last update to this patch set so I'll refresh everyone's memories (including myself) with an overview. Apologies in advance for a long email...
The goal is to allow HOT updates under two new conditions:
* when an indexed expression has not changed
* when possible for a partial index
Expression Indexes and HOT updates
==========================================================
Indexes on expressions such as:
CREATE INDEX test1_lower_col1_idx ON test1 (lower(col1));
CREATE INDEX people_names ON people ((first_name || ' ' || last_name));
CREATE INDEX names ON people((docs->>'user'));
These are not currently candidates for HOT updates because the attributes they reference are added to the hotblockingattrs bitmapset that is compared for overlap with the modified_attrs bitmap in heap_update() as created by HeapDetermineColumnsInfo(). Logically this exclusion makes sense because expressions used to form indexes can only contain references to functions that are IMMUTABLE. This then allows for a simple and efficient method for determining if the update is a candidate for the HOT path, a quick check for overlap of two bitmapsets and you have an answer.
But there is a common case that is overlooked. The third example above is an expression index that references a JSONB attribute 'docs' field 'name'. Imagine a simple bit of JSON stored in the 'docs' JSONB column changed by an UPDATE from:
{ "user": "scott", "password": "tiger" }
to:
{ "user": "scott", "password": "$ecret" }
this will not use the HOT path in today's code because the attribute for 'docs' will be referenced in the UPDATE statement and that attribute's content did change so it will be in the modified_attrs which will overlap with the hotblockingattrs set. The result is that it is not possible to use the HOT update path in heap_update() if there is an indexed JSONB column in that statement. This has a huge impact on performance and bloat. It's not hard to see that the index doesn't require an update, the portion of the document used to form the index tuple isn't changing.
This patch addressed this problem by evaluating the expression on the index using the current and new tuple and then comparing them. If there is no change to the index tuple then the HOT path is still an option for the update whereas before this was not the case.
There is at least one major challenge to this approach left to solve, it invokes UDFs while holding BUFFER_LOCK_EXCLUSIVE which we learn from heap_attr_equals() "we cannot safely invoke user-defined functions while holding exclusive buffer lock."
So, for performance an safety reasons it makes sense to try to limit the expressions that can be evaluated to those that can't self-deadlock trying to pin the same buffer and are roughly constant time and fast so we don't hold that lock for very long. Additionally it would be nice to ensure that we really have to do the evaluation in the first place.
I looked for ways to limit the types of expressions to evaluate so as to only take this approach when necessary, but I wasn't able to identify a good way forward. The problem is that it is possible to author a UDF similar to the json/jsonb getter functions json_extract_path() that extract a portion of a datum that is then used when creating an index. There is currently no demarcation on functions similar to "IMMUTABLE" (for example "EXTRACTOR") that would indicate that's what the UDF does nor does it strictly fit within the category of volatility as I don't think of this as information for the optimizer to use. Ideally you'd only want to evaluate the expression when any function in the expression is known to extract a portion of the datum. Were that the case you could require for HOT updates in the presence of expression indexes that at least one function in the expression be an "EXTRACTOR". I might still do this depending on reaction to it here, I'm seeking ideas.
Another idea was to change up the strategy a bit and not check expressions at all, but rather find a method to keep the attribute out of the modified_attrs set to begin with. This would mean that HeapDeterminColumnsInfo() would have to change, any maybe that's a good idea to try out too. It could even morph into something that can test equality using type-specific equality tests supporting custom types and invoke index access method-specific path extraction functions. In some ways this feels more "correct" to me in that HeapDeterminColumnsInfo()'s goal is to reduce the set of attributes in play from the interesting set to the modified set and if what is interesting is an index on a field within a document that isn't modified then it shouldn't be in the set. This comes with challenges related to that comment mentioned earlier in heap_attr_equals() and the fact that index access methods don't have a way to supply their equality op or provide a way to call a path operator should one exist. But the benefit would be that we expand HOT updates a bit further than my initial goals.
I'm not wedded to the use of a reloption to disable expression checks, nor do I love the name I choose ("expression_checks"). I'm interested to hear if I should simple remove this.
Another benefit to changing HeapDeterminColumnsInfo() is that it is what provides information to the PHOT patch during heap pruning to record which attributes were changed. As it stands now, this patch and the PHOT patch when combined result in some undesirable index scan results.
Partial Indexes and HOT updates
==========================================================
Partial indexes are a problem when it comes to HOT updates because they are part of the modified_attrs set returned by HeapDeterminColumnsInfo() even in cases where the value being set in the UPDATE does not satisfy the partial index predicate. Take for example:
CREATE TABLE example (a int, b int);
CREATE INDEX idx_a ON example(a);
CREATE INDEX idx_b ON example(b) WHERE b > 100;
INSERT INTO example (a, b) VALUES (1, 50);
UPDATE example SET b = 60 WHERE a = 1;
What happened? Was the update HOT? No. Why? Because the predicate isn't taken into account at the time the decision for HOT (or not) is made, that happens later during ExecInsertIndexTuples(). Today this update would trigger new index entries for both idx_a and idx_b when neither were required.
This patch changes this behavior by evaluating the predicate for the partial index earlier during heap_update(). When both the existing and updated tuples fall outside of the predicate the HOT update path is still an option. To limit scope creep I've not implemented the case when the existing was within the index's predicate but now the updated falls outside the predicate because that would require adding an optional delete function to the index access method (something that I think would be generally useful at some point). When that happens this patch does not allow a HOT update forcing the new tuple in the heap to not be redirected so later on index scan the CTID from the index will be ignored.
Summary
==========================================================
Attached find v17 of this patch rebased to this morning's (EDT) HEAD passing make world and formatted.
any and all feedback welcome.
-greg
Attachments:
[text/x-patch] v17-0001-Expand-HOT-update-path-to-include-expression-and.patch (134.4K, 2-v17-0001-Expand-HOT-update-path-to-include-expression-and.patch)
download | inline diff:
From 4057bfe4636de795bb14afa230d77cb7ead13b34 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Mon, 27 Jan 2025 13:28:59 -0500
Subject: [PATCH v17] Expand HOT update path to include expression and partial
indexes.
This patch extends the cases where HOT updates are possible in the
heapam. Expression indexes have historically been considered "hot
blocking" because the functions used within an expression must be marked
IMMUTABLE and so any changed attribute referenced within the expression
logically requires a new index tuple. However this logic doesn't hold
true in cases where only a portion of an attribute is used to form the
index tuple as is commonly the case with JSONB columns or other index
access methods that have custom operators which extract a subset of an
attribute when forming the index tuple. In these cases while an
attribute in an UPDATE statement may have changed as determined by
HeapDetermineColumnsInfo() and is part of the modified_attrs set the
result of the expression used to form the index tuple may be unchanged
from the previous version. This oversight makes it impossible to have a
HOT update when an update includes a JSONB column. This patch corrects
that. The index expression is evaluated to form the before and after
index datums and compares them to determine if the mutation to the
attribute(s) referenced in the expression is in fact a change that
requires inserting a new index tuple or not.
Secondly, updates to attributes referenced by partial indexes are
effectively "HOT blocking" even when they fall outside of the index's
predicate forcing all indexes on the relation to be updated. This patch
changes that behavior allowing HOT updates only in cases where the
predicate on the index is not satisfied for either the current or the
new tuple.
It is important to note that this patch is an evolution of a previous
patch
https://postgr.es/m/4d9928ee-a9e6-15f9-9c82-5981f13ffca6%40postgrespro.ru
applied in c203d6cf81b4d7e43edb2b75ec1b741ba48e04e0 and later reverted
in 05f84605dbeb9cf8279a157234b24bbb706c5256.
Signed-off-by: Greg Burd <[email protected]>
---
doc/src/sgml/ref/create_table.sgml | 18 +
doc/src/sgml/storage.sgml | 29 +-
src/backend/access/common/reloptions.c | 12 +-
src/backend/access/heap/README.HOT | 45 +-
src/backend/access/heap/heapam.c | 61 +-
src/backend/access/heap/heapam_handler.c | 15 +-
src/backend/access/table/tableam.c | 5 +-
src/backend/catalog/index.c | 48 +
src/backend/catalog/indexing.c | 18 +-
src/backend/commands/copyfrom.c | 2 +-
src/backend/executor/execIndexing.c | 212 +++-
src/backend/executor/execPartition.c | 2 +-
src/backend/executor/execReplication.c | 10 +-
src/backend/executor/nodeModifyTable.c | 31 +-
src/backend/replication/logical/worker.c | 6 +-
src/backend/utils/cache/relcache.c | 81 +-
src/bin/psql/tab-complete.in.c | 2 +-
src/include/access/heapam.h | 6 +-
src/include/access/tableam.h | 33 +-
src/include/catalog/index.h | 1 +
src/include/executor/executor.h | 8 +-
src/include/nodes/execnodes.h | 13 +
src/include/utils/rel.h | 10 +
src/include/utils/relcache.h | 1 +
.../regress/expected/heap_hot_updates.out | 1048 +++++++++++++++++
src/test/regress/parallel_schedule | 5 +
src/test/regress/sql/heap_hot_updates.sql | 718 +++++++++++
src/tools/pgindent/typedefs.list | 2 +-
28 files changed, 2302 insertions(+), 140 deletions(-)
create mode 100644 src/test/regress/expected/heap_hot_updates.out
create mode 100644 src/test/regress/sql/heap_hot_updates.sql
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index dc000e913c1..bece90dc192 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -1997,6 +1997,24 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry id="reloption-expression-checks" xreflabel="expression_checks">
+ <term><literal>expression_checks</literal> (<type>boolean</type>)
+ <indexterm>
+ <primary><varname>expression_checks</varname> storage parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Enables or disables evaulation of predicate expressions on partial
+ indexes or expressions used to define indexes during updates.
+ If <literal>true</literal>, then these expressions are evaluated during
+ updates to data within the heap relation against the old and new values
+ and then compared to determine if <acronym>HOT</acronym> updates are
+ allowable or not. The default value is <literal>true</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect2>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index 61250799ec0..6b340e885f2 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -1097,10 +1097,10 @@ data. Empty in ordinary tables.</entry>
<itemizedlist>
<listitem>
<para>
- The update does not modify any columns referenced by the table's indexes,
- not including summarizing indexes. The only summarizing index method in
- the core <productname>PostgreSQL</productname> distribution is <link
- linkend="brin">BRIN</link>.
+ The update does not modify index keys, or when using a summarized
+ index. The only summarizing index method in the core
+ <productname>PostgreSQL</productname> distribution is
+ <link linkend="brin">BRIN</link>.
</para>
</listitem>
<listitem>
@@ -1138,6 +1138,27 @@ data. Empty in ordinary tables.</entry>
</itemizedlist>
</para>
+ <para>
+ <acronym>HOT</acronym> updates can occur when the expression used to define
+ an index shows no changes to the indexed value. To determine this requires
+ that the expression be evaulated for the old and new values to be stored in
+ the index and then compared. This allows for <acronym>HOT</acronym> updates
+ when data indexed within JSONB columns is unchanged. To disable this
+ behavior and avoid the overhead of evaluating the expression during updates
+ set the <literal>expression_checks</literal> option to false for the table.
+ </para>
+
+ <para>
+ <acronym>HOT</acronym> updates can also occur when updated values are not
+ within the predicate of a partial index. However, <acronym>HOT</acronym>
+ updates are not possible when the updated value and the current value differ
+ with regards to the predicate. To determine this requires that the predicate
+ expression be evaluated for the old and new values to be stored in the index
+ and then compared. To disable this behavior and avoid the overhead of
+ evaluating the expression during updates set
+ the <literal>expression_checks</literal> option to false for the table.
+ </para>
+
<para>
You can increase the likelihood of sufficient page space for
<acronym>HOT</acronym> updates by decreasing a table's <link
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 50747c16396..407d5dcf0c9 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -166,6 +166,15 @@ static relopt_bool boolRelOpts[] =
},
true
},
+ {
+ {
+ "expression_checks",
+ "When disabled prevents checking expressions on indexes and predicates on partial indexes for changes that might influence heap-only tuple (HOT) updates.",
+ RELOPT_KIND_HEAP,
+ ShareUpdateExclusiveLock
+ },
+ true
+ },
/* list terminator */
{{NULL}}
};
@@ -1915,7 +1924,8 @@ default_reloptions(Datum reloptions, bool validate, relopt_kind kind)
{"vacuum_truncate", RELOPT_TYPE_BOOL,
offsetof(StdRdOptions, vacuum_truncate), offsetof(StdRdOptions, vacuum_truncate_set)},
{"vacuum_max_eager_freeze_failure_rate", RELOPT_TYPE_REAL,
- offsetof(StdRdOptions, vacuum_max_eager_freeze_failure_rate)}
+ offsetof(StdRdOptions, vacuum_max_eager_freeze_failure_rate)},
+ {"expression_checks", RELOPT_TYPE_BOOL, offsetof(StdRdOptions, expression_checks)}
};
return (bytea *) build_reloptions(reloptions, validate, kind,
diff --git a/src/backend/access/heap/README.HOT b/src/backend/access/heap/README.HOT
index 74e407f375a..2340d82053f 100644
--- a/src/backend/access/heap/README.HOT
+++ b/src/backend/access/heap/README.HOT
@@ -36,7 +36,7 @@ HOT solves this problem for two restricted but useful special cases:
First, where a tuple is repeatedly updated in ways that do not change
its indexed columns. (Here, "indexed column" means any column referenced
at all in an index definition, including for example columns that are
-tested in a partial-index predicate but are not stored in the index.)
+tested in a partial-index predicate, that has materially changed.)
Second, where the modified columns are only used in indexes that do not
contain tuple IDs, but maintain summaries of the indexed data by block.
@@ -133,28 +133,34 @@ Note: we can use a "dead" line pointer for any DELETEd tuple,
whether it was part of a HOT chain or not. This allows space reclamation
in advance of running VACUUM for plain DELETEs as well as HOT updates.
-The requirement for doing a HOT update is that indexes which point to
-the root line pointer (and thus need to be cleaned up by VACUUM when the
-tuple is dead) do not reference columns which are updated in that HOT
-chain. Summarizing indexes (such as BRIN) are assumed to have no
-references to individual tuples and thus are ignored when checking HOT
-applicability. The updated columns are checked at execution time by
-comparing the binary representation of the old and new values. We insist
-on bitwise equality rather than using datatype-specific equality routines.
-The main reason to avoid the latter is that there might be multiple
-notions of equality for a datatype, and we don't know exactly which one
-is relevant for the indexes at hand. We assume that bitwise equality
-guarantees equality for all purposes.
+The requirement for doing a HOT update is that indexes which point to the root
+line pointer (and thus need to be cleaned up by VACUUM when the tuple is dead)
+do not reference columns which are updated in that HOT chain.
+
+Summarizing indexes (such as BRIN) are assumed to have no references to
+individual tuples and thus are ignored when checking HOT applicability.
+
+Expressions on indexes are evaluated and the results used when checking for
+changes. This allows for the JSONB datatype to have HOT updates when the
+indexed portion of the document are not modified.
+
+Partial index expressions are evaluated, HOT updates are allowed when the
+updated index values do not satisfy the predicate.
+
+The updated columns are checked at execution time by comparing the binary
+representation of the old and new values. We insist on bitwise equality rather
+than using datatype-specific equality routines. The main reason to avoid the
+latter is that there might be multiple notions of equality for a datatype, and
+we don't know exactly which one is relevant for the indexes at hand. We assume
+that bitwise equality guarantees equality for all purposes.
If any columns that are included by non-summarizing indexes are updated,
the HOT optimization is not applied, and the new tuple is inserted into
all indexes of the table. If none of the updated columns are included in
the table's indexes, the HOT optimization is applied and no indexes are
updated. If instead the updated columns are only indexed by summarizing
-indexes, the HOT optimization is applied, but the update is propagated to
-all summarizing indexes. (Realistically, we only need to propagate the
-update to the indexes that contain the updated values, but that is yet to
-be implemented.)
+indexes, the HOT optimization is applied and the update is propagated to
+all of the summarizing indexes.
Abort Cases
-----------
@@ -477,8 +483,9 @@ Heap-only tuple
HOT-safe
A proposed tuple update is said to be HOT-safe if it changes
- none of the tuple's indexed columns. It will only become an
- actual HOT update if we can find room on the same page for
+ none of the tuple's indexed columns or if the changes remain
+ outside of a partial index's predicate. It will only become
+ an actual HOT update if we can find room on the same page for
the new tuple version.
HOT update
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 0dcd6ee817e..9edd18808f8 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -3238,13 +3238,14 @@ simple_heap_delete(Relation relation, ItemPointer tid)
TM_Result
heap_update(Relation relation, ItemPointer otid, HeapTuple newtup,
CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ TM_FailureData *tmfd, UpdateContext *updateCxt)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
+ LockTupleMode *lockmode = &updateCxt->lockmode;
Bitmapset *hot_attrs;
Bitmapset *sum_attrs;
+ Bitmapset *exp_attrs;
Bitmapset *key_attrs;
Bitmapset *id_attrs;
Bitmapset *interesting_attrs;
@@ -3323,6 +3324,8 @@ heap_update(Relation relation, ItemPointer otid, HeapTuple newtup,
INDEX_ATTR_BITMAP_HOT_BLOCKING);
sum_attrs = RelationGetIndexAttrBitmap(relation,
INDEX_ATTR_BITMAP_SUMMARIZED);
+ exp_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_EXPRESSION);
key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
id_attrs = RelationGetIndexAttrBitmap(relation,
INDEX_ATTR_BITMAP_IDENTITY_KEY);
@@ -3384,10 +3387,11 @@ heap_update(Relation relation, ItemPointer otid, HeapTuple newtup,
tmfd->ctid = *otid;
tmfd->xmax = InvalidTransactionId;
tmfd->cmax = InvalidCommandId;
- *update_indexes = TU_None;
+ updateCxt->updateIndexes = TU_None;
bms_free(hot_attrs);
bms_free(sum_attrs);
+ bms_free(exp_attrs);
bms_free(key_attrs);
bms_free(id_attrs);
/* modified_attrs not yet initialized */
@@ -3685,10 +3689,11 @@ l2:
UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
if (vmbuffer != InvalidBuffer)
ReleaseBuffer(vmbuffer);
- *update_indexes = TU_None;
+ updateCxt->updateIndexes = TU_None;
bms_free(hot_attrs);
bms_free(sum_attrs);
+ bms_free(exp_attrs);
bms_free(key_attrs);
bms_free(id_attrs);
bms_free(modified_attrs);
@@ -4005,23 +4010,39 @@ l2:
if (newbuf == buffer)
{
+ ResultRelInfo *resultRelInfo = updateCxt->rri;
+ EState *estate = updateCxt->estate;
+
/*
- * Since the new tuple is going into the same page, we might be able
- * to do a HOT update. Check if any of the index columns have been
- * changed.
+ * hot_attrs includes indexes with expressions and indexes with
+ * predicates that may not be impacted by this change. If the
+ * modified attributes in this update don't overlap with any
+ * attributes referenced by indexes on the relation then we can use
+ * the HOT update path. If they do overlap, then check to see if the
+ * overlap is exclusively due to attributes that are only referenced
+ * within expressions. If that is the case, the HOT update path may
+ * be possible iff the expression indexes are unchanged by this update
+ * or, with partial indexes, both the new and the old heap tuples
+ * don't satisfy the partial index predicate expression (meaning they
+ * are both outside of the scope of the index).
*/
- if (!bms_overlap(modified_attrs, hot_attrs))
+ if (!bms_overlap(modified_attrs, hot_attrs) ||
+ (bms_is_subset(modified_attrs, exp_attrs) &&
+ !ExecExprIndexesRequireUpdates(relation,
+ resultRelInfo,
+ modified_attrs,
+ estate,
+ resultRelInfo->ri_oldTupleSlot,
+ resultRelInfo->ri_newTupleSlot)))
{
use_hot_update = true;
/*
- * If none of the columns that are used in hot-blocking indexes
- * were updated, we can apply HOT, but we do still need to check
- * if we need to update the summarizing indexes, and update those
- * indexes if the columns were updated, or we may fail to detect
- * e.g. value bound changes in BRIN minmax indexes.
+ * If all modified attributes were only referenced by summarizing
+ * indexes then we remain HOT, but we need to update those indexes
+ * to ensure that they are consistent with the new tuple.
*/
- if (bms_overlap(modified_attrs, sum_attrs))
+ if (bms_is_subset(modified_attrs, sum_attrs))
summarized_update = true;
}
}
@@ -4192,18 +4213,19 @@ l2:
if (use_hot_update)
{
if (summarized_update)
- *update_indexes = TU_Summarizing;
+ updateCxt->updateIndexes = TU_Summarizing;
else
- *update_indexes = TU_None;
+ updateCxt->updateIndexes = TU_None;
}
else
- *update_indexes = TU_All;
+ updateCxt->updateIndexes = TU_All;
if (old_key_tuple != NULL && old_key_copied)
heap_freetuple(old_key_tuple);
bms_free(hot_attrs);
bms_free(sum_attrs);
+ bms_free(exp_attrs);
bms_free(key_attrs);
bms_free(id_attrs);
bms_free(modified_attrs);
@@ -4481,16 +4503,15 @@ HeapDetermineColumnsInfo(Relation relation,
*/
void
simple_heap_update(Relation relation, ItemPointer otid, HeapTuple tup,
- TU_UpdateIndexes *update_indexes)
+ UpdateContext *updateCxt)
{
TM_Result result;
TM_FailureData tmfd;
- LockTupleMode lockmode;
result = heap_update(relation, otid, tup,
GetCurrentCommandId(true), InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, updateCxt);
switch (result)
{
case TM_SelfModified:
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index cb4bc35c93e..ae03ed5e718 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -316,8 +316,7 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd,
- LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
+ bool wait, TM_FailureData *tmfd, UpdateContext *updateCxt)
{
bool shouldFree = true;
HeapTuple tuple = ExecFetchSlotHeapTuple(slot, true, &shouldFree);
@@ -328,7 +327,7 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
tuple->t_tableOid = slot->tts_tableOid;
result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
- tmfd, lockmode, update_indexes);
+ tmfd, updateCxt);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
@@ -343,14 +342,14 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
*/
if (result != TM_Ok)
{
- Assert(*update_indexes == TU_None);
- *update_indexes = TU_None;
+ Assert(updateCxt->updateIndexes == TU_None);
+ updateCxt->updateIndexes = TU_None;
}
else if (!HeapTupleIsHeapOnly(tuple))
- Assert(*update_indexes == TU_All);
+ Assert(updateCxt->updateIndexes == TU_All);
else
- Assert((*update_indexes == TU_Summarizing) ||
- (*update_indexes == TU_None));
+ Assert((updateCxt->updateIndexes == TU_Summarizing) ||
+ (updateCxt->updateIndexes == TU_None));
if (shouldFree)
pfree(tuple);
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index a56c5eceb14..1eeeebadc09 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -336,17 +336,16 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
- TU_UpdateIndexes *update_indexes)
+ UpdateContext *updateCxt)
{
TM_Result result;
TM_FailureData tmfd;
- LockTupleMode lockmode;
result = table_tuple_update(rel, otid, slot,
GetCurrentCommandId(true),
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, updateCxt);
switch (result)
{
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index aa216683b74..cace52120af 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2467,6 +2467,8 @@ BuildIndexInfo(Relation index)
&ii->ii_ExclusionStrats);
}
+ ii->ii_IndexAttrByVal = NULL;
+
return ii;
}
@@ -2707,6 +2709,52 @@ BuildSpeculativeIndexInfo(Relation index, IndexInfo *ii)
}
}
+/* ----------------
+ * BuildExpressionIndexInfo
+ * Add extra state to IndexInfo record
+ *
+ * For expression indexes updates may not change the indexed value allowing
+ * for a HOT update. Add information to the IndexInfo to allow for checking
+ * if the indexed value has changed.
+ *
+ * Do this processing here rather than in BuildIndexInfo() to not incur the
+ * overhead in the common non-expression cases.
+ * ----------------
+ */
+void
+BuildExpressionIndexInfo(Relation index, IndexInfo *ii)
+{
+ int i;
+ int indnkeyatts;
+
+ /*
+ * Expressions are not allowed on non-key attributes, so we can skip them
+ * as they should show up in the index HOT-blocking attributes.
+ */
+ indnkeyatts = IndexRelationGetNumberOfKeyAttributes(index);
+
+ /*
+ * Collect attributes used by the index, their len and if they are by
+ * value.
+ */
+ for (i = 0; i < indnkeyatts; i++)
+ {
+ CompactAttribute *attr = TupleDescCompactAttr(RelationGetDescr(index), i);
+
+ ii->ii_IndexAttrLen[i] = attr->attlen;
+ if (attr->attbyval)
+ ii->ii_IndexAttrByVal = bms_add_member(ii->ii_IndexAttrByVal, i);
+ }
+
+ /* collect attributes used in the expression */
+ if (ii->ii_Expressions)
+ pull_varattnos((Node *) ii->ii_Expressions, 1, &ii->ii_ExpressionsAttrs);
+
+ /* collect attributes used in the predicate */
+ if (ii->ii_Predicate)
+ pull_varattnos((Node *) ii->ii_Predicate, 1, &ii->ii_PredicateAttrs);
+}
+
/* ----------------
* FormIndexDatum
* Construct values[] and isnull[] arrays for a new index tuple.
diff --git a/src/backend/catalog/indexing.c b/src/backend/catalog/indexing.c
index 25c4b6bdc87..c413c099da8 100644
--- a/src/backend/catalog/indexing.c
+++ b/src/backend/catalog/indexing.c
@@ -49,7 +49,7 @@ CatalogOpenIndexes(Relation heapRel)
resultRelInfo->ri_RelationDesc = heapRel;
resultRelInfo->ri_TrigDesc = NULL; /* we don't fire triggers */
- ExecOpenIndices(resultRelInfo, false);
+ ExecOpenIndices(resultRelInfo, false, false);
return resultRelInfo;
}
@@ -313,15 +313,17 @@ void
CatalogTupleUpdate(Relation heapRel, ItemPointer otid, HeapTuple tup)
{
CatalogIndexState indstate;
- TU_UpdateIndexes updateIndexes = TU_All;
+ UpdateContext updateCxt = {0};
+
+ updateCxt.updateIndexes = TU_All;
CatalogTupleCheckConstraints(heapRel, tup);
indstate = CatalogOpenIndexes(heapRel);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ simple_heap_update(heapRel, otid, tup, &updateCxt);
- CatalogIndexInsert(indstate, tup, updateIndexes);
+ CatalogIndexInsert(indstate, tup, updateCxt.updateIndexes);
CatalogCloseIndexes(indstate);
}
@@ -337,13 +339,15 @@ void
CatalogTupleUpdateWithInfo(Relation heapRel, ItemPointer otid, HeapTuple tup,
CatalogIndexState indstate)
{
- TU_UpdateIndexes updateIndexes = TU_All;
+ UpdateContext updateCxt = {0};
+
+ updateCxt.updateIndexes = TU_All;
CatalogTupleCheckConstraints(heapRel, tup);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ simple_heap_update(heapRel, otid, tup, &updateCxt);
- CatalogIndexInsert(indstate, tup, updateIndexes);
+ CatalogIndexInsert(indstate, tup, updateCxt.updateIndexes);
}
/*
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index fbbbc09a97b..f8271aa8866 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -921,7 +921,7 @@ CopyFrom(CopyFromState cstate)
/* Verify the named relation is a valid target for INSERT */
CheckValidResultRel(resultRelInfo, CMD_INSERT, NIL);
- ExecOpenIndices(resultRelInfo, false);
+ ExecOpenIndices(resultRelInfo, false, false);
/*
* Set up a ModifyTableState so we can let FDW(s) init themselves for
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index ca33a854278..940311dc0d6 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -113,10 +113,13 @@
#include "catalog/index.h"
#include "executor/executor.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/optimizer.h"
#include "storage/lmgr.h"
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
+#include "utils/datum.h"
+#include "utils/lsyscache.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
typedef enum
@@ -157,7 +160,7 @@ static void ExecWithoutOverlapsNotEmpty(Relation rel, NameData attname, Datum at
* ----------------------------------------------------------------
*/
void
-ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative)
+ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative, bool update)
{
Relation resultRelation = resultRelInfo->ri_RelationDesc;
List *indexoidlist;
@@ -220,6 +223,13 @@ ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative)
if (speculative && ii->ii_Unique && !indexDesc->rd_index->indisexclusion)
BuildSpeculativeIndexInfo(indexDesc, ii);
+ /*
+ * If the index uses expressions then let's populate the additional
+ * information nessaary to evaluate them for changes during updates.
+ */
+ if (update && (ii->ii_Expressions || ii->ii_Predicate))
+ BuildExpressionIndexInfo(indexDesc, ii);
+
relationDescs[i] = indexDesc;
indexInfoArray[i] = ii;
i++;
@@ -382,19 +392,33 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
ExprState *predicate;
/*
- * If predicate state not set up yet, create it (in the estate's
- * per-query context)
+ * It is possible that we've already checked the predicate, if so
+ * then avoid the duplicate work.
*/
- predicate = indexInfo->ii_PredicateState;
- if (predicate == NULL)
+ if (indexInfo->ii_CheckedPredicate)
{
- predicate = ExecPrepareQual(indexInfo->ii_Predicate, estate);
- indexInfo->ii_PredicateState = predicate;
+ /* Skip this index-update if the predicate isn't satisfied */
+ if (!indexInfo->ii_PredicateSatisfied)
+ continue;
}
+ else
+ {
- /* Skip this index-update if the predicate isn't satisfied */
- if (!ExecQual(predicate, econtext))
- continue;
+ /*
+ * If predicate state not set up yet, create it (in the
+ * estate's per-query context)
+ */
+ predicate = indexInfo->ii_PredicateState;
+ if (predicate == NULL)
+ {
+ predicate = ExecPrepareQual(indexInfo->ii_Predicate, estate);
+ indexInfo->ii_PredicateState = predicate;
+ }
+
+ /* Skip this index-update if the predicate isn't satisfied */
+ if (!ExecQual(predicate, econtext))
+ continue;
+ }
}
/*
@@ -1095,6 +1119,9 @@ index_unchanged_by_update(ResultRelInfo *resultRelInfo, EState *estate,
if (hasexpression)
{
+ if (indexInfo->ii_IndexUnchanged)
+ return true;
+
indexInfo->ii_IndexUnchanged = false;
return false;
}
@@ -1172,3 +1199,168 @@ ExecWithoutOverlapsNotEmpty(Relation rel, NameData attname, Datum attval, char t
errmsg("empty WITHOUT OVERLAPS value found in column \"%s\" in relation \"%s\"",
NameStr(attname), RelationGetRelationName(rel))));
}
+
+/*
+ *
+ * This will first determine if the index has a predicate and if so if the
+ * update satisfies that or not. Then, if necessary, we compare old and
+ * new values of the indexed expression and help determine if it is possible
+ * to use a HOT update or not.
+ *
+ * 'resultRelInfo' is the table with the indexes we should examine.
+ * 'modified' is the set of attributes that are modified by the update.
+ * 'estate' is the executor state for the update.
+ * 'old_tts' is a slot with the old tuple.
+ * 'new_tts' is a slot with the new tuple.
+ *
+ * Returns true iff none of the indexes on this relation require updating.
+ *
+ * When the changes in new tuple impact a value stored in an index we must
+ * return true. When an index has a predicate that is not satisfied by either
+ * the new or old tuples then that index is unchanged. When an index has a
+ * predicate that is satisfied by both the old and new tuples then we can
+ * proceed and check to see if the indexed values were changed or not.
+ */
+bool
+ExecExprIndexesRequireUpdates(Relation relation,
+ ResultRelInfo *resultRelInfo,
+ Bitmapset *modifiedAttrs,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts)
+{
+ bool expression_checks = RelationGetExpressionChecks(relation);
+ bool result = false;
+ IndexInfo *indexInfo;
+ TupleTableSlot *save_scantuple;
+ ExprContext *econtext = NULL;
+
+ if (resultRelInfo == NULL || estate == NULL ||
+ old_tts == NULL || new_tts == NULL ||
+ !expression_checks || IsolationIsSerializable())
+ return true;
+
+ econtext = GetPerTupleExprContext(estate);
+
+ /*
+ * Examine each index on this relation to see if it is affected by the
+ * changes in newtup. If any index is changed, we must not use a HOT
+ * update.
+ */
+ for (int i = 0; i < resultRelInfo->ri_NumIndices; i++)
+ {
+ indexInfo = resultRelInfo->ri_IndexRelationInfo[i];
+
+ /*
+ * If this is a partial index it has a predicate, evaluate the
+ * expression to determine if we need to include it or not.
+ */
+ if (bms_overlap(indexInfo->ii_PredicateAttrs, modifiedAttrs))
+ {
+ ExprState *pstate;
+ bool old_tuple_qualifies,
+ new_tuple_qualifies;
+
+ pstate = ExecPrepareQual(indexInfo->ii_Predicate, estate);
+
+ /*
+ * Here the term "qualifies" means "satisfies the predicate
+ * condition of the partial index".
+ */
+ save_scantuple = econtext->ecxt_scantuple;
+ econtext->ecxt_scantuple = old_tts;
+ old_tuple_qualifies = ExecQual(pstate, econtext);
+
+ econtext->ecxt_scantuple = new_tts;
+ new_tuple_qualifies = ExecQual(pstate, econtext);
+ econtext->ecxt_scantuple = save_scantuple;
+
+ indexInfo->ii_CheckedPredicate = true;
+ indexInfo->ii_PredicateSatisfied = new_tuple_qualifies;
+
+ /*
+ * If neither the old nor the new tuples satisfy the predicate we
+ * can be sure that this index doesn't need updating, continue to
+ * the next index.
+ */
+ if ((new_tuple_qualifies == false) && (old_tuple_qualifies == false))
+ continue;
+
+ /*
+ * If there is a transition between indexed and not indexed,
+ * that's enough to require an index update.
+ */
+ if (new_tuple_qualifies != old_tuple_qualifies)
+ {
+ result = true;
+ break;
+ }
+
+ /*
+ * Otherwise the old and new values exist in the index, but did
+ * they get updated? We don't yet know, so proceed with the next
+ * statement in the loop to find out.
+ */
+ }
+
+ /*
+ * Indexes with expressions may or may not have changed, it is
+ * impossible to know without exercising their expression and
+ * reviewing index tuple state for changes. This is a lot of work,
+ * but because all indexes on JSONB columns fall into this category it
+ * can be worth it to avoid index updates and remain on the HOT update
+ * path when possible.
+ */
+ if (bms_overlap(indexInfo->ii_ExpressionsAttrs, modifiedAttrs))
+ {
+ Datum old_values[INDEX_MAX_KEYS];
+ bool old_isnull[INDEX_MAX_KEYS];
+ Datum new_values[INDEX_MAX_KEYS];
+ bool new_isnull[INDEX_MAX_KEYS];
+
+ save_scantuple = econtext->ecxt_scantuple;
+ econtext->ecxt_scantuple = old_tts;
+ FormIndexDatum(indexInfo,
+ old_tts,
+ estate,
+ old_values,
+ old_isnull);
+ econtext->ecxt_scantuple = new_tts;
+ FormIndexDatum(indexInfo,
+ new_tts,
+ estate,
+ new_values,
+ new_isnull);
+ econtext->ecxt_scantuple = save_scantuple;
+
+ for (int j = 0; j < indexInfo->ii_NumIndexKeyAttrs; j++)
+ {
+ if (old_isnull[j] != new_isnull[j])
+ {
+ result = true;
+ break;
+ }
+ else if (!old_isnull[j])
+ {
+ int16 elmlen = indexInfo->ii_IndexAttrLen[j];
+ bool elmbyval = bms_is_member(j, indexInfo->ii_IndexAttrByVal);
+
+ if (!datum_image_eq(old_values[j], new_values[j],
+ elmbyval, elmlen))
+ {
+ result = true;
+ break;
+ }
+ }
+ }
+
+ indexInfo->ii_CheckedUnchanged = true;
+ indexInfo->ii_IndexUnchanged = !result;
+
+ if (result)
+ break;
+ }
+ }
+
+ return result;
+}
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 514eae1037d..2886705ed39 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -539,7 +539,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
leaf_part_rri->ri_IndexRelationDescs == NULL)
ExecOpenIndices(leaf_part_rri,
(node != NULL &&
- node->onConflictAction != ONCONFLICT_NONE));
+ node->onConflictAction != ONCONFLICT_NONE), false);
/*
* Build WITH CHECK OPTION constraints for the partition. Note that we
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index 53ddd25c42d..3c97fce0100 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -677,7 +677,7 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (!skip_tuple)
{
List *recheckIndexes = NIL;
- TU_UpdateIndexes update_indexes;
+ UpdateContext updateCxt = {0};
List *conflictindexes;
bool conflict = false;
@@ -693,17 +693,19 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ updateCxt.estate = estate;
+ updateCxt.rri = resultRelInfo;
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
- &update_indexes);
+ &updateCxt);
conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
- if (resultRelInfo->ri_NumIndices > 0 && (update_indexes != TU_None))
+ if (resultRelInfo->ri_NumIndices > 0 && (updateCxt.updateIndexes != TU_None))
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
slot, estate, true,
conflictindexes ? true : false,
&conflict, conflictindexes,
- (update_indexes == TU_Summarizing));
+ (updateCxt.updateIndexes == TU_Summarizing));
/*
* Refer to the comments above the call to CheckAndReportConflict() in
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 54da8e7995b..b159ecb7b15 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -116,21 +116,6 @@ typedef struct ModifyTableContext
TupleTableSlot *cpUpdateReturningSlot;
} ModifyTableContext;
-/*
- * Context struct containing output data specific to UPDATE operations.
- */
-typedef struct UpdateContext
-{
- bool crossPartUpdate; /* was it a cross-partition update? */
- TU_UpdateIndexes updateIndexes; /* Which index updates are required? */
-
- /*
- * Lock mode to acquire on the latest tuple version before performing
- * EvalPlanQual on it
- */
- LockTupleMode lockmode;
-} UpdateContext;
-
static void ExecBatchInsert(ModifyTableState *mtstate,
ResultRelInfo *resultRelInfo,
@@ -890,7 +875,7 @@ ExecInsert(ModifyTableContext *context,
*/
if (resultRelationDesc->rd_rel->relhasindex &&
resultRelInfo->ri_IndexRelationDescs == NULL)
- ExecOpenIndices(resultRelInfo, onconflict != ONCONFLICT_NONE);
+ ExecOpenIndices(resultRelInfo, onconflict != ONCONFLICT_NONE, false);
/*
* BEFORE ROW INSERT Triggers.
@@ -2105,7 +2090,7 @@ ExecUpdatePrologue(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
*/
if (resultRelationDesc->rd_rel->relhasindex &&
resultRelInfo->ri_IndexRelationDescs == NULL)
- ExecOpenIndices(resultRelInfo, false);
+ ExecOpenIndices(resultRelInfo, false, true);
/* BEFORE ROW UPDATE triggers */
if (resultRelInfo->ri_TrigDesc &&
@@ -2303,8 +2288,7 @@ lreplace:
estate->es_snapshot,
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
- &context->tmfd, &updateCxt->lockmode,
- &updateCxt->updateIndexes);
+ &context->tmfd, updateCxt);
return result;
}
@@ -2322,6 +2306,7 @@ ExecUpdateEpilogue(ModifyTableContext *context, UpdateContext *updateCxt,
{
ModifyTableState *mtstate = context->mtstate;
List *recheckIndexes = NIL;
+ bool onlySummarizing = updateCxt->updateIndexes == TU_Summarizing;
/* insert index entries for tuple if necessary */
if (resultRelInfo->ri_NumIndices > 0 && (updateCxt->updateIndexes != TU_None))
@@ -2329,7 +2314,7 @@ ExecUpdateEpilogue(ModifyTableContext *context, UpdateContext *updateCxt,
slot, context->estate,
true, false,
NULL, NIL,
- (updateCxt->updateIndexes == TU_Summarizing));
+ onlySummarizing);
/* AFTER ROW UPDATE Triggers */
ExecARUpdateTriggers(context->estate, resultRelInfo,
@@ -2465,6 +2450,9 @@ ExecUpdate(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
UpdateContext updateCxt = {0};
TM_Result result;
+ updateCxt.estate = estate;
+ updateCxt.rri = resultRelInfo;
+
/*
* abort the operation if not running transactions
*/
@@ -3153,6 +3141,9 @@ lmerge_matched:
TM_Result result;
UpdateContext updateCxt = {0};
+ updateCxt.rri = resultRelInfo;
+ updateCxt.estate = estate;
+
/*
* Test condition, if any.
*
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index fd11805a44c..51054693f24 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -2452,7 +2452,7 @@ apply_handle_insert(StringInfo s)
{
ResultRelInfo *relinfo = edata->targetRelInfo;
- ExecOpenIndices(relinfo, false);
+ ExecOpenIndices(relinfo, false, false);
apply_handle_insert_internal(edata, relinfo, remoteslot);
ExecCloseIndices(relinfo);
}
@@ -2675,7 +2675,7 @@ apply_handle_update_internal(ApplyExecutionData *edata,
MemoryContext oldctx;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
- ExecOpenIndices(relinfo, false);
+ ExecOpenIndices(relinfo, false, true);
found = FindReplTupleInLocalRel(edata, localrel,
&relmapentry->remoterel,
@@ -2819,7 +2819,7 @@ apply_handle_delete(StringInfo s)
{
ResultRelInfo *relinfo = edata->targetRelInfo;
- ExecOpenIndices(relinfo, false);
+ ExecOpenIndices(relinfo, false, false);
apply_handle_delete_internal(edata, relinfo,
remoteslot, rel->localindexoid);
ExecCloseIndices(relinfo);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 559ba9cdb2c..b7e78fb3bf1 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -64,6 +64,7 @@
#include "catalog/pg_type.h"
#include "catalog/schemapg.h"
#include "catalog/storage.h"
+#include "catalog/index.h"
#include "commands/policy.h"
#include "commands/publicationcmds.h"
#include "commands/trigger.h"
@@ -2482,6 +2483,7 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
bms_free(relation->rd_idattr);
bms_free(relation->rd_hotblockingattr);
bms_free(relation->rd_summarizedattr);
+ bms_free(relation->rd_expressionattr);
if (relation->rd_pubdesc)
pfree(relation->rd_pubdesc);
if (relation->rd_options)
@@ -5283,6 +5285,7 @@ RelationGetIndexPredicate(Relation relation)
* index (empty if FULL)
* INDEX_ATTR_BITMAP_HOT_BLOCKING Columns that block updates from being HOT
* INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
+ * INDEX_ATTR_BITMAP_EXPRESSION Columns included in expresion indexes
*
* Attribute numbers are offset by FirstLowInvalidHeapAttributeNumber so that
* we can include system attributes (e.g., OID) in the bitmap representation.
@@ -5305,8 +5308,9 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
Bitmapset *uindexattrs; /* columns in unique indexes */
Bitmapset *pkindexattrs; /* columns in the primary index */
Bitmapset *idindexattrs; /* columns in the replica identity */
- Bitmapset *hotblockingattrs; /* columns with HOT blocking indexes */
- Bitmapset *summarizedattrs; /* columns with summarizing indexes */
+ Bitmapset *idx_attrs; /* columns referenced by indexes */
+ Bitmapset *expr_attrs; /* columns referenced by index expressions */
+ Bitmapset *sum_attrs; /* columns with summarizing indexes */
List *indexoidlist;
List *newindexoidlist;
Oid relpkindex;
@@ -5329,6 +5333,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
return bms_copy(relation->rd_hotblockingattr);
case INDEX_ATTR_BITMAP_SUMMARIZED:
return bms_copy(relation->rd_summarizedattr);
+ case INDEX_ATTR_BITMAP_EXPRESSION:
+ return bms_copy(relation->rd_expressionattr);
default:
elog(ERROR, "unknown attrKind %u", attrKind);
}
@@ -5371,8 +5377,9 @@ restart:
uindexattrs = NULL;
pkindexattrs = NULL;
idindexattrs = NULL;
- hotblockingattrs = NULL;
- summarizedattrs = NULL;
+ idx_attrs = NULL;
+ expr_attrs = NULL;
+ sum_attrs = NULL;
foreach(l, indexoidlist)
{
Oid indexOid = lfirst_oid(l);
@@ -5386,6 +5393,7 @@ restart:
bool isPK; /* primary key */
bool isIDKey; /* replica identity index */
Bitmapset **attrs;
+ Bitmapset **exprattrs;
indexDesc = index_open(indexOid, AccessShareLock);
@@ -5429,20 +5437,26 @@ restart:
* decide which bitmap we'll update in the following loop.
*/
if (indexDesc->rd_indam->amsummarizing)
- attrs = &summarizedattrs;
+ {
+ attrs = &sum_attrs;
+ exprattrs = &sum_attrs;
+ }
else
- attrs = &hotblockingattrs;
+ {
+ attrs = &idx_attrs;
+ exprattrs = &expr_attrs;
+ }
/* Collect simple attribute references */
for (i = 0; i < indexDesc->rd_index->indnatts; i++)
{
- int attrnum = indexDesc->rd_index->indkey.values[i];
+ int attridx = indexDesc->rd_index->indkey.values[i];
/*
* Since we have covering indexes with non-key columns, we must
* handle them accurately here. non-key columns must be added into
- * hotblockingattrs or summarizedattrs, since they are in index,
- * and update shouldn't miss them.
+ * idx_attrs or sum_attrs, since they are in index, and update
+ * shouldn't miss them.
*
* Summarizing indexes do not block HOT, but do need to be updated
* when the column value changes, thus require a separate
@@ -5452,30 +5466,28 @@ restart:
* key or identity key. Hence we do not include them into
* uindexattrs, pkindexattrs and idindexattrs bitmaps.
*/
- if (attrnum != 0)
+ if (attridx != 0)
{
- *attrs = bms_add_member(*attrs,
- attrnum - FirstLowInvalidHeapAttributeNumber);
+ AttrNumber attrnum = attridx - FirstLowInvalidHeapAttributeNumber;
+
+ *attrs = bms_add_member(*attrs, attrnum);
if (isKey && i < indexDesc->rd_index->indnkeyatts)
- uindexattrs = bms_add_member(uindexattrs,
- attrnum - FirstLowInvalidHeapAttributeNumber);
+ uindexattrs = bms_add_member(uindexattrs, attrnum);
if (isPK && i < indexDesc->rd_index->indnkeyatts)
- pkindexattrs = bms_add_member(pkindexattrs,
- attrnum - FirstLowInvalidHeapAttributeNumber);
+ pkindexattrs = bms_add_member(pkindexattrs, attrnum);
if (isIDKey && i < indexDesc->rd_index->indnkeyatts)
- idindexattrs = bms_add_member(idindexattrs,
- attrnum - FirstLowInvalidHeapAttributeNumber);
+ idindexattrs = bms_add_member(idindexattrs, attrnum);
}
}
/* Collect all attributes used in expressions, too */
- pull_varattnos(indexExpressions, 1, attrs);
+ pull_varattnos(indexExpressions, 1, exprattrs);
/* Collect all attributes in the index predicate, too */
- pull_varattnos(indexPredicate, 1, attrs);
+ pull_varattnos(indexPredicate, 1, exprattrs);
index_close(indexDesc, AccessShareLock);
}
@@ -5503,12 +5515,24 @@ restart:
bms_free(uindexattrs);
bms_free(pkindexattrs);
bms_free(idindexattrs);
- bms_free(hotblockingattrs);
- bms_free(summarizedattrs);
+ bms_free(idx_attrs);
+ bms_free(expr_attrs);
+ bms_free(sum_attrs);
goto restart;
}
+ /*
+ * HOT-blocking attributes should include all columns that are part of the
+ * index except attributes only referenced in expressions, including
+ * expressions used to form partial indexes. So, we need to remove the
+ * expression-only attributes from the HOT-blocking columns bitmap as
+ * those will be checked separately.
+ */
+ expr_attrs = bms_del_members(expr_attrs, idx_attrs);
+ idx_attrs = bms_add_members(idx_attrs, expr_attrs);
+ expr_attrs = bms_add_members(expr_attrs, sum_attrs);
+
/* Don't leak the old values of these bitmaps, if any */
relation->rd_attrsvalid = false;
bms_free(relation->rd_keyattr);
@@ -5521,6 +5545,8 @@ restart:
relation->rd_hotblockingattr = NULL;
bms_free(relation->rd_summarizedattr);
relation->rd_summarizedattr = NULL;
+ bms_free(relation->rd_expressionattr);
+ relation->rd_expressionattr = NULL;
/*
* Now save copies of the bitmaps in the relcache entry. We intentionally
@@ -5533,8 +5559,9 @@ restart:
relation->rd_keyattr = bms_copy(uindexattrs);
relation->rd_pkattr = bms_copy(pkindexattrs);
relation->rd_idattr = bms_copy(idindexattrs);
- relation->rd_hotblockingattr = bms_copy(hotblockingattrs);
- relation->rd_summarizedattr = bms_copy(summarizedattrs);
+ relation->rd_hotblockingattr = bms_copy(idx_attrs);
+ relation->rd_summarizedattr = bms_copy(sum_attrs);
+ relation->rd_expressionattr = bms_copy(expr_attrs);
relation->rd_attrsvalid = true;
MemoryContextSwitchTo(oldcxt);
@@ -5548,9 +5575,11 @@ restart:
case INDEX_ATTR_BITMAP_IDENTITY_KEY:
return idindexattrs;
case INDEX_ATTR_BITMAP_HOT_BLOCKING:
- return hotblockingattrs;
+ return idx_attrs;
case INDEX_ATTR_BITMAP_SUMMARIZED:
- return summarizedattrs;
+ return sum_attrs;
+ case INDEX_ATTR_BITMAP_EXPRESSION:
+ return expr_attrs;
default:
elog(ERROR, "unknown attrKind %u", attrKind);
return NULL;
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index 8c2ea0b9587..07d1cffdefa 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -3007,7 +3007,7 @@ match_previous_words(int pattern_id,
COMPLETE_WITH("(");
/* ALTER TABLESPACE <foo> SET|RESET ( */
else if (Matches("ALTER", "TABLESPACE", MatchAny, "SET|RESET", "("))
- COMPLETE_WITH("seq_page_cost", "random_page_cost",
+ COMPLETE_WITH("seq_page_cost", "random_page_cost", "expression_checks",
"effective_io_concurrency", "maintenance_io_concurrency");
/* ALTER TEXT SEARCH */
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index a2bd5a897f8..5dd0922f52f 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -22,6 +22,7 @@
#include "access/table.h" /* for backward compatibility */
#include "access/tableam.h"
#include "commands/vacuum.h"
+#include "executor/executor.h"
#include "nodes/lockoptions.h"
#include "nodes/primnodes.h"
#include "storage/bufpage.h"
@@ -329,8 +330,7 @@ extern void heap_abort_speculative(Relation relation, ItemPointer tid);
extern TM_Result heap_update(Relation relation, ItemPointer otid,
HeapTuple newtup,
CommandId cid, Snapshot crosscheck, bool wait,
- struct TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+ struct TM_FailureData *tmfd, UpdateContext *updateCxt);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -365,7 +365,7 @@ extern bool heap_tuple_needs_eventual_freeze(HeapTupleHeader tuple);
extern void simple_heap_insert(Relation relation, HeapTuple tup);
extern void simple_heap_delete(Relation relation, ItemPointer tid);
extern void simple_heap_update(Relation relation, ItemPointer otid,
- HeapTuple tup, TU_UpdateIndexes *update_indexes);
+ HeapTuple tup, UpdateContext *updateCxt);
extern TransactionId heap_index_delete_tuples(Relation rel,
TM_IndexDeleteOp *delstate);
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 1c9e802a6b1..932f8ff6c4c 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -118,6 +118,28 @@ typedef enum TU_UpdateIndexes
TU_Summarizing,
} TU_UpdateIndexes;
+/*
+ * Data specific to processing UPDATE operations.
+ *
+ * When table_tuple_update is called some storage managers, notably heapam,
+ * can at times avoid index updates. In the heapam this is known as a HOT
+ * update. This struct is used to provide the state required to test for
+ * HOT updates and to communicate that decision on to the index AMs.
+ */
+typedef struct UpdateContext
+{
+ TU_UpdateIndexes updateIndexes; /* Which index updates are required? */
+ struct ResultRelInfo *rri; /* ResultRelInfo for the updated table. */
+ struct EState *estate; /* EState used within the update. */
+ bool crossPartUpdate; /* Was it a cross-partition update? */
+
+ /*
+ * Lock mode to acquire on the latest tuple version before performing
+ * EvalPlanQual on it
+ */
+ LockTupleMode lockmode;
+} UpdateContext;
+
/*
* When table_tuple_update, table_tuple_delete, or table_tuple_lock fail
* because the target tuple is already outdated, they fill in this struct to
@@ -542,8 +564,7 @@ typedef struct TableAmRoutine
Snapshot crosscheck,
bool wait,
TM_FailureData *tmfd,
- LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+ UpdateContext *updateCxt);
/* see table_tuple_lock() for reference about parameters */
TM_Result (*tuple_lock) (Relation rel,
@@ -1494,13 +1515,11 @@ table_tuple_delete(Relation rel, ItemPointer tid, CommandId cid,
static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ bool wait, TM_FailureData *tmfd, UpdateContext *updateCxt)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
- wait, tmfd,
- lockmode, update_indexes);
+ wait, tmfd, updateCxt);
}
/*
@@ -2001,7 +2020,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
- TU_UpdateIndexes *update_indexes);
+ UpdateContext *updateCxt);
/* ----------------------------------------------------------------------------
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 4daa8bef5ee..fdbf47f607b 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -132,6 +132,7 @@ extern bool CompareIndexInfo(const IndexInfo *info1, const IndexInfo *info2,
const AttrMap *attmap);
extern void BuildSpeculativeIndexInfo(Relation index, IndexInfo *ii);
+extern void BuildExpressionIndexInfo(Relation index, IndexInfo *indexInfo);
extern void FormIndexDatum(IndexInfo *indexInfo,
TupleTableSlot *slot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 104b059544d..9cffdaa8561 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -731,7 +731,7 @@ extern Bitmapset *ExecGetAllUpdatedCols(ResultRelInfo *relinfo, EState *estate);
/*
* prototypes from functions in execIndexing.c
*/
-extern void ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative);
+extern void ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative, bool update);
extern void ExecCloseIndices(ResultRelInfo *resultRelInfo);
extern List *ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
TupleTableSlot *slot, EState *estate,
@@ -749,6 +749,12 @@ extern void check_exclusion_constraint(Relation heap, Relation index,
ItemPointer tupleid,
const Datum *values, const bool *isnull,
EState *estate, bool newIndex);
+extern bool ExecExprIndexesRequireUpdates(Relation relation,
+ ResultRelInfo *resultRelInfo,
+ Bitmapset *modifiedAttrs,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
/*
* prototypes from functions in execReplication.c
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index e107d6e5f81..bc6d1919103 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -180,11 +180,20 @@ typedef struct IndexInfo
List *ii_Expressions; /* list of Expr */
/* exec state for expressions, or NIL if none */
List *ii_ExpressionsState; /* list of ExprState */
+ /* attributes referenced by expressions, or NULL if none */
+ Bitmapset *ii_ExpressionsAttrs;
+
+ /* index attribute length */
+ uint16 ii_IndexAttrLen[INDEX_MAX_KEYS];
+ /* is the index attribute by-value */
+ Bitmapset *ii_IndexAttrByVal;
/* partial-index predicate, or NIL if none */
List *ii_Predicate; /* list of Expr */
/* exec state for expressions, or NIL if none */
ExprState *ii_PredicateState;
+ /* attributes referenced by the predicate, or NULL if none */
+ Bitmapset *ii_PredicateAttrs;
/* Per-column exclusion operators, or NULL if none */
Oid *ii_ExclusionOps; /* array with one entry per column */
@@ -208,6 +217,10 @@ typedef struct IndexInfo
bool ii_CheckedUnchanged;
/* aminsert hint, cached for retail inserts */
bool ii_IndexUnchanged;
+ /* partial index predicate determined yet? */
+ bool ii_CheckedPredicate;
+ /* amupdate hint used to avoid rechecking predicate */
+ bool ii_PredicateSatisfied;
/* are we doing a concurrent index build? */
bool ii_Concurrent;
/* did we detect any broken HOT chains? */
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index b552359915f..27ed8db903d 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -164,6 +164,7 @@ typedef struct RelationData
Bitmapset *rd_idattr; /* included in replica identity index */
Bitmapset *rd_hotblockingattr; /* cols blocking HOT update */
Bitmapset *rd_summarizedattr; /* cols indexed by summarizing indexes */
+ Bitmapset *rd_expressionattr; /* indexed cols referenced by expressions */
PublicationDesc *rd_pubdesc; /* publication descriptor, or NULL */
@@ -348,6 +349,7 @@ typedef struct StdRdOptions
StdRdOptIndexCleanup vacuum_index_cleanup; /* controls index vacuuming */
bool vacuum_truncate; /* enables vacuum to truncate a relation */
bool vacuum_truncate_set; /* whether vacuum_truncate is set */
+ bool expression_checks; /* use expression to checks for changes */
/*
* Fraction of pages in a relation that vacuum can eagerly scan and fail
@@ -409,6 +411,14 @@ typedef struct StdRdOptions
((relation)->rd_options ? \
((StdRdOptions *) (relation)->rd_options)->parallel_workers : (defaultpw))
+/*
+ * RelationGetExpressionChecks
+ * Returns the relation's expression_checks reloption setting.
+ */
+#define RelationGetExpressionChecks(relation) \
+ ((relation)->rd_options ? \
+ ((StdRdOptions *) (relation)->rd_options)->expression_checks : true)
+
/* ViewOptions->check_option values */
typedef enum ViewOptCheckOption
{
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 3561c6bef0b..4b312bc8d06 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -71,6 +71,7 @@ typedef enum IndexAttrBitmapKind
INDEX_ATTR_BITMAP_IDENTITY_KEY,
INDEX_ATTR_BITMAP_HOT_BLOCKING,
INDEX_ATTR_BITMAP_SUMMARIZED,
+ INDEX_ATTR_BITMAP_EXPRESSION,
} IndexAttrBitmapKind;
extern Bitmapset *RelationGetIndexAttrBitmap(Relation relation,
diff --git a/src/test/regress/expected/heap_hot_updates.out b/src/test/regress/expected/heap_hot_updates.out
new file mode 100644
index 00000000000..7d22befc34a
--- /dev/null
+++ b/src/test/regress/expected/heap_hot_updates.out
@@ -0,0 +1,1048 @@
+-- Create a function to measure HOT updates
+CREATE OR REPLACE FUNCTION check_hot_updates(
+ expected INT,
+ p_table_name TEXT DEFAULT 't',
+ p_schema_name TEXT DEFAULT current_schema()
+)
+RETURNS TABLE (
+ table_name TEXT,
+ total_updates BIGINT,
+ hot_updates BIGINT,
+ hot_update_percentage NUMERIC,
+ matches_expected BOOLEAN,
+ has_indexes BOOLEAN,
+ index_count INT,
+ fillfactor INT
+)
+LANGUAGE plpgsql
+AS $$
+DECLARE
+ v_relid oid;
+ v_qualified_name TEXT;
+ v_hot_updates BIGINT;
+ v_updates BIGINT;
+ v_xact_hot_updates BIGINT;
+ v_xact_updates BIGINT;
+BEGIN
+
+ -- We need to wait for statistics to update
+ PERFORM pg_stat_force_next_flush();
+
+ -- Construct qualified name
+ v_qualified_name := quote_ident(p_schema_name) || '.' || quote_ident(p_table_name);
+
+ -- Get the OID using regclass
+ v_relid := v_qualified_name::regclass;
+
+ IF v_relid IS NULL THEN
+ RAISE EXCEPTION 'Table %.% not found', p_schema_name, p_table_name;
+ END IF;
+
+ -- Get cumulative stats
+ v_hot_updates := COALESCE(pg_stat_get_tuples_hot_updated(v_relid), 0);
+ v_updates := COALESCE(pg_stat_get_tuples_updated(v_relid), 0);
+
+ -- Get current transaction stats
+ v_xact_hot_updates := COALESCE(pg_stat_get_xact_tuples_hot_updated(v_relid), 0);
+ v_xact_updates := COALESCE(pg_stat_get_xact_tuples_updated(v_relid), 0);
+
+ -- Combine stats
+ v_hot_updates := v_hot_updates + v_xact_hot_updates;
+ v_updates := v_updates + v_xact_updates;
+
+ RETURN QUERY
+ SELECT
+ p_table_name::TEXT,
+ v_updates::BIGINT as total_updates,
+ v_hot_updates::BIGINT as hot_updates,
+ CASE
+ WHEN v_updates > 0 THEN
+ ROUND((v_hot_updates::numeric / v_updates::numeric * 100)::numeric, 2)
+ ELSE 0
+ END as hot_update_percentage,
+ (v_hot_updates = expected)::BOOLEAN as matches_expected,
+ (EXISTS (
+ SELECT 1 FROM pg_index WHERE indrelid = v_relid
+ ))::BOOLEAN as has_indexes,
+ (
+ SELECT COUNT(*)::INT
+ FROM pg_index
+ WHERE indrelid = v_relid
+ ) as index_count,
+ COALESCE(
+ (
+ SELECT (regexp_match(array_to_string(reloptions, ','), 'fillfactor=(\d+)'))[1]::int
+ FROM pg_class
+ WHERE oid = v_relid
+ ),
+ 100
+ ) as fillfactor;
+END;
+$$;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table will have two columns and two indexes, one on the primary key
+-- id and one on the expression (docs->>'name'). That means that the indexed
+-- attributes are 'id' and 'docs'.
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t((docs->>'name'));
+INSERT INTO t VALUES (1, '{"name": "john", "data": "some data"}');
+-- Disable expression checks.
+ALTER TABLE t SET (expression_checks = false);
+SELECT reloptions FROM pg_class WHERE relname = 't';
+ reloptions
+----------------------------------------------------------------
+ {autovacuum_enabled=off,fillfactor=70,expression_checks=false}
+(1 row)
+
+-- While the indexed attribute "name" is unchanged we've disabled expression
+-- checks so this update should not go HOT as the system can't determine if
+-- the indexed attribute has changed without evaluating the expression.
+update t set docs='{"name": "john", "data": "something else"}' where id=1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 1 | 0 | 0.00 | t | t | 2 | 70
+(1 row)
+
+-- Re-enable expression checks.
+ALTER TABLE t SET (expression_checks = true);
+SELECT reloptions FROM pg_class WHERE relname = 't';
+ reloptions
+---------------------------------------------------------------
+ {autovacuum_enabled=off,fillfactor=70,expression_checks=true}
+(1 row)
+
+-- The indexed attribute "name" with value "john" is unchanged, expect a HOT update.
+UPDATE t SET docs='{"name": "john", "data": "some other data"}' WHERE id=1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 2 | 1 | 50.00 | t | t | 2 | 70
+(1 row)
+
+-- The following update changes the indexed attribute "name", this should not be a HOT update.
+UPDATE t SET docs='{"name": "smith", "data": "some other data"}' WHERE id=1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 3 | 1 | 33.33 | t | t | 2 | 70
+(1 row)
+
+-- Now, this update does not change the indexed attribute "name" from "smith", this should be HOT.
+UPDATE t SET docs='{"name": "smith", "data": "some more data"}' WHERE id=1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 4 | 2 | 50.00 | t | t | 2 | 70
+(1 row)
+
+DROP TABLE t;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table is the same as the previous one but it has a third index. The
+-- index 'colindex' isn't an expression index, it indexes the entire value
+-- in the docs column. There are still only two indexed attributes for this
+-- relation, the same two as before. The presence of an index on the entire
+-- value of the docs column should prevent HOT updates for any updates to any
+-- portion of JSONB content in that column.
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t((docs->>'name'));
+CREATE INDEX t_docs_col_idx ON t(docs);
+INSERT INTO t VALUES (1, '{"name": "john", "data": "some data"}');
+-- This update doesn't change the value of the expression index, but it does
+-- change the content of the docs column and so should not be HOT because the
+-- indexed value changed as a result of the update.
+UPDATE t SET docs='{"name": "john", "data": "some other data"}' WHERE id=1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 1 | 0 | 0.00 | t | t | 3 | 70
+(1 row)
+
+DROP TABLE t;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- The table has one column docs and two indexes. They are both expression
+-- indexes referencing the same column attribute (docs) but one is a partial
+-- index.
+CREATE TABLE t (docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+INSERT INTO t (docs) VALUES ('{"a": 0, "b": 0}');
+INSERT INTO t (docs) SELECT jsonb_build_object('b', n) FROM generate_series(100, 10000) as n;
+CREATE INDEX t_idx_a ON t ((docs->>'a'));
+CREATE INDEX t_idx_b ON t ((docs->>'b')) WHERE (docs->>'b')::numeric > 9;
+-- We're using BTREE indexes and for this test we want to make sure that they remain
+-- in sync with changes to our relation. Force the choice of index scans below so
+-- that we know we're checking the index's understanding of what values should be
+-- in the index or not.
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+-- Leave 'a' unchanged but modify 'b' to a value outside of the index predicate.
+-- This should be a HOT update because neither index is changed.
+UPDATE t SET docs = jsonb_build_object('a', 0, 'b', 1) WHERE (docs->>'a')::numeric = 0;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 1 | 1 | 100.00 | t | t | 2 | 70
+(1 row)
+
+-- Let's check to make sure that the index does not contain a value for 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ QUERY PLAN
+--------------------------------------------------------------
+ Index Scan using t_idx_b on t
+ Filter: (((docs ->> 'b'::text))::numeric < '100'::numeric)
+(2 rows)
+
+SELECT * FROM t WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ docs
+------
+(0 rows)
+
+-- Leave 'a' unchanged but modify 'b' to a value within the index predicate.
+-- This represents a change for field 'b' from unindexed to indexed and so
+-- this should not take the HOT path.
+UPDATE t SET docs = jsonb_build_object('a', 0, 'b', 10) WHERE (docs->>'a')::numeric = 0;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 2 | 1 | 50.00 | t | t | 2 | 70
+(1 row)
+
+-- Let's check to make sure that the index contains the new value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ QUERY PLAN
+--------------------------------------------------------------
+ Index Scan using t_idx_b on t
+ Filter: (((docs ->> 'b'::text))::numeric < '100'::numeric)
+(2 rows)
+
+SELECT * FROM t WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ docs
+-------------------
+ {"a": 0, "b": 10}
+(1 row)
+
+-- This update modifies the value of 'a', an indexed field, so it also cannot
+-- be a HOT update.
+UPDATE t SET docs = jsonb_build_object('a', 1, 'b', 10) WHERE (docs->>'b')::numeric = 10;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 3 | 1 | 33.33 | t | t | 2 | 70
+(1 row)
+
+-- This update changes both 'a' and 'b' to new values that require index updates,
+-- this cannot use the HOT path.
+UPDATE t SET docs = jsonb_build_object('a', 2, 'b', 12) WHERE (docs->>'b')::numeric = 10;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 4 | 1 | 25.00 | t | t | 2 | 70
+(1 row)
+
+-- Let's check to make sure that the index contains the new value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ QUERY PLAN
+--------------------------------------------------------------
+ Index Scan using t_idx_b on t
+ Filter: (((docs ->> 'b'::text))::numeric < '100'::numeric)
+(2 rows)
+
+SELECT * FROM t WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ docs
+-------------------
+ {"a": 2, "b": 12}
+(1 row)
+
+-- This update changes 'b' to a value outside its predicate requiring that
+-- we remove it from the index. That's a transition that can't be done
+-- during a HOT update.
+UPDATE t SET docs = jsonb_build_object('a', 2, 'b', 1) WHERE (docs->>'b')::numeric = 12;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 5 | 1 | 20.00 | t | t | 2 | 70
+(1 row)
+
+-- Let's check to make sure that the index no longer contains the value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ QUERY PLAN
+--------------------------------------------------------------
+ Index Scan using t_idx_b on t
+ Filter: (((docs ->> 'b'::text))::numeric < '100'::numeric)
+(2 rows)
+
+SELECT * FROM t WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+ docs
+------
+(0 rows)
+
+DROP TABLE t;
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- Tests to ensure that HOT updates are not performed when multiple indexed
+-- attributes are updated.
+CREATE TABLE t(a INT, b INT) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx_a ON t(a);
+CREATE INDEX t_idx_b ON t(abs(b));
+INSERT INTO t VALUES (1, -1);
+-- Both are updated, the second is an expression index with an unchanged
+-- index value. The change to the index on a should prevent HOT updates.
+UPDATE t SET a = 2, b = 1 WHERE a = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 1 | 0 | 0.00 | t | t | 2 | 70
+(1 row)
+
+DROP TABLE t;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- Tests to check the expression_checks reloption behavior.
+--
+CREATE TABLE t(a INT, b INT) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx_a ON t(abs(a)) WHERE abs(a) > 10;
+CREATE INDEX t_idx_b ON t(abs(b));
+INSERT INTO t VALUES (-1, -1), (-2, -2), (-3, -3), (-4, -4);
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+-- Disable expression checks on indexes and partial index predicates.
+ALTER TABLE t SET (expression_checks = false);
+-- Before and after values of a are outside the predicate of the index and
+-- the indexed value of b hasn't changed however we've disabled expression
+-- checks so this should not be a HOT update.
+-- (-1, -1) -> (-5, -1)
+UPDATE t SET a = -5, b = -1 WHERE a = -1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 1 | 0 | 0.00 | t | t | 2 | 70
+(1 row)
+
+-- Enable expression checks on indexes, but not on predicates yet.
+ALTER TABLE t SET (expression_checks = true);
+-- The indexed value of b hasn't changed, this should be a HOT update.
+-- (-5, -1) -> (-5, 1)
+UPDATE t SET b = 1 WHERE a = -5;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 2 | 1 | 50.00 | t | t | 2 | 70
+(1 row)
+
+-- Now that we're not checking the predicate of the partial index, this
+-- update of a from -5 to 5 should be HOT because we should ignore the
+-- predicate and check the expression and find it unchanged.
+-- (-5, 1) -> (5, 1)
+UPDATE t SET a = 5 WHERE a = -5;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 3 | 2 | 66.67 | t | t | 2 | 70
+(1 row)
+
+-- This update meets the critera for the partial index and should not
+-- be HOT. Let's make sure of that and check the index as well.
+-- (-4, -4) -> (-11, -4)
+UPDATE t SET a = -11 WHERE a = -4;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 4 | 2 | 50.00 | t | t | 2 | 70
+(1 row)
+
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE abs(a) > 10;
+ QUERY PLAN
+-------------------------------
+ Index Scan using t_idx_a on t
+(1 row)
+
+SELECT * FROM t WHERE abs(a) > 10;
+ a | b
+-----+----
+ -11 | -4
+(1 row)
+
+-- (-11, -4) -> (11, -4)
+UPDATE t SET a = 11 WHERE a = -11;
+SELECT * FROM check_hot_updates(3);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 5 | 3 | 60.00 | t | t | 2 | 70
+(1 row)
+
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE abs(a) > 10;
+ QUERY PLAN
+-------------------------------
+ Index Scan using t_idx_a on t
+(1 row)
+
+SELECT * FROM t WHERE abs(a) > 10;
+ a | b
+----+----
+ 11 | -4
+(1 row)
+
+-- (11, -4) -> (-4, -4)
+UPDATE t SET a = -4 WHERE a = 11;
+SELECT * FROM check_hot_updates(3);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 6 | 3 | 50.00 | t | t | 2 | 70
+(1 row)
+
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE abs(a) > 10;
+ QUERY PLAN
+-------------------------------
+ Index Scan using t_idx_a on t
+(1 row)
+
+SELECT * FROM t WHERE abs(a) > 10;
+ a | b
+---+---
+(0 rows)
+
+-- This update of a from 5 to -1 is HOT despite that attribute
+-- being indexed because the before and after values for the
+-- partial index predicate are outside the index definition.
+-- (5, 1) -> (-1, 1)
+UPDATE t SET a = -1 WHERE a = 5;
+SELECT * FROM check_hot_updates(4);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 7 | 4 | 57.14 | t | t | 2 | 70
+(1 row)
+
+-- This update of a from -2 to -1 with predicate checks enabled should be
+-- HOT because the before/after values of a are both outside the predicate
+-- of the partial index.
+-- (-1, 1) -> (-2, 1)
+UPDATE t SET a = -2 WHERE a = -1;
+SELECT * FROM check_hot_updates(5);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 8 | 5 | 62.50 | t | t | 2 | 70
+(1 row)
+
+-- The indexed value for b isn't changing, this should be HOT.
+-- (-2, -2) -> (-2, 2)
+UPDATE t SET b = 2 WHERE b = -2;
+SELECT * FROM check_hot_updates(6);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 9 | 6 | 66.67 | t | t | 2 | 70
+(1 row)
+
+EXPLAIN (COSTS OFF) SELECT abs(b) FROM t;
+ QUERY PLAN
+------------------
+ Seq Scan on t
+ Disabled: true
+(2 rows)
+
+SELECT abs(b) FROM t;
+ abs
+-----
+ 3
+ 4
+ 1
+ 2
+(4 rows)
+
+-- Before and after values for a are outside the predicate of the index,
+-- and because we're checking this should be HOT.
+-- (-2, 1) -> (5, 1)
+-- (-2, -2) -> (5, -2)
+UPDATE t SET a = 5 WHERE a = -2;
+SELECT * FROM check_hot_updates(8);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 11 | 8 | 72.73 | t | t | 2 | 70
+(1 row)
+
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE abs(a) > 10;
+ QUERY PLAN
+-------------------------------
+ Index Scan using t_idx_a on t
+(1 row)
+
+SELECT * FROM t WHERE abs(a) > 10;
+ a | b
+---+---
+(0 rows)
+
+SELECT * FROM t;
+ a | b
+----+----
+ -3 | -3
+ -4 | -4
+ 5 | 1
+ 5 | 2
+(4 rows)
+
+DROP TABLE t;
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- The tests here examines the behavior of HOT updates when the relation
+-- has a JSONB column with an index on the field 'a' and the partial index
+-- expression on a different JSONB field 'b'.
+CREATE TABLE t(docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t((docs->'a')) WHERE (docs->'b')::integer = 1;
+INSERT INTO t VALUES ('{"a": 1, "b": 1}');
+EXPLAIN (COSTS OFF) SELECT * FROM t;
+ QUERY PLAN
+---------------
+ Seq Scan on t
+(1 row)
+
+SELECT * FROM t;
+ docs
+------------------
+ {"a": 1, "b": 1}
+(1 row)
+
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->'b')::integer = 1;
+ QUERY PLAN
+----------------------------------
+ Index Scan using t_docs_idx on t
+(1 row)
+
+SELECT * FROM t WHERE (docs->'b')::integer = 1;
+ docs
+------------------
+ {"a": 1, "b": 1}
+(1 row)
+
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 0 | 0 | 0 | t | t | 1 | 70
+(1 row)
+
+UPDATE t SET docs='{"a": 1, "b": 0}';
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 1 | 0 | 0.00 | t | t | 1 | 70
+(1 row)
+
+SELECT * FROM t WHERE (docs->'b')::integer = 1;
+ docs
+------
+(0 rows)
+
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+DROP TABLE t;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- Tests for multi-column indexes
+--
+CREATE TABLE t(id INT, docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t(id, (docs->'a'));
+INSERT INTO t VALUES (1, '{"a": 1, "b": 1}');
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer > 0;
+ QUERY PLAN
+------------------------------------------------
+ Index Scan using t_docs_idx on t
+ Index Cond: (id > 0)
+ Filter: (((docs -> 'a'::text))::integer > 0)
+(3 rows)
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer > 0;
+ id | docs
+----+------------------
+ 1 | {"a": 1, "b": 1}
+(1 row)
+
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 0 | 0 | 0 | t | t | 1 | 70
+(1 row)
+
+-- Changing the id attribute which is an indexed attribute should
+-- prevent HOT updates.
+UPDATE t SET id = 2;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 1 | 0 | 0.00 | t | t | 1 | 70
+(1 row)
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer > 0;
+ id | docs
+----+------------------
+ 2 | {"a": 1, "b": 1}
+(1 row)
+
+-- Changing the docs->'a' field in the indexed attribute 'docs'
+-- should prevent HOT updates.
+UPDATE t SET docs='{"a": -2, "b": 1}';
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 2 | 0 | 0.00 | t | t | 1 | 70
+(1 row)
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer < 0;
+ id | docs
+----+-------------------
+ 2 | {"a": -2, "b": 1}
+(1 row)
+
+-- Leaving the docs->'a' attribute unchanged means that the expression
+-- is unchanged and because the 'id' attribute isn't in the modified
+-- set the indexed tuple is unchanged, this can go HOT.
+UPDATE t SET docs='{"a": -2, "b": 2}';
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 3 | 1 | 33.33 | t | t | 1 | 70
+(1 row)
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer < 0;
+ id | docs
+----+-------------------
+ 2 | {"a": -2, "b": 2}
+(1 row)
+
+-- Here we change the 'id' attribute and the 'docs' attribute setting
+-- the expression docs->'a' to a new value, this cannot be a HOT update.
+UPDATE t SET id = 3, docs='{"a": 3, "b": 3}';
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ t | 4 | 1 | 25.00 | t | t | 1 | 70
+(1 row)
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer > 0;
+ id | docs
+----+------------------
+ 3 | {"a": 3, "b": 3}
+(1 row)
+
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+DROP TABLE t;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table has a single column 'email' and a unique constraint on it that
+-- should preclude HOT updates.
+CREATE TABLE users (
+ user_id serial primary key,
+ name VARCHAR(255) NOT NULL,
+ email VARCHAR(255) NOT NULL,
+ EXCLUDE USING btree (lower(email) WITH =)
+);
+-- Add some data to the table and then update it in ways that should and should
+-- not be HOT updates.
+INSERT INTO users (name, email) VALUES
+('user1', '[email protected]'),
+('user2', '[email protected]'),
+('taken', '[email protected]'),
+('you', '[email protected]'),
+('taken', '[email protected]');
+-- Should fail because of the unique constraint on the email column.
+UPDATE users SET email = '[email protected]' WHERE email = '[email protected]';
+ERROR: conflicting key value violates exclusion constraint "users_lower_excl"
+DETAIL: Key (lower(email::text))=([email protected]) conflicts with existing key (lower(email::text))=([email protected]).
+SELECT * FROM check_hot_updates(0, 'users');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ users | 1 | 0 | 0.00 | t | t | 2 | 100
+(1 row)
+
+-- Should succeed because the email column is not being updated and should go HOT.
+UPDATE users SET name = 'foo' WHERE email = '[email protected]';
+SELECT * FROM check_hot_updates(1, 'users');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ users | 2 | 1 | 50.00 | t | t | 2 | 100
+(1 row)
+
+-- Create a partial index on the email column, updates
+CREATE INDEX idx_users_email_no_example ON users (lower(email)) WHERE lower(email) LIKE '%@example.com%';
+-- An update that changes the email column but not the indexed portion of it and falls outside the constraint.
+-- Shouldn't be a HOT update because of the exclusion constraint.
+UPDATE users SET email = '[email protected]' WHERE name = 'you';
+SELECT * FROM check_hot_updates(1, 'users');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ users | 3 | 1 | 33.33 | t | t | 3 | 100
+(1 row)
+
+-- An update that changes the email column but not the indexed portion of it and falls within the constraint.
+-- Again, should fail constraint and fail to be a HOT update.
+UPDATE users SET email = '[email protected]' WHERE name = 'you';
+ERROR: conflicting key value violates exclusion constraint "users_lower_excl"
+DETAIL: Key (lower(email::text))=([email protected]) conflicts with existing key (lower(email::text))=([email protected]).
+SELECT * FROM check_hot_updates(1, 'users');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ users | 4 | 1 | 25.00 | t | t | 3 | 100
+(1 row)
+
+DROP TABLE users;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- Another test of constraints spoiling HOT updates, this time with a range.
+CREATE TABLE events (
+ id serial primary key,
+ name VARCHAR(255) NOT NULL,
+ event_time tstzrange,
+ constraint no_screening_time_overlap exclude using gist (
+ event_time WITH &&
+ )
+);
+-- Add two non-overlapping events.
+INSERT INTO events (id, event_time, name)
+VALUES
+ (1, '["2023-01-01 19:00:00", "2023-01-01 20:45:00"]', 'event1'),
+ (2, '["2023-01-01 21:00:00", "2023-01-01 21:45:00"]', 'event2');
+-- Update the first event to overlap with the second, should fail the constraint and not be HOT.
+UPDATE events SET event_time = '["2023-01-01 20:00:00", "2023-01-01 21:45:00"]' WHERE id = 1;
+ERROR: conflicting key value violates exclusion constraint "no_screening_time_overlap"
+DETAIL: Key (event_time)=(["Sun Jan 01 20:00:00 2023 PST","Sun Jan 01 21:45:00 2023 PST"]) conflicts with existing key (event_time)=(["Sun Jan 01 21:00:00 2023 PST","Sun Jan 01 21:45:00 2023 PST"]).
+SELECT * FROM check_hot_updates(0, 'events');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ events | 1 | 0 | 0.00 | t | t | 2 | 100
+(1 row)
+
+-- Update the first event to not overlap with the second, again not HOT due to the constraint.
+UPDATE events SET event_time = '["2023-01-01 22:00:00", "2023-01-01 22:45:00"]' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 'events');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ events | 2 | 0 | 0.00 | t | t | 2 | 100
+(1 row)
+
+-- Update the first event to not overlap with the second, this time we're HOT because we don't overlap with the constraint.
+UPDATE events SET name = 'new name here' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 'events');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ events | 3 | 1 | 33.33 | t | t | 2 | 100
+(1 row)
+
+DROP TABLE events;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- A test to ensure that only modified summarizing indexes are updated, not
+-- all of them.
+CREATE TABLE ex (id SERIAL primary key, att1 JSONB, att2 text, att3 text, att4 text) WITH (fillfactor = 60);
+CREATE INDEX ex_expr1_idx ON ex USING btree((att1->'data'));
+CREATE INDEX ex_sumr1_idx ON ex USING BRIN(att2);
+CREATE INDEX ex_expr2_idx ON ex USING btree((att1->'a'));
+CREATE INDEX ex_expr3_idx ON ex USING btree((att1->'b'));
+CREATE INDEX ex_expr4_idx ON ex USING btree((att1->'c'));
+CREATE INDEX ex_sumr2_idx ON ex USING BRIN(att3);
+CREATE INDEX ex_sumr3_idx ON ex USING BRIN(att4);
+CREATE INDEX ex_expr5_idx ON ex USING btree((att1->'d'));
+INSERT INTO ex (att1, att2) VALUES ('{"data": []}'::json, 'nothing special');
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+--------------+-----------------+------+------
+ 1 | {"data": []} | nothing special | |
+(1 row)
+
+-- Update att2 and att4 both are BRIN/summarizing indexes, this should be a HOT update and
+-- only update two of the three summarizing indexes.
+UPDATE ex SET att2 = 'special indeed', att4 = 'whatever';
+SELECT * FROM check_hot_updates(1, 'ex');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ ex | 1 | 1 | 100.00 | t | t | 9 | 60
+(1 row)
+
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+--------------+----------------+------+----------
+ 1 | {"data": []} | special indeed | | whatever
+(1 row)
+
+-- Update att1 and att2, only one is BRIN/summarizing, this should NOT be a HOT update.
+UPDATE ex SET att1 = att1 || '{"data": "howdy"}', att2 = 'special, so special';
+SELECT * FROM check_hot_updates(1, 'ex');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ ex | 2 | 1 | 50.00 | t | t | 9 | 60
+(1 row)
+
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+-------------------+---------------------+------+----------
+ 1 | {"data": "howdy"} | special, so special | | whatever
+(1 row)
+
+-- Update att2, att3, and att4 all are BRIN/summarizing indexes, this should be a HOT update
+-- and yet still update all three summarizing indexes.
+UPDATE ex SET att2 = 'a', att3 = 'b', att4 = 'c';
+SELECT * FROM check_hot_updates(2, 'ex');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ ex | 3 | 2 | 66.67 | t | t | 9 | 60
+(1 row)
+
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+-------------------+------+------+------
+ 1 | {"data": "howdy"} | a | b | c
+(1 row)
+
+-- Update att1, att2, and att3 all modified values are BRIN/summarizing indexes, this should be a HOT update
+-- and yet still update all three summarizing indexes.
+UPDATE ex SET att1 = '{"data": "howdy"}', att2 = 'd', att3 = 'e';
+SELECT * FROM check_hot_updates(3, 'ex');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ ex | 4 | 3 | 75.00 | t | t | 9 | 60
+(1 row)
+
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+-------------------+------+------+------
+ 1 | {"data": "howdy"} | d | e | c
+(1 row)
+
+DROP TABLE ex;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- A test to ensure that summarizing indexes are not updated when they don't
+-- change, but are updated when they do while not prefent HOT updates.
+CREATE TABLE ex (att1 JSONB, att2 text) WITH (fillfactor = 60);
+CREATE INDEX ex_expr1_idx ON ex USING btree((att1->'data'));
+CREATE INDEX ex_sumr1_idx ON ex USING BRIN(att2);
+INSERT INTO ex VALUES ('{"data": []}', 'nothing special');
+-- Update the unindexed value of att1, this should be a HOT update and and should
+-- update the summarizing index.
+UPDATE ex SET att1 = att1 || '{"status": "stalemate"}';
+SELECT * FROM check_hot_updates(1, 'ex');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ ex | 1 | 1 | 100.00 | t | t | 2 | 60
+(1 row)
+
+-- Update the indexed value of att2, a summarized value, this is a summarized
+-- only update and should use the HOT path while still triggering an update to
+-- the summarizing BRIN index.
+UPDATE ex SET att2 = 'special indeed';
+SELECT * FROM check_hot_updates(2, 'ex');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ ex | 2 | 2 | 100.00 | t | t | 2 | 60
+(1 row)
+
+-- Update to att1 doesn't change the indexed value while the update to att2 does,
+-- this again is a summarized only update and should use the HOT path as well as
+-- trigger an update to the BRIN index.
+UPDATE ex SET att1 = att1 || '{"status": "checkmate"}', att2 = 'special, so special';
+SELECT * FROM check_hot_updates(3, 'ex');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ ex | 3 | 3 | 100.00 | t | t | 2 | 60
+(1 row)
+
+-- This updates both indexes, the expression index on att1 and the summarizing
+-- index on att2. This should not be a HOT update because there are modified
+-- indexes and only some are summarized, not all. This should force all
+-- indexes to be updated.
+UPDATE ex SET att1 = att1 || '{"data": [1,2,3]}', att2 = 'do you want to play a game?';
+SELECT * FROM check_hot_updates(4, 'ex');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ ex | 4 | 3 | 75.00 | f | t | 2 | 60
+(1 row)
+
+DROP TABLE ex;
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This test is for a table with a custom type and a custom operators on
+-- the BTREE index. The question is, when comparing values for equality
+-- to determine if there are changes on the index or not... shouldn't we
+-- be using the custom operators?
+-- Create a type
+CREATE TYPE my_custom_type AS (val int);
+-- Comparison functions (returns boolean)
+CREATE FUNCTION my_custom_lt(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val < b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_le(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val <= b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_eq(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val = b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_ge(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val >= b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_gt(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val > b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_ne(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val != b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+-- Comparison function (returns -1, 0, 1)
+CREATE FUNCTION my_custom_cmp(a my_custom_type, b my_custom_type) RETURNS int AS $$
+BEGIN
+ IF a.val < b.val THEN
+ RETURN -1;
+ ELSIF a.val > b.val THEN
+ RETURN 1;
+ ELSE
+ RETURN 0;
+ END IF;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+-- Create the operators
+CREATE OPERATOR < (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_lt,
+ COMMUTATOR = >,
+ NEGATOR = >=
+);
+CREATE OPERATOR <= (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_le,
+ COMMUTATOR = >=,
+ NEGATOR = >
+);
+CREATE OPERATOR = (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_eq,
+ COMMUTATOR = =,
+ NEGATOR = <>
+);
+CREATE OPERATOR >= (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_ge,
+ COMMUTATOR = <=,
+ NEGATOR = <
+);
+CREATE OPERATOR > (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_gt,
+ COMMUTATOR = <,
+ NEGATOR = <=
+);
+CREATE OPERATOR <> (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_ne,
+ COMMUTATOR = <>,
+ NEGATOR = =
+);
+-- Create the operator class (including the support function)
+CREATE OPERATOR CLASS my_custom_ops
+ DEFAULT FOR TYPE my_custom_type USING btree AS
+ OPERATOR 1 <,
+ OPERATOR 2 <=,
+ OPERATOR 3 =,
+ OPERATOR 4 >=,
+ OPERATOR 5 >,
+ FUNCTION 1 my_custom_cmp(my_custom_type, my_custom_type);
+-- Create the table
+CREATE TABLE my_table (
+ id int,
+ custom_val my_custom_type
+);
+-- Insert some data
+INSERT INTO my_table (id, custom_val) VALUES
+(1, ROW(3)::my_custom_type),
+(2, ROW(1)::my_custom_type),
+(3, ROW(4)::my_custom_type),
+(4, ROW(2)::my_custom_type);
+-- Create a function to use when indexing
+CREATE OR REPLACE FUNCTION abs_val(val my_custom_type) RETURNS int AS $$
+BEGIN
+ RETURN abs(val.val);
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+-- Create the index
+CREATE INDEX idx_custom_val_abs ON my_table (abs_val(custom_val));
+-- Update 1
+UPDATE my_table SET custom_val = ROW(5)::my_custom_type WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 'my_table');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ my_table | 1 | 0 | 0.00 | t | t | 1 | 100
+(1 row)
+
+-- Update 2
+UPDATE my_table SET custom_val = ROW(0)::my_custom_type WHERE custom_val < ROW(3)::my_custom_type;
+SELECT * FROM check_hot_updates(0, 'my_table');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ my_table | 3 | 0 | 0.00 | t | t | 1 | 100
+(1 row)
+
+-- Update 3
+UPDATE my_table SET custom_val = ROW(6)::my_custom_type WHERE id = 3;
+SELECT * FROM check_hot_updates(0, 'my_table');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ my_table | 4 | 0 | 0.00 | t | t | 1 | 100
+(1 row)
+
+-- Update 4
+UPDATE my_table SET id = 5 WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+ pg_stat_get_xact_tuples_hot_updated
+-------------------------------------
+ 1
+(1 row)
+
+SELECT * FROM check_hot_updates(0, 'my_table');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected | has_indexes | index_count | fillfactor
+------------+---------------+-------------+-----------------------+------------------+-------------+-------------+------------
+ my_table | 5 | 1 | 20.00 | f | t | 1 | 100
+(1 row)
+
+-- Query using the index
+EXPLAIN (COSTS OFF) SELECT * FROM my_table WHERE abs_val(custom_val) = 6;
+ QUERY PLAN
+-------------------------------------
+ Seq Scan on my_table
+ Filter: (abs_val(custom_val) = 6)
+(2 rows)
+
+SELECT * FROM my_table WHERE abs_val(custom_val) = 6;
+ id | custom_val
+----+------------
+ 3 | (6)
+(1 row)
+
+-- Clean up
+DROP TABLE my_table CASCADE;
+DROP OPERATOR CLASS my_custom_ops USING btree CASCADE;
+DROP OPERATOR < (my_custom_type, my_custom_type);
+DROP OPERATOR <= (my_custom_type, my_custom_type);
+DROP OPERATOR = (my_custom_type, my_custom_type);
+DROP OPERATOR >= (my_custom_type, my_custom_type);
+DROP OPERATOR > (my_custom_type, my_custom_type);
+DROP OPERATOR <> (my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_lt(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_le(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_eq(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_ge(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_gt(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_ne(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_cmp(my_custom_type, my_custom_type);
+DROP FUNCTION abs_val(my_custom_type);
+DROP TYPE my_custom_type CASCADE;
+DROP FUNCTION check_hot_updates(int, text, text);
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index a424be2a6bf..f712acd9a20 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -68,6 +68,11 @@ test: select_into select_distinct select_distinct_on select_implicit select_havi
# ----------
test: brin gin gist spgist privileges init_privs security_label collate matview lock replica_identity rowsecurity object_address tablesample groupingsets drop_operator password identity generated_stored join_hash
+# ----------
+# Another group of parallel tests
+# ----------
+test: heap_hot_updates
+
# ----------
# Additional BRIN tests
# ----------
diff --git a/src/test/regress/sql/heap_hot_updates.sql b/src/test/regress/sql/heap_hot_updates.sql
new file mode 100644
index 00000000000..7016e9eabd0
--- /dev/null
+++ b/src/test/regress/sql/heap_hot_updates.sql
@@ -0,0 +1,718 @@
+-- Create a function to measure HOT updates
+CREATE OR REPLACE FUNCTION check_hot_updates(
+ expected INT,
+ p_table_name TEXT DEFAULT 't',
+ p_schema_name TEXT DEFAULT current_schema()
+)
+RETURNS TABLE (
+ table_name TEXT,
+ total_updates BIGINT,
+ hot_updates BIGINT,
+ hot_update_percentage NUMERIC,
+ matches_expected BOOLEAN,
+ has_indexes BOOLEAN,
+ index_count INT,
+ fillfactor INT
+)
+LANGUAGE plpgsql
+AS $$
+DECLARE
+ v_relid oid;
+ v_qualified_name TEXT;
+ v_hot_updates BIGINT;
+ v_updates BIGINT;
+ v_xact_hot_updates BIGINT;
+ v_xact_updates BIGINT;
+BEGIN
+
+ -- We need to wait for statistics to update
+ PERFORM pg_stat_force_next_flush();
+
+ -- Construct qualified name
+ v_qualified_name := quote_ident(p_schema_name) || '.' || quote_ident(p_table_name);
+
+ -- Get the OID using regclass
+ v_relid := v_qualified_name::regclass;
+
+ IF v_relid IS NULL THEN
+ RAISE EXCEPTION 'Table %.% not found', p_schema_name, p_table_name;
+ END IF;
+
+ -- Get cumulative stats
+ v_hot_updates := COALESCE(pg_stat_get_tuples_hot_updated(v_relid), 0);
+ v_updates := COALESCE(pg_stat_get_tuples_updated(v_relid), 0);
+
+ -- Get current transaction stats
+ v_xact_hot_updates := COALESCE(pg_stat_get_xact_tuples_hot_updated(v_relid), 0);
+ v_xact_updates := COALESCE(pg_stat_get_xact_tuples_updated(v_relid), 0);
+
+ -- Combine stats
+ v_hot_updates := v_hot_updates + v_xact_hot_updates;
+ v_updates := v_updates + v_xact_updates;
+
+ RETURN QUERY
+ SELECT
+ p_table_name::TEXT,
+ v_updates::BIGINT as total_updates,
+ v_hot_updates::BIGINT as hot_updates,
+ CASE
+ WHEN v_updates > 0 THEN
+ ROUND((v_hot_updates::numeric / v_updates::numeric * 100)::numeric, 2)
+ ELSE 0
+ END as hot_update_percentage,
+ (v_hot_updates = expected)::BOOLEAN as matches_expected,
+ (EXISTS (
+ SELECT 1 FROM pg_index WHERE indrelid = v_relid
+ ))::BOOLEAN as has_indexes,
+ (
+ SELECT COUNT(*)::INT
+ FROM pg_index
+ WHERE indrelid = v_relid
+ ) as index_count,
+ COALESCE(
+ (
+ SELECT (regexp_match(array_to_string(reloptions, ','), 'fillfactor=(\d+)'))[1]::int
+ FROM pg_class
+ WHERE oid = v_relid
+ ),
+ 100
+ ) as fillfactor;
+END;
+$$;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table will have two columns and two indexes, one on the primary key
+-- id and one on the expression (docs->>'name'). That means that the indexed
+-- attributes are 'id' and 'docs'.
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t((docs->>'name'));
+INSERT INTO t VALUES (1, '{"name": "john", "data": "some data"}');
+
+-- Disable expression checks.
+ALTER TABLE t SET (expression_checks = false);
+SELECT reloptions FROM pg_class WHERE relname = 't';
+
+-- While the indexed attribute "name" is unchanged we've disabled expression
+-- checks so this update should not go HOT as the system can't determine if
+-- the indexed attribute has changed without evaluating the expression.
+update t set docs='{"name": "john", "data": "something else"}' where id=1;
+SELECT * FROM check_hot_updates(0);
+
+-- Re-enable expression checks.
+ALTER TABLE t SET (expression_checks = true);
+SELECT reloptions FROM pg_class WHERE relname = 't';
+
+-- The indexed attribute "name" with value "john" is unchanged, expect a HOT update.
+UPDATE t SET docs='{"name": "john", "data": "some other data"}' WHERE id=1;
+SELECT * FROM check_hot_updates(1);
+
+-- The following update changes the indexed attribute "name", this should not be a HOT update.
+UPDATE t SET docs='{"name": "smith", "data": "some other data"}' WHERE id=1;
+SELECT * FROM check_hot_updates(1);
+
+-- Now, this update does not change the indexed attribute "name" from "smith", this should be HOT.
+UPDATE t SET docs='{"name": "smith", "data": "some more data"}' WHERE id=1;
+SELECT * FROM check_hot_updates(2);
+
+DROP TABLE t;
+
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table is the same as the previous one but it has a third index. The
+-- index 'colindex' isn't an expression index, it indexes the entire value
+-- in the docs column. There are still only two indexed attributes for this
+-- relation, the same two as before. The presence of an index on the entire
+-- value of the docs column should prevent HOT updates for any updates to any
+-- portion of JSONB content in that column.
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t((docs->>'name'));
+CREATE INDEX t_docs_col_idx ON t(docs);
+INSERT INTO t VALUES (1, '{"name": "john", "data": "some data"}');
+
+-- This update doesn't change the value of the expression index, but it does
+-- change the content of the docs column and so should not be HOT because the
+-- indexed value changed as a result of the update.
+UPDATE t SET docs='{"name": "john", "data": "some other data"}' WHERE id=1;
+SELECT * FROM check_hot_updates(0);
+DROP TABLE t;
+
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- The table has one column docs and two indexes. They are both expression
+-- indexes referencing the same column attribute (docs) but one is a partial
+-- index.
+CREATE TABLE t (docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+INSERT INTO t (docs) VALUES ('{"a": 0, "b": 0}');
+INSERT INTO t (docs) SELECT jsonb_build_object('b', n) FROM generate_series(100, 10000) as n;
+CREATE INDEX t_idx_a ON t ((docs->>'a'));
+CREATE INDEX t_idx_b ON t ((docs->>'b')) WHERE (docs->>'b')::numeric > 9;
+
+-- We're using BTREE indexes and for this test we want to make sure that they remain
+-- in sync with changes to our relation. Force the choice of index scans below so
+-- that we know we're checking the index's understanding of what values should be
+-- in the index or not.
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+
+-- Leave 'a' unchanged but modify 'b' to a value outside of the index predicate.
+-- This should be a HOT update because neither index is changed.
+UPDATE t SET docs = jsonb_build_object('a', 0, 'b', 1) WHERE (docs->>'a')::numeric = 0;
+SELECT * FROM check_hot_updates(1);
+-- Let's check to make sure that the index does not contain a value for 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+SELECT * FROM t WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+
+-- Leave 'a' unchanged but modify 'b' to a value within the index predicate.
+-- This represents a change for field 'b' from unindexed to indexed and so
+-- this should not take the HOT path.
+UPDATE t SET docs = jsonb_build_object('a', 0, 'b', 10) WHERE (docs->>'a')::numeric = 0;
+SELECT * FROM check_hot_updates(1);
+-- Let's check to make sure that the index contains the new value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+SELECT * FROM t WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+
+-- This update modifies the value of 'a', an indexed field, so it also cannot
+-- be a HOT update.
+UPDATE t SET docs = jsonb_build_object('a', 1, 'b', 10) WHERE (docs->>'b')::numeric = 10;
+SELECT * FROM check_hot_updates(1);
+
+-- This update changes both 'a' and 'b' to new values that require index updates,
+-- this cannot use the HOT path.
+UPDATE t SET docs = jsonb_build_object('a', 2, 'b', 12) WHERE (docs->>'b')::numeric = 10;
+SELECT * FROM check_hot_updates(1);
+-- Let's check to make sure that the index contains the new value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+SELECT * FROM t WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+
+-- This update changes 'b' to a value outside its predicate requiring that
+-- we remove it from the index. That's a transition that can't be done
+-- during a HOT update.
+UPDATE t SET docs = jsonb_build_object('a', 2, 'b', 1) WHERE (docs->>'b')::numeric = 12;
+SELECT * FROM check_hot_updates(1);
+-- Let's check to make sure that the index no longer contains the value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+SELECT * FROM t WHERE (docs->>'b')::numeric > 9 AND (docs->>'b')::numeric < 100;
+
+DROP TABLE t;
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- Tests to ensure that HOT updates are not performed when multiple indexed
+-- attributes are updated.
+CREATE TABLE t(a INT, b INT) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx_a ON t(a);
+CREATE INDEX t_idx_b ON t(abs(b));
+INSERT INTO t VALUES (1, -1);
+
+-- Both are updated, the second is an expression index with an unchanged
+-- index value. The change to the index on a should prevent HOT updates.
+UPDATE t SET a = 2, b = 1 WHERE a = 1;
+SELECT * FROM check_hot_updates(0);
+
+DROP TABLE t;
+
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- Tests to check the expression_checks reloption behavior.
+--
+CREATE TABLE t(a INT, b INT) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx_a ON t(abs(a)) WHERE abs(a) > 10;
+CREATE INDEX t_idx_b ON t(abs(b));
+INSERT INTO t VALUES (-1, -1), (-2, -2), (-3, -3), (-4, -4);
+
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+
+-- Disable expression checks on indexes and partial index predicates.
+ALTER TABLE t SET (expression_checks = false);
+
+-- Before and after values of a are outside the predicate of the index and
+-- the indexed value of b hasn't changed however we've disabled expression
+-- checks so this should not be a HOT update.
+-- (-1, -1) -> (-5, -1)
+UPDATE t SET a = -5, b = -1 WHERE a = -1;
+SELECT * FROM check_hot_updates(0);
+
+-- Enable expression checks on indexes, but not on predicates yet.
+ALTER TABLE t SET (expression_checks = true);
+
+-- The indexed value of b hasn't changed, this should be a HOT update.
+-- (-5, -1) -> (-5, 1)
+UPDATE t SET b = 1 WHERE a = -5;
+SELECT * FROM check_hot_updates(1);
+
+-- Now that we're not checking the predicate of the partial index, this
+-- update of a from -5 to 5 should be HOT because we should ignore the
+-- predicate and check the expression and find it unchanged.
+-- (-5, 1) -> (5, 1)
+UPDATE t SET a = 5 WHERE a = -5;
+SELECT * FROM check_hot_updates(2);
+
+-- This update meets the critera for the partial index and should not
+-- be HOT. Let's make sure of that and check the index as well.
+-- (-4, -4) -> (-11, -4)
+UPDATE t SET a = -11 WHERE a = -4;
+SELECT * FROM check_hot_updates(2);
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE abs(a) > 10;
+SELECT * FROM t WHERE abs(a) > 10;
+
+-- (-11, -4) -> (11, -4)
+UPDATE t SET a = 11 WHERE a = -11;
+SELECT * FROM check_hot_updates(3);
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE abs(a) > 10;
+SELECT * FROM t WHERE abs(a) > 10;
+
+-- (11, -4) -> (-4, -4)
+UPDATE t SET a = -4 WHERE a = 11;
+SELECT * FROM check_hot_updates(3);
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE abs(a) > 10;
+SELECT * FROM t WHERE abs(a) > 10;
+
+-- This update of a from 5 to -1 is HOT despite that attribute
+-- being indexed because the before and after values for the
+-- partial index predicate are outside the index definition.
+-- (5, 1) -> (-1, 1)
+UPDATE t SET a = -1 WHERE a = 5;
+SELECT * FROM check_hot_updates(4);
+
+-- This update of a from -2 to -1 with predicate checks enabled should be
+-- HOT because the before/after values of a are both outside the predicate
+-- of the partial index.
+-- (-1, 1) -> (-2, 1)
+UPDATE t SET a = -2 WHERE a = -1;
+SELECT * FROM check_hot_updates(5);
+
+-- The indexed value for b isn't changing, this should be HOT.
+-- (-2, -2) -> (-2, 2)
+UPDATE t SET b = 2 WHERE b = -2;
+SELECT * FROM check_hot_updates(6);
+EXPLAIN (COSTS OFF) SELECT abs(b) FROM t;
+SELECT abs(b) FROM t;
+
+-- Before and after values for a are outside the predicate of the index,
+-- and because we're checking this should be HOT.
+-- (-2, 1) -> (5, 1)
+-- (-2, -2) -> (5, -2)
+UPDATE t SET a = 5 WHERE a = -2;
+SELECT * FROM check_hot_updates(8);
+
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE abs(a) > 10;
+SELECT * FROM t WHERE abs(a) > 10;
+
+SELECT * FROM t;
+
+DROP TABLE t;
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- The tests here examines the behavior of HOT updates when the relation
+-- has a JSONB column with an index on the field 'a' and the partial index
+-- expression on a different JSONB field 'b'.
+CREATE TABLE t(docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t((docs->'a')) WHERE (docs->'b')::integer = 1;
+INSERT INTO t VALUES ('{"a": 1, "b": 1}');
+
+EXPLAIN (COSTS OFF) SELECT * FROM t;
+SELECT * FROM t;
+
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->'b')::integer = 1;
+SELECT * FROM t WHERE (docs->'b')::integer = 1;
+
+SELECT * FROM check_hot_updates(0);
+
+UPDATE t SET docs='{"a": 1, "b": 0}';
+SELECT * FROM check_hot_updates(0);
+
+SELECT * FROM t WHERE (docs->'b')::integer = 1;
+
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+
+DROP TABLE t;
+
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- Tests for multi-column indexes
+--
+CREATE TABLE t(id INT, docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t(id, (docs->'a'));
+INSERT INTO t VALUES (1, '{"a": 1, "b": 1}');
+
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer > 0;
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer > 0;
+
+SELECT * FROM check_hot_updates(0);
+
+-- Changing the id attribute which is an indexed attribute should
+-- prevent HOT updates.
+UPDATE t SET id = 2;
+SELECT * FROM check_hot_updates(0);
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer > 0;
+
+-- Changing the docs->'a' field in the indexed attribute 'docs'
+-- should prevent HOT updates.
+UPDATE t SET docs='{"a": -2, "b": 1}';
+SELECT * FROM check_hot_updates(0);
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer < 0;
+
+-- Leaving the docs->'a' attribute unchanged means that the expression
+-- is unchanged and because the 'id' attribute isn't in the modified
+-- set the indexed tuple is unchanged, this can go HOT.
+UPDATE t SET docs='{"a": -2, "b": 2}';
+SELECT * FROM check_hot_updates(1);
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer < 0;
+
+-- Here we change the 'id' attribute and the 'docs' attribute setting
+-- the expression docs->'a' to a new value, this cannot be a HOT update.
+UPDATE t SET id = 3, docs='{"a": 3, "b": 3}';
+SELECT * FROM check_hot_updates(1);
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer > 0;
+
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+
+DROP TABLE t;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This table has a single column 'email' and a unique constraint on it that
+-- should preclude HOT updates.
+CREATE TABLE users (
+ user_id serial primary key,
+ name VARCHAR(255) NOT NULL,
+ email VARCHAR(255) NOT NULL,
+ EXCLUDE USING btree (lower(email) WITH =)
+);
+
+-- Add some data to the table and then update it in ways that should and should
+-- not be HOT updates.
+INSERT INTO users (name, email) VALUES
+('user1', '[email protected]'),
+('user2', '[email protected]'),
+('taken', '[email protected]'),
+('you', '[email protected]'),
+('taken', '[email protected]');
+
+-- Should fail because of the unique constraint on the email column.
+UPDATE users SET email = '[email protected]' WHERE email = '[email protected]';
+SELECT * FROM check_hot_updates(0, 'users');
+
+-- Should succeed because the email column is not being updated and should go HOT.
+UPDATE users SET name = 'foo' WHERE email = '[email protected]';
+SELECT * FROM check_hot_updates(1, 'users');
+
+-- Create a partial index on the email column, updates
+CREATE INDEX idx_users_email_no_example ON users (lower(email)) WHERE lower(email) LIKE '%@example.com%';
+
+-- An update that changes the email column but not the indexed portion of it and falls outside the constraint.
+-- Shouldn't be a HOT update because of the exclusion constraint.
+UPDATE users SET email = '[email protected]' WHERE name = 'you';
+SELECT * FROM check_hot_updates(1, 'users');
+
+-- An update that changes the email column but not the indexed portion of it and falls within the constraint.
+-- Again, should fail constraint and fail to be a HOT update.
+UPDATE users SET email = '[email protected]' WHERE name = 'you';
+SELECT * FROM check_hot_updates(1, 'users');
+
+DROP TABLE users;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- Another test of constraints spoiling HOT updates, this time with a range.
+CREATE TABLE events (
+ id serial primary key,
+ name VARCHAR(255) NOT NULL,
+ event_time tstzrange,
+ constraint no_screening_time_overlap exclude using gist (
+ event_time WITH &&
+ )
+);
+
+-- Add two non-overlapping events.
+INSERT INTO events (id, event_time, name)
+VALUES
+ (1, '["2023-01-01 19:00:00", "2023-01-01 20:45:00"]', 'event1'),
+ (2, '["2023-01-01 21:00:00", "2023-01-01 21:45:00"]', 'event2');
+
+-- Update the first event to overlap with the second, should fail the constraint and not be HOT.
+UPDATE events SET event_time = '["2023-01-01 20:00:00", "2023-01-01 21:45:00"]' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 'events');
+
+-- Update the first event to not overlap with the second, again not HOT due to the constraint.
+UPDATE events SET event_time = '["2023-01-01 22:00:00", "2023-01-01 22:45:00"]' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 'events');
+
+-- Update the first event to not overlap with the second, this time we're HOT because we don't overlap with the constraint.
+UPDATE events SET name = 'new name here' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 'events');
+
+DROP TABLE events;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- A test to ensure that only modified summarizing indexes are updated, not
+-- all of them.
+CREATE TABLE ex (id SERIAL primary key, att1 JSONB, att2 text, att3 text, att4 text) WITH (fillfactor = 60);
+CREATE INDEX ex_expr1_idx ON ex USING btree((att1->'data'));
+CREATE INDEX ex_sumr1_idx ON ex USING BRIN(att2);
+CREATE INDEX ex_expr2_idx ON ex USING btree((att1->'a'));
+CREATE INDEX ex_expr3_idx ON ex USING btree((att1->'b'));
+CREATE INDEX ex_expr4_idx ON ex USING btree((att1->'c'));
+CREATE INDEX ex_sumr2_idx ON ex USING BRIN(att3);
+CREATE INDEX ex_sumr3_idx ON ex USING BRIN(att4);
+CREATE INDEX ex_expr5_idx ON ex USING btree((att1->'d'));
+INSERT INTO ex (att1, att2) VALUES ('{"data": []}'::json, 'nothing special');
+
+SELECT * FROM ex;
+
+-- Update att2 and att4 both are BRIN/summarizing indexes, this should be a HOT update and
+-- only update two of the three summarizing indexes.
+UPDATE ex SET att2 = 'special indeed', att4 = 'whatever';
+SELECT * FROM check_hot_updates(1, 'ex');
+SELECT * FROM ex;
+
+-- Update att1 and att2, only one is BRIN/summarizing, this should NOT be a HOT update.
+UPDATE ex SET att1 = att1 || '{"data": "howdy"}', att2 = 'special, so special';
+SELECT * FROM check_hot_updates(1, 'ex');
+SELECT * FROM ex;
+
+-- Update att2, att3, and att4 all are BRIN/summarizing indexes, this should be a HOT update
+-- and yet still update all three summarizing indexes.
+UPDATE ex SET att2 = 'a', att3 = 'b', att4 = 'c';
+SELECT * FROM check_hot_updates(2, 'ex');
+SELECT * FROM ex;
+
+-- Update att1, att2, and att3 all modified values are BRIN/summarizing indexes, this should be a HOT update
+-- and yet still update all three summarizing indexes.
+UPDATE ex SET att1 = '{"data": "howdy"}', att2 = 'd', att3 = 'e';
+SELECT * FROM check_hot_updates(3, 'ex');
+SELECT * FROM ex;
+
+DROP TABLE ex;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- A test to ensure that summarizing indexes are not updated when they don't
+-- change, but are updated when they do while not prefent HOT updates.
+CREATE TABLE ex (att1 JSONB, att2 text) WITH (fillfactor = 60);
+CREATE INDEX ex_expr1_idx ON ex USING btree((att1->'data'));
+CREATE INDEX ex_sumr1_idx ON ex USING BRIN(att2);
+INSERT INTO ex VALUES ('{"data": []}', 'nothing special');
+
+-- Update the unindexed value of att1, this should be a HOT update and and should
+-- update the summarizing index.
+UPDATE ex SET att1 = att1 || '{"status": "stalemate"}';
+SELECT * FROM check_hot_updates(1, 'ex');
+
+-- Update the indexed value of att2, a summarized value, this is a summarized
+-- only update and should use the HOT path while still triggering an update to
+-- the summarizing BRIN index.
+UPDATE ex SET att2 = 'special indeed';
+SELECT * FROM check_hot_updates(2, 'ex');
+
+-- Update to att1 doesn't change the indexed value while the update to att2 does,
+-- this again is a summarized only update and should use the HOT path as well as
+-- trigger an update to the BRIN index.
+UPDATE ex SET att1 = att1 || '{"status": "checkmate"}', att2 = 'special, so special';
+SELECT * FROM check_hot_updates(3, 'ex');
+
+-- This updates both indexes, the expression index on att1 and the summarizing
+-- index on att2. This should not be a HOT update because there are modified
+-- indexes and only some are summarized, not all. This should force all
+-- indexes to be updated.
+UPDATE ex SET att1 = att1 || '{"data": [1,2,3]}', att2 = 'do you want to play a game?';
+SELECT * FROM check_hot_updates(4, 'ex');
+
+DROP TABLE ex;
+
+-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
+-- This test is for a table with a custom type and a custom operators on
+-- the BTREE index. The question is, when comparing values for equality
+-- to determine if there are changes on the index or not... shouldn't we
+-- be using the custom operators?
+
+-- Create a type
+CREATE TYPE my_custom_type AS (val int);
+
+-- Comparison functions (returns boolean)
+CREATE FUNCTION my_custom_lt(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val < b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_le(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val <= b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_eq(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val = b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_ge(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val >= b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_gt(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val > b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_ne(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val != b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+-- Comparison function (returns -1, 0, 1)
+CREATE FUNCTION my_custom_cmp(a my_custom_type, b my_custom_type) RETURNS int AS $$
+BEGIN
+ IF a.val < b.val THEN
+ RETURN -1;
+ ELSIF a.val > b.val THEN
+ RETURN 1;
+ ELSE
+ RETURN 0;
+ END IF;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+-- Create the operators
+CREATE OPERATOR < (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_lt,
+ COMMUTATOR = >,
+ NEGATOR = >=
+);
+
+CREATE OPERATOR <= (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_le,
+ COMMUTATOR = >=,
+ NEGATOR = >
+);
+
+CREATE OPERATOR = (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_eq,
+ COMMUTATOR = =,
+ NEGATOR = <>
+);
+
+CREATE OPERATOR >= (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_ge,
+ COMMUTATOR = <=,
+ NEGATOR = <
+);
+
+CREATE OPERATOR > (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_gt,
+ COMMUTATOR = <,
+ NEGATOR = <=
+);
+
+CREATE OPERATOR <> (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_ne,
+ COMMUTATOR = <>,
+ NEGATOR = =
+);
+
+-- Create the operator class (including the support function)
+CREATE OPERATOR CLASS my_custom_ops
+ DEFAULT FOR TYPE my_custom_type USING btree AS
+ OPERATOR 1 <,
+ OPERATOR 2 <=,
+ OPERATOR 3 =,
+ OPERATOR 4 >=,
+ OPERATOR 5 >,
+ FUNCTION 1 my_custom_cmp(my_custom_type, my_custom_type);
+
+-- Create the table
+CREATE TABLE my_table (
+ id int,
+ custom_val my_custom_type
+);
+
+-- Insert some data
+INSERT INTO my_table (id, custom_val) VALUES
+(1, ROW(3)::my_custom_type),
+(2, ROW(1)::my_custom_type),
+(3, ROW(4)::my_custom_type),
+(4, ROW(2)::my_custom_type);
+
+-- Create a function to use when indexing
+CREATE OR REPLACE FUNCTION abs_val(val my_custom_type) RETURNS int AS $$
+BEGIN
+ RETURN abs(val.val);
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+-- Create the index
+CREATE INDEX idx_custom_val_abs ON my_table (abs_val(custom_val));
+
+-- Update 1
+UPDATE my_table SET custom_val = ROW(5)::my_custom_type WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 'my_table');
+
+-- Update 2
+UPDATE my_table SET custom_val = ROW(0)::my_custom_type WHERE custom_val < ROW(3)::my_custom_type;
+SELECT * FROM check_hot_updates(0, 'my_table');
+
+-- Update 3
+UPDATE my_table SET custom_val = ROW(6)::my_custom_type WHERE id = 3;
+SELECT * FROM check_hot_updates(0, 'my_table');
+
+-- Update 4
+UPDATE my_table SET id = 5 WHERE id = 1;
+SELECT pg_stat_get_xact_tuples_hot_updated('my_table'::regclass);
+SELECT * FROM check_hot_updates(0, 'my_table');
+
+-- Query using the index
+EXPLAIN (COSTS OFF) SELECT * FROM my_table WHERE abs_val(custom_val) = 6;
+SELECT * FROM my_table WHERE abs_val(custom_val) = 6;
+
+-- Clean up
+DROP TABLE my_table CASCADE;
+DROP OPERATOR CLASS my_custom_ops USING btree CASCADE;
+DROP OPERATOR < (my_custom_type, my_custom_type);
+DROP OPERATOR <= (my_custom_type, my_custom_type);
+DROP OPERATOR = (my_custom_type, my_custom_type);
+DROP OPERATOR >= (my_custom_type, my_custom_type);
+DROP OPERATOR > (my_custom_type, my_custom_type);
+DROP OPERATOR <> (my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_lt(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_le(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_eq(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_ge(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_gt(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_ne(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_cmp(my_custom_type, my_custom_type);
+DROP FUNCTION abs_val(my_custom_type);
+DROP TYPE my_custom_type CASCADE;
+
+DROP FUNCTION check_hot_updates(int, text, text);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 220e5a4f6b3..b5cdf2ac89a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2967,6 +2967,7 @@ TSVectorStat
TState
TStatus
TStoreState
+UpdateContext
TU_UpdateIndexes
TXNEntryFile
TYPCATEGORY
@@ -3154,7 +3155,6 @@ UniqueState
UnlistenStmt
UnresolvedTup
UnresolvedTupData
-UpdateContext
UpdateStmt
UpgradeTask
UpgradeTaskProcessCB
--
2.49.0
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
@ 2025-10-07 21:36 ` Greg Burd <[email protected]>
2025-10-08 20:48 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2025-10-09 19:27 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
0 siblings, 2 replies; 37+ messages in thread
From: Greg Burd @ 2025-10-07 21:36 UTC (permalink / raw)
To: Burd, Greg <[email protected]>; +Cc: pgsql-hackers
--68e587cb_25cd3729_41f9
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
On Jul 2 2025, at 2:10 pm, Greg Burd <[email protected]> wrote:
> The goal is to allow HOT updates under two new conditions:
> * when an indexed expression has not changed
> * when possible for a partial index
This is still true. :)
This patch has languished for nearly 2+ months at v17. Why? Primarily
due to feedback that although the idea had merit, there was a critical
flaw in the approach that made it a non-starter. The flaw was that I'd
been executing expressions while holding both the pin and a lock on the
buffer, which is not a great idea (self dead lock, etc.). This was
pointed out to me (thanks Robert Haas!) and so I needed to re-think my approach.
I put the patch aside for a while, then this past week at PGConf.dev/NYC
I heard interest from a few people (Jeff Davis, Nathan Bossart) who
encouraged me to move the code executing the expressions to just before
acquiring the lock but after pinning the buffer. The theory being that
my new code using the old/new tts to form and test the index tuples
resulting from executing expressions was using the resultsRelInfo struct
created during plan execution, not the information found on the page,
and so was safe without the lock.
This proved tricky because I had been using the modified_attrs and
expr_attrs as a test to avoid exercising expressions when unnecessary.
Calling HeapDetermineColumnsInfo() outside the buffer lock to get
modified_attrs proved to be a problem as it examines an oldtup that is
cobbled together from the elements on the page, requiring the lock I was
trying to avoid.
After reviewing how updates work in the executor, I discovered that
during execution the new tuple slot is populated with the information
from ExecBuildUpdateProjection() and the old tuple, but that most
importantly for this use case that function created a bitmap of the
modified columns (the columns specified in the update). This bitmap
isn't the same as the one produced by HeapDetermineColumnsInfo() as the
latter excludes attributes that are not changed after testing equality
with the helper function heap_attr_equals() where as the former will
include attributes that appear in the update but are the same value as
before. This, happily, is immaterial for the purposes of my function
ExecExprIndexesRequireUpdates() which simply needs to check to see if
index tuples generated are unchanged. So I had all I needed to run the
checks ahead of acquiring the lock on the buffer.
So, this led to v18 (attached), which passes test wold including a
number of new tests for the various corner cases relative to HOT updates
for expressions.
There is much room for improvement, and your suggestions are welcome.
I'll find time to quantify the benefit of this patch for the targeted
use cases and to ensure that all other cases see no regressions.
I need to review the tests I've added to ensure that they are the
minimal set required, that they communicate effectively their purpose,
etc. For now, it's more shotgun than scalpel, .... I'll get to it.
I added a reloption "expression_checks" to disable this new code path.
Good idea or bad precedent?
In execIndexing I special case for IsolationIsSerializable() and I can't
remember why now but I do recall one isolation test failing... I'll
check on this and get back to the thread. Or maybe you know why that
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-07 21:36 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
@ 2025-10-08 20:48 ` Nathan Bossart <[email protected]>
2025-10-09 19:08 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2025-10-09 20:57 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
1 sibling, 2 replies; 37+ messages in thread
From: Nathan Bossart @ 2025-10-08 20:48 UTC (permalink / raw)
To: Greg Burd <[email protected]>; +Cc: Burd, Greg <[email protected]>; pgsql-hackers
On Tue, Oct 07, 2025 at 05:36:11PM -0400, Greg Burd wrote:
> I put the patch aside for a while, then this past week at PGConf.dev/NYC
> I heard interest from a few people (Jeff Davis, Nathan Bossart) who
> encouraged me to move the code executing the expressions to just before
> acquiring the lock but after pinning the buffer. The theory being that
> my new code using the old/new tts to form and test the index tuples
> resulting from executing expressions was using the resultsRelInfo struct
> created during plan execution, not the information found on the page,
> and so was safe without the lock.
An open question (at least from me) is whether this is safe. I'm not
familiar enough with this area of code yet to confidently determine that.
> After reviewing how updates work in the executor, I discovered that
> during execution the new tuple slot is populated with the information
> from ExecBuildUpdateProjection() and the old tuple, but that most
> importantly for this use case that function created a bitmap of the
> modified columns (the columns specified in the update). This bitmap
> isn't the same as the one produced by HeapDetermineColumnsInfo() as the
> latter excludes attributes that are not changed after testing equality
> with the helper function heap_attr_equals() where as the former will
> include attributes that appear in the update but are the same value as
> before. This, happily, is immaterial for the purposes of my function
> ExecExprIndexesRequireUpdates() which simply needs to check to see if
> index tuples generated are unchanged. So I had all I needed to run the
> checks ahead of acquiring the lock on the buffer.
Nice.
> There is much room for improvement, and your suggestions are welcome.
A general and predictable suggestion is to find ways to break this into
smaller pieces. As-is, this patch would take me an enormous amount of time
to review in any depth. If we can break off some smaller pieces that we
can scrutinize and commit independently, we can start making forward
progress sooner. The UpdateContext and reloption stuff are examples of
things that might be possible to split into independent patches.
> I'll find time to quantify the benefit of this patch for the targeted
> use cases and to ensure that all other cases see no regressions.
Looking forward to these results. This should also help us decide whether
to set expression_checks by default.
> I added a reloption "expression_checks" to disable this new code path.
> Good idea or bad precedent?
If there are cases where the added overhead outweighs the benefits (which
seems like it must be true some of the time), then I think we must have a
way to opt-out (or maybe even opt-in). In fact, I'd advise adding a GUC to
complement the reloption so that users can configure it at higher levels.
> In execIndexing I special case for IsolationIsSerializable() and I can't
> remember why now but I do recall one isolation test failing... I'll
> check on this and get back to the thread. Or maybe you know why that
I didn't follow this.
> I'd like not to build, then rebuild index tuples for these expressions
> but I can't think of a way to do that without a palloc(), this is
> avoided today.
Is the avoidance of palloc() a strict rule? Is this discussed in the code
anywhere?
--
nathan
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-07 21:36 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-08 20:48 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
@ 2025-10-09 19:08 ` Jeff Davis <[email protected]>
2025-10-09 21:13 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
1 sibling, 1 reply; 37+ messages in thread
From: Jeff Davis @ 2025-10-09 19:08 UTC (permalink / raw)
To: Nathan Bossart <[email protected]>; Greg Burd <[email protected]>; +Cc: Burd, Greg <[email protected]>; pgsql-hackers
On Wed, 2025-10-08 at 15:48 -0500, Nathan Bossart wrote:
> > The theory being that
> > my new code using the old/new tts to form and test the index tuples
> > resulting from executing expressions was using the resultsRelInfo
> > struct
> > created during plan execution, not the information found on the
> > page,
> > and so was safe without the lock.
>
> An open question (at least from me) is whether this is safe. I'm not
> familiar enough with this area of code yet to confidently determine
> that.
The optimization requires that the expression evaluates to the same
thing on the old and new tuples. That determination doesn't have
anything to do with a lock on the buffer, so long as the old tuple
isn't pruned away or something. And clearly it won't be pruned, because
we're in the process of updating it, so we have a snapshot that can see
it.
There might be subtleties in other parts of the proposal, but the above
determination can be made safely without a buffer lock.
>
> > I added a reloption "expression_checks" to disable this new code
> > path.
> > Good idea or bad precedent?
>
> If there are cases where the added overhead outweighs the benefits
> (which
> seems like it must be true some of the time), then I think we must
> have a
> way to opt-out (or maybe even opt-in). In fact, I'd advise adding a
> GUC to
> complement the reloption so that users can configure it at higher
> levels.
I'll push back against this. For now I'm fine with developer options to
make testing easier, but we should find a way to make this work well
without tuning.
Regards,
Jeff Davis
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-07 21:36 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-08 20:48 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2025-10-09 19:08 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
@ 2025-10-09 21:13 ` Greg Burd <[email protected]>
0 siblings, 0 replies; 37+ messages in thread
From: Greg Burd @ 2025-10-09 21:13 UTC (permalink / raw)
To: Jeff Davis <[email protected]>; +Cc: Nathan Bossart <[email protected]>; pgsql-hackers
> On Oct 9, 2025, at 3:08 PM, Jeff Davis <[email protected]> wrote:
>
> On Wed, 2025-10-08 at 15:48 -0500, Nathan Bossart wrote:
>>> The theory being that
>>> my new code using the old/new tts to form and test the index tuples
>>> resulting from executing expressions was using the resultsRelInfo
>>> struct
>>> created during plan execution, not the information found on the
>>> page,
>>> and so was safe without the lock.
>>
>> An open question (at least from me) is whether this is safe. I'm not
>> familiar enough with this area of code yet to confidently determine
>> that.
Hey Jeff,
Thanks for the nudge at PGConf.dev in NYC and for the follow-up here.
> The optimization requires that the expression evaluates to the same
> thing on the old and new tuples. That determination doesn't have
> anything to do with a lock on the buffer, so long as the old tuple
> isn't pruned away or something. And clearly it won't be pruned, because
> we're in the process of updating it, so we have a snapshot that can see
> it.
Right, I test that the expression on the index evaluates to the same
value when forming an index tuple for old/new slots.
> There might be subtleties in other parts of the proposal, but the above
> determination can be made safely without a buffer lock.
>
>>
>>> I added a reloption "expression_checks" to disable this new code
>>> path.
>>> Good idea or bad precedent?
>>
>> If there are cases where the added overhead outweighs the benefits
>> (which
>> seems like it must be true some of the time), then I think we must
>> have a
>> way to opt-out (or maybe even opt-in). In fact, I'd advise adding a
>> GUC to
>> complement the reloption so that users can configure it at higher
>> levels.
>
> I'll push back against this. For now I'm fine with developer options to
> make testing easier, but we should find a way to make this work well
> without tuning.
I'm aligned with this, the reloption evolved from a GUC and I'm more of
the opinion that neither should exist and that the overhead of this be
minimized and so require no tuning or consideration by the end user.
best.
-greg
> Regards,
> Jeff Davis
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-07 21:36 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-08 20:48 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
@ 2025-10-09 20:57 ` Greg Burd <[email protected]>
1 sibling, 0 replies; 37+ messages in thread
From: Greg Burd @ 2025-10-09 20:57 UTC (permalink / raw)
To: Nathan Bossart <[email protected]>; +Cc: pgsql-hackers
> On Oct 8, 2025, at 4:48 PM, Nathan Bossart <[email protected]> wrote:
>
> On Tue, Oct 07, 2025 at 05:36:11PM -0400, Greg Burd wrote:
>> I put the patch aside for a while, then this past week at PGConf.dev/NYC
>> I heard interest from a few people (Jeff Davis, Nathan Bossart) who
>> encouraged me to move the code executing the expressions to just before
>> acquiring the lock but after pinning the buffer. The theory being that
>> my new code using the old/new tts to form and test the index tuples
>> resulting from executing expressions was using the resultsRelInfo struct
>> created during plan execution, not the information found on the page,
>> and so was safe without the lock.
Thanks for taking a look Nathan.
>
> An open question (at least from me) is whether this is safe. I'm not
> familiar enough with this area of code yet to confidently determine that.
My read is that it is safe because we're testing the content of two
TupleTableSlots both formed in the executor. The function uses only
that information and doesn't reference data on the page at all.
>> After reviewing how updates work in the executor, I discovered that
>> during execution the new tuple slot is populated with the information
>> from ExecBuildUpdateProjection() and the old tuple, but that most
>> importantly for this use case that function created a bitmap of the
>> modified columns (the columns specified in the update). This bitmap
>> isn't the same as the one produced by HeapDetermineColumnsInfo() as the
>> latter excludes attributes that are not changed after testing equality
>> with the helper function heap_attr_equals() where as the former will
>> include attributes that appear in the update but are the same value as
>> before. This, happily, is immaterial for the purposes of my function
>> ExecExprIndexesRequireUpdates() which simply needs to check to see if
>> index tuples generated are unchanged. So I had all I needed to run the
>> checks ahead of acquiring the lock on the buffer.
>
> Nice.
Handy indeed. I'm not at all a fan of increasing the size of a plan node
but it's only by a little... and for a good cause.
>> There is much room for improvement, and your suggestions are welcome.
>
> A general and predictable suggestion is to find ways to break this into
> smaller pieces. As-is, this patch would take me an enormous amount of time
> to review in any depth. If we can break off some smaller pieces that we
> can scrutinize and commit independently, we can start making forward
> progress sooner. The UpdateContext and reloption stuff are examples of
> things that might be possible to split into independent patches.
Fair, I'll try.
>> I'll find time to quantify the benefit of this patch for the targeted
>> use cases and to ensure that all other cases see no regressions.
>
> Looking forward to these results. This should also help us decide whether
> to set expression_checks by default.
In past my test results were very positive for cases where this helped avoid
heap and index bloat and almost immeasurably small even for cases where we
were doing the work to test but ultimately unable to take the HOT path.
This will require new tests as the code has changed quite a bit.
>> I added a reloption "expression_checks" to disable this new code path.
>> Good idea or bad precedent?
>
> If there are cases where the added overhead outweighs the benefits (which
> seems like it must be true some of the time), then I think we must have a
> way to opt-out (or maybe even opt-in). In fact, I'd advise adding a GUC to
> complement the reloption so that users can configure it at higher levels.
This evolved from a GUC to a reloption and I'd rather it go away entirely.
I hear your concern, but I've yet to measure a perceptable impact and I'll
try hard to keep it that way as this matures. Assuming that's the case,
I'd like to eliminate the potentially confusing tuning knob.
>> In execIndexing I special case for IsolationIsSerializable() and I can't
>> remember why now but I do recall one isolation test failing... I'll
>> check on this and get back to the thread. Or maybe you know why that
>
> I didn't follow this.
More later if/when I can reproduce it and understand it better myself.
>> I'd like not to build, then rebuild index tuples for these expressions
>> but I can't think of a way to do that without a palloc(), this is
>> avoided today.
>
> Is the avoidance of palloc() a strict rule? Is this discussed in the code
> anywhere?
Not that I know of, just my paranoid self trying to avoid it on a path that
didn't have it before.
> --
> nathan
best.
-greg
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-07 21:36 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
@ 2025-10-09 19:27 ` Jeff Davis <[email protected]>
2025-10-14 17:46 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-11-16 18:53 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
1 sibling, 2 replies; 37+ messages in thread
From: Jeff Davis @ 2025-10-09 19:27 UTC (permalink / raw)
To: Greg Burd <[email protected]>; Burd, Greg <[email protected]>; +Cc: pgsql-hackers
On Tue, 2025-10-07 at 17:36 -0400, Greg Burd wrote:
> After reviewing how updates work in the executor, I discovered that
> during execution the new tuple slot is populated with the information
> from ExecBuildUpdateProjection() and the old tuple, but that most
> importantly for this use case that function created a bitmap of the
> modified columns (the columns specified in the update). This bitmap
> isn't the same as the one produced by HeapDetermineColumnsInfo() as
> the
> latter excludes attributes that are not changed after testing
> equality
> with the helper function heap_attr_equals() where as the former will
> include attributes that appear in the update but are the same value
> as
> before. This, happily, is immaterial for the purposes of my function
> ExecExprIndexesRequireUpdates() which simply needs to check to see if
> index tuples generated are unchanged. So I had all I needed to run
> the
> checks ahead of acquiring the lock on the buffer.
You're still calling ExecExprIndexesRequireUpdates() from within
heap_update(). Can't you do that inside of ExecUpdatePrologue() or
thereabouts?
Regards,
Jeff Davis
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-07 21:36 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-09 19:27 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
@ 2025-10-14 17:46 ` Greg Burd <[email protected]>
2025-10-14 18:43 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
1 sibling, 1 reply; 37+ messages in thread
From: Greg Burd @ 2025-10-14 17:46 UTC (permalink / raw)
To: Jeff Davis <[email protected]>; +Cc: pgsql-hackers
> On Oct 9, 2025, at 3:27 PM, Jeff Davis <[email protected]> wrote:
>
> On Tue, 2025-10-07 at 17:36 -0400, Greg Burd wrote:
>> After reviewing how updates work in the executor, I discovered that
>> during execution the new tuple slot is populated with the information
>> from ExecBuildUpdateProjection() and the old tuple, but that most
>> importantly for this use case that function created a bitmap of the
>> modified columns (the columns specified in the update). This bitmap
>> isn't the same as the one produced by HeapDetermineColumnsInfo() as
>> the
>> latter excludes attributes that are not changed after testing
>> equality
>> with the helper function heap_attr_equals() where as the former will
>> include attributes that appear in the update but are the same value
>> as
>> before. This, happily, is immaterial for the purposes of my function
>> ExecExprIndexesRequireUpdates() which simply needs to check to see if
>> index tuples generated are unchanged. So I had all I needed to run
>> the
>> checks ahead of acquiring the lock on the buffer.
>
> You're still calling ExecExprIndexesRequireUpdates() from within
> heap_update(). Can't you do that inside of ExecUpdatePrologue() or
> thereabouts?
Hey Jeff,
I'm trying to knit this into the executor layer but that is tricky because
the concept of HOT is very heap-specific, so the executor should be
ignorant of the heap's specific needs (right?). Right now, I am considering
adding a step in ExecUpdatePrologue() just after opening the indexes.
The idea I'm toying with is to have a new function on all TupleTableSlots
that examines the before/after slots for an update and the set of updated
attributes and returns a Bitmapset of the changed attributes that overlap
with indexes and so should trigger index updates in ExecUpdateEpilogue().
That way for heap we'd have something like:
Bitmapset *tts_heap_getidxattr(ResultRelInfo *info,
TupleTableSlot *updated,
TupleTableSlot *existing,
Bitmapset *updated_attrs)
{
some combo of HeapDeterminColumnsInfo() and
ExecExprIndexesRequireUpdates()
returns the set of indexed attrs that this update changed
}
So, attributes only referenced by expressions where the expression
produces the same value for the updated and existing slots would be
removed from the set.
Interestingly, summarizing indexes that don't overlap with changed
attributes won't be updated (and that's a good thing).
Problem is we're not yet accounting for what is about to happen in
ExecUpdateAct() when calling into the heap_update(). That's where
heap tries to fit the new tuple onto the same page. That might be
possible with large tuples thanks to TOAST, it's impossible to say
before getting into this function with the page locked.
So, for updates we include the modified_attrs in the UpdateContext
which is available to heap_update(). If the heap code decides to
go HOT, great unset all attributes in the modified_attrs except any
that are only summarizing. If the heap can't go HOT, fine, add
the indexed attrs back into modified_attrs which should trigger all
indexes to be updated.
This gets rid of TU_UpdateIndexes enum and allows only modified
summarizing indexes to be updated on the HOT path. Two additional
benefits IMO.
at least, that's what I'm trying out now,
-greg
> Regards,
> Jeff Davis
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-07 21:36 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-09 19:27 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2025-10-14 17:46 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
@ 2025-10-14 18:43 ` Jeff Davis <[email protected]>
0 siblings, 0 replies; 37+ messages in thread
From: Jeff Davis @ 2025-10-14 18:43 UTC (permalink / raw)
To: Greg Burd <[email protected]>; +Cc: pgsql-hackers
On Tue, 2025-10-14 at 13:46 -0400, Greg Burd wrote:
> I'm trying to knit this into the executor layer but that is tricky
> because
> the concept of HOT is very heap-specific, so the executor should be
> ignorant of the heap's specific needs (right?).
It's wrong for the executor to say "do a HOT update" but it's OK for
the executor to say "this is the set of indexes that might have a new
key after the update". If that set is empty, then the heap can choose
to do a HOT update.
> Right now, I am considering
> adding a step in ExecUpdatePrologue() just after opening the indexes.
Seems like a reasonable place.
> The idea I'm toying with is to have a new function on all
> TupleTableSlots...
>
> That way for heap we'd have something like:
> Bitmapset *tts_heap_getidxattr(ResultRelInfo *info,
> TupleTableSlot *updated,
> TupleTableSlot *existing,
> Bitmapset *updated_attrs)
> {
> some combo of HeapDeterminColumnsInfo() and
> ExecExprIndexesRequireUpdates()
>
> returns the set of indexed attrs that this update changed
> }
Why is this a generic method for all slots? Do we need to reuse it
somewhere else? I would have expected just a static method in
nodeModifyTable.c that does just what's needed.
And to be precise, it's the set of indexed attrs where the update might
have created a new key, right? The whole point is that we don't care if
the indexed attr has been changed, so long as it doesn't create a new
index key.
> Interestingly, summarizing indexes that don't overlap with changed
> attributes won't be updated (and that's a good thing).
Nice.
> Problem is we're not yet accounting for what is about to happen in
> ExecUpdateAct() when calling into the heap_update(). That's where
> heap tries to fit the new tuple onto the same page. That might be
> possible with large tuples thanks to TOAST, it's impossible to say
> before getting into this function with the page locked.
I don't see why that's a problem. The executor can pass down the list
of indexed attrs that might have created new keys after the update,
then heap_update uses that information (along with other factors, like
if it fits on the same page) to determine whether to perform a HOT
update or not.
> So, for updates we include the modified_attrs in the UpdateContext
> which is available to heap_update().
It doesn't look like UpdateContext is currently available to
heap_update(). We might need to change the signature. But I think it's
fine to change the signature if it results in a cleaner design --
tableam extensions often need source changes when new major versions
are released.
> If the heap code decides to
> go HOT, great unset all attributes in the modified_attrs except any
> that are only summarizing. If the heap can't go HOT, fine, add
> the indexed attrs back into modified_attrs which should trigger all
> indexes to be updated.
IIUC, that sounds like a good plan.
> This gets rid of TU_UpdateIndexes enum and allows only modified
> summarizing indexes to be updated on the HOT path. Two additional
> benefits IMO.
I'm not sure that I understand, but I'll look at that after we sort out
some of the other details.
Regards,
Jeff Davis
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-07 21:36 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-09 19:27 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
@ 2025-11-16 18:53 ` Greg Burd <[email protected]>
2025-11-19 18:00 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
1 sibling, 1 reply; 37+ messages in thread
From: Greg Burd @ 2025-11-16 18:53 UTC (permalink / raw)
To: pgsql-hackers
Hello again.
This idea started over a year ago for me while working on a project that
used JSONB and had horrible "bloat" with update times that were not
fantastic. The root cause, expression indexes prevent HOT updates and
all indexes on JSONB are by definition, expressions. That's the backstory.
The idea for the solution came from a patch [1] applied [2], then later
reverted [3], that basically evaluated the before/after tuple
expressions in heap_update() at the point where newbuff == buffer just
before deciding to use_hot_update or not. When the evaluated
expressions produced equal results using a binary comparison the index
didn't need to be updated. While this approach sorta worked, but it was
reverted for a few reasons, here's Tom's summary: "The problem here is
that [the code that checks for equality] thinks that the type of the
index column is identical to the type of the source datum for it, which
is not true for any opclass making use of the opckeytype property. [4]"
Still, expanding the domain of what might go HOT seems like a good goal
to me and it was at the heart of the issues I was facing on the project
using JSONB, so I kept at it.
The patches I've sent on this thread have evolved from that first idea.
First they addressed the specific issue raised by Tom. Then they
expanded to include partial indexes as well. This worked, and I helped
to ship a fork of Postgres used this approach and solved customer issues
with the JSONB use case. Customer experience was better, no unnecessary
index updates when indexed data wasn't modified meant no more heap/index
bloat and faster update times. Vacuum could finally keep up and
performance and storage overhead was much better.
For this patch set v1 through v17 were essentially work that
refined/reworked that original approach, but there was a serious flaw
that I didn't fully appreciate until around v16/17. I was evaluating
the expressions while holding a lock on the buffer page which a) expands
the time the lock is held, and b) opens the door to self-deadlock. No bueno.
Then there was v18, a quick work-around for that. I moved the call that
invokes the executor to the beginning of heap_update() before taking the
lock on the page. To do this I had to find the set of updated
attributes, which I discovered was available in the executor as the
updated tuple is created. This was a viable fix, but didn't really go
far enough and was a bit hackish IMO. Jeff Davis and others challenged
me to move the work to identify what's changed into the executor and
clean it up. I'm a sucker for a challenge.
More generally, this idea made sense. While Postgres has many places
where logic is tightly coupled to the way the heap works this didn't
have to be one. To make this work I needed the update path in the
executor to be interested in:
a) knowing what columns were specified in the UPDATE statement and those
impacted by before/after triggers,
b) reducing that set to those attributes known to be both indexed and to
have changed value,
c) finding which of those (and possibly other) attributes that force new
index updates.
Why? We'll, that code already exists in a few places and in some cases
is replicated; for (a) there is ExecGetAllUpdatedCols(), for (b)
HeapDetermineColumnsInfo() and index_unchanged_by_update().
An interesting thing to note is that HeapDetermineColumnsInfo() might
return a set that includes columns not returned by
ExecGetAllUpdatedCols() because HeapDetermineColumnsInfo() iterates over
all indexed attributes looking for changes and that might find an
indexed attribute that was changed by heap_modify_tuple() but not
knowable by ExecGetAllUpdatedCols(). This happens in tsvector code, see
tsvector_op.c tsvector_update_trigger() where if (update_needed)
heap_modify_tuple_by_cols(). That column isn't known to ExecGetAllUpdatedCols().
HeapDetermineColumnsInfo() is also critical when modifying catalog
tuples. Catalog tuples are modified using either Form/GETSTRUCT or
values/nulls/replaces then using heap_modify_tuple() and calling into
CatalogTupleUpdate() which calls simple_heap_update() that calls
heap_update() where we find HeapDetermineColumnsInfo(). The interesting
thing here is that when modifying catalog tuples there is knowledge of
what attributes are changed, but that knowledge isn't preserved and
passed into CatalogTupleUpdate(), rather it is re-discovered in
HeapDetermineColumnsInfo(). That's how catalog tuples are able to take
the HOT path, they re-use that same logic. There is a fix for that [5]
too (and I really hope that lands in master ASAP), but that's not the
subject of this thread.
HeapDetermineColumnsInfo() also helps inform a few other decisions in
heap_update(), but these have to happen after taking the buffer lock and
are very heap-specific, namely:
1. Do either the replica identity key attributes overlap with the
modified index attributes or do they need to be stored externally, this
is passed on to ExtractReplicaIdentity() to find out if we must augment
the WAL log or not.
2. Are there any modified indexed attributes that intersect with the
primary keys of the relation, if not lower the lock mode to enable
multixact to work.
HeapDetermineColumnsInfo() also takes a pragmatic approach to testing
for equality when looking for modified indexed attributes, it uses
datumIsEqual() which boils down to a simple memcmp() of the before/after
HeapTuple datum. This is fine in most cases, but limits the scope of
what can be HOT.
Interestingly, this requirement of binary equality has leaked into other
parts of the code, namely nbtree's deduplication of TIDs on page split.
That code uses binary equality as well. A nbtree index with collation
for case insensitive must store both "A" and "a" despite those being
type-equal because they are not binary equivalent. More on this later.
At this point, I had my sights set on HeapDetermineColumnsInfo(). I
felt that what it was doing should move into the executor, well as much
of that work as possible, and outside of the buffer lock. This would
also open the door for removal of redundant code. My thought was that
the table AM update API should have an additional argument, the
"modified indexed attributes" or "mix_attrs", passed in.
So, here we are at the door of v19... let's begin.
0001 - Reorganize heap update logic
This is preparatory work for the larger goal in that heap_update()
serves two masters: CatalogTupleUpdate()/simple_heap_update() and
heap_tuple_update() and in reality, they were different but needed most
of the same logic that happens at the start of heap_update(). This
patch splits that logic out and moves it into heap_tuple_update() and
simple_heap_update(). Functionally nothing changes. That's the meat of
this patch.Reorganize heap update logic
0002 - Track changed indexed columns in the executor during UPDATEs
This is the first core set of changes, it doesn't expand HOT updates but
it does restructure where HeapDetermineColumnsInfo()'s core work happens.
A new function ExecCheckIndexedAttrsForChanges() in nodeModifyTable.c is
now responsible for checking for changes between Datum in the old/new
TupleTableSlots. This is different from before in that we're not
checking the new HeapTuple Datum verses the HeapTuple we read from the
buffer page while holding the lock on that page.
An update starts off by reading the existing tuple using the table AM.
Then a new updated tuple is created as the set of changes to the old.
Then the new TupleTableSlot is the combination of the existing one we
just read and the changes we just recorded. So, in the executor before
calling into the table AM's update function we have a pin on the buffer
and the before/after TupleTableSlots for this update. So, I've put the
call to my new ExecCheckIndexedAttrsForChanges() function just before
calling table_tuple_update() and I've added the "mix_attrs" into that
call which get passed on to the heap in heap_tuple_update() and then
heap_update() and all is well.
Why is this safe? The way I read heap_update() is that it has always
historically had code to deal with cases where the tuple is concurrently
updated and react accordingly thanks to HeapTupleSatisfiesUpdate() which
remains where it was in heap_update(). Visibility checks happened when
we first read the tuple to form the updated tuple and later in
heap_update() when we call HeapTupleSatisfiesVisibility() to check for
transaction-snapshot mode RI updates.
So, this new update path from the executor into the table AM seems to me
to be okay and almost functionally equivalent. But there is one big
change to discuss before moving to the simple_heap_update() path.
In nodeModifyTable.c tts_attr_equal() replaces heap_attr_equal()
changing the test for equality when calling into heap_tuple_update().
In the past we used datumIsEqual(), essentially a binary comparison
using memcmp(), now the comparison code in tts_attr_equal uses
type-specific equality function when available and falls back to
datumIsEqual() when not.
The other parts of HeapDetermineColumnsInfo() remain in the code, but
they still happen within the simple_heap_update() and
heap_tuple_update() code. That's where you'll find that after the
buffer is locked we do (1) and (2) from above. This keeps the
heap-specific work in the heap, but we've moved some work up into the
executor and outside the buffer lock.
While this accomplishes the goal of removing HeapDetermineColumnsInfo()
from heap_update() on the path that uses the table AM API
heap_tuple_update() but it doesn't on the simple_heap_update() path.
That remains the same as it was in the previous patch. Ideally, my
patch to restructure how catalog tuples are updated [5] is committed and
we can fully remove HeapDetermineColumnsInfo() and likely speed up all
catalog updates in the process. That's what motivated [5], please take
a look, it required a huge number of changes so I thought it deserved a
life/thread of its own.
Finally, there is the ExecSimpleRelationUpdate() path and
slot_modify_data(). On this path we know what attributes are being
updated, so we just check to see if they changed and then intersect that
with the set of indexed attributes and we have our modified indexed
attributes set to pass into simple_heap_update().
0003 - Replace index_unchanged_by_update with ri_ChangedIndexedCols
This patch removes the function index_unchanged_by_update() in
execIndexing.c and simply re-uses the modified indexed attributes that
we've stashed away in ResultRelInfo as ri_ChangedIndexedCols. This
provides a hint when calling into the index AM's index_insert() function
indicating if the UPDATE was without logical change to the data or not.
We've done that check, we don't need to do it again.
0004 - Enable HOT updates for expression and partial indexes
This finally gets us back to where this project started, but on much
more firm ground than before because we're not going to self-deadlock.
The idea has grown from a small function into something larger, but only
out of necessity.
In this patch I add ExecWhichIndexesRequireUpdates() in execIndexing.c
which implements (c) finding the set of attributes that force new index
updates. This set can be very different from the modified indexed
attributes. We know that some attributes are not equal to their
previous versions, but does that mean that the index that references
that attribute needs a new index tuple? It may, or it may not. Here's
the comment on that function that explains:
/*
* ExecWhichIndexesRequireUpdates
*
* Determine which indexes need updating given modified indexed attributes.
* This function is a companion to ExecCheckIndexedAttrsForChanges().
On the
* surface, they appear similar but they are doing two very different things.
*
* For a standard index on a set of attributes this is the intersection of
* the mix_attrs and the index attrs (key, expression, but not predicate).
*
* For expression indexes and indexes which implement the amcomparedatums()
* index AM API we'll need to form index datum and compare each
attribute to
* see if any actually changed.
*
* For expression indexes the result of the expression might not change
at all,
* this is common with JSONB columns which require expression indexes
and where
* it is commonplace to index a field within a document and have updates that
* generally don't update that field.
*
* Partial indexes won't trigger index tuples when the old/new tuples
are both
* outside of the predicate range.
*
* For nbtree the amcomparedatums() API is critical as it requires that key
* attributes are equal when they memcmp(), which might not be the case when
* using type-specific comparison or factoring in collation which might make
* an index case insensitive.
*
* All of this is to say that the goal is for the executor to know,
ahead of
* calling into the table AM for the update and before calling into the index
* AM for inserting new index tuples, which attributes at a minimum will
* necessitate a new index tuple.
*
...
*/
Whereas before we were comparing Datum in the table relation, now we're
comparing Datum in the index relation. Index AMs are free to store what
they want, we need to know if what's changed and referenced by the index
means that the index needs a new tuple or not.
In the case of a JSONB expression index (or any expression index) the
expression is evaluated when calling FormIndexDatum(). The result of
the expression is the Datum in the values/isnull arrays. Then we need
to compare them to see if they changed. This can be done using the same
tts_attr_equal() function, but with the attribute desc of the index, not
table relation.
In some cases that's not enough, for instance nbtree and it's more
stringent equality requirements. For that reason (and another coming up
in a second) we need a new optional index AM API, amcomparedatums().
Indexes implementing this function have the ability to compare two
Datums for equality in what ever way they want. For nbtree, that's a
binary comparison.
The new case here that this also supports is for indexes like GIN/RUM
where there is an opclass that can extract zero or more pieces from the
attribute and form multiple index entries when needed. Those extracted
pieces might have odd equality rules as well. This opens the door for
index implementations to provide information that will help inform heap
when making HOT decisions that didn't exist before.
Take for example the Linux Foundation's DocumentDB [6] project [7] which
aims to be an open source alternative to MongoDB built on top of
PostgreSQL. One of the pieces of that project is its "extended RUM"
index AM implementation. This index extracts portions of the BSON
documents stored and forms index keys from that. Here is an example:
CREATE INDEX documents_rum_index_14002
ON documentdb_data.documents_14001
USING documentdb_rum
(document bson_rum_single_path_ops (path=a, iswildcard='true', tl='2699'))
An index on the "document" column which uses the
"bson_rum_single_path_ops" opclass to extract a portion of the BSON
document that matches "path=a".
For this to potentially be stored by heap as a HOT update we need to
know that what changed within that document didn't intersect with the
"path=a" and if it did that the new value(s) were all equal to the old
values. Equality in DocumentDB isn't what you think, it's quite odd and
specific to BSON and rules defined by MongoDB so it's important to allow
the index AM to execute what's necessary for it's use case.
For the more common JSONB use case we can now have heap make an informed
decision about HOT updates after evaluating the expression. For
GIN/RUM/etc. implementation there is a path to HOT. For nbtree there is
a way to maintain its requirements.
Is there a cost to all this? Yes, of course. There is net new work
being done on some paths. IndexFormDatum() will be called more
frequently and sometimes twice for the same thing. This could be
improved, there might be a way to cache that information. But to have
tests of old/new we'll have to do that work at least twice.
Is there a benefit? Yes, of course. Some redundant code paths are gone
and in the end we've increased the number of cases where HOT updates are
a possibility. This especially helps out users of JSONB, but not only them.
What's left undone?
* I need to check code coverage so that I might
* create tests covering all the new cases
* update the README.HOT documentation, wiki, etc.
* performance...
For performance I'd like to examine some worst cases as in lots of
indexes that have a lot of new code to exercise and all but the last
index would allow for a HOT update. That should represent the maximum
amount of new overhead for this code. Then, the other side of the
equation, how much does this help JSONB? I think that is something to
measure in terms of TPS as well as "bloat" avoided and time spent vacuuming.
I also don't like the TU_Updating enum, I think it's a leaky abstraction
and really pointless now. I'd like to remove it in favor of the bitmap
of attributes known to force index tuples to be inserted. Maybe I'll
layer that into the next set.
In the end, this is a lot of work and I believe that it moves the ball
forward. I'll have more metrics on that soon I hope, but I wanted to
get the conversation re-started ASAP as we're late in the v19 cycle.
Finally, the "elephant" in the room (ha!) is PHOT[9]/WARM[10][11]. Yes,
some of this work does help make a solution to allow HOT updates when
only updating a subset of indexes closer to reality, I'd be lying not to
mention that here, it is a big part of my overall plan for my next year
and (I hope) v20 and I need this work to get there. Specifically, PHOT
is in part based on HeapDetermineColumnsInfo() and I needed to make that
more truthful for it to work.
I hope you see the value in this work and will partner with me to
finalize it and get it into master.
best.
-greg
[1] https://www.postgresql.org/message-id/flat/4d9928ee-a9e6-15f9-9c82-5981f13ffca6%40postgrespro.ru
[2] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=c203d6cf8
[3] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=05f84605dbeb9cf8279a157234b24bbb706c5...
[4] https://www.postgresql.org/message-id/2877.1541538838%40sss.pgh.pa.us
[5] https://www.postgresql.org/message-id/flat/[email protected]
[6] https://www.linuxfoundation.org/press/linux-foundation-welcomes-documentdb-to-advance-open-developer...
[7] https://github.com/documentdb/documentdb
[8] https://github.com/documentdb/documentdb/tree/main/pg_documentdb_extended_rum
[9] https://www.postgresql.org/message-id/flat/2ECBBCA0-4D8D-4841-8872-4A5BBDC063D2%40amazon.com
[10] https://www.postgresql.org/message-id/flat/CABOikdMop5Rb_RnS2xFdAXMZGSqcJ-P-BY2ruMd%2BbuUkJ4iDPw%40m...
[11]
https://www.postgresql.org/message-id/flat/CABOikdMNy6yowA%2BwTGK9RVd8iw%2BCzqHeQSGpW7Yka_4RSZ_LOQ%4...
Attachments:
[application/octet-stream] v19-0001-Reorganize-heap-update-logic.patch (47.6K, 2-v19-0001-Reorganize-heap-update-logic.patch)
download | inline diff:
From 6d76d6ad5bc7077727f92bdb9c33b78ca3d3c14e Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 2 Nov 2025 11:36:20 -0500
Subject: [PATCH v19 1/4] Reorganize heap update logic
This commit refactors the interaction between heap_tuple_update(),
heap_update(), and simple_heap_update() to improve code organization
and flexibility. The changes are functionally equivalent to the
previous implementation and have no performance impact.
The primary motivation is to prepare for upcoming modifications to
how and where modified attributes are identified during the update
path, particularly for catalog updates.
As part of this reorganization, the handling of replica identity key
attributes has been adjusted. Instead of fetching a second copy of
the bitmap during an update operation, the caller is now required to
provide it. This change applies to both heap_update() and
heap_delete().
No user-visible changes.
---
src/backend/access/heap/heapam.c | 568 +++++++++++------------
src/backend/access/heap/heapam_handler.c | 117 ++++-
src/include/access/heapam.h | 24 +-
3 files changed, 410 insertions(+), 299 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 4b0c49f4bb0..aff47481345 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -39,18 +39,24 @@
#include "access/syncscan.h"
#include "access/valid.h"
#include "access/visibilitymap.h"
+#include "access/xact.h"
#include "access/xloginsert.h"
+#include "catalog/catalog.h"
#include "catalog/pg_database.h"
#include "catalog/pg_database_d.h"
#include "commands/vacuum.h"
+#include "nodes/bitmapset.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
+#include "storage/bufmgr.h"
+#include "storage/itemptr.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
#include "storage/procarray.h"
#include "utils/datum.h"
#include "utils/injection_point.h"
#include "utils/inval.h"
+#include "utils/relcache.h"
#include "utils/spccache.h"
#include "utils/syscache.h"
@@ -62,16 +68,8 @@ static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
HeapTuple newtup, HeapTuple old_key_tuple,
bool all_visible_cleared, bool new_all_visible_cleared);
#ifdef USE_ASSERT_CHECKING
-static void check_lock_if_inplace_updateable_rel(Relation relation,
- const ItemPointerData *otid,
- HeapTuple newtup);
static void check_inplace_rel_lock(HeapTuple oldtup);
#endif
-static Bitmapset *HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external);
static bool heap_acquire_tuplock(Relation relation, const ItemPointerData *tid,
LockTupleMode mode, LockWaitPolicy wait_policy,
bool *have_tuple_lock);
@@ -103,10 +101,10 @@ static bool ConditionalMultiXactIdWait(MultiXactId multi, MultiXactStatus status
static void index_delete_sort(TM_IndexDeleteOp *delstate);
static int bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate);
static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
-static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
+static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp,
+ Bitmapset *rid_attrs, bool key_required,
bool *copy);
-
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
* corresponding MultiXactStatuses (one to merely lock tuples, another one to
@@ -2799,6 +2797,7 @@ heap_delete(Relation relation, const ItemPointerData *tid,
Buffer buffer;
Buffer vmbuffer = InvalidBuffer;
TransactionId new_xmax;
+ Bitmapset *rid_attrs;
uint16 new_infomask,
new_infomask2;
bool have_tuple_lock = false;
@@ -2811,6 +2810,8 @@ heap_delete(Relation relation, const ItemPointerData *tid,
AssertHasSnapshotForToast(relation);
+ rid_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
/*
* Forbid this during a parallel operation, lest it allocate a combo CID.
* Other workers might need that combo CID for visibility checks, and we
@@ -3014,6 +3015,7 @@ l1:
UnlockTupleTuplock(relation, &(tp.t_self), LockTupleExclusive);
if (vmbuffer != InvalidBuffer)
ReleaseBuffer(vmbuffer);
+ bms_free(rid_attrs);
return result;
}
@@ -3035,7 +3037,10 @@ l1:
* Compute replica identity tuple before entering the critical section so
* we don't PANIC upon a memory allocation failure.
*/
- old_key_tuple = ExtractReplicaIdentity(relation, &tp, true, &old_key_copied);
+ old_key_tuple = ExtractReplicaIdentity(relation, &tp, rid_attrs,
+ true, &old_key_copied);
+ bms_free(rid_attrs);
+ rid_attrs = NULL;
/*
* If this is the first possibly-multixact-able operation in the current
@@ -3247,7 +3252,10 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* heap_update - replace a tuple
*
* See table_tuple_update() for an explanation of the parameters, except that
- * this routine directly takes a tuple rather than a slot.
+ * this routine directly takes a heap tuple rather than a slot.
+ *
+ * It's required that the caller has acquired the pin and lock on the buffer.
+ * That lock and pin will be managed here, not in the caller.
*
* In the failure cases, the routine fills *tmfd with the tuple's t_ctid,
* t_xmax (resolving a possible MultiXact, if necessary), and t_cmax (the last
@@ -3255,30 +3263,21 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* generated by another transaction).
*/
TM_Result
-heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+heap_update(Relation relation, HeapTupleData *oldtup,
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
+ Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
+ Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
+ Bitmapset *mix_attrs, Buffer *vmbuffer,
+ bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
- Bitmapset *hot_attrs;
- Bitmapset *sum_attrs;
- Bitmapset *key_attrs;
- Bitmapset *id_attrs;
- Bitmapset *interesting_attrs;
- Bitmapset *modified_attrs;
- ItemId lp;
- HeapTupleData oldtup;
HeapTuple heaptup;
HeapTuple old_key_tuple = NULL;
bool old_key_copied = false;
- Page page;
- BlockNumber block;
MultiXactStatus mxact_status;
- Buffer buffer,
- newbuf,
- vmbuffer = InvalidBuffer,
+ Buffer newbuf,
vmbuffer_new = InvalidBuffer;
bool need_toast;
Size newtupsize,
@@ -3292,7 +3291,6 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
bool all_visible_cleared_new = false;
bool checked_lockers;
bool locker_remains;
- bool id_has_external = false;
TransactionId xmax_new_tuple,
xmax_old_tuple;
uint16 infomask_old_tuple,
@@ -3300,144 +3298,13 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
infomask_new_tuple,
infomask2_new_tuple;
- Assert(ItemPointerIsValid(otid));
-
- /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
- Assert(HeapTupleHeaderGetNatts(newtup->t_data) <=
- RelationGetNumberOfAttributes(relation));
-
+ Assert(BufferIsLockedByMe(buffer));
+ Assert(ItemIdIsNormal(lp));
AssertHasSnapshotForToast(relation);
- /*
- * Forbid this during a parallel operation, lest it allocate a combo CID.
- * Other workers might need that combo CID for visibility checks, and we
- * have no provision for broadcasting it to them.
- */
- if (IsInParallelMode())
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
- errmsg("cannot update tuples during a parallel operation")));
-
-#ifdef USE_ASSERT_CHECKING
- check_lock_if_inplace_updateable_rel(relation, otid, newtup);
-#endif
-
- /*
- * Fetch the list of attributes to be checked for various operations.
- *
- * For HOT considerations, this is wasted effort if we fail to update or
- * have to put the new tuple on a different page. But we must compute the
- * list before obtaining buffer lock --- in the worst case, if we are
- * doing an update on one of the relevant system catalogs, we could
- * deadlock if we try to fetch the list later. In any case, the relcache
- * caches the data so this is usually pretty cheap.
- *
- * We also need columns used by the replica identity and columns that are
- * considered the "key" of rows in the table.
- *
- * Note that we get copies of each bitmap, so we need not worry about
- * relcache flush happening midway through.
- */
- hot_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_HOT_BLOCKING);
- sum_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_SUMMARIZED);
- key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
- id_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
- interesting_attrs = NULL;
- interesting_attrs = bms_add_members(interesting_attrs, hot_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, sum_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, key_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, id_attrs);
-
- block = ItemPointerGetBlockNumber(otid);
- INJECTION_POINT("heap_update-before-pin", NULL);
- buffer = ReadBuffer(relation, block);
- page = BufferGetPage(buffer);
-
- /*
- * Before locking the buffer, pin the visibility map page if it appears to
- * be necessary. Since we haven't got the lock yet, someone else might be
- * in the middle of changing this, so we'll need to recheck after we have
- * the lock.
- */
- if (PageIsAllVisible(page))
- visibilitymap_pin(relation, block, &vmbuffer);
-
- LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
-
- lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
-
- /*
- * Usually, a buffer pin and/or snapshot blocks pruning of otid, ensuring
- * we see LP_NORMAL here. When the otid origin is a syscache, we may have
- * neither a pin nor a snapshot. Hence, we may see other LP_ states, each
- * of which indicates concurrent pruning.
- *
- * Failing with TM_Updated would be most accurate. However, unlike other
- * TM_Updated scenarios, we don't know the successor ctid in LP_UNUSED and
- * LP_DEAD cases. While the distinction between TM_Updated and TM_Deleted
- * does matter to SQL statements UPDATE and MERGE, those SQL statements
- * hold a snapshot that ensures LP_NORMAL. Hence, the choice between
- * TM_Updated and TM_Deleted affects only the wording of error messages.
- * Settle on TM_Deleted, for two reasons. First, it avoids complicating
- * the specification of when tmfd->ctid is valid. Second, it creates
- * error log evidence that we took this branch.
- *
- * Since it's possible to see LP_UNUSED at otid, it's also possible to see
- * LP_NORMAL for a tuple that replaced LP_UNUSED. If it's a tuple for an
- * unrelated row, we'll fail with "duplicate key value violates unique".
- * XXX if otid is the live, newer version of the newtup row, we'll discard
- * changes originating in versions of this catalog row after the version
- * the caller got from syscache. See syscache-update-pruned.spec.
- */
- if (!ItemIdIsNormal(lp))
- {
- Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
-
- UnlockReleaseBuffer(buffer);
- Assert(!have_tuple_lock);
- if (vmbuffer != InvalidBuffer)
- ReleaseBuffer(vmbuffer);
- tmfd->ctid = *otid;
- tmfd->xmax = InvalidTransactionId;
- tmfd->cmax = InvalidCommandId;
- *update_indexes = TU_None;
-
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- /* modified_attrs not yet initialized */
- bms_free(interesting_attrs);
- return TM_Deleted;
- }
-
- /*
- * Fill in enough data in oldtup for HeapDetermineColumnsInfo to work
- * properly.
- */
- oldtup.t_tableOid = RelationGetRelid(relation);
- oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
- oldtup.t_len = ItemIdGetLength(lp);
- oldtup.t_self = *otid;
-
- /* the new tuple is ready, except for this: */
+ /* The new tuple is ready, except for this */
newtup->t_tableOid = RelationGetRelid(relation);
- /*
- * Determine columns modified by the update. Additionally, identify
- * whether any of the unmodified replica identity key attributes in the
- * old tuple is externally stored or not. This is required because for
- * such attributes the flattened value won't be WAL logged as part of the
- * new tuple so we must include it as part of the old_key_tuple. See
- * ExtractReplicaIdentity.
- */
- modified_attrs = HeapDetermineColumnsInfo(relation, interesting_attrs,
- id_attrs, &oldtup,
- newtup, &id_has_external);
-
/*
* If we're not updating any "key" column, we can grab a weaker lock type.
* This allows for more concurrency when we are running simultaneously
@@ -3449,7 +3316,7 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
* is updates that don't manipulate key columns, not those that
* serendipitously arrive at the same key values.
*/
- if (!bms_overlap(modified_attrs, key_attrs))
+ if (!bms_overlap(mix_attrs, pk_attrs))
{
*lockmode = LockTupleNoKeyExclusive;
mxact_status = MultiXactStatusNoKeyUpdate;
@@ -3473,17 +3340,10 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
key_intact = false;
}
- /*
- * Note: beyond this point, use oldtup not otid to refer to old tuple.
- * otid may very well point at newtup->t_self, which we will overwrite
- * with the new tuple's location, so there's great risk of confusion if we
- * use otid anymore.
- */
-
l2:
checked_lockers = false;
locker_remains = false;
- result = HeapTupleSatisfiesUpdate(&oldtup, cid, buffer);
+ result = HeapTupleSatisfiesUpdate(oldtup, cid, buffer);
/* see below about the "no wait" case */
Assert(result != TM_BeingModified || wait);
@@ -3515,8 +3375,8 @@ l2:
*/
/* must copy state data before unlocking buffer */
- xwait = HeapTupleHeaderGetRawXmax(oldtup.t_data);
- infomask = oldtup.t_data->t_infomask;
+ xwait = HeapTupleHeaderGetRawXmax(oldtup->t_data);
+ infomask = oldtup->t_data->t_infomask;
/*
* Now we have to do something about the existing locker. If it's a
@@ -3556,13 +3416,12 @@ l2:
* requesting a lock and already have one; avoids deadlock).
*/
if (!current_is_member)
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &oldtup->t_self, *lockmode,
LockWaitBlock, &have_tuple_lock);
/* wait for multixact */
MultiXactIdWait((MultiXactId) xwait, mxact_status, infomask,
- relation, &oldtup.t_self, XLTW_Update,
- &remain);
+ relation, &oldtup->t_self, XLTW_Update, &remain);
checked_lockers = true;
locker_remains = remain != 0;
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
@@ -3572,9 +3431,9 @@ l2:
* could update this tuple before we get to this point. Check
* for xmax change, and start over if so.
*/
- if (xmax_infomask_changed(oldtup.t_data->t_infomask,
+ if (xmax_infomask_changed(oldtup->t_data->t_infomask,
infomask) ||
- !TransactionIdEquals(HeapTupleHeaderGetRawXmax(oldtup.t_data),
+ !TransactionIdEquals(HeapTupleHeaderGetRawXmax(oldtup->t_data),
xwait))
goto l2;
}
@@ -3599,8 +3458,8 @@ l2:
* before this one, which are important to keep in case this
* subxact aborts.
*/
- if (!HEAP_XMAX_IS_LOCKED_ONLY(oldtup.t_data->t_infomask))
- update_xact = HeapTupleGetUpdateXid(oldtup.t_data);
+ if (!HEAP_XMAX_IS_LOCKED_ONLY(oldtup->t_data->t_infomask))
+ update_xact = HeapTupleGetUpdateXid(oldtup->t_data);
else
update_xact = InvalidTransactionId;
@@ -3641,9 +3500,9 @@ l2:
* lock.
*/
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &oldtup->t_self, *lockmode,
LockWaitBlock, &have_tuple_lock);
- XactLockTableWait(xwait, relation, &oldtup.t_self,
+ XactLockTableWait(xwait, relation, &oldtup->t_self,
XLTW_Update);
checked_lockers = true;
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
@@ -3653,20 +3512,20 @@ l2:
* other xact could update this tuple before we get to this point.
* Check for xmax change, and start over if so.
*/
- if (xmax_infomask_changed(oldtup.t_data->t_infomask, infomask) ||
+ if (xmax_infomask_changed(oldtup->t_data->t_infomask, infomask) ||
!TransactionIdEquals(xwait,
- HeapTupleHeaderGetRawXmax(oldtup.t_data)))
+ HeapTupleHeaderGetRawXmax(oldtup->t_data)))
goto l2;
/* Otherwise check if it committed or aborted */
- UpdateXmaxHintBits(oldtup.t_data, buffer, xwait);
- if (oldtup.t_data->t_infomask & HEAP_XMAX_INVALID)
+ UpdateXmaxHintBits(oldtup->t_data, buffer, xwait);
+ if (oldtup->t_data->t_infomask & HEAP_XMAX_INVALID)
can_continue = true;
}
if (can_continue)
result = TM_Ok;
- else if (!ItemPointerEquals(&oldtup.t_self, &oldtup.t_data->t_ctid))
+ else if (!ItemPointerEquals(&oldtup->t_self, &oldtup->t_data->t_ctid))
result = TM_Updated;
else
result = TM_Deleted;
@@ -3679,39 +3538,33 @@ l2:
result == TM_Updated ||
result == TM_Deleted ||
result == TM_BeingModified);
- Assert(!(oldtup.t_data->t_infomask & HEAP_XMAX_INVALID));
+ Assert(!(oldtup->t_data->t_infomask & HEAP_XMAX_INVALID));
Assert(result != TM_Updated ||
- !ItemPointerEquals(&oldtup.t_self, &oldtup.t_data->t_ctid));
+ !ItemPointerEquals(&oldtup->t_self, &oldtup->t_data->t_ctid));
}
if (crosscheck != InvalidSnapshot && result == TM_Ok)
{
/* Perform additional check for transaction-snapshot mode RI updates */
- if (!HeapTupleSatisfiesVisibility(&oldtup, crosscheck, buffer))
+ if (!HeapTupleSatisfiesVisibility(oldtup, crosscheck, buffer))
result = TM_Updated;
}
if (result != TM_Ok)
{
- tmfd->ctid = oldtup.t_data->t_ctid;
- tmfd->xmax = HeapTupleHeaderGetUpdateXid(oldtup.t_data);
+ tmfd->ctid = oldtup->t_data->t_ctid;
+ tmfd->xmax = HeapTupleHeaderGetUpdateXid(oldtup->t_data);
if (result == TM_SelfModified)
- tmfd->cmax = HeapTupleHeaderGetCmax(oldtup.t_data);
+ tmfd->cmax = HeapTupleHeaderGetCmax(oldtup->t_data);
else
tmfd->cmax = InvalidCommandId;
UnlockReleaseBuffer(buffer);
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
- if (vmbuffer != InvalidBuffer)
- ReleaseBuffer(vmbuffer);
+ UnlockTupleTuplock(relation, &oldtup->t_self, *lockmode);
+ if (*vmbuffer != InvalidBuffer)
+ ReleaseBuffer(*vmbuffer);
*update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
return result;
}
@@ -3724,10 +3577,10 @@ l2:
* tuple has been locked or updated under us, but hopefully it won't
* happen very often.
*/
- if (vmbuffer == InvalidBuffer && PageIsAllVisible(page))
+ if (*vmbuffer == InvalidBuffer && PageIsAllVisible(page))
{
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- visibilitymap_pin(relation, block, &vmbuffer);
+ visibilitymap_pin(relation, block, vmbuffer);
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
goto l2;
}
@@ -3738,9 +3591,9 @@ l2:
* If the tuple we're updating is locked, we need to preserve the locking
* info in the old tuple's Xmax. Prepare a new Xmax value for this.
*/
- compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
- oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2,
+ compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup->t_data),
+ oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2,
xid, *lockmode, true,
&xmax_old_tuple, &infomask_old_tuple,
&infomask2_old_tuple);
@@ -3752,12 +3605,12 @@ l2:
* tuple. (In rare cases that might also be InvalidTransactionId and yet
* not have the HEAP_XMAX_INVALID bit set; that's fine.)
*/
- if ((oldtup.t_data->t_infomask & HEAP_XMAX_INVALID) ||
- HEAP_LOCKED_UPGRADED(oldtup.t_data->t_infomask) ||
+ if ((oldtup->t_data->t_infomask & HEAP_XMAX_INVALID) ||
+ HEAP_LOCKED_UPGRADED(oldtup->t_data->t_infomask) ||
(checked_lockers && !locker_remains))
xmax_new_tuple = InvalidTransactionId;
else
- xmax_new_tuple = HeapTupleHeaderGetRawXmax(oldtup.t_data);
+ xmax_new_tuple = HeapTupleHeaderGetRawXmax(oldtup->t_data);
if (!TransactionIdIsValid(xmax_new_tuple))
{
@@ -3772,7 +3625,7 @@ l2:
* Note that since we're doing an update, the only possibility is that
* the lockers had FOR KEY SHARE lock.
*/
- if (oldtup.t_data->t_infomask & HEAP_XMAX_IS_MULTI)
+ if (oldtup->t_data->t_infomask & HEAP_XMAX_IS_MULTI)
{
GetMultiXactIdHintBits(xmax_new_tuple, &infomask_new_tuple,
&infomask2_new_tuple);
@@ -3800,7 +3653,7 @@ l2:
* Replace cid with a combo CID if necessary. Note that we already put
* the plain cid into the new tuple.
*/
- HeapTupleHeaderAdjustCmax(oldtup.t_data, &cid, &iscombo);
+ HeapTupleHeaderAdjustCmax(oldtup->t_data, &cid, &iscombo);
/*
* If the toaster needs to be activated, OR if the new tuple will not fit
@@ -3817,12 +3670,12 @@ l2:
relation->rd_rel->relkind != RELKIND_MATVIEW)
{
/* toast table entries should never be recursively toasted */
- Assert(!HeapTupleHasExternal(&oldtup));
+ Assert(!HeapTupleHasExternal(oldtup));
Assert(!HeapTupleHasExternal(newtup));
need_toast = false;
}
else
- need_toast = (HeapTupleHasExternal(&oldtup) ||
+ need_toast = (HeapTupleHasExternal(oldtup) ||
HeapTupleHasExternal(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
@@ -3855,9 +3708,9 @@ l2:
* updating, because the potentially created multixact would otherwise
* be wrong.
*/
- compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
- oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2,
+ compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup->t_data),
+ oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2,
xid, *lockmode, false,
&xmax_lock_old_tuple, &infomask_lock_old_tuple,
&infomask2_lock_old_tuple);
@@ -3867,18 +3720,18 @@ l2:
START_CRIT_SECTION();
/* Clear obsolete visibility flags ... */
- oldtup.t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
- oldtup.t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
- HeapTupleClearHotUpdated(&oldtup);
+ oldtup->t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
+ oldtup->t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
+ HeapTupleClearHotUpdated(oldtup);
/* ... and store info about transaction updating this tuple */
Assert(TransactionIdIsValid(xmax_lock_old_tuple));
- HeapTupleHeaderSetXmax(oldtup.t_data, xmax_lock_old_tuple);
- oldtup.t_data->t_infomask |= infomask_lock_old_tuple;
- oldtup.t_data->t_infomask2 |= infomask2_lock_old_tuple;
- HeapTupleHeaderSetCmax(oldtup.t_data, cid, iscombo);
+ HeapTupleHeaderSetXmax(oldtup->t_data, xmax_lock_old_tuple);
+ oldtup->t_data->t_infomask |= infomask_lock_old_tuple;
+ oldtup->t_data->t_infomask2 |= infomask2_lock_old_tuple;
+ HeapTupleHeaderSetCmax(oldtup->t_data, cid, iscombo);
/* temporarily make it look not-updated, but locked */
- oldtup.t_data->t_ctid = oldtup.t_self;
+ oldtup->t_data->t_ctid = oldtup->t_self;
/*
* Clear all-frozen bit on visibility map if needed. We could
@@ -3887,7 +3740,7 @@ l2:
* worthwhile.
*/
if (PageIsAllVisible(page) &&
- visibilitymap_clear(relation, block, vmbuffer,
+ visibilitymap_clear(relation, block, *vmbuffer,
VISIBILITYMAP_ALL_FROZEN))
cleared_all_frozen = true;
@@ -3901,10 +3754,10 @@ l2:
XLogBeginInsert();
XLogRegisterBuffer(0, buffer, REGBUF_STANDARD);
- xlrec.offnum = ItemPointerGetOffsetNumber(&oldtup.t_self);
+ xlrec.offnum = ItemPointerGetOffsetNumber(&oldtup->t_self);
xlrec.xmax = xmax_lock_old_tuple;
- xlrec.infobits_set = compute_infobits(oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2);
+ xlrec.infobits_set = compute_infobits(oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2);
xlrec.flags =
cleared_all_frozen ? XLH_LOCK_ALL_FROZEN_CLEARED : 0;
XLogRegisterData(&xlrec, SizeOfHeapLock);
@@ -3926,7 +3779,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = heap_toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = heap_toast_insert_or_update(relation, newtup, oldtup, 0);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
@@ -3962,20 +3815,20 @@ l2:
/* It doesn't fit, must use RelationGetBufferForTuple. */
newbuf = RelationGetBufferForTuple(relation, heaptup->t_len,
buffer, 0, NULL,
- &vmbuffer_new, &vmbuffer,
+ &vmbuffer_new, vmbuffer,
0);
/* We're all done. */
break;
}
/* Acquire VM page pin if needed and we don't have it. */
- if (vmbuffer == InvalidBuffer && PageIsAllVisible(page))
- visibilitymap_pin(relation, block, &vmbuffer);
+ if (*vmbuffer == InvalidBuffer && PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, vmbuffer);
/* Re-acquire the lock on the old tuple's page. */
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
/* Re-check using the up-to-date free space */
pagefree = PageGetHeapFreeSpace(page);
if (newtupsize > pagefree ||
- (vmbuffer == InvalidBuffer && PageIsAllVisible(page)))
+ (*vmbuffer == InvalidBuffer && PageIsAllVisible(page)))
{
/*
* Rats, it doesn't fit anymore, or somebody just now set the
@@ -4013,7 +3866,7 @@ l2:
* will include checking the relation level, there is no benefit to a
* separate check for the new tuple.
*/
- CheckForSerializableConflictIn(relation, &oldtup.t_self,
+ CheckForSerializableConflictIn(relation, &oldtup->t_self,
BufferGetBlockNumber(buffer));
/*
@@ -4021,7 +3874,6 @@ l2:
* has enough space for the new tuple. If they are the same buffer, only
* one pin is held.
*/
-
if (newbuf == buffer)
{
/*
@@ -4029,7 +3881,7 @@ l2:
* to do a HOT update. Check if any of the index columns have been
* changed.
*/
- if (!bms_overlap(modified_attrs, hot_attrs))
+ if (!bms_overlap(mix_attrs, hot_attrs))
{
use_hot_update = true;
@@ -4040,7 +3892,7 @@ l2:
* indexes if the columns were updated, or we may fail to detect
* e.g. value bound changes in BRIN minmax indexes.
*/
- if (bms_overlap(modified_attrs, sum_attrs))
+ if (bms_overlap(mix_attrs, sum_attrs))
summarized_update = true;
}
}
@@ -4057,10 +3909,8 @@ l2:
* logged. Pass old key required as true only if the replica identity key
* columns are modified or it has external data.
*/
- old_key_tuple = ExtractReplicaIdentity(relation, &oldtup,
- bms_overlap(modified_attrs, id_attrs) ||
- id_has_external,
- &old_key_copied);
+ old_key_tuple = ExtractReplicaIdentity(relation, oldtup, rid_attrs,
+ rep_id_key_required, &old_key_copied);
/* NO EREPORT(ERROR) from here till changes are logged */
START_CRIT_SECTION();
@@ -4082,7 +3932,7 @@ l2:
if (use_hot_update)
{
/* Mark the old tuple as HOT-updated */
- HeapTupleSetHotUpdated(&oldtup);
+ HeapTupleSetHotUpdated(oldtup);
/* And mark the new tuple as heap-only */
HeapTupleSetHeapOnly(heaptup);
/* Mark the caller's copy too, in case different from heaptup */
@@ -4091,7 +3941,7 @@ l2:
else
{
/* Make sure tuples are correctly marked as not-HOT */
- HeapTupleClearHotUpdated(&oldtup);
+ HeapTupleClearHotUpdated(oldtup);
HeapTupleClearHeapOnly(heaptup);
HeapTupleClearHeapOnly(newtup);
}
@@ -4100,17 +3950,17 @@ l2:
/* Clear obsolete visibility flags, possibly set by ourselves above... */
- oldtup.t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
- oldtup.t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
+ oldtup->t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
+ oldtup->t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
/* ... and store info about transaction updating this tuple */
Assert(TransactionIdIsValid(xmax_old_tuple));
- HeapTupleHeaderSetXmax(oldtup.t_data, xmax_old_tuple);
- oldtup.t_data->t_infomask |= infomask_old_tuple;
- oldtup.t_data->t_infomask2 |= infomask2_old_tuple;
- HeapTupleHeaderSetCmax(oldtup.t_data, cid, iscombo);
+ HeapTupleHeaderSetXmax(oldtup->t_data, xmax_old_tuple);
+ oldtup->t_data->t_infomask |= infomask_old_tuple;
+ oldtup->t_data->t_infomask2 |= infomask2_old_tuple;
+ HeapTupleHeaderSetCmax(oldtup->t_data, cid, iscombo);
/* record address of new tuple in t_ctid of old one */
- oldtup.t_data->t_ctid = heaptup->t_self;
+ oldtup->t_data->t_ctid = heaptup->t_self;
/* clear PD_ALL_VISIBLE flags, reset all visibilitymap bits */
if (PageIsAllVisible(BufferGetPage(buffer)))
@@ -4118,7 +3968,7 @@ l2:
all_visible_cleared = true;
PageClearAllVisible(BufferGetPage(buffer));
visibilitymap_clear(relation, BufferGetBlockNumber(buffer),
- vmbuffer, VISIBILITYMAP_VALID_BITS);
+ *vmbuffer, VISIBILITYMAP_VALID_BITS);
}
if (newbuf != buffer && PageIsAllVisible(BufferGetPage(newbuf)))
{
@@ -4143,12 +3993,12 @@ l2:
*/
if (RelationIsAccessibleInLogicalDecoding(relation))
{
- log_heap_new_cid(relation, &oldtup);
+ log_heap_new_cid(relation, oldtup);
log_heap_new_cid(relation, heaptup);
}
recptr = log_heap_update(relation, buffer,
- newbuf, &oldtup, heaptup,
+ newbuf, oldtup, heaptup,
old_key_tuple,
all_visible_cleared,
all_visible_cleared_new);
@@ -4173,7 +4023,7 @@ l2:
* both tuple versions in one call to inval.c so we can avoid redundant
* sinval messages.)
*/
- CacheInvalidateHeapTuple(relation, &oldtup, heaptup);
+ CacheInvalidateHeapTuple(relation, oldtup, heaptup);
/* Now we can release the buffer(s) */
if (newbuf != buffer)
@@ -4181,14 +4031,14 @@ l2:
ReleaseBuffer(buffer);
if (BufferIsValid(vmbuffer_new))
ReleaseBuffer(vmbuffer_new);
- if (BufferIsValid(vmbuffer))
- ReleaseBuffer(vmbuffer);
+ if (BufferIsValid(*vmbuffer))
+ ReleaseBuffer(*vmbuffer);
/*
* Release the lmgr tuple lock, if we had it.
*/
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &oldtup->t_self, *lockmode);
pgstat_count_heap_update(relation, use_hot_update, newbuf != buffer);
@@ -4221,13 +4071,6 @@ l2:
if (old_key_tuple != NULL && old_key_copied)
heap_freetuple(old_key_tuple);
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
-
return TM_Ok;
}
@@ -4236,7 +4079,7 @@ l2:
* Confirm adequate lock held during heap_update(), per rules from
* README.tuplock section "Locking to write inplace-updated tables".
*/
-static void
+void
check_lock_if_inplace_updateable_rel(Relation relation,
const ItemPointerData *otid,
HeapTuple newtup)
@@ -4408,7 +4251,7 @@ heap_attr_equals(TupleDesc tupdesc, int attrnum, Datum value1, Datum value2,
* listed as interesting) of the old tuple is a member of external_cols and is
* stored externally.
*/
-static Bitmapset *
+Bitmapset *
HeapDetermineColumnsInfo(Relation relation,
Bitmapset *interesting_cols,
Bitmapset *external_cols,
@@ -4491,25 +4334,175 @@ HeapDetermineColumnsInfo(Relation relation,
}
/*
- * simple_heap_update - replace a tuple
- *
- * This routine may be used to update a tuple when concurrent updates of
- * the target tuple are not expected (for example, because we have a lock
- * on the relation associated with the tuple). Any failure is reported
- * via ereport().
+ * This routine may be used to update a tuple when concurrent updates of the
+ * target tuple are not expected (for example, because we have a lock on the
+ * relation associated with the tuple). Any failure is reported via ereport().
*/
void
-simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup,
+simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TM_FailureData tmfd;
LockTupleMode lockmode;
+ Buffer buffer;
+ Buffer vmbuffer = InvalidBuffer;
+ Page page;
+ BlockNumber block;
+ Bitmapset *hot_attrs,
+ *sum_attrs,
+ *pk_attrs,
+ *rid_attrs,
+ *mix_attrs,
+ *idx_attrs;
+ ItemId lp;
+ HeapTupleData oldtup;
+ bool rep_id_key_required = false;
+
+ Assert(ItemPointerIsValid(otid));
+
+ /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
+ Assert(HeapTupleHeaderGetNatts(tuple->t_data) <=
+ RelationGetNumberOfAttributes(relation));
+
+ /*
+ * Forbid this during a parallel operation, lest it allocate a combo CID.
+ * Other workers might need that combo CID for visibility checks, and we
+ * have no provision for broadcasting it to them.
+ */
+ if (IsInParallelMode())
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
+ errmsg("cannot update tuples during a parallel operation")));
+
+#ifdef USE_ASSERT_CHECKING
+ check_lock_if_inplace_updateable_rel(relation, otid, tuple);
+#endif
+
+ /*
+ * Fetch the list of attributes to be checked for various operations.
+ *
+ * For HOT considerations, this is wasted effort if we fail to update or
+ * have to put the new tuple on a different page. But we must compute the
+ * list before obtaining buffer lock --- in the worst case, if we are
+ * doing an update on one of the relevant system catalogs, we could
+ * deadlock if we try to fetch the list later. In any case, the relcache
+ * caches the data so this is usually pretty cheap.
+ *
+ * We also need columns used by the replica identity and columns that are
+ * considered the "key" of rows in the table.
+ *
+ * Note that we get copies of each bitmap, so we need not worry about
+ * relcache flush happening midway through.
+ */
+ hot_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_HOT_BLOCKING);
+ sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+ pk_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
+ rid_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
+ idx_attrs = bms_copy(hot_attrs);
+ idx_attrs = bms_add_members(idx_attrs, sum_attrs);
+ idx_attrs = bms_add_members(idx_attrs, pk_attrs);
+ idx_attrs = bms_add_members(idx_attrs, rid_attrs);
+
+ block = ItemPointerGetBlockNumber(otid);
+ INJECTION_POINT("heap_update-before-pin", NULL);
+ buffer = ReadBuffer(relation, block);
+ page = BufferGetPage(buffer);
+
+ /*
+ * Before locking the buffer, pin the visibility map page if it appears to
+ * be necessary. Since we haven't got the lock yet, someone else might be
+ * in the middle of changing this, so we'll need to recheck after we have
+ * the lock.
+ */
+ if (PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, &vmbuffer);
+
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+
+ lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
+
+ /*
+ * Usually, a buffer pin and/or snapshot blocks pruning of otid, ensuring
+ * we see LP_NORMAL here. When the otid origin is a syscache, we may have
+ * neither a pin nor a snapshot. Hence, we may see other LP_ states, each
+ * of which indicates concurrent pruning.
+ *
+ * Failing with TM_Updated would be most accurate. However, unlike other
+ * TM_Updated scenarios, we don't know the successor ctid in LP_UNUSED and
+ * LP_DEAD cases. While the distinction between TM_Updated and TM_Deleted
+ * does matter to SQL statements UPDATE and MERGE, those SQL statements
+ * hold a snapshot that ensures LP_NORMAL. Hence, the choice between
+ * TM_Updated and TM_Deleted affects only the wording of error messages.
+ * Settle on TM_Deleted, for two reasons. First, it avoids complicating
+ * the specification of when tmfd->ctid is valid. Second, it creates
+ * error log evidence that we took this branch.
+ *
+ * Since it's possible to see LP_UNUSED at otid, it's also possible to see
+ * LP_NORMAL for a tuple that replaced LP_UNUSED. If it's a tuple for an
+ * unrelated row, we'll fail with "duplicate key value violates unique".
+ * XXX if otid is the live, newer version of the newtup row, we'll discard
+ * changes originating in versions of this catalog row after the version
+ * the caller got from syscache. See syscache-update-pruned.spec.
+ */
+ if (!ItemIdIsNormal(lp))
+ {
+ Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
+
+ UnlockReleaseBuffer(buffer);
+ if (vmbuffer != InvalidBuffer)
+ ReleaseBuffer(vmbuffer);
+ *update_indexes = TU_None;
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(idx_attrs);
+ /* mix_attrs not yet initialized */
+
+ elog(ERROR, "tuple concurrently deleted");
+
+ return;
+ }
+
+ /*
+ * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
+ * then pass that on to heap_update.
+ */
+ oldtup.t_tableOid = RelationGetRelid(relation);
+ oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
+ oldtup.t_len = ItemIdGetLength(lp);
+ oldtup.t_self = *otid;
+
+ mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
+ &oldtup, tuple, &rep_id_key_required);
+
+ /*
+ * We'll need to WAL log the replica identity attributes if either they
+ * overlap with the modified indexed attributes or, as we've checked for
+ * just now in HeapDetermineColumnsInfo, they were unmodified external
+ * indexed attributes.
+ */
+ rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+
+ result = heap_update(relation, &oldtup, tuple, GetCurrentCommandId(true),
+ InvalidSnapshot, true /* wait for commit */ , &tmfd, &lockmode,
+ buffer, page, block, lp, hot_attrs, sum_attrs, pk_attrs,
+ rid_attrs, mix_attrs, &vmbuffer, rep_id_key_required,
+ update_indexes);
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(mix_attrs);
+ bms_free(idx_attrs);
- result = heap_update(relation, otid, tup,
- GetCurrentCommandId(true), InvalidSnapshot,
- true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
switch (result)
{
case TM_SelfModified:
@@ -9149,12 +9142,11 @@ log_heap_new_cid(Relation relation, HeapTuple tup)
* the same tuple that was passed in.
*/
static HeapTuple
-ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
- bool *copy)
+ExtractReplicaIdentity(Relation relation, HeapTuple tp, Bitmapset *rid_attrs,
+ bool key_required, bool *copy)
{
TupleDesc desc = RelationGetDescr(relation);
char replident = relation->rd_rel->relreplident;
- Bitmapset *idattrs;
HeapTuple key_tuple;
bool nulls[MaxHeapAttributeNumber];
Datum values[MaxHeapAttributeNumber];
@@ -9185,17 +9177,13 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
if (!key_required)
return NULL;
- /* find out the replica identity columns */
- idattrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
-
/*
* If there's no defined replica identity columns, treat as !key_required.
* (This case should not be reachable from heap_update, since that should
* calculate key_required accurately. But heap_delete just passes
* constant true for key_required, so we can hit this case in deletes.)
*/
- if (bms_is_empty(idattrs))
+ if (bms_is_empty(rid_attrs))
return NULL;
/*
@@ -9208,7 +9196,7 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
for (int i = 0; i < desc->natts; i++)
{
if (bms_is_member(i + 1 - FirstLowInvalidHeapAttributeNumber,
- idattrs))
+ rid_attrs))
Assert(!nulls[i]);
else
nulls[i] = true;
@@ -9217,8 +9205,6 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
key_tuple = heap_form_tuple(desc, values, nulls);
*copy = true;
- bms_free(idattrs);
-
/*
* If the tuple, which by here only contains indexed columns, still has
* toasted columns, force them to be inlined. This is somewhat unlikely
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bcbac844bb6..1cf9a18775d 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -44,6 +44,7 @@
#include "storage/procarray.h"
#include "storage/smgr.h"
#include "utils/builtins.h"
+#include "utils/injection_point.h"
#include "utils/rel.h"
static void reform_and_rewrite_tuple(HeapTuple tuple,
@@ -312,23 +313,133 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
return heap_delete(relation, tid, cid, crosscheck, wait, tmfd, changingPart);
}
-
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd,
LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
{
+ bool rep_id_key_required = false;
bool shouldFree = true;
HeapTuple tuple = ExecFetchSlotHeapTuple(slot, true, &shouldFree);
+ HeapTupleData oldtup;
+ Buffer buffer;
+ Buffer vmbuffer = InvalidBuffer;
+ Page page;
+ BlockNumber block;
+ ItemId lp;
+ Bitmapset *hot_attrs,
+ *sum_attrs,
+ *pk_attrs,
+ *rid_attrs,
+ *mix_attrs,
+ *idx_attrs;
TM_Result result;
+ Assert(ItemPointerIsValid(otid));
+
+ /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
+ Assert(HeapTupleHeaderGetNatts(tuple->t_data) <=
+ RelationGetNumberOfAttributes(relation));
+
+ /*
+ * Forbid this during a parallel operation, lest it allocate a combo CID.
+ * Other workers might need that combo CID for visibility checks, and we
+ * have no provision for broadcasting it to them.
+ */
+ if (IsInParallelMode())
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
+ errmsg("cannot update tuples during a parallel operation")));
+
+#ifdef USE_ASSERT_CHECKING
+ check_lock_if_inplace_updateable_rel(relation, otid, tuple);
+#endif
+
+ /*
+ * Fetch the list of attributes to be checked for various operations.
+ *
+ * For HOT considerations, this is wasted effort if we fail to update or
+ * have to put the new tuple on a different page. But we must compute the
+ * list before obtaining buffer lock --- in the worst case, if we are
+ * doing an update on one of the relevant system catalogs, we could
+ * deadlock if we try to fetch the list later. In any case, the relcache
+ * caches the data so this is usually pretty cheap.
+ *
+ * We also need columns used by the replica identity and columns that are
+ * considered the "key" of rows in the table.
+ *
+ * Note that we get copies of each bitmap, so we need not worry about
+ * relcache flush happening midway through.
+ */
+ hot_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_HOT_BLOCKING);
+ sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+ pk_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
+ rid_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
+ idx_attrs = bms_copy(hot_attrs);
+ idx_attrs = bms_add_members(idx_attrs, sum_attrs);
+ idx_attrs = bms_add_members(idx_attrs, pk_attrs);
+ idx_attrs = bms_add_members(idx_attrs, rid_attrs);
+
+ block = ItemPointerGetBlockNumber(otid);
+ INJECTION_POINT("heap_update-before-pin", NULL);
+ buffer = ReadBuffer(relation, block);
+ page = BufferGetPage(buffer);
+
+ /*
+ * Before locking the buffer, pin the visibility map page if it appears to
+ * be necessary. Since we haven't got the lock yet, someone else might be
+ * in the middle of changing this, so we'll need to recheck after we have
+ * the lock.
+ */
+ if (PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, &vmbuffer);
+
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+
+ lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
+
+ Assert(ItemIdIsNormal(lp));
+
+ /*
+ * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
+ * then pass that on to heap_update.
+ */
+ oldtup.t_tableOid = RelationGetRelid(relation);
+ oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
+ oldtup.t_len = ItemIdGetLength(lp);
+ oldtup.t_self = *otid;
+
+ mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
+ &oldtup, tuple, &rep_id_key_required);
+
+ /*
+ * We'll need to WAL log the replica identity attributes if either they
+ * overlap with the modified indexed attributes or, as we've checked for
+ * just now in HeapDetermineColumnsInfo, they were unmodified external
+ * indexed attributes.
+ */
+ rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
- result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
- tmfd, lockmode, update_indexes);
+ result = heap_update(relation, &oldtup, tuple, cid, crosscheck, wait, tmfd, lockmode,
+ buffer, page, block, lp, hot_attrs, sum_attrs, pk_attrs,
+ rid_attrs, mix_attrs, &vmbuffer, rep_id_key_required, update_indexes);
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(mix_attrs);
+ bms_free(idx_attrs);
+
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 909db73b7bb..41d541aa6b2 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -321,11 +321,13 @@ extern TM_Result heap_delete(Relation relation, const ItemPointerData *tid,
TM_FailureData *tmfd, bool changingPart);
extern void heap_finish_speculative(Relation relation, const ItemPointerData *tid);
extern void heap_abort_speculative(Relation relation, const ItemPointerData *tid);
-extern TM_Result heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+extern TM_Result heap_update(Relation relation, HeapTupleData *oldtup,
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
+ Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
+ Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
+ Bitmapset *mix_attrs, Buffer *vmbuffer,
+ bool rep_id_key_required, TU_UpdateIndexes *update_indexes);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -391,6 +393,18 @@ extern void log_heap_prune_and_freeze(Relation relation, Buffer buffer,
OffsetNumber *dead, int ndead,
OffsetNumber *unused, int nunused);
+/* in heap/heapam.c */
+extern Bitmapset *HeapDetermineColumnsInfo(Relation relation,
+ Bitmapset *interesting_cols,
+ Bitmapset *external_cols,
+ HeapTuple oldtup, HeapTuple newtup,
+ bool *has_external);
+#ifdef USE_ASSERT_CHECKING
+extern void check_lock_if_inplace_updateable_rel(Relation relation,
+ const ItemPointerData *otid,
+ HeapTuple newtup);
+#endif
+
/* in heap/vacuumlazy.c */
extern void heap_vacuum_rel(Relation rel,
const VacuumParams params, BufferAccessStrategy bstrategy);
--
2.49.0
[application/octet-stream] v19-0002-Track-changed-indexed-columns-in-the-executor-du.patch (34.3K, 3-v19-0002-Track-changed-indexed-columns-in-the-executor-du.patch)
download | inline diff:
From 69d226a736a77ca83e8df2e617226552c431da13 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 26 Oct 2025 10:49:25 -0400
Subject: [PATCH v19 2/4] Track changed indexed columns in the executor during
UPDATEs
Refactor executor update logic to determine which indexed columns have
actually changed during an UPDATE operation rather than leaving this up
to HeapDetermineColumnsInfo in heap_update. This enables the comparison
to happen without taking a lock on the page and opens the door to reuse
in other code paths.
Because heap_update now requires the caller to provide the modified
indexed columns simple_heap_update has become a tad more complex. It is
frequently called from CatalogTupleUpdate which either updates heap
tuples via their form or using heap_modify_tuple. In both cases the
caller does know the modified set of attributes, but sadly those
attributes are lost before being provided to simple_heap_update. Due to
that the "simple" path has to retain the HeapDetermineColumnsInfo logic
of old (for now). In order for that to work it was necessary to split
the (overly large) heap_update call itself up. This moves up into
simple_heap_update and heap_tuple_update a bit of what existed in
heap_update itself. Ideally this will be cleaned up once
CatalogTupleUpdate paths are all recording modified attributes
correctly, when that happens the "simple" path can be simplified again.
ExecCheckIndexedAttrsForChanges replaces HeapDeterminesColumnsInfo and
tts_attr_equal replaces heap_attr_equal changing the test for equality
when calling into heap_tuple_update (but not simple_heap_update). In
the past we used datumIsEqual(), essentially a binary comparison using
memcmp(), now the comparison code in tts_attr_equal uses type-specific
equality function when available and falls back to datumIsEqual() when
not. This change in equality testing has some intended implications and
opens the door for more HOT updates (foreshadowing). For instance,
indexes with collation information allowing more HOT updates when the
index is specified to be case insensitive.
This change forced some logic changes in execReplication on the update
paths is now it is required to have knowledge of the set of attributes
that are both changed and referenced by indexes. Luckilly, the this is
available within calls to slot_modify_data() where LogicalRepTupleData
is processed and has a set of updated attributes. In this case rather
than using ExecCheckIndexedAttrsForChanges we can preseve what
slot_modify_data() identifies as the modified set and then intersect
that with the set of indexes on the relation and get the correct set of
modified indexed attributes required on heap_update().
---
src/backend/access/heap/heapam.c | 12 +-
src/backend/access/heap/heapam_handler.c | 72 +++++--
src/backend/access/table/tableam.c | 5 +-
src/backend/executor/execMain.c | 1 +
src/backend/executor/execReplication.c | 7 +
src/backend/executor/nodeModifyTable.c | 247 ++++++++++++++++++++++-
src/backend/nodes/bitmapset.c | 4 +
src/backend/replication/logical/worker.c | 72 ++++++-
src/backend/utils/cache/relcache.c | 15 ++
src/include/access/tableam.h | 8 +-
src/include/executor/executor.h | 5 +
src/include/nodes/execnodes.h | 1 +
src/include/utils/rel.h | 1 +
src/include/utils/relcache.h | 1 +
14 files changed, 415 insertions(+), 36 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index aff47481345..1cdb72b3a7a 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -3263,12 +3263,12 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* generated by another transaction).
*/
TM_Result
-heap_update(Relation relation, HeapTupleData *oldtup,
- HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
- Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
- Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
- Bitmapset *mix_attrs, Buffer *vmbuffer,
+heap_update(Relation relation, HeapTupleData *oldtup, HeapTuple newtup,
+ CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode,
+ Buffer buffer, Page page, BlockNumber block, ItemId lp,
+ Bitmapset *hot_attrs, Bitmapset *sum_attrs, Bitmapset *pk_attrs,
+ Bitmapset *rid_attrs, Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 1cf9a18775d..ef08e1d3e10 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -315,9 +315,12 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
- CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd,
- LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
+ CommandId cid, Snapshot snapshot,
+ Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd,
+ LockTupleMode *lockmode,
+ Bitmapset *mix_attrs,
+ TU_UpdateIndexes *update_indexes)
{
bool rep_id_key_required = false;
bool shouldFree = true;
@@ -332,7 +335,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
*sum_attrs,
*pk_attrs,
*rid_attrs,
- *mix_attrs,
*idx_attrs;
TM_Result result;
@@ -414,16 +416,61 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
oldtup.t_len = ItemIdGetLength(lp);
oldtup.t_self = *otid;
- mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
- &oldtup, tuple, &rep_id_key_required);
-
/*
- * We'll need to WAL log the replica identity attributes if either they
- * overlap with the modified indexed attributes or, as we've checked for
- * just now in HeapDetermineColumnsInfo, they were unmodified external
- * indexed attributes.
+ * We'll need to include the replica identity key when either the identity
+ * key attributes overlap with the modified index attributes or when the
+ * replica identity attributes are stored externally. This is required
+ * because for such attributes the flattened value won't be WAL logged as
+ * part of the new tuple so we must determine if we need to extract and
+ * include them as part of the old_key_tuple (see ExtractReplicaIdentity).
*/
- rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+ rep_id_key_required = bms_overlap(mix_attrs, rid_attrs);
+ if (!rep_id_key_required)
+ {
+ Bitmapset *attrs;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ int attidx = -1;
+
+ /*
+ * We don't own idx_attrs so we'll copy it and remove the modified set
+ * to reduce the attributes we need to test in the while loop and
+ * avoid a two branches in the loop.
+ */
+ attrs = bms_difference(idx_attrs, mix_attrs);
+ attrs = bms_int_members(attrs, rid_attrs);
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /*
+ * attidx is zero-based, attrnum is the normal attribute number
+ */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value;
+ bool isnull;
+
+ /*
+ * System attributes are not added into interesting_attrs in
+ * relcache
+ */
+ Assert(attrnum > 0);
+
+ value = heap_getattr(&oldtup, attrnum, tupdesc, &isnull);
+
+ /* No need to check attributes that can't be stored externally */
+ if (isnull ||
+ TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
+ continue;
+
+ /* Check if the old tuple's attribute is stored externally */
+ if (VARATT_IS_EXTERNAL((struct varlena *) DatumGetPointer(value)))
+ {
+ rep_id_key_required = true;
+ break;
+ }
+ }
+
+ bms_free(attrs);
+ }
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
@@ -437,7 +484,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
bms_free(sum_attrs);
bms_free(pk_attrs);
bms_free(rid_attrs);
- bms_free(mix_attrs);
bms_free(idx_attrs);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 5e41404937e..dadcf03ed24 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -336,6 +336,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
+ Bitmapset *modified_indexed_cols,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -346,7 +347,9 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
GetCurrentCommandId(true),
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, &lockmode,
+ modified_indexed_cols,
+ update_indexes);
switch (result)
{
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 27c9eec697b..6b7b6bc8019 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1282,6 +1282,7 @@ InitResultRelInfo(ResultRelInfo *resultRelInfo,
/* The following fields are set later if needed */
resultRelInfo->ri_RowIdAttNo = 0;
resultRelInfo->ri_extraUpdatedCols = NULL;
+ resultRelInfo->ri_ChangedIndexedCols = NULL;
resultRelInfo->ri_projectNew = NULL;
resultRelInfo->ri_newTupleSlot = NULL;
resultRelInfo->ri_oldTupleSlot = NULL;
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index def32774c90..2709e2db0f2 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -32,6 +32,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
#include "utils/typcache.h"
@@ -936,7 +937,13 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ /*
+ * We're not going to call ExecCheckIndexedAttrsForChanges here
+ * because we've already identified the changes earlier on thanks to
+ * slot_modify_data.
+ */
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
+ resultRelInfo->ri_ChangedIndexedCols,
&update_indexes);
conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 4c5647ac38a..4d1cf50e369 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -17,6 +17,7 @@
* ExecModifyTable - retrieve the next tuple from the node
* ExecEndModifyTable - shut down the ModifyTable node
* ExecReScanModifyTable - rescan the ModifyTable node
+ * ExecCheckIndexedAttrsForChanges - find set of updated indexed columns
*
* NOTES
* The ModifyTable node receives input from its outerPlan, which is
@@ -54,11 +55,14 @@
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/tupconvert.h"
+#include "access/tupdesc.h"
#include "access/xact.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
#include "executor/nodeModifyTable.h"
+#include "executor/tuptable.h"
#include "foreign/fdwapi.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
@@ -68,6 +72,8 @@
#include "storage/lmgr.h"
#include "utils/builtins.h"
#include "utils/datum.h"
+#include "utils/float.h"
+#include "utils/lsyscache.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
@@ -176,6 +182,219 @@ static TupleTableSlot *ExecMergeNotMatched(ModifyTableContext *context,
bool canSetTag);
+/*
+ * Compare two datums using the type's default equality operator.
+ *
+ * Returns true if the values are equal according to the type's equality
+ * operator, false otherwise. Falls back to binary comparison if no
+ * type-specific operator is available.
+ *
+ * This function uses the TypeCache infrastructure which caches operator
+ * lookups for efficiency.
+ */
+bool
+tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
+ Datum value1, Datum value2)
+{
+ TypeCacheEntry *typentry;
+
+ LOCAL_FCINFO(fcinfo, 2);
+ Datum result;
+
+ /*
+ * Fast path for common types to avoid even the type cache lookup. These
+ * types have simple equality semantics.
+ */
+ switch (typid)
+ {
+ case INT2OID:
+ return DatumGetInt16(value1) == DatumGetInt16(value2);
+ case INT4OID:
+ return DatumGetInt32(value1) == DatumGetInt32(value2);
+ case INT8OID:
+ return DatumGetInt64(value1) == DatumGetInt64(value2);
+ case FLOAT4OID:
+ return !float4_cmp_internal(DatumGetFloat4(value1), DatumGetFloat4(value2));
+ case FLOAT8OID:
+ return !float8_cmp_internal(DatumGetFloat8(value1), DatumGetFloat8(value2));
+ case BOOLOID:
+ return DatumGetBool(value1) == DatumGetBool(value2);
+ case OIDOID:
+ case REGPROCOID:
+ case REGPROCEDUREOID:
+ case REGOPEROID:
+ case REGOPERATOROID:
+ case REGCLASSOID:
+ case REGTYPEOID:
+ case REGROLEOID:
+ case REGNAMESPACEOID:
+ case REGCONFIGOID:
+ case REGDICTIONARYOID:
+ return DatumGetObjectId(value1) == DatumGetObjectId(value2);
+ case CHAROID:
+ return DatumGetChar(value1) == DatumGetChar(value2);
+ default:
+ /* Continue to type cache lookup */
+ break;
+ }
+
+ /*
+ * Look up the type's equality operator using the type cache. Request both
+ * the operator OID and the function info for efficiency.
+ */
+ typentry = lookup_type_cache(typid,
+ TYPECACHE_EQ_OPR | TYPECACHE_EQ_OPR_FINFO);
+
+ /*
+ * If no equality operator is available, fall back to binary comparison.
+ * This handles types that don't have proper equality operators defined.
+ */
+ if (!OidIsValid(typentry->eq_opr))
+ return datumIsEqual(value1, value2, typbyval, typlen);
+
+ /*
+ * Use the cached function info if available, otherwise look it up. The
+ * type cache keeps this around so subsequent calls are fast.
+ */
+ if (typentry->eq_opr_finfo.fn_addr == NULL)
+ {
+ Oid eq_proc = get_opcode(typentry->eq_opr);
+
+ if (!OidIsValid(eq_proc))
+ /* Shouldn't happen, but fall back to binary comparison */
+ return datumIsEqual(value1, value2, typbyval, typlen);
+
+ fmgr_info_cxt(eq_proc, &typentry->eq_opr_finfo,
+ CacheMemoryContext);
+ }
+
+ /* Set up function call */
+ InitFunctionCallInfoData(*fcinfo, &typentry->eq_opr_finfo, 2,
+ collation, NULL, NULL);
+
+ fcinfo->args[0].value = value1;
+ fcinfo->args[0].isnull = false;
+ fcinfo->args[1].value = value2;
+ fcinfo->args[1].isnull = false;
+
+ /* Invoke the equality operator */
+ result = FunctionCallInvoke(fcinfo);
+
+ /*
+ * If the function returned NULL (shouldn't happen for equality ops),
+ * treat as not equal for safety.
+ */
+ if (fcinfo->isnull)
+ return false;
+
+ return DatumGetBool(result);
+}
+
+/*
+ * Determine which updated attributes actually changed values between old and
+ * new tuples and are referenced by indexes on the relation.
+ *
+ * Returns a Bitmapset of attribute offsets (0-based, adjusted by
+ * FirstLowInvalidHeapAttributeNumber) or NULL if no attributes changed.
+ */
+Bitmapset *
+ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
+ TupleTableSlot *tts_old,
+ TupleTableSlot *tts_new)
+{
+ Relation relation = relinfo->ri_RelationDesc;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ Bitmapset *indexed_attrs;
+ Bitmapset *modified = NULL;
+ int attidx;
+
+ /* If no indexes, we're done */
+ if (relinfo->ri_NumIndices == 0)
+ return NULL;
+
+ /*
+ * Get the set of index key attributes. This includes summarizing,
+ * expression indexes and attributes mentioned in the predicate of a
+ * partition but not those in INCLUDING.
+ */
+ indexed_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_INDEXED);
+ Assert(!bms_is_empty(indexed_attrs));
+
+ /*
+ * NOTE: It is important to scan all indexed attributes in the tuples
+ * because ExecGetAllUpdatedCols won't include columns that may have been
+ * modified via heap_modify_tuple_by_col which is the case in
+ * tsvector_update_trigger.
+ */
+ attidx = -1;
+ while ((attidx = bms_next_member(indexed_attrs, attidx)) >= 0)
+ {
+ /* attidx is zero-based, attrnum is the normal attribute number */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Form_pg_attribute attr;
+ bool oldnull,
+ newnull;
+ Datum oldval,
+ newval;
+
+ /*
+ * If it's a whole-tuple reference, record as modified. It's not
+ * really worth supporting this case, since it could only succeed
+ * after a no-op update, which is hardly a case worth optimizing for.
+ */
+ if (attrnum == 0)
+ {
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /*
+ * Likewise, include in the modified set any system attribute other
+ * than tableOID; we cannot expect these to be consistent in a HOT
+ * chain, or even to be set correctly yet in the new tuple.
+ */
+ if (attrnum < 0)
+ {
+ if (attrnum != TableOidAttributeNumber)
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /* Extract values from both slots */
+ oldval = slot_getattr(tts_old, attrnum, &oldnull);
+ newval = slot_getattr(tts_new, attrnum, &newnull);
+
+ /* If one value is NULL and the other is not, they are not equal */
+ if (oldnull != newnull)
+ {
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /* If both are NULL, consider them equal */
+ if (oldnull)
+ continue;
+
+ /* Get attribute metadata */
+ Assert(attrnum > 0 && attrnum <= tupdesc->natts);
+ attr = TupleDescAttr(tupdesc, attrnum - 1);
+
+ /* Compare using type-specific equality operator */
+ if (!tts_attr_equal(attr->atttypid,
+ attr->attcollation,
+ attr->attbyval,
+ attr->attlen,
+ oldval,
+ newval))
+ modified = bms_add_member(modified, attidx);
+ }
+
+ bms_free(indexed_attrs);
+
+ return modified;
+}
+
/*
* Verify that the tuples to be produced by INSERT match the
* target relation's rowtype
@@ -2168,8 +2387,8 @@ ExecUpdatePrepareSlot(ResultRelInfo *resultRelInfo,
*/
static TM_Result
ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
- ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *slot,
- bool canSetTag, UpdateContext *updateCxt)
+ ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *oldSlot,
+ TupleTableSlot *slot, bool canSetTag, UpdateContext *updateCxt)
{
EState *estate = context->estate;
Relation resultRelationDesc = resultRelInfo->ri_RelationDesc;
@@ -2291,6 +2510,16 @@ lreplace:
if (resultRelationDesc->rd_att->constr)
ExecConstraints(resultRelInfo, slot, estate);
+ /*
+ * Identify which, if any, indexed attributes were modified here so that
+ * we might reuse it in a few places.
+ */
+ bms_free(resultRelInfo->ri_ChangedIndexedCols);
+ resultRelInfo->ri_ChangedIndexedCols = NULL;
+
+ resultRelInfo->ri_ChangedIndexedCols =
+ ExecCheckIndexedAttrsForChanges(resultRelInfo, oldSlot, slot);
+
/*
* replace the heap tuple
*
@@ -2306,6 +2535,7 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
+ resultRelInfo->ri_ChangedIndexedCols,
&updateCxt->updateIndexes);
return result;
@@ -2524,8 +2754,9 @@ ExecUpdate(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
*/
redo_act:
lockedtid = *tupleid;
- result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, slot,
- canSetTag, &updateCxt);
+
+ result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, oldSlot,
+ slot, canSetTag, &updateCxt);
/*
* If ExecUpdateAct reports that a cross-partition update was done,
@@ -3222,8 +3453,8 @@ lmerge_matched:
Assert(oldtuple == NULL);
result = ExecUpdateAct(context, resultRelInfo, tupleid,
- NULL, newslot, canSetTag,
- &updateCxt);
+ NULL, resultRelInfo->ri_oldTupleSlot,
+ newslot, canSetTag, &updateCxt);
/*
* As in ExecUpdate(), if ExecUpdateAct() reports that a
@@ -3248,6 +3479,7 @@ lmerge_matched:
tupleid, NULL, newslot);
mtstate->mt_merge_updated += 1;
}
+
break;
case CMD_DELETE:
@@ -4333,7 +4565,7 @@ ExecModifyTable(PlanState *pstate)
* For UPDATE/DELETE/MERGE, fetch the row identity info for the tuple
* to be updated/deleted/merged. For a heap relation, that's a TID;
* otherwise we may have a wholerow junk attr that carries the old
- * tuple in toto. Keep this in step with the part of
+ * tuple in total. Keep this in step with the part of
* ExecInitModifyTable that sets up ri_RowIdAttNo.
*/
if (operation == CMD_UPDATE || operation == CMD_DELETE ||
@@ -4509,6 +4741,7 @@ ExecModifyTable(PlanState *pstate)
/* Now apply the update. */
slot = ExecUpdate(&context, resultRelInfo, tupleid, oldtuple,
oldSlot, slot, node->canSetTag);
+
if (tuplock)
UnlockTuple(resultRelInfo->ri_RelationDesc, tupleid,
InplaceUpdateTupleLock);
diff --git a/src/backend/nodes/bitmapset.c b/src/backend/nodes/bitmapset.c
index b4ecf0b0390..9014990267a 100644
--- a/src/backend/nodes/bitmapset.c
+++ b/src/backend/nodes/bitmapset.c
@@ -238,6 +238,10 @@ bms_make_singleton(int x)
void
bms_free(Bitmapset *a)
{
+#if USE_ASSERT_CHECKING
+ Assert(bms_is_valid_set(a));
+#endif
+
if (a)
pfree(a);
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 93970c6af29..b363eaa49cc 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -243,6 +243,8 @@
*/
#include "postgres.h"
+#include "access/sysattr.h"
+#include "nodes/bitmapset.h"
#include <sys/stat.h>
#include <unistd.h>
@@ -275,7 +277,6 @@
#include "replication/logicalrelation.h"
#include "replication/logicalworker.h"
#include "replication/origin.h"
-#include "replication/slot.h"
#include "replication/walreceiver.h"
#include "replication/worker_internal.h"
#include "rewrite/rewriteHandler.h"
@@ -291,6 +292,7 @@
#include "utils/memutils.h"
#include "utils/pg_lsn.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/rls.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
@@ -1110,15 +1112,18 @@ slot_store_data(TupleTableSlot *slot, LogicalRepRelMapEntry *rel,
* "slot" is filled with a copy of the tuple in "srcslot", replacing
* columns provided in "tupleData" and leaving others as-is.
*
+ * Returns a bitmap of the modified columns.
+ *
* Caution: unreplaced pass-by-ref columns in "slot" will point into the
* storage for "srcslot". This is OK for current usage, but someday we may
* need to materialize "slot" at the end to make it independent of "srcslot".
*/
-static void
+static Bitmapset *
slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
LogicalRepRelMapEntry *rel,
LogicalRepTupleData *tupleData)
{
+ Bitmapset *modified = NULL;
int natts = slot->tts_tupleDescriptor->natts;
int i;
@@ -1195,6 +1200,28 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
slot->tts_isnull[i] = true;
}
+ /*
+ * Determine if the replicated value changed the local value by
+ * comparing slots. This is a subset of
+ * ExecCheckIndexedAttrsForChanges.
+ */
+ if (srcslot->tts_isnull[i] != slot->tts_isnull[i])
+ {
+ /* One is NULL, the other is not so the value changed */
+ modified = bms_add_member(modified, i + 1 - FirstLowInvalidHeapAttributeNumber);
+ }
+ else if (!srcslot->tts_isnull[i])
+ {
+ /* Both are not NULL, compare their values */
+ if (!tts_attr_equal(att->atttypid,
+ att->attcollation,
+ att->attbyval,
+ att->attlen,
+ srcslot->tts_values[i],
+ slot->tts_values[i]))
+ modified = bms_add_member(modified, i + 1 - FirstLowInvalidHeapAttributeNumber);
+ }
+
/* Reset attnum for error callback */
apply_error_callback_arg.remote_attnum = -1;
}
@@ -1202,6 +1229,8 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
/* And finally, declare that "slot" contains a valid virtual tuple */
ExecStoreVirtualTuple(slot);
+
+ return modified;
}
/*
@@ -2918,6 +2947,7 @@ apply_handle_update_internal(ApplyExecutionData *edata,
ConflictTupleInfo conflicttuple = {0};
bool found;
MemoryContext oldctx;
+ Bitmapset *indexed = NULL;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
ExecOpenIndices(relinfo, false);
@@ -2934,6 +2964,8 @@ apply_handle_update_internal(ApplyExecutionData *edata,
*/
if (found)
{
+ Bitmapset *modified = NULL;
+
/*
* Report the conflict if the tuple was modified by a different
* origin.
@@ -2957,15 +2989,29 @@ apply_handle_update_internal(ApplyExecutionData *edata,
/* Process and store remote tuple in the slot */
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot, localslot, relmapentry, newtup);
+ modified = slot_modify_data(remoteslot, localslot, relmapentry, newtup);
MemoryContextSwitchTo(oldctx);
+ /*
+ * Normally we'd call ExecCheckIndexedAttrForChanges but here we have
+ * the record of changed columns in the replication state, so let's
+ * use that instead.
+ */
+ indexed = RelationGetIndexAttrBitmap(relinfo->ri_RelationDesc,
+ INDEX_ATTR_BITMAP_INDEXED);
+
+ bms_free(relinfo->ri_ChangedIndexedCols);
+ relinfo->ri_ChangedIndexedCols = bms_int_members(modified, indexed);
+ bms_free(indexed);
+
EvalPlanQualSetSlot(&epqstate, remoteslot);
InitConflictIndexes(relinfo);
- /* Do the actual update. */
+ /* First check privileges */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
+
+ /* Then do the actual update. */
ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
remoteslot);
}
@@ -3455,6 +3501,8 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
bool found;
EPQState epqstate;
ConflictTupleInfo conflicttuple = {0};
+ Bitmapset *modified = NULL;
+ Bitmapset *indexed;
/* Get the matching local tuple from the partition. */
found = FindReplTupleInLocalRel(edata, partrel,
@@ -3523,8 +3571,8 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
* remoteslot_part.
*/
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot_part, localslot, part_entry,
- newtup);
+ modified = slot_modify_data(remoteslot_part, localslot, part_entry,
+ newtup);
MemoryContextSwitchTo(oldctx);
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
@@ -3549,6 +3597,18 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
EvalPlanQualSetSlot(&epqstate, remoteslot_part);
TargetPrivilegesCheck(partrelinfo->ri_RelationDesc,
ACL_UPDATE);
+
+ /*
+ * Normally we'd call ExecCheckIndexedAttrForChanges but
+ * here we have the record of changed columns in the
+ * replication state, so let's use that instead.
+ */
+ indexed = RelationGetIndexAttrBitmap(partrelinfo->ri_RelationDesc,
+ INDEX_ATTR_BITMAP_INDEXED);
+ bms_free(partrelinfo->ri_ChangedIndexedCols);
+ partrelinfo->ri_ChangedIndexedCols = bms_int_members(modified, indexed);
+ bms_free(indexed);
+
ExecSimpleRelationUpdate(partrelinfo, estate, &epqstate,
localslot, remoteslot_part);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 915d0bc9084..32825596be1 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -2482,6 +2482,7 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
bms_free(relation->rd_idattr);
bms_free(relation->rd_hotblockingattr);
bms_free(relation->rd_summarizedattr);
+ bms_free(relation->rd_indexedattr);
if (relation->rd_pubdesc)
pfree(relation->rd_pubdesc);
if (relation->rd_options)
@@ -5283,6 +5284,7 @@ RelationGetIndexPredicate(Relation relation)
* index (empty if FULL)
* INDEX_ATTR_BITMAP_HOT_BLOCKING Columns that block updates from being HOT
* INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
+ * INDEX_ATTR_BITMAP_INDEXED Columns referenced by indexes
*
* Attribute numbers are offset by FirstLowInvalidHeapAttributeNumber so that
* we can include system attributes (e.g., OID) in the bitmap representation.
@@ -5307,6 +5309,7 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
Bitmapset *idindexattrs; /* columns in the replica identity */
Bitmapset *hotblockingattrs; /* columns with HOT blocking indexes */
Bitmapset *summarizedattrs; /* columns with summarizing indexes */
+ Bitmapset *indexedattrs; /* columns referenced by indexes */
List *indexoidlist;
List *newindexoidlist;
Oid relpkindex;
@@ -5329,6 +5332,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
return bms_copy(relation->rd_hotblockingattr);
case INDEX_ATTR_BITMAP_SUMMARIZED:
return bms_copy(relation->rd_summarizedattr);
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return bms_copy(relation->rd_indexedattr);
default:
elog(ERROR, "unknown attrKind %u", attrKind);
}
@@ -5373,6 +5378,7 @@ restart:
idindexattrs = NULL;
hotblockingattrs = NULL;
summarizedattrs = NULL;
+ indexedattrs = NULL;
foreach(l, indexoidlist)
{
Oid indexOid = lfirst_oid(l);
@@ -5505,10 +5511,14 @@ restart:
bms_free(idindexattrs);
bms_free(hotblockingattrs);
bms_free(summarizedattrs);
+ bms_free(indexedattrs);
goto restart;
}
+ /* Combine all index attributes */
+ indexedattrs = bms_union(hotblockingattrs, summarizedattrs);
+
/* Don't leak the old values of these bitmaps, if any */
relation->rd_attrsvalid = false;
bms_free(relation->rd_keyattr);
@@ -5521,6 +5531,8 @@ restart:
relation->rd_hotblockingattr = NULL;
bms_free(relation->rd_summarizedattr);
relation->rd_summarizedattr = NULL;
+ bms_free(relation->rd_indexedattr);
+ relation->rd_indexedattr = NULL;
/*
* Now save copies of the bitmaps in the relcache entry. We intentionally
@@ -5535,6 +5547,7 @@ restart:
relation->rd_idattr = bms_copy(idindexattrs);
relation->rd_hotblockingattr = bms_copy(hotblockingattrs);
relation->rd_summarizedattr = bms_copy(summarizedattrs);
+ relation->rd_indexedattr = bms_copy(indexedattrs);
relation->rd_attrsvalid = true;
MemoryContextSwitchTo(oldcxt);
@@ -5551,6 +5564,8 @@ restart:
return hotblockingattrs;
case INDEX_ATTR_BITMAP_SUMMARIZED:
return summarizedattrs;
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return indexedattrs;
default:
elog(ERROR, "unknown attrKind %u", attrKind);
return NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index e16bf025692..8a5931a3118 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -549,6 +549,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
+ Bitmapset *updated_cols,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1502,12 +1503,12 @@ static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ Bitmapset *updated_cols, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
- wait, tmfd,
- lockmode, update_indexes);
+ wait, tmfd, lockmode,
+ updated_cols, update_indexes);
}
/*
@@ -2010,6 +2011,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
+ Bitmapset *modified_indexe_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index fa2b657fb2f..993dc0e6ced 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -800,5 +800,10 @@ extern ResultRelInfo *ExecLookupResultRelByOid(ModifyTableState *node,
Oid resultoid,
bool missing_ok,
bool update_cache);
+extern Bitmapset *ExecCheckIndexedAttrsForChanges(ResultRelInfo *resultRelInfo,
+ TupleTableSlot *tts_old,
+ TupleTableSlot *tts_new);
+extern bool tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
+ Datum value1, Datum value2);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 18ae8f0d4bb..8b08e0045ba 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -498,6 +498,7 @@ typedef struct ResultRelInfo
Bitmapset *ri_extraUpdatedCols;
/* true if the above has been computed */
bool ri_extraUpdatedCols_valid;
+ Bitmapset *ri_ChangedIndexedCols;
/* Projection to generate new tuple in an INSERT/UPDATE */
ProjectionInfo *ri_projectNew;
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 80286076a11..b23a7306e69 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -164,6 +164,7 @@ typedef struct RelationData
Bitmapset *rd_idattr; /* included in replica identity index */
Bitmapset *rd_hotblockingattr; /* cols blocking HOT update */
Bitmapset *rd_summarizedattr; /* cols indexed by summarizing indexes */
+ Bitmapset *rd_indexedattr; /* all cols referenced by indexes */
PublicationDesc *rd_pubdesc; /* publication descriptor, or NULL */
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 3561c6bef0b..d3fbb8b093a 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -71,6 +71,7 @@ typedef enum IndexAttrBitmapKind
INDEX_ATTR_BITMAP_IDENTITY_KEY,
INDEX_ATTR_BITMAP_HOT_BLOCKING,
INDEX_ATTR_BITMAP_SUMMARIZED,
+ INDEX_ATTR_BITMAP_INDEXED,
} IndexAttrBitmapKind;
extern Bitmapset *RelationGetIndexAttrBitmap(Relation relation,
--
2.49.0
[application/octet-stream] v19-0003-Replace-index_unchanged_by_update-with-ri_Change.patch (8.3K, 4-v19-0003-Replace-index_unchanged_by_update-with-ri_Change.patch)
download | inline diff:
From 8687e77751d0ee16d4d9495828099b918426423b Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Fri, 31 Oct 2025 14:55:25 -0400
Subject: [PATCH v19 3/4] Replace index_unchanged_by_update with
ri_ChangedIndexedCols
In execIndexing on updates we'd like to pass a hint to the indexing code
when the indexed attributes are unchanged. This commit replaces the now
redundant code in index_unchanged_by_update with the same information
found earlier in the update path.
---
src/backend/catalog/toasting.c | 2 -
src/backend/executor/execIndexing.c | 156 +---------------------------
src/backend/nodes/makefuncs.c | 2 -
src/include/nodes/execnodes.h | 4 -
4 files changed, 1 insertion(+), 163 deletions(-)
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 874a8fc89ad..5d819bda54a 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -300,8 +300,6 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
indexInfo->ii_Unique = true;
indexInfo->ii_NullsNotDistinct = false;
indexInfo->ii_ReadyForInserts = true;
- indexInfo->ii_CheckedUnchanged = false;
- indexInfo->ii_IndexUnchanged = false;
indexInfo->ii_Concurrent = false;
indexInfo->ii_BrokenHotChain = false;
indexInfo->ii_ParallelWorkers = 0;
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 401606f840a..fb1bc3a480d 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -138,11 +138,6 @@ static bool check_exclusion_or_unique_constraint(Relation heap, Relation index,
static bool index_recheck_constraint(Relation index, const Oid *constr_procs,
const Datum *existing_values, const bool *existing_isnull,
const Datum *new_values);
-static bool index_unchanged_by_update(ResultRelInfo *resultRelInfo,
- EState *estate, IndexInfo *indexInfo,
- Relation indexRelation);
-static bool index_expression_changed_walker(Node *node,
- Bitmapset *allUpdatedCols);
static void ExecWithoutOverlapsNotEmpty(Relation rel, NameData attname, Datum attval,
char typtype, Oid atttypid);
@@ -440,10 +435,7 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* index. If we're being called as part of an UPDATE statement,
* consider if the 'indexUnchanged' = true hint should be passed.
*/
- indexUnchanged = update && index_unchanged_by_update(resultRelInfo,
- estate,
- indexInfo,
- indexRelation);
+ indexUnchanged = update && bms_is_empty(resultRelInfo->ri_ChangedIndexedCols);
satisfiesConstraint =
index_insert(indexRelation, /* index relation */
@@ -993,152 +985,6 @@ index_recheck_constraint(Relation index, const Oid *constr_procs,
return true;
}
-/*
- * Check if ExecInsertIndexTuples() should pass indexUnchanged hint.
- *
- * When the executor performs an UPDATE that requires a new round of index
- * tuples, determine if we should pass 'indexUnchanged' = true hint for one
- * single index.
- */
-static bool
-index_unchanged_by_update(ResultRelInfo *resultRelInfo, EState *estate,
- IndexInfo *indexInfo, Relation indexRelation)
-{
- Bitmapset *updatedCols;
- Bitmapset *extraUpdatedCols;
- Bitmapset *allUpdatedCols;
- bool hasexpression = false;
- List *idxExprs;
-
- /*
- * Check cache first
- */
- if (indexInfo->ii_CheckedUnchanged)
- return indexInfo->ii_IndexUnchanged;
- indexInfo->ii_CheckedUnchanged = true;
-
- /*
- * Check for indexed attribute overlap with updated columns.
- *
- * Only do this for key columns. A change to a non-key column within an
- * INCLUDE index should not be counted here. Non-key column values are
- * opaque payload state to the index AM, a little like an extra table TID.
- *
- * Note that row-level BEFORE triggers won't affect our behavior, since
- * they don't affect the updatedCols bitmaps generally. It doesn't seem
- * worth the trouble of checking which attributes were changed directly.
- */
- updatedCols = ExecGetUpdatedCols(resultRelInfo, estate);
- extraUpdatedCols = ExecGetExtraUpdatedCols(resultRelInfo, estate);
- for (int attr = 0; attr < indexInfo->ii_NumIndexKeyAttrs; attr++)
- {
- int keycol = indexInfo->ii_IndexAttrNumbers[attr];
-
- if (keycol <= 0)
- {
- /*
- * Skip expressions for now, but remember to deal with them later
- * on
- */
- hasexpression = true;
- continue;
- }
-
- if (bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
- updatedCols) ||
- bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
- extraUpdatedCols))
- {
- /* Changed key column -- don't hint for this index */
- indexInfo->ii_IndexUnchanged = false;
- return false;
- }
- }
-
- /*
- * When we get this far and index has no expressions, return true so that
- * index_insert() call will go on to pass 'indexUnchanged' = true hint.
- *
- * The _absence_ of an indexed key attribute that overlaps with updated
- * attributes (in addition to the total absence of indexed expressions)
- * shows that the index as a whole is logically unchanged by UPDATE.
- */
- if (!hasexpression)
- {
- indexInfo->ii_IndexUnchanged = true;
- return true;
- }
-
- /*
- * Need to pass only one bms to expression_tree_walker helper function.
- * Avoid allocating memory in common case where there are no extra cols.
- */
- if (!extraUpdatedCols)
- allUpdatedCols = updatedCols;
- else
- allUpdatedCols = bms_union(updatedCols, extraUpdatedCols);
-
- /*
- * We have to work slightly harder in the event of indexed expressions,
- * but the principle is the same as before: try to find columns (Vars,
- * actually) that overlap with known-updated columns.
- *
- * If we find any matching Vars, don't pass hint for index. Otherwise
- * pass hint.
- */
- idxExprs = RelationGetIndexExpressions(indexRelation);
- hasexpression = index_expression_changed_walker((Node *) idxExprs,
- allUpdatedCols);
- list_free(idxExprs);
- if (extraUpdatedCols)
- bms_free(allUpdatedCols);
-
- if (hasexpression)
- {
- indexInfo->ii_IndexUnchanged = false;
- return false;
- }
-
- /*
- * Deliberately don't consider index predicates. We should even give the
- * hint when result rel's "updated tuple" has no corresponding index
- * tuple, which is possible with a partial index (provided the usual
- * conditions are met).
- */
- indexInfo->ii_IndexUnchanged = true;
- return true;
-}
-
-/*
- * Indexed expression helper for index_unchanged_by_update().
- *
- * Returns true when Var that appears within allUpdatedCols located.
- */
-static bool
-index_expression_changed_walker(Node *node, Bitmapset *allUpdatedCols)
-{
- if (node == NULL)
- return false;
-
- if (IsA(node, Var))
- {
- Var *var = (Var *) node;
-
- if (bms_is_member(var->varattno - FirstLowInvalidHeapAttributeNumber,
- allUpdatedCols))
- {
- /* Var was updated -- indicates that we should not hint */
- return true;
- }
-
- /* Still haven't found a reason to not pass the hint */
- return false;
- }
-
- return expression_tree_walker(node, index_expression_changed_walker,
- allUpdatedCols);
-}
-
/*
* ExecWithoutOverlapsNotEmpty - raise an error if the tuple has an empty
* range or multirange in the given attribute.
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index e2d9e9be41a..d69dc090aa4 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -845,8 +845,6 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
n->ii_Unique = unique;
n->ii_NullsNotDistinct = nulls_not_distinct;
n->ii_ReadyForInserts = isready;
- n->ii_CheckedUnchanged = false;
- n->ii_IndexUnchanged = false;
n->ii_Concurrent = concurrent;
n->ii_Summarizing = summarizing;
n->ii_WithoutOverlaps = withoutoverlaps;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 8b08e0045ba..898368fb8cb 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -202,10 +202,6 @@ typedef struct IndexInfo
bool ii_NullsNotDistinct;
/* is it valid for inserts? */
bool ii_ReadyForInserts;
- /* IndexUnchanged status determined yet? */
- bool ii_CheckedUnchanged;
- /* aminsert hint, cached for retail inserts */
- bool ii_IndexUnchanged;
/* are we doing a concurrent index build? */
bool ii_Concurrent;
/* did we detect any broken HOT chains? */
--
2.49.0
[application/octet-stream] v19-0004-Enable-HOT-updates-for-expression-and-partial-in.patch (95.5K, 5-v19-0004-Enable-HOT-updates-for-expression-and-partial-in.patch)
download | inline diff:
From 05c4e601b85be2fd79b642de2e1c194b5ad7ea80 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 26 Oct 2025 10:49:25 -0400
Subject: [PATCH v19 4/4] Enable HOT updates for expression and partial indexes
Currently, PostgreSQL conservatively prevents HOT (Heap-Only Tuple)
updates whenever any indexed column changes, even if the indexed
portion of that column remains identical. This is overly restrictive
for expression indexes (where f(column) might not change even when
column changes) and partial indexes (where both old and new tuples
might fall outside the predicate).
This patch introduces several improvements to enable HOT updates in
these cases:
Add amcomparedatums callback to IndexAmRoutine. This allows index
access methods like GIN to provide custom logic for comparing datums
by extracting and comparing index keys rather than comparing the raw
datums. GIN indexes now implement gincomparedatums() which extracts
keys from both datums and compares the resulting key sets.
Add ExecWhichIndexesRequireUpdates() to refine the set of modified
attributes and determine precisely which indexes need updating. For
partial indexes, this checks whether both old and new tuples satisfy
or fail the predicate. For expression indexes, this uses type-specific
equality operators to compare computed values. For extraction-based
indexes (GIN/RUM), this delegates to amcomparedatums.
Modify heap update paths to use the refined modified indexed attrs
bitmapset returned by ExecWhichIndexesRequireUpdates(). This allows
HOT updates when indexes don't actually require updating, while still
preventing HOT updates when they do.
Importantly, table access methods can still signal using TU_Update if
all, none, or only summarizing indexes should be updated. While the
executor layer now owns determining what has changed due to an update
and is interested in only updating the minimum number of indexes
possible, the table AM can override that while performing
table_tuple_update(), which is what heap does.
This optimization significantly improves update performance for tables
with expression indexes, partial indexes, and GIN/GiST indexes on
complex data types like JSONB and tsvector, while maintaining correct
index semantics. Minimal additional overhead due to type-specific
equality checking should be washed out by the benefits of updating
indexes fewer times.
---
src/backend/access/gin/ginutil.c | 94 ++-
src/backend/access/heap/heapam.c | 10 +-
src/backend/access/heap/heapam_handler.c | 6 +-
src/backend/access/nbtree/nbtree.c | 38 ++
src/backend/access/table/tableam.c | 4 +-
src/backend/bootstrap/bootstrap.c | 8 +
src/backend/catalog/index.c | 57 ++
src/backend/catalog/indexing.c | 16 +-
src/backend/catalog/toasting.c | 4 +
src/backend/executor/execIndexing.c | 326 ++++++++-
src/backend/executor/nodeModifyTable.c | 61 +-
src/backend/nodes/makefuncs.c | 4 +
src/include/access/amapi.h | 28 +
src/include/access/gin.h | 3 +
src/include/access/heapam.h | 6 +-
src/include/access/nbtree.h | 4 +
src/include/access/tableam.h | 8 +-
src/include/catalog/index.h | 1 +
src/include/executor/executor.h | 5 +
src/include/nodes/execnodes.h | 19 +
.../expected/hot_expression_indexes.out | 644 ++++++++++++++++++
src/test/regress/parallel_schedule | 6 +
.../regress/sql/hot_expression_indexes.sql | 491 +++++++++++++
src/tools/pgindent/typedefs.list | 1 +
24 files changed, 1794 insertions(+), 50 deletions(-)
create mode 100644 src/test/regress/expected/hot_expression_indexes.out
create mode 100644 src/test/regress/sql/hot_expression_indexes.sql
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index 78f7b7a2495..85e25ed73e8 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -26,6 +26,7 @@
#include "storage/indexfsm.h"
#include "utils/builtins.h"
#include "utils/index_selfuncs.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/typcache.h"
@@ -78,6 +79,7 @@ ginhandler(PG_FUNCTION_ARGS)
amroutine->amproperty = NULL;
amroutine->ambuildphasename = ginbuildphasename;
amroutine->amvalidate = ginvalidate;
+ amroutine->amcomparedatums = gincomparedatums;
amroutine->amadjustmembers = ginadjustmembers;
amroutine->ambeginscan = ginbeginscan;
amroutine->amrescan = ginrescan;
@@ -477,13 +479,6 @@ cmpEntries(const void *a, const void *b, void *arg)
return res;
}
-
-/*
- * Extract the index key values from an indexable item
- *
- * The resulting key values are sorted, and any duplicates are removed.
- * This avoids generating redundant index entries.
- */
Datum *
ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
@@ -729,3 +724,88 @@ ginbuildphasename(int64 phasenum)
return NULL;
}
}
+
+/*
+ * gincomparedatums - Compare two datums to determine if they produce identical keys
+ *
+ * This function extracts keys from both old_datum and new_datum using the
+ * opclass's extractValue function, then compares the extracted key arrays.
+ * Returns true if the key sets are identical (same keys, same counts).
+ *
+ * This enables HOT updates for GIN indexes when the indexed portions of a
+ * value haven't changed, even if the value itself has changed.
+ *
+ * Example: JSONB column with GIN index. If an update changes a non-indexed
+ * key in the JSONB document, the extracted keys are identical and we can
+ * do a HOT update.
+ */
+bool
+gincomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull)
+{
+ GinState ginstate;
+ Datum *old_keys;
+ Datum *new_keys;
+ GinNullCategory *old_categories;
+ GinNullCategory *new_categories;
+ int32 old_nkeys;
+ int32 new_nkeys;
+ MemoryContext tmpcontext;
+ MemoryContext oldcontext;
+ bool result = true;
+
+ /* Handle NULL cases */
+ if (old_isnull != new_isnull)
+ return false;
+ if (old_isnull)
+ return true;
+
+ /* Create temporary context for extraction work */
+ tmpcontext = AllocSetContextCreate(CurrentMemoryContext,
+ "GIN datum comparison",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(tmpcontext);
+
+ initGinState(&ginstate, index);
+
+ /*
+ * Extract keys from both datums using existing GIN infrastructure.
+ */
+ old_keys = ginExtractEntries(&ginstate, attnum, old_datum, old_isnull,
+ &old_nkeys, &old_categories);
+ new_keys = ginExtractEntries(&ginstate, attnum, new_datum, new_isnull,
+ &new_nkeys, &new_categories);
+
+ /* Different number of keys → definitely different */
+ if (old_nkeys != new_nkeys)
+ {
+ result = false;
+ goto cleanup;
+ }
+
+ /*
+ * Compare the sorted key arrays element-by-element. Since both arrays are
+ * already sorted by ginExtractEntries, we can do a simple O(n)
+ * comparison.
+ */
+ for (int i = 0; i < old_nkeys; i++)
+ {
+ int cmp = ginCompareEntries(&ginstate, attnum,
+ old_keys[i], old_categories[i],
+ new_keys[i], new_categories[i]);
+
+ if (cmp != 0)
+ {
+ result = false;
+ break;
+ }
+ }
+
+cleanup:
+ /* Clean up */
+ MemoryContextSwitchTo(oldcontext);
+ MemoryContextDelete(tmpcontext);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 1cdb72b3a7a..5b0ff13b13d 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -3268,7 +3268,7 @@ heap_update(Relation relation, HeapTupleData *oldtup, HeapTuple newtup,
TM_FailureData *tmfd, LockTupleMode *lockmode,
Buffer buffer, Page page, BlockNumber block, ItemId lp,
Bitmapset *hot_attrs, Bitmapset *sum_attrs, Bitmapset *pk_attrs,
- Bitmapset *rid_attrs, Bitmapset *mix_attrs, Buffer *vmbuffer,
+ Bitmapset *rid_attrs, const Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -4337,8 +4337,9 @@ HeapDetermineColumnsInfo(Relation relation,
* This routine may be used to update a tuple when concurrent updates of the
* target tuple are not expected (for example, because we have a lock on the
* relation associated with the tuple). Any failure is reported via ereport().
+ * Returns the set of modified indexed attributes.
*/
-void
+Bitmapset *
simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
@@ -4467,7 +4468,7 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
elog(ERROR, "tuple concurrently deleted");
- return;
+ return NULL;
}
/*
@@ -4500,7 +4501,6 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
bms_free(sum_attrs);
bms_free(pk_attrs);
bms_free(rid_attrs);
- bms_free(mix_attrs);
bms_free(idx_attrs);
switch (result)
@@ -4526,6 +4526,8 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
elog(ERROR, "unrecognized heap_update status: %u", result);
break;
}
+
+ return mix_attrs;
}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index ef08e1d3e10..7527809ec08 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -319,7 +319,7 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
Snapshot crosscheck, bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
- Bitmapset *mix_attrs,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes)
{
bool rep_id_key_required = false;
@@ -407,10 +407,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
Assert(ItemIdIsNormal(lp));
- /*
- * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
- * then pass that on to heap_update.
- */
oldtup.t_tableOid = RelationGetRelid(relation);
oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
oldtup.t_len = ItemIdGetLength(lp);
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index fdff960c130..73cc3208757 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -155,6 +155,7 @@ bthandler(PG_FUNCTION_ARGS)
amroutine->amproperty = btproperty;
amroutine->ambuildphasename = btbuildphasename;
amroutine->amvalidate = btvalidate;
+ amroutine->amcomparedatums = btcomparedatums;
amroutine->amadjustmembers = btadjustmembers;
amroutine->ambeginscan = btbeginscan;
amroutine->amrescan = btrescan;
@@ -1795,3 +1796,40 @@ bttranslatecmptype(CompareType cmptype, Oid opfamily)
return InvalidStrategy;
}
}
+
+/*
+ * btcomparedatums - Compare two datums for equality
+ *
+ * This function is necessary because nbtree requires that keys that are not
+ * binary identical not be "equal". Other indexes might allow "A" and "a" to
+ * be "equal" when collation is case insensative, but not nbtree. Why? Well,
+ * nbtree deduplicates TIDs on page split and the way it accomplish that is by
+ * doing a binary comparison of the keys.
+ */
+
+bool
+btcomparedatums(Relation index, int attrnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull)
+{
+ TupleDesc desc = RelationGetDescr(index);
+ CompactAttribute *att;
+
+ /*
+ * If one value is NULL and other is not, then they are certainly not
+ * equal
+ */
+ if (old_isnull != new_isnull)
+ return false;
+
+ /*
+ * If both are NULL, they can be considered equal.
+ */
+ if (old_isnull)
+ return true;
+
+ /* We do simple binary comparison of the two datums */
+ Assert(attrnum <= desc->natts);
+ att = TupleDescCompactAttr(desc, attrnum - 1);
+ return datumIsEqual(old_datum, new_datum, att->attbyval, att->attlen);
+}
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index dadcf03ed24..ef7736bfa76 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -336,7 +336,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
- Bitmapset *modified_indexed_cols,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -348,7 +348,7 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
&tmfd, &lockmode,
- modified_indexed_cols,
+ mix_attrs,
update_indexes);
switch (result)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index fc8638c1b61..329c110d0bf 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -961,10 +961,18 @@ index_register(Oid heap,
newind->il_info->ii_Expressions =
copyObject(indexInfo->ii_Expressions);
newind->il_info->ii_ExpressionsState = NIL;
+ /* expression attrs will likely be null, but may as well copy it */
+ newind->il_info->ii_ExpressionsAttrs =
+ copyObject(indexInfo->ii_ExpressionsAttrs);
/* predicate will likely be null, but may as well copy it */
newind->il_info->ii_Predicate =
copyObject(indexInfo->ii_Predicate);
newind->il_info->ii_PredicateState = NULL;
+ /* predicate attrs will likely be null, but may as well copy it */
+ newind->il_info->ii_PredicateAttrs =
+ copyObject(indexInfo->ii_PredicateAttrs);
+ newind->il_info->ii_CheckedPredicate = false;
+ newind->il_info->ii_PredicateSatisfied = false;
/* no exclusion constraints at bootstrap time, so no need to copy */
Assert(indexInfo->ii_ExclusionOps == NULL);
Assert(indexInfo->ii_ExclusionProcs == NULL);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 5d9db167e59..29b8cc4badd 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -27,6 +27,7 @@
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/relscan.h"
+#include "access/sysattr.h"
#include "access/tableam.h"
#include "access/toast_compression.h"
#include "access/transam.h"
@@ -58,6 +59,7 @@
#include "commands/trigger.h"
#include "executor/executor.h"
#include "miscadmin.h"
+#include "nodes/execnodes.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/optimizer.h"
@@ -2414,6 +2416,61 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* ----------------------------------------------------------------
*/
+/* ----------------
+ * BuildUpdateIndexInfo
+ *
+ * For expression indexes updates may not change the indexed value allowing
+ * for a HOT update. Add information to the IndexInfo to allow for checking
+ * if the indexed value has changed.
+ *
+ * Do this processing here rather than in BuildIndexInfo() to not incur the
+ * overhead in the common non-expression cases.
+ * ----------------
+ */
+void
+BuildUpdateIndexInfo(ResultRelInfo *resultRelInfo)
+{
+ for (int j = 0; j < resultRelInfo->ri_NumIndices; j++)
+ {
+ int i;
+ int indnkeyatts;
+ Bitmapset *attrs = NULL;
+ IndexInfo *ii = resultRelInfo->ri_IndexRelationInfo[j];
+
+ /*
+ * Expressions are not allowed on non-key attributes, so we can skip
+ * them as they should show up in the index HOT-blocking attributes.
+ */
+ indnkeyatts = ii->ii_NumIndexKeyAttrs;
+
+ /* Collect key attributes used by the index */
+ for (i = 0; i < indnkeyatts; i++)
+ {
+ AttrNumber attnum = ii->ii_IndexAttrNumbers[i];
+
+ if (attnum != 0)
+ attrs = bms_add_member(attrs, attnum - FirstLowInvalidHeapAttributeNumber);
+ }
+
+ /* Collect attributes used in the expression */
+ if (ii->ii_Expressions)
+ pull_varattnos((Node *) ii->ii_Expressions,
+ resultRelInfo->ri_RangeTableIndex,
+ &ii->ii_ExpressionsAttrs);
+
+ /* Collect attributes used in the predicate */
+ if (ii->ii_Predicate)
+ pull_varattnos((Node *) ii->ii_Predicate,
+ resultRelInfo->ri_RangeTableIndex,
+ &ii->ii_PredicateAttrs);
+
+ ii->ii_IndexedAttrs = bms_union(attrs, ii->ii_ExpressionsAttrs);
+
+ /* All indexes should index *something*! */
+ Assert(!bms_is_empty(ii->ii_IndexedAttrs));
+ }
+}
+
/* ----------------
* BuildIndexInfo
* Construct an IndexInfo record for an open index
diff --git a/src/backend/catalog/indexing.c b/src/backend/catalog/indexing.c
index 004c5121000..a361c215490 100644
--- a/src/backend/catalog/indexing.c
+++ b/src/backend/catalog/indexing.c
@@ -102,7 +102,7 @@ CatalogIndexInsert(CatalogIndexState indstate, HeapTuple heapTuple,
* Get information from the state structure. Fall out if nothing to do.
*/
numIndexes = indstate->ri_NumIndices;
- if (numIndexes == 0)
+ if (numIndexes == 0 || updateIndexes == TU_None)
return;
relationDescs = indstate->ri_IndexRelationDescs;
indexInfoArray = indstate->ri_IndexRelationInfo;
@@ -314,15 +314,18 @@ CatalogTupleUpdate(Relation heapRel, const ItemPointerData *otid, HeapTuple tup)
{
CatalogIndexState indstate;
TU_UpdateIndexes updateIndexes = TU_All;
+ Bitmapset *updatedAttrs;
CatalogTupleCheckConstraints(heapRel, tup);
indstate = CatalogOpenIndexes(heapRel);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
-
+ updatedAttrs = simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = updatedAttrs;
CatalogIndexInsert(indstate, tup, updateIndexes);
+
CatalogCloseIndexes(indstate);
+ bms_free(updatedAttrs);
}
/*
@@ -338,12 +341,15 @@ CatalogTupleUpdateWithInfo(Relation heapRel, const ItemPointerData *otid, HeapTu
CatalogIndexState indstate)
{
TU_UpdateIndexes updateIndexes = TU_All;
+ Bitmapset *updatedAttrs;
CatalogTupleCheckConstraints(heapRel, tup);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
-
+ updatedAttrs = simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = updatedAttrs;
CatalogIndexInsert(indstate, tup, updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = NULL;
+ bms_free(updatedAttrs);
}
/*
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 5d819bda54a..c665aa744b3 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -292,8 +292,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
indexInfo->ii_IndexAttrNumbers[1] = 2;
indexInfo->ii_Expressions = NIL;
indexInfo->ii_ExpressionsState = NIL;
+ indexInfo->ii_ExpressionsAttrs = NULL;
indexInfo->ii_Predicate = NIL;
indexInfo->ii_PredicateState = NULL;
+ indexInfo->ii_PredicateAttrs = NULL;
+ indexInfo->ii_CheckedPredicate = false;
+ indexInfo->ii_PredicateSatisfied = false;
indexInfo->ii_ExclusionOps = NULL;
indexInfo->ii_ExclusionProcs = NULL;
indexInfo->ii_ExclusionStrats = NULL;
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index fb1bc3a480d..736543147e7 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -109,11 +109,15 @@
#include "access/genam.h"
#include "access/relscan.h"
#include "access/tableam.h"
+#include "access/sysattr.h"
#include "access/xact.h"
#include "catalog/index.h"
#include "executor/executor.h"
+#include "nodes/bitmapset.h"
+#include "nodes/execnodes.h"
#include "nodes/nodeFuncs.h"
#include "storage/lmgr.h"
+#include "utils/datum.h"
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
@@ -318,8 +322,8 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
Relation heapRelation;
IndexInfo **indexInfoArray;
ExprContext *econtext;
- Datum values[INDEX_MAX_KEYS];
- bool isnull[INDEX_MAX_KEYS];
+ Datum loc_values[INDEX_MAX_KEYS];
+ bool loc_isnull[INDEX_MAX_KEYS];
Assert(ItemPointerIsValid(tupleid));
@@ -343,13 +347,13 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
/* Arrange for econtext's scan tuple to be the tuple under test */
econtext->ecxt_scantuple = slot;
- /*
- * for each index, form and insert the index tuple
- */
+ /* Insert into each index that needs updating */
for (i = 0; i < numIndices; i++)
{
Relation indexRelation = relationDescs[i];
IndexInfo *indexInfo;
+ Datum *values;
+ bool *isnull;
bool applyNoDupErr;
IndexUniqueCheck checkUnique;
bool indexUnchanged;
@@ -366,7 +370,7 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
/*
* Skip processing of non-summarizing indexes if we only update
- * summarizing indexes
+ * summarizing indexes or if this index is unchanged.
*/
if (onlySummarizing && !indexInfo->ii_Summarizing)
continue;
@@ -387,8 +391,15 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
indexInfo->ii_PredicateState = predicate;
}
+ /* Check the index predicate if we haven't done so earlier on */
+ if (!indexInfo->ii_CheckedPredicate)
+ {
+ indexInfo->ii_PredicateSatisfied = ExecQual(predicate, econtext);
+ indexInfo->ii_CheckedPredicate = true;
+ }
+
/* Skip this index-update if the predicate isn't satisfied */
- if (!ExecQual(predicate, econtext))
+ if (!indexInfo->ii_PredicateSatisfied)
continue;
}
@@ -396,11 +407,10 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* FormIndexDatum fills in its values and isnull parameters with the
* appropriate values for the column(s) of the index.
*/
- FormIndexDatum(indexInfo,
- slot,
- estate,
- values,
- isnull);
+ FormIndexDatum(indexInfo, slot, estate, loc_values, loc_isnull);
+
+ values = loc_values;
+ isnull = loc_isnull;
/* Check whether to apply noDupErr to this index */
applyNoDupErr = noDupErr &&
@@ -435,7 +445,9 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* index. If we're being called as part of an UPDATE statement,
* consider if the 'indexUnchanged' = true hint should be passed.
*/
- indexUnchanged = update && bms_is_empty(resultRelInfo->ri_ChangedIndexedCols);
+ indexUnchanged = update &&
+ !bms_overlap(indexInfo->ii_IndexedAttrs,
+ resultRelInfo->ri_ChangedIndexedCols);
satisfiesConstraint =
index_insert(indexRelation, /* index relation */
@@ -604,7 +616,12 @@ ExecCheckIndexConstraints(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
checkedIndex = true;
/* Check for partial index */
- if (indexInfo->ii_Predicate != NIL)
+ if (indexInfo->ii_CheckedPredicate && !indexInfo->ii_PredicateSatisfied)
+ {
+ /* We've already checked and the predicate wasn't satisfied. */
+ continue;
+ }
+ else if (indexInfo->ii_Predicate != NIL)
{
ExprState *predicate;
@@ -1018,3 +1035,284 @@ ExecWithoutOverlapsNotEmpty(Relation rel, NameData attname, Datum attval, char t
errmsg("empty WITHOUT OVERLAPS value found in column \"%s\" in relation \"%s\"",
NameStr(attname), RelationGetRelationName(rel))));
}
+
+/*
+ * ExecWhichIndexesRequireUpdates
+ *
+ * Determine which indexes need updating given modified indexed attributes.
+ * This function is a companion to ExecCheckIndexedAttrsForChanges(). On the
+ * surface, they appear similar but they are doing two very different things.
+ *
+ * For a standard index on a set of attributes this is the intersection of
+ * the mix_attrs and the index attrs (key, expression, but not predicate).
+ *
+ * For expression indexes and indexes which implement the amcomparedatums()
+ * index AM API we'll need to form index datum and compare each attribute to
+ * see if any actually changed.
+ *
+ * For expression indexes the result of the expression might not change at all,
+ * this is common with JSONB columns which require expression indexes and where
+ * it is commonplace to index a field within a document and have updates that
+ * generally don't update that field.
+ *
+ * Partial indexes won't trigger index tuples when the old/new tuples are both
+ * outside of the predicate range.
+ *
+ * For nbtree the amcomparedatums() API is critical as it requires that key
+ * attributes are equal when they memcmp(), which might not be the case when
+ * using type-specific comparison or factoring in collation which might make
+ * an index case insensitive.
+ *
+ * All of this is to say that the goal is for the executor to know, ahead of
+ * calling into the table AM for the update and before calling into the index
+ * AM for inserting new index tuples, which attributes at a minimum will
+ * necessitate a new index tuple.
+ *
+ * The mix_attrs parameter contains attributes that:
+ * 1. Were refereced by the UPDATE statement and are known to have been modified
+ * 2. Are referenced by at least one index (expression, predicate, or otherwise)
+ *
+ * This function refines that set by determining which indexes actually
+ * need updates and only retaining the attributes that indicate that:
+ * - Partial index predicates: If both tuples fall outside, no update needed
+ * - Expression indexes: If expression results are identical, no update needed
+ * - Extraction indexes (GIN, RUM, etc): If extracted values are identical, no update needed
+ *
+ * Returns a refined Bitmapset of attributes that force index updates.
+ */
+Bitmapset *
+ExecWhichIndexesRequireUpdates(ResultRelInfo *relinfo,
+ Bitmapset *mix_attrs,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts)
+{
+ Bitmapset *result_attrs = NULL;
+ ExprContext *econtext = GetPerTupleExprContext(estate);
+ TupleTableSlot *save_scantuple;
+ int i;
+
+ if (relinfo->ri_NumIndices == 0 || bms_is_empty(mix_attrs))
+ return NULL;
+
+ /* Check each index */
+ for (i = 0; i < relinfo->ri_NumIndices; i++)
+ {
+ Relation indexRel = relinfo->ri_IndexRelationDescs[i];
+ IndexInfo *indexInfo = relinfo->ri_IndexRelationInfo[i];
+ IndexAmRoutine *amroutine = indexRel->rd_indam;
+ bool has_expressions = (indexInfo->ii_Expressions != NIL);
+ bool has_am_compare = (amroutine->amcomparedatums != NULL);
+ bool is_partial = (indexInfo->ii_Predicate != NIL);
+ Bitmapset *idx_attrs = indexInfo->ii_IndexedAttrs;
+ Bitmapset *pre_attrs = indexInfo->ii_PredicateAttrs;
+
+ /* Check partial index predicate iff attrs overlap with modified */
+ if (is_partial &&
+ !bms_is_empty((pre_attrs = bms_intersect(pre_attrs, mix_attrs))))
+ {
+ ExprState *pstate;
+ bool old_qualifies,
+ new_qualifies;
+
+ if (!indexInfo->ii_CheckedPredicate)
+ pstate = ExecPrepareQual(indexInfo->ii_Predicate, estate);
+ else
+ pstate = indexInfo->ii_PredicateState;
+
+ save_scantuple = econtext->ecxt_scantuple;
+
+ econtext->ecxt_scantuple = old_tts;
+ old_qualifies = ExecQual(pstate, econtext);
+
+ econtext->ecxt_scantuple = new_tts;
+ new_qualifies = ExecQual(pstate, econtext);
+
+ econtext->ecxt_scantuple = save_scantuple;
+
+ indexInfo->ii_CheckedPredicate = true;
+ indexInfo->ii_PredicateState = pstate;
+ indexInfo->ii_PredicateSatisfied = new_qualifies;
+
+ /* Both outside predicate, index doesn't need update */
+ if (!old_qualifies && !new_qualifies)
+ {
+ bms_free(pre_attrs);
+ continue;
+ }
+
+ /* Predicate transition, must update index, add predicate attrs */
+ if (old_qualifies != new_qualifies)
+ {
+ /*
+ * We can say for sure that the index needs update, so we can
+ * add in the attributes from the predicate that are also in
+ * the mix_attrs set, but we don't yet know if there are other
+ * attributes this index references that are modified (in the
+ * mix_attrs) and force index updates. The only way to know
+ * if to test them one by one.
+ */
+ result_attrs = bms_add_members(result_attrs, pre_attrs);
+ bms_free(pre_attrs);
+ }
+ }
+
+ /*
+ * If we've got a result set equal to our modified set then we've
+ * identified that all the attributes on the index need to trigger new
+ * index tuples. No need to keep checking, we're done not just with
+ * this index but in general, break out of the loop here.
+ */
+ if (bms_equal(result_attrs, mix_attrs))
+ break;
+
+ /*
+ * NOTE: While it feels like we could avoid checking any further when
+ * the indexed attributes (key and expression) do not overlap with the
+ * modified indexed attributes (mix_attrs) we can't. Why? Well, it
+ * turns out that it is allowable for index AMs to have a different
+ * notion of equality. For instance, nbtree requires that datum
+ * equality be based on binary comparison, not anything type-specific.
+ * So, it is possible that the set of mix_attrs from
+ * ExecCheckIndexedAttrsForChanges() found that the new value of an
+ * attribute was equal to the old value despite it having a different
+ * binary representation.
+ *
+ * XXX: maybe that's something we should enforce and change nbtree?
+ */
+
+ /*
+ * Expression index or extraction-based index require us to form index
+ * datums/tuples and compare. We've done all we can to avoid this
+ * overhead now it's time to bite the bullet and get it done.
+ */
+ if (has_expressions || has_am_compare)
+ {
+ Datum old_values[INDEX_MAX_KEYS];
+ bool old_isnull[INDEX_MAX_KEYS];
+ Datum new_values[INDEX_MAX_KEYS];
+ bool new_isnull[INDEX_MAX_KEYS];
+ TupleDesc indexdesc = RelationGetDescr(indexRel);
+
+ save_scantuple = econtext->ecxt_scantuple;
+
+ /* Evaluate expressions (if any) to get base datums */
+ econtext->ecxt_scantuple = old_tts;
+ FormIndexDatum(indexInfo, old_tts, estate,
+ old_values, old_isnull);
+
+ econtext->ecxt_scantuple = new_tts;
+ FormIndexDatum(indexInfo, new_tts, estate,
+ new_values, new_isnull);
+
+ econtext->ecxt_scantuple = save_scantuple;
+
+ /* Compare the index key datums */
+ for (int j = 0; j < indexInfo->ii_NumIndexKeyAttrs; j++)
+ {
+ AttrNumber attrnum = indexInfo->ii_IndexAttrNumbers[j];
+ bool values_equal;
+
+ /*
+ * Skip attributes that we've already identified as triggering
+ * an index update.
+ */
+ if (attrnum > 0 &&
+ bms_is_member(attrnum - FirstLowInvalidHeapAttributeNumber, result_attrs))
+ continue;
+
+ /* A change to/from NULL, record this attribute */
+ if (old_isnull[j] != new_isnull[j])
+ {
+ if (attrnum == 0)
+ result_attrs = bms_add_members(result_attrs, indexInfo->ii_ExpressionsAttrs);
+ else
+ result_attrs = bms_add_member(result_attrs, attrnum - FirstLowInvalidHeapAttributeNumber);
+ continue;
+ }
+ /* Both NULL, no change */
+ if (old_isnull[j])
+ continue;
+
+ /*
+ * Use index AM's comparison function if present when
+ * comparing the datum formed when creating an index key. This
+ * is different from the comparison of datum in the tuple
+ * destined for a storage AM, which is why we only need to use
+ * this here.
+ */
+ if (has_am_compare)
+ {
+ /*
+ * For nbtree to properly deduplicate TIDs on page split
+ * it must treat equality as binary comparison. So it is
+ * vital that we call it's comparedatums() function.
+ *
+ * In the case of GIN/RUM indexes they too behave
+ * differently and can even extract one or more portions
+ * of the datum when forming index tuples. We'd like to
+ * know if this update needs to trigger one or more index
+ * tuples, so we let the index AM perform their extraction
+ * and compare the results.
+ *
+ * There may be other similar index AM implementation with
+ * extraction where indexes are built using only part(s)
+ * of the Datum and might even need to invoke
+ * type-specific equality operators.
+ *
+ * NOTE: For AM comparison, pass the 1-based index
+ * attribute number. The AM's compare function expects the
+ * same numbering as used internally by the AM.
+ */
+ values_equal = amroutine->amcomparedatums(indexRel, j + 1,
+ old_values[j], old_isnull[j],
+ new_values[j], new_isnull[j]);
+ }
+ else
+ {
+ /*
+ * Expression index without custom AM comparison. Compare
+ * the expression results using type-specific equality via
+ * the TypeCache.
+ */
+ Form_pg_attribute attr = TupleDescAttr(indexdesc, j);
+
+ values_equal = tts_attr_equal(attr->atttypid,
+ attr->attcollation,
+ attr->attbyval,
+ attr->attlen,
+ old_values[j],
+ new_values[j]);
+ }
+
+ if (!values_equal)
+ {
+ if (attrnum == 0)
+ result_attrs = bms_add_members(result_attrs, indexInfo->ii_ExpressionsAttrs);
+ else
+ result_attrs = bms_add_member(result_attrs, attrnum - FirstLowInvalidHeapAttributeNumber);
+ }
+ }
+ }
+ else
+ {
+ /*
+ * Here we know that we're reviewing an index that doesn't have a
+ * partial predicate, isn't an expression index, and doesn't have
+ * a amcomparedatums() implementation. Attributes that overlap
+ * with those known to have changed are the ones we need to
+ * record.
+ */
+
+ /*
+ * NOTE: we intentionally copy via intersect and then free rather
+ * than modify on the set we're pointing at in IndexInfo.
+ */
+ idx_attrs = bms_intersect(idx_attrs, mix_attrs);
+ result_attrs = bms_add_members(result_attrs, idx_attrs);
+ bms_free(idx_attrs);
+ }
+ }
+
+ return result_attrs;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 4d1cf50e369..191748cdce8 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -54,10 +54,12 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/sysattr.h"
#include "access/tableam.h"
#include "access/tupconvert.h"
#include "access/tupdesc.h"
#include "access/xact.h"
+#include "catalog/index.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -75,6 +77,7 @@
#include "utils/float.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
@@ -2395,6 +2398,12 @@ ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
bool partition_constraint_failed;
TM_Result result;
+ /* the set of modified indexed attributes */
+ Bitmapset *mix_attrs = NULL;
+
+ /* the set of index attributes that trigger new index datum */
+ Bitmapset *iru_attrs = NULL;
+
updateCxt->crossPartUpdate = false;
/*
@@ -2517,13 +2526,48 @@ lreplace:
bms_free(resultRelInfo->ri_ChangedIndexedCols);
resultRelInfo->ri_ChangedIndexedCols = NULL;
- resultRelInfo->ri_ChangedIndexedCols =
- ExecCheckIndexedAttrsForChanges(resultRelInfo, oldSlot, slot);
+ /*
+ * At this point we know the set of attributes specified in the UPDATE
+ * statement and those referenced by triggers, so we have a complete view
+ * of the UPDATE attributes on the table. We could get this set via the
+ * ExecGetUpdatedCols() function, but we'll need to review all indexed
+ * attributes because extensions could just directly heap_modify_tuple()
+ * an attribute not known to ExecGetUpdatedCols().
+ *
+ * We want to know which, if any, attributes that are referenced by an
+ * index have changed value. This set of attributes will dictate the
+ * minimum number of indexes we need to update.
+ */
+ mix_attrs = ExecCheckIndexedAttrsForChanges(resultRelInfo, oldSlot, slot);
/*
- * replace the heap tuple
+ * During updates we'll need a bit more information in IndexInfo but we've
+ * delayed adding it until here. We check to ensure that there are
+ * indexes, that something has changed that is indexed, and that the first
+ * index doesn't yet have ii_IndexedAttrs set as a way to ensure we only
+ * build this when needed and only once. We don't build this in
+ * ExecOpenIndicies() as it is unnecessary overhead when not performing an
+ * update.
+ */
+ if (resultRelInfo->ri_NumIndices > 0 &&
+ bms_is_empty(resultRelInfo->ri_IndexRelationInfo[0]->ii_IndexedAttrs))
+ BuildUpdateIndexInfo(resultRelInfo);
+
+ /*
+ * The next step is to identify which indexes, at a minimum, require new
+ * index tuples. You might think that you could simply intersect the
+ * index key attributes with the modified attributes and be done, but then
+ * you'd have missed a few cases (expressions, partial, indexes with
+ * operators that index only a portion of a datum or many different
+ * portions of it).
+ */
+ iru_attrs = ExecWhichIndexesRequireUpdates(resultRelInfo, mix_attrs, estate,
+ oldSlot, slot);
+
+ /*
+ * Call into the table AM to update the heap tuple.
*
- * Note: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
+ * NOTE: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
* the row to be updated is visible to that snapshot, and throw a
* can't-serialize error if not. This is a special-case behavior needed
* for referential integrity updates in transaction-snapshot mode
@@ -2535,9 +2579,14 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
- resultRelInfo->ri_ChangedIndexedCols,
+ iru_attrs,
&updateCxt->updateIndexes);
+ Assert(bms_is_empty(resultRelInfo->ri_ChangedIndexedCols));
+ resultRelInfo->ri_ChangedIndexedCols = iru_attrs;
+
+ bms_free(mix_attrs);
+
return result;
}
@@ -2555,7 +2604,7 @@ ExecUpdateEpilogue(ModifyTableContext *context, UpdateContext *updateCxt,
ModifyTableState *mtstate = context->mtstate;
List *recheckIndexes = NIL;
- /* insert index entries for tuple if necessary */
+ /* Insert index entries for tuple if necessary */
if (resultRelInfo->ri_NumIndices > 0 && (updateCxt->updateIndexes != TU_None))
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
slot, context->estate,
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index d69dc090aa4..e9a53b95caf 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -855,10 +855,14 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
/* expressions */
n->ii_Expressions = expressions;
n->ii_ExpressionsState = NIL;
+ n->ii_ExpressionsAttrs = NULL;
/* predicates */
n->ii_Predicate = predicates;
n->ii_PredicateState = NULL;
+ n->ii_PredicateAttrs = NULL;
+ n->ii_CheckedPredicate = false;
+ n->ii_PredicateSatisfied = false;
/* exclusion constraints */
n->ii_ExclusionOps = NULL;
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index 63dd41c1f21..9bdf73eda59 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -211,6 +211,33 @@ typedef void (*ammarkpos_function) (IndexScanDesc scan);
/* restore marked scan position */
typedef void (*amrestrpos_function) (IndexScanDesc scan);
+/*
+ * amcomparedatums - Compare datums to determine if index update is needed
+ *
+ * This function compares old_datum and new_datum to determine if they would
+ * produce different index entries. For extraction-based indexes (GIN, RUM),
+ * this should:
+ * 1. Extract keys from old_datum using the opclass's extractValue function
+ * 2. Extract keys from new_datum using the opclass's extractValue function
+ * 3. Compare the two sets of keys using appropriate equality operators
+ * 4. Return true if the sets are equal (no index update needed)
+ *
+ * The comparison should account for:
+ * - Different numbers of extracted keys
+ * - NULL values
+ * - Type-specific equality (not just binary equality)
+ * - Opclass parameters (e.g., path in bson_rum_single_path_ops)
+ *
+ * For the DocumentDB example with path='a', this would extract values at
+ * path 'a' from both old and new BSON documents and compare them using
+ * BSON's equality operator.
+ */
+/* identify if updated datums would produce one or more index entries */
+typedef bool (*amcomparedatums_function) (Relation indexRelation,
+ int attno,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* Callback function signatures - for parallel index scans.
*/
@@ -313,6 +340,7 @@ typedef struct IndexAmRoutine
amendscan_function amendscan;
ammarkpos_function ammarkpos; /* can be NULL */
amrestrpos_function amrestrpos; /* can be NULL */
+ amcomparedatums_function amcomparedatums; /* can be NULL */
/* interface functions to support parallel index scans */
amestimateparallelscan_function amestimateparallelscan; /* can be NULL */
diff --git a/src/include/access/gin.h b/src/include/access/gin.h
index 13ea91922ef..2f265f4816c 100644
--- a/src/include/access/gin.h
+++ b/src/include/access/gin.h
@@ -100,6 +100,9 @@ extern PGDLLIMPORT int gin_pending_list_limit;
extern void ginGetStats(Relation index, GinStatsData *stats);
extern void ginUpdateStats(Relation index, const GinStatsData *stats,
bool is_build);
+extern bool gincomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
extern void _gin_parallel_build_main(dsm_segment *seg, shm_toc *toc);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 41d541aa6b2..59db389a546 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -326,7 +326,7 @@ extern TM_Result heap_update(Relation relation, HeapTupleData *oldtup,
TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
- Bitmapset *mix_attrs, Buffer *vmbuffer,
+ const Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
@@ -361,8 +361,8 @@ extern bool heap_tuple_needs_eventual_freeze(HeapTupleHeader tuple);
extern void simple_heap_insert(Relation relation, HeapTuple tup);
extern void simple_heap_delete(Relation relation, const ItemPointerData *tid);
-extern void simple_heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple tup, TU_UpdateIndexes *update_indexes);
+extern Bitmapset *simple_heap_update(Relation relation, const ItemPointerData *otid,
+ HeapTuple tup, TU_UpdateIndexes *update_indexes);
extern TransactionId heap_index_delete_tuples(Relation rel,
TM_IndexDeleteOp *delstate);
diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h
index 16be5c7a9c1..42bd329eaad 100644
--- a/src/include/access/nbtree.h
+++ b/src/include/access/nbtree.h
@@ -1210,6 +1210,10 @@ extern int btgettreeheight(Relation rel);
extern CompareType bttranslatestrategy(StrategyNumber strategy, Oid opfamily);
extern StrategyNumber bttranslatecmptype(CompareType cmptype, Oid opfamily);
+extern bool btcomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* prototypes for internal functions in nbtree.c
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 8a5931a3118..2b9206ff24a 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -549,7 +549,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
- Bitmapset *updated_cols,
+ const Bitmapset *updated_cols,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1503,12 +1503,12 @@ static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- Bitmapset *updated_cols, TU_UpdateIndexes *update_indexes)
+ const Bitmapset *mix_cols, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
wait, tmfd, lockmode,
- updated_cols, update_indexes);
+ mix_cols, update_indexes);
}
/*
@@ -2011,7 +2011,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
- Bitmapset *modified_indexe_attrs,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index dda95e54903..8d364f8b30f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -132,6 +132,7 @@ extern bool CompareIndexInfo(const IndexInfo *info1, const IndexInfo *info2,
const AttrMap *attmap);
extern void BuildSpeculativeIndexInfo(Relation index, IndexInfo *ii);
+extern void BuildUpdateIndexInfo(ResultRelInfo *resultRelInfo);
extern void FormIndexDatum(IndexInfo *indexInfo,
TupleTableSlot *slot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 993dc0e6ced..dda48f17605 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -739,6 +739,11 @@ extern Bitmapset *ExecGetAllUpdatedCols(ResultRelInfo *relinfo, EState *estate);
*/
extern void ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative);
extern void ExecCloseIndices(ResultRelInfo *resultRelInfo);
+extern Bitmapset *ExecWhichIndexesRequireUpdates(ResultRelInfo *relinfo,
+ Bitmapset *mix_attrs,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
extern List *ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
TupleTableSlot *slot, EState *estate,
bool update,
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 898368fb8cb..d8e88817206 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -174,15 +174,29 @@ typedef struct IndexInfo
*/
AttrNumber ii_IndexAttrNumbers[INDEX_MAX_KEYS];
+ /*
+ * All key, expression, sumarizing, and partition attributes referenced by
+ * this index
+ */
+ Bitmapset *ii_IndexedAttrs;
+
/* expr trees for expression entries, or NIL if none */
List *ii_Expressions; /* list of Expr */
/* exec state for expressions, or NIL if none */
List *ii_ExpressionsState; /* list of ExprState */
+ /* attributes exclusively referenced by expression indexes */
+ Bitmapset *ii_ExpressionsAttrs;
/* partial-index predicate, or NIL if none */
List *ii_Predicate; /* list of Expr */
/* exec state for expressions, or NIL if none */
ExprState *ii_PredicateState;
+ /* attributes referenced by the predicate */
+ Bitmapset *ii_PredicateAttrs;
+ /* partial index predicate determined yet? */
+ bool ii_CheckedPredicate;
+ /* amupdate hint used to avoid rechecking predicate */
+ bool ii_PredicateSatisfied;
/* Per-column exclusion operators, or NULL if none */
Oid *ii_ExclusionOps; /* array with one entry per column */
@@ -494,6 +508,11 @@ typedef struct ResultRelInfo
Bitmapset *ri_extraUpdatedCols;
/* true if the above has been computed */
bool ri_extraUpdatedCols_valid;
+
+ /*
+ * For UPDATE a Bitmapset of the attributes that are both indexed and have
+ * changed in value.
+ */
Bitmapset *ri_ChangedIndexedCols;
/* Projection to generate new tuple in an INSERT/UPDATE */
diff --git a/src/test/regress/expected/hot_expression_indexes.out b/src/test/regress/expected/hot_expression_indexes.out
new file mode 100644
index 00000000000..d3eb07742c6
--- /dev/null
+++ b/src/test/regress/expected/hot_expression_indexes.out
@@ -0,0 +1,644 @@
+-- ================================================================
+-- Test Suite for HOT Updates with Expression and Partial Indexes
+-- ================================================================
+-- Setup: Create function to measure HOT updates
+CREATE OR REPLACE FUNCTION check_hot_updates(
+ expected INT,
+ p_table_name TEXT DEFAULT 't',
+ p_schema_name TEXT DEFAULT current_schema()
+)
+RETURNS TABLE (
+ table_name TEXT,
+ total_updates BIGINT,
+ hot_updates BIGINT,
+ hot_update_percentage NUMERIC,
+ matches_expected BOOLEAN
+)
+LANGUAGE plpgsql
+AS $$
+DECLARE
+ v_relid oid;
+ v_qualified_name TEXT;
+ v_hot_updates BIGINT;
+ v_updates BIGINT;
+ v_xact_hot_updates BIGINT;
+ v_xact_updates BIGINT;
+BEGIN
+ -- Force statistics update
+ PERFORM pg_stat_force_next_flush();
+
+ -- Get table OID
+ v_qualified_name := quote_ident(p_schema_name) || '.' || quote_ident(p_table_name);
+ v_relid := v_qualified_name::regclass;
+
+ IF v_relid IS NULL THEN
+ RAISE EXCEPTION 'Table %.% not found', p_schema_name, p_table_name;
+ END IF;
+
+ -- Get cumulative + transaction stats
+ v_hot_updates := COALESCE(pg_stat_get_tuples_hot_updated(v_relid), 0);
+ v_updates := COALESCE(pg_stat_get_tuples_updated(v_relid), 0);
+ v_xact_hot_updates := COALESCE(pg_stat_get_xact_tuples_hot_updated(v_relid), 0);
+ v_xact_updates := COALESCE(pg_stat_get_xact_tuples_updated(v_relid), 0);
+
+ v_hot_updates := v_hot_updates + v_xact_hot_updates;
+ v_updates := v_updates + v_xact_updates;
+
+ RETURN QUERY
+ SELECT
+ p_table_name::TEXT,
+ v_updates::BIGINT,
+ v_hot_updates::BIGINT,
+ CASE WHEN v_updates > 0
+ THEN ROUND((v_hot_updates::numeric / v_updates::numeric * 100)::numeric, 2)
+ ELSE 0
+ END,
+ (v_hot_updates = expected)::BOOLEAN;
+END;
+$$;
+-- ================================================================
+-- Basic JSONB Expression Index
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_name_idx ON t((docs->>'name'));
+INSERT INTO t VALUES (1, '{"name": "alice", "age": 30}');
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET docs = '{"name": "alice", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update indexed JSONB field - should NOT be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update non-indexed field again - should be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 32}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 2 | 66.67 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Partial Index with Predicate Transitions
+-- ================================================================
+CREATE TABLE t(id INT, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_value_idx ON t(value) WHERE value > 10;
+INSERT INTO t VALUES (1, 5);
+-- Both outside predicate - should be HOT
+UPDATE t SET value = 8 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET value = 15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Both inside predicate, value changes - should NOT be HOT
+UPDATE t SET value = 20 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Transition out of predicate - should NOT be HOT
+UPDATE t SET value = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+-- Both outside predicate again - should be HOT
+UPDATE t SET value = 3 WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 5 | 2 | 40.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Expression Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((docs->>'status'))
+ WHERE (docs->>'priority')::int > 5;
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3}');
+-- Both outside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 4}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 10}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Inside predicate, status changes - should NOT be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 10}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Inside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 8}';
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 2 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Index on JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data);
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "database"]}');
+-- Change tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Change tags again - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Add field without changing existing keys - GIN keys changed (added "note"), NOT HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "note": "test"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Index with Unchanged Keys
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create GIN index on specific path
+CREATE INDEX t_gin_idx ON t USING gin((data->'tags'));
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "sql"], "status": "active"}');
+-- Change non-indexed field - GIN keys on 'tags' unchanged, should be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change indexed tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN with jsonb_path_ops
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data jsonb_path_ops);
+INSERT INTO t VALUES (1, '{"user": {"name": "alice"}, "tags": ["a", "b"]}');
+-- Change value at different path - keys changed, NOT HOT
+UPDATE t SET data = '{"user": {"name": "bob"}, "tags": ["a", "b"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- BRIN Index
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- BRIN tracks min/max per block range
+CREATE INDEX t_brin_idx ON t USING brin(value);
+INSERT INTO t VALUES (1, 100, 'initial');
+-- Update non-indexed column - BRIN doesn't care, should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update indexed column but stay in same range - should still be HOT
+-- (Note: BRIN tracks ranges, not exact values)
+UPDATE t SET value = 105 WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Change to significantly different value - still HOT for single row
+-- (BRIN summary won't change for single-row updates in same block)
+UPDATE t SET value = 1000 WHERE id = 1;
+SELECT * FROM check_hot_updates(3, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 3 | 100.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Multi-Column Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, a INT, b INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t(id, abs(a), abs(b));
+INSERT INTO t VALUES (1, -5, -10);
+-- Change sign but not abs value - should be HOT
+UPDATE t SET a = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change abs value - should NOT be HOT
+UPDATE t SET b = -15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Change id - should NOT be HOT
+UPDATE t SET id = 2 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Mixed Index Types (BRIN + Expression)
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_idx ON t USING brin(value);
+CREATE INDEX t_expr_idx ON t((data->>'status'));
+INSERT INTO t VALUES (1, 100, '{"status": "active"}');
+-- Update only BRIN column - should be HOT
+UPDATE t SET value = 200 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update only expression column - should NOT be HOT
+UPDATE t SET data = '{"status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update both - should NOT be HOT
+UPDATE t SET value = 300, data = '{"status": "pending"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Expression with COLLATION
+-- ================================================================
+CREATE TABLE t(id INT, name TEXT COLLATE "C")
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_lower_idx ON t(lower(name));
+INSERT INTO t VALUES (1, 'ALICE');
+-- Change case but not lowercase value - should be HOT
+UPDATE t SET name = 'Alice' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change lowercase value - should NOT be HOT
+UPDATE t SET name = 'BOB' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Array Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_array_len_idx ON t(array_length(tags, 1));
+INSERT INTO t VALUES (1, ARRAY['a', 'b', 'c']);
+-- Same length, different elements - should be HOT
+UPDATE t SET tags = ARRAY['d', 'e', 'f'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Different length - should NOT be HOT
+UPDATE t SET tags = ARRAY['d', 'e'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Nested JSONB Expression
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_nested_idx ON t((data->'user'->>'name'));
+INSERT INTO t VALUES ('{"user": {"name": "alice", "age": 30}}');
+-- Change nested non-indexed field - should be HOT
+UPDATE t SET data = '{"user": {"name": "alice", "age": 31}}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change nested indexed field - should NOT be HOT
+UPDATE t SET data = '{"user": {"name": "bob", "age": 31}}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Complex Predicate on Multiple JSONB Fields
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((data->>'status'))
+ WHERE (data->>'priority')::int > 5
+ AND (data->>'active')::boolean = true;
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3, "active": true}');
+-- Outside predicate (priority too low) - should be HOT
+UPDATE t SET data = '{"status": "done", "priority": 3, "active": true}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": true}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Inside predicate, change to outside (active = false) - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": false}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- TOASTed Values in Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, large_text TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_substr_idx ON t(substr(large_text, 1, 10));
+INSERT INTO t VALUES (1, repeat('x', 5000) || 'identifier');
+-- Change end of string, prefix unchanged - should be HOT
+UPDATE t SET large_text = repeat('x', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change prefix - should NOT be HOT
+UPDATE t SET large_text = repeat('y', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- TEST: GIN with TOASTed TEXT (tsvector)
+-- ================================================================
+CREATE TABLE t(id INT, content TEXT, search_vec tsvector)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create trigger to maintain tsvector
+CREATE TRIGGER tsvectorupdate_toast
+ BEFORE INSERT OR UPDATE ON t
+ FOR EACH ROW EXECUTE FUNCTION
+ tsvector_update_trigger(search_vec, 'pg_catalog.english', content);
+CREATE INDEX t_gin ON t USING gin(search_vec);
+-- Insert with large content (will be TOASTed)
+INSERT INTO t (id, content) VALUES
+ (1, repeat('important keyword ', 1000) || repeat('filler text ', 10000));
+-- Verify initial state
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('important');
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+-- IMPORTANT: The BEFORE UPDATE trigger modifies search_vec, so by the time
+-- ExecWhichIndexesRequireUpdates() runs, search_vec has already changed.
+-- This means the comparison sees old tsvector vs. trigger-modified tsvector,
+-- not the natural progression. HOT won't happen because the trigger changed
+-- the indexed column.
+-- Update: Even though content keywords unchanged, trigger still fires
+UPDATE t
+SET content = repeat('important keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (trigger modifies search_vec, blocking HOT)
+-- This is actually correct behavior - the trigger updated an indexed column
+-- Update: Change indexed keywords
+UPDATE t
+SET content = repeat('critical keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (index keys changed)
+-- Verify query correctness
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('critical');
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+DROP TABLE t CASCADE;
+-- ================================================================
+-- TEST: GIN with TOASTed JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin((data->'tags'));
+-- Insert with TOASTed JSONB
+INSERT INTO t (id, data) VALUES
+ (1, jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('x', 10000)
+ ));
+-- Update: Change large_field, tags unchanged - should be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT update
+-- Update: Change tags - should NOT be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "sql"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Expected: Still 1 HOT
+-- Verify correctness
+SELECT count(*) FROM t WHERE data->'tags' @> '["database"]'::jsonb;
+ count
+-------
+ 0
+(1 row)
+
+-- Expected: 0 rows
+SELECT count(*) FROM t WHERE data->'tags' @> '["sql"]'::jsonb;
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+DROP TABLE t CASCADE;
+-- ================================================================
+-- TEST: GIN with Array of Large Strings
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin(tags);
+-- Insert with large array elements (might be TOASTed)
+INSERT INTO t (id, tags) VALUES
+ (1, ARRAY[repeat('tag1', 1000), repeat('tag2', 1000)]);
+-- Update: Change to different large values - NOT HOT
+UPDATE t
+SET tags = ARRAY[repeat('tag3', 1000), repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (keys actually changed)
+-- Update: Keep same tag values, just reorder - SHOULD BE HOT
+-- (GIN is order-insensitive: both [tag3,tag4] and [tag4,tag3]
+-- extract to the same sorted key set ['tag3','tag4'])
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000), repeat('tag3', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Expected: 1 HOT (GIN keys semantically identical)
+-- Update: Remove an element - NOT HOT (keys changed)
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Expected: Still 1 HOT (not this one)
+DROP TABLE t CASCADE;
+-- Cleanup
+DROP FUNCTION check_hot_updates(int, text, text);
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index f56482fb9f1..4459625a59b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -125,6 +125,12 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression compression_lz4 memoize stats predicate numa eager_aggregate
+
+# ----------
+# Another group of parallel tests, these focused on heap HOT updates
+# ----------
+test: hot_expression_indexes
+
# event_trigger depends on create_am and cannot run concurrently with
# any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/sql/hot_expression_indexes.sql b/src/test/regress/sql/hot_expression_indexes.sql
new file mode 100644
index 00000000000..5dcadbde465
--- /dev/null
+++ b/src/test/regress/sql/hot_expression_indexes.sql
@@ -0,0 +1,491 @@
+-- ================================================================
+-- Test Suite for HOT Updates with Expression and Partial Indexes
+-- ================================================================
+
+-- Setup: Create function to measure HOT updates
+CREATE OR REPLACE FUNCTION check_hot_updates(
+ expected INT,
+ p_table_name TEXT DEFAULT 't',
+ p_schema_name TEXT DEFAULT current_schema()
+)
+RETURNS TABLE (
+ table_name TEXT,
+ total_updates BIGINT,
+ hot_updates BIGINT,
+ hot_update_percentage NUMERIC,
+ matches_expected BOOLEAN
+)
+LANGUAGE plpgsql
+AS $$
+DECLARE
+ v_relid oid;
+ v_qualified_name TEXT;
+ v_hot_updates BIGINT;
+ v_updates BIGINT;
+ v_xact_hot_updates BIGINT;
+ v_xact_updates BIGINT;
+BEGIN
+ -- Force statistics update
+ PERFORM pg_stat_force_next_flush();
+
+ -- Get table OID
+ v_qualified_name := quote_ident(p_schema_name) || '.' || quote_ident(p_table_name);
+ v_relid := v_qualified_name::regclass;
+
+ IF v_relid IS NULL THEN
+ RAISE EXCEPTION 'Table %.% not found', p_schema_name, p_table_name;
+ END IF;
+
+ -- Get cumulative + transaction stats
+ v_hot_updates := COALESCE(pg_stat_get_tuples_hot_updated(v_relid), 0);
+ v_updates := COALESCE(pg_stat_get_tuples_updated(v_relid), 0);
+ v_xact_hot_updates := COALESCE(pg_stat_get_xact_tuples_hot_updated(v_relid), 0);
+ v_xact_updates := COALESCE(pg_stat_get_xact_tuples_updated(v_relid), 0);
+
+ v_hot_updates := v_hot_updates + v_xact_hot_updates;
+ v_updates := v_updates + v_xact_updates;
+
+ RETURN QUERY
+ SELECT
+ p_table_name::TEXT,
+ v_updates::BIGINT,
+ v_hot_updates::BIGINT,
+ CASE WHEN v_updates > 0
+ THEN ROUND((v_hot_updates::numeric / v_updates::numeric * 100)::numeric, 2)
+ ELSE 0
+ END,
+ (v_hot_updates = expected)::BOOLEAN;
+END;
+$$;
+
+-- ================================================================
+-- Basic JSONB Expression Index
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_name_idx ON t((docs->>'name'));
+INSERT INTO t VALUES (1, '{"name": "alice", "age": 30}');
+
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET docs = '{"name": "alice", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update indexed JSONB field - should NOT be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update non-indexed field again - should be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 32}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Partial Index with Predicate Transitions
+-- ================================================================
+CREATE TABLE t(id INT, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_value_idx ON t(value) WHERE value > 10;
+INSERT INTO t VALUES (1, 5);
+
+-- Both outside predicate - should be HOT
+UPDATE t SET value = 8 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET value = 15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Both inside predicate, value changes - should NOT be HOT
+UPDATE t SET value = 20 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Transition out of predicate - should NOT be HOT
+UPDATE t SET value = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Both outside predicate again - should be HOT
+UPDATE t SET value = 3 WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Expression Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((docs->>'status'))
+ WHERE (docs->>'priority')::int > 5;
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3}');
+
+-- Both outside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 4}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 10}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Inside predicate, status changes - should NOT be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 10}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Inside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 8}';
+SELECT * FROM check_hot_updates(2, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN Index on JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data);
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "database"]}');
+
+-- Change tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+-- Change tags again - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+-- Add field without changing existing keys - GIN keys changed (added "note"), NOT HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "note": "test"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN Index with Unchanged Keys
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create GIN index on specific path
+CREATE INDEX t_gin_idx ON t USING gin((data->'tags'));
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "sql"], "status": "active"}');
+
+-- Change non-indexed field - GIN keys on 'tags' unchanged, should be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change indexed tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN with jsonb_path_ops
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data jsonb_path_ops);
+INSERT INTO t VALUES (1, '{"user": {"name": "alice"}, "tags": ["a", "b"]}');
+
+-- Change value at different path - keys changed, NOT HOT
+UPDATE t SET data = '{"user": {"name": "bob"}, "tags": ["a", "b"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- BRIN Index
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- BRIN tracks min/max per block range
+CREATE INDEX t_brin_idx ON t USING brin(value);
+INSERT INTO t VALUES (1, 100, 'initial');
+
+-- Update non-indexed column - BRIN doesn't care, should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update indexed column but stay in same range - should still be HOT
+-- (Note: BRIN tracks ranges, not exact values)
+UPDATE t SET value = 105 WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+
+-- Change to significantly different value - still HOT for single row
+-- (BRIN summary won't change for single-row updates in same block)
+UPDATE t SET value = 1000 WHERE id = 1;
+SELECT * FROM check_hot_updates(3, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Multi-Column Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, a INT, b INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t(id, abs(a), abs(b));
+INSERT INTO t VALUES (1, -5, -10);
+
+-- Change sign but not abs value - should be HOT
+UPDATE t SET a = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change abs value - should NOT be HOT
+UPDATE t SET b = -15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change id - should NOT be HOT
+UPDATE t SET id = 2 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Mixed Index Types (BRIN + Expression)
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_idx ON t USING brin(value);
+CREATE INDEX t_expr_idx ON t((data->>'status'));
+INSERT INTO t VALUES (1, 100, '{"status": "active"}');
+
+-- Update only BRIN column - should be HOT
+UPDATE t SET value = 200 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update only expression column - should NOT be HOT
+UPDATE t SET data = '{"status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update both - should NOT be HOT
+UPDATE t SET value = 300, data = '{"status": "pending"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Expression with COLLATION
+-- ================================================================
+CREATE TABLE t(id INT, name TEXT COLLATE "C")
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_lower_idx ON t(lower(name));
+INSERT INTO t VALUES (1, 'ALICE');
+
+-- Change case but not lowercase value - should be HOT
+UPDATE t SET name = 'Alice' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change lowercase value - should NOT be HOT
+UPDATE t SET name = 'BOB' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Array Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_array_len_idx ON t(array_length(tags, 1));
+INSERT INTO t VALUES (1, ARRAY['a', 'b', 'c']);
+
+-- Same length, different elements - should be HOT
+UPDATE t SET tags = ARRAY['d', 'e', 'f'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Different length - should NOT be HOT
+UPDATE t SET tags = ARRAY['d', 'e'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Nested JSONB Expression
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_nested_idx ON t((data->'user'->>'name'));
+INSERT INTO t VALUES ('{"user": {"name": "alice", "age": 30}}');
+
+-- Change nested non-indexed field - should be HOT
+UPDATE t SET data = '{"user": {"name": "alice", "age": 31}}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change nested indexed field - should NOT be HOT
+UPDATE t SET data = '{"user": {"name": "bob", "age": 31}}';
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Complex Predicate on Multiple JSONB Fields
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((data->>'status'))
+ WHERE (data->>'priority')::int > 5
+ AND (data->>'active')::boolean = true;
+
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3, "active": true}');
+
+-- Outside predicate (priority too low) - should be HOT
+UPDATE t SET data = '{"status": "done", "priority": 3, "active": true}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": true}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Inside predicate, change to outside (active = false) - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": false}';
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- TOASTed Values in Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, large_text TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_substr_idx ON t(substr(large_text, 1, 10));
+
+INSERT INTO t VALUES (1, repeat('x', 5000) || 'identifier');
+
+-- Change end of string, prefix unchanged - should be HOT
+UPDATE t SET large_text = repeat('x', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change prefix - should NOT be HOT
+UPDATE t SET large_text = repeat('y', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- TEST: GIN with TOASTed TEXT (tsvector)
+-- ================================================================
+CREATE TABLE t(id INT, content TEXT, search_vec tsvector)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+-- Create trigger to maintain tsvector
+CREATE TRIGGER tsvectorupdate_toast
+ BEFORE INSERT OR UPDATE ON t
+ FOR EACH ROW EXECUTE FUNCTION
+ tsvector_update_trigger(search_vec, 'pg_catalog.english', content);
+
+CREATE INDEX t_gin ON t USING gin(search_vec);
+
+-- Insert with large content (will be TOASTed)
+INSERT INTO t (id, content) VALUES
+ (1, repeat('important keyword ', 1000) || repeat('filler text ', 10000));
+
+-- Verify initial state
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('important');
+-- Expected: 1 row
+
+-- IMPORTANT: The BEFORE UPDATE trigger modifies search_vec, so by the time
+-- ExecWhichIndexesRequireUpdates() runs, search_vec has already changed.
+-- This means the comparison sees old tsvector vs. trigger-modified tsvector,
+-- not the natural progression. HOT won't happen because the trigger changed
+-- the indexed column.
+
+-- Update: Even though content keywords unchanged, trigger still fires
+UPDATE t
+SET content = repeat('important keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+-- Expected: 0 HOT (trigger modifies search_vec, blocking HOT)
+-- This is actually correct behavior - the trigger updated an indexed column
+
+-- Update: Change indexed keywords
+UPDATE t
+SET content = repeat('critical keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+-- Expected: 0 HOT (index keys changed)
+
+-- Verify query correctness
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('critical');
+-- Expected: 1 row
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- TEST: GIN with TOASTed JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin((data->'tags'));
+
+-- Insert with TOASTed JSONB
+INSERT INTO t (id, data) VALUES
+ (1, jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('x', 10000)
+ ));
+
+-- Update: Change large_field, tags unchanged - should be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: 1 HOT update
+
+-- Update: Change tags - should NOT be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "sql"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: Still 1 HOT
+
+-- Verify correctness
+SELECT count(*) FROM t WHERE data->'tags' @> '["database"]'::jsonb;
+-- Expected: 0 rows
+SELECT count(*) FROM t WHERE data->'tags' @> '["sql"]'::jsonb;
+-- Expected: 1 row
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- TEST: GIN with Array of Large Strings
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin(tags);
+
+-- Insert with large array elements (might be TOASTed)
+INSERT INTO t (id, tags) VALUES
+ (1, ARRAY[repeat('tag1', 1000), repeat('tag2', 1000)]);
+
+-- Update: Change to different large values - NOT HOT
+UPDATE t
+SET tags = ARRAY[repeat('tag3', 1000), repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+-- Expected: 0 HOT (keys actually changed)
+
+-- Update: Keep same tag values, just reorder - SHOULD BE HOT
+-- (GIN is order-insensitive: both [tag3,tag4] and [tag4,tag3]
+-- extract to the same sorted key set ['tag3','tag4'])
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000), repeat('tag3', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: 1 HOT (GIN keys semantically identical)
+
+-- Update: Remove an element - NOT HOT (keys changed)
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: Still 1 HOT (not this one)
+
+DROP TABLE t CASCADE;
+
+-- Cleanup
+DROP FUNCTION check_hot_updates(int, text, text);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 23bce72ae64..52ef8f10b35 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -390,6 +390,7 @@ CachedFunctionCompileCallback
CachedFunctionDeleteCallback
CachedFunctionHashEntry
CachedFunctionHashKey
+CachedIndexDatum
CachedPlan
CachedPlanSource
CallContext
--
2.49.0
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-07 21:36 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-09 19:27 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2025-11-16 18:53 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
@ 2025-11-19 18:00 ` Greg Burd <[email protected]>
2025-11-19 18:21 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-11-21 15:25 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
0 siblings, 2 replies; 37+ messages in thread
From: Greg Burd @ 2025-11-19 18:00 UTC (permalink / raw)
To: pgsql-hackers
On Nov 16 2025, at 1:53 pm, Greg Burd <[email protected]> wrote:
> 0004 - Enable HOT updates for expression and partial indexes
>
> This finally gets us back to where this project started, but on much
> more firm ground than before because we're not going to self-deadlock.
> The idea has grown from a small function into something larger, but only
> out of necessity.
>
> In this patch I add ExecWhichIndexesRequireUpdates() in execIndexing.c
> which implements (c) finding the set of attributes that force new index
> updates. This set can be very different from the modified indexed
> attributes. We know that some attributes are not equal to their
> previous versions, but does that mean that the index that references
> that attribute needs a new index tuple? It may, or it may not. Here's
> the comment on that function that explains:
>
> /*
> * ExecWhichIndexesRequireUpdates
> *
> * Determine which indexes need updating given modified indexed attributes.
> * This function is a companion to ExecCheckIndexedAttrsForChanges().
> On the
> * surface, they appear similar but they are doing two very different things.
> *
> * For a standard index on a set of attributes this is the intersection of
> * the mix_attrs and the index attrs (key, expression, but not predicate).
> *
> * For expression indexes and indexes which implement the amcomparedatums()
> * index AM API we'll need to form index datum and compare each
> attribute to
> * see if any actually changed.
> *
> * For expression indexes the result of the expression might not change
> at all,
> * this is common with JSONB columns which require expression indexes
> and where
> * it is commonplace to index a field within a document and have
> updates that
> * generally don't update that field.
> *
> * Partial indexes won't trigger index tuples when the old/new tuples
> are both
> * outside of the predicate range.
> *
> * For nbtree the amcomparedatums() API is critical as it requires that key
> * attributes are equal when they memcmp(), which might not be the case when
> * using type-specific comparison or factoring in collation which might make
> * an index case insensitive.
> *
> * All of this is to say that the goal is for the executor to know,
> ahead of
> * calling into the table AM for the update and before calling into the index
> * AM for inserting new index tuples, which attributes at a minimum will
> * necessitate a new index tuple.
> *
> ...
> */
Attached are rebased (d5b4f3a6d4e) patches with the only changes
happening in the last patch in the series.
0004 - Enable HOT updates for expression and partial indexes
I was never happy with the dual functions
ExecCheckIndexedAttrsForChanges() and ExecWhichIndexesRequireUpdates(),
it felt like too much overhead and duplication of effort. While
updating my tests, adding a few cases, I found that there was also a
flaw in the logic. So, time to rewrite and combine them.
What did I discover? Before the logic was to find the set of modified
indexed attributes then review all the indexes for changed attributes
using FormIndexDatum() and comparing before/after to see if expressions
really changed the value to be indexed or not. The first pass didn't
take into account expressions, the second did. So, an expression index
over JSONB data wouldn't extract and test the field within the document,
it was just comparing the entire document before/after using the jsonb
comparison function, no bueno.
This approach wraps both functions into one somewhat simplified
function. The logic is basically, iterate over the indexes reviewing
indexed attributes for changes. Along the way we call into the new
index AM's comparison function when present, otherwise we find and use
the proper type-specific comparison function for the datum. At the end
of the function we have our Bitmapset of attributes that should trigger
new index tuples.
> What's left undone?
>
> * I need to check code coverage so that I might
I did this and it was quite good, I'll do it again for this new series
but it's nice to see that the tests are exercising the vast majority of
the code paths.
> * create tests covering all the new cases
I think the coverage is good, maybe even redundant or overly complex in places.
> * update the README.HOT documentation, wiki, etc.
Soon, I hope to have this approach solid and under review before
solidifying the docs.
> * performance...
Still as yet unmeasured, I know that there is more work per-update to
perform these checks, so some overhead, but I don't know if that
overhead is more than before with HeapDetermineColumnsInfo() and
index_unchanged_by_update(). Those two functions did essentially the
same thing, only with binary comparison (datumIsEqual()). I need to
measure that. What about doing all this work outside of the buffer lock
in heap_update()? Surely that'll give back a bit or at least add to
concurrency. Forming index tuples a few extra times and evaluating the
expressions 3 times rather than 1 is going to hurt, I think I can come
up with a way to cache the formed datum and use it later on, but is that
worth it? Complex expressions, yes. Also, what about expressions that
expect to be executed once... and now are 3x? That's what forced my
update to the insert-conflict-specconflict.out test, but AFAICT there is
no way to test if an expression's value is going to change on update
without exercising it once for the old tuple and once for the new tuple.
Even if it were possible for an index to provide the key it might have
changed after the expression evaluation (as is the case in hash), so I
don't think this is avoidable. Maybe that's reason enough to add a
reloption to disable the expression evaluation piece of this? Given
that it might create a logic or performance regression. The flip side
is the potential to use the HOT path, that's a real savings.
One concerning thing is that nbtree's assumption that key attributes for
TIDs must use binary comparison for equality. This means that for our
common case (heap/btree) there is more work per-update than before,
which is why I need to measure. I could look into eliminating the
nbtree requirement, I don't understand it too well as yet by I believe
that on page split there is an attempt to deduplicate TIDs into a
TIDBitmap and the test for when that's possible is datumIsEqual(). If
that were the same as in this new code, possibly evening using
tts_attr_equal(), then... I don't know, I'll have to investigate. Chime
in here if you can educate me on this one. :)
best.
-greg
Attachments:
[application/octet-stream] v21-0001-Reorganize-heap-update-logic.patch (47.6K, 2-v21-0001-Reorganize-heap-update-logic.patch)
download | inline diff:
From 0de059ae17042a76594610e1b7b35dbb2db9415c Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 2 Nov 2025 11:36:20 -0500
Subject: [PATCH v21 1/4] Reorganize heap update logic
This commit refactors the interaction between heap_tuple_update(),
heap_update(), and simple_heap_update() to improve code organization
and flexibility. The changes are functionally equivalent to the
previous implementation and have no performance impact.
The primary motivation is to prepare for upcoming modifications to
how and where modified attributes are identified during the update
path, particularly for catalog updates.
As part of this reorganization, the handling of replica identity key
attributes has been adjusted. Instead of fetching a second copy of
the bitmap during an update operation, the caller is now required to
provide it. This change applies to both heap_update() and
heap_delete().
No user-visible changes.
---
src/backend/access/heap/heapam.c | 568 +++++++++++------------
src/backend/access/heap/heapam_handler.c | 117 ++++-
src/include/access/heapam.h | 24 +-
3 files changed, 410 insertions(+), 299 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 4b0c49f4bb0..aff47481345 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -39,18 +39,24 @@
#include "access/syncscan.h"
#include "access/valid.h"
#include "access/visibilitymap.h"
+#include "access/xact.h"
#include "access/xloginsert.h"
+#include "catalog/catalog.h"
#include "catalog/pg_database.h"
#include "catalog/pg_database_d.h"
#include "commands/vacuum.h"
+#include "nodes/bitmapset.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
+#include "storage/bufmgr.h"
+#include "storage/itemptr.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
#include "storage/procarray.h"
#include "utils/datum.h"
#include "utils/injection_point.h"
#include "utils/inval.h"
+#include "utils/relcache.h"
#include "utils/spccache.h"
#include "utils/syscache.h"
@@ -62,16 +68,8 @@ static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
HeapTuple newtup, HeapTuple old_key_tuple,
bool all_visible_cleared, bool new_all_visible_cleared);
#ifdef USE_ASSERT_CHECKING
-static void check_lock_if_inplace_updateable_rel(Relation relation,
- const ItemPointerData *otid,
- HeapTuple newtup);
static void check_inplace_rel_lock(HeapTuple oldtup);
#endif
-static Bitmapset *HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external);
static bool heap_acquire_tuplock(Relation relation, const ItemPointerData *tid,
LockTupleMode mode, LockWaitPolicy wait_policy,
bool *have_tuple_lock);
@@ -103,10 +101,10 @@ static bool ConditionalMultiXactIdWait(MultiXactId multi, MultiXactStatus status
static void index_delete_sort(TM_IndexDeleteOp *delstate);
static int bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate);
static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
-static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
+static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp,
+ Bitmapset *rid_attrs, bool key_required,
bool *copy);
-
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
* corresponding MultiXactStatuses (one to merely lock tuples, another one to
@@ -2799,6 +2797,7 @@ heap_delete(Relation relation, const ItemPointerData *tid,
Buffer buffer;
Buffer vmbuffer = InvalidBuffer;
TransactionId new_xmax;
+ Bitmapset *rid_attrs;
uint16 new_infomask,
new_infomask2;
bool have_tuple_lock = false;
@@ -2811,6 +2810,8 @@ heap_delete(Relation relation, const ItemPointerData *tid,
AssertHasSnapshotForToast(relation);
+ rid_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
/*
* Forbid this during a parallel operation, lest it allocate a combo CID.
* Other workers might need that combo CID for visibility checks, and we
@@ -3014,6 +3015,7 @@ l1:
UnlockTupleTuplock(relation, &(tp.t_self), LockTupleExclusive);
if (vmbuffer != InvalidBuffer)
ReleaseBuffer(vmbuffer);
+ bms_free(rid_attrs);
return result;
}
@@ -3035,7 +3037,10 @@ l1:
* Compute replica identity tuple before entering the critical section so
* we don't PANIC upon a memory allocation failure.
*/
- old_key_tuple = ExtractReplicaIdentity(relation, &tp, true, &old_key_copied);
+ old_key_tuple = ExtractReplicaIdentity(relation, &tp, rid_attrs,
+ true, &old_key_copied);
+ bms_free(rid_attrs);
+ rid_attrs = NULL;
/*
* If this is the first possibly-multixact-able operation in the current
@@ -3247,7 +3252,10 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* heap_update - replace a tuple
*
* See table_tuple_update() for an explanation of the parameters, except that
- * this routine directly takes a tuple rather than a slot.
+ * this routine directly takes a heap tuple rather than a slot.
+ *
+ * It's required that the caller has acquired the pin and lock on the buffer.
+ * That lock and pin will be managed here, not in the caller.
*
* In the failure cases, the routine fills *tmfd with the tuple's t_ctid,
* t_xmax (resolving a possible MultiXact, if necessary), and t_cmax (the last
@@ -3255,30 +3263,21 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* generated by another transaction).
*/
TM_Result
-heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+heap_update(Relation relation, HeapTupleData *oldtup,
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
+ Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
+ Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
+ Bitmapset *mix_attrs, Buffer *vmbuffer,
+ bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
- Bitmapset *hot_attrs;
- Bitmapset *sum_attrs;
- Bitmapset *key_attrs;
- Bitmapset *id_attrs;
- Bitmapset *interesting_attrs;
- Bitmapset *modified_attrs;
- ItemId lp;
- HeapTupleData oldtup;
HeapTuple heaptup;
HeapTuple old_key_tuple = NULL;
bool old_key_copied = false;
- Page page;
- BlockNumber block;
MultiXactStatus mxact_status;
- Buffer buffer,
- newbuf,
- vmbuffer = InvalidBuffer,
+ Buffer newbuf,
vmbuffer_new = InvalidBuffer;
bool need_toast;
Size newtupsize,
@@ -3292,7 +3291,6 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
bool all_visible_cleared_new = false;
bool checked_lockers;
bool locker_remains;
- bool id_has_external = false;
TransactionId xmax_new_tuple,
xmax_old_tuple;
uint16 infomask_old_tuple,
@@ -3300,144 +3298,13 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
infomask_new_tuple,
infomask2_new_tuple;
- Assert(ItemPointerIsValid(otid));
-
- /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
- Assert(HeapTupleHeaderGetNatts(newtup->t_data) <=
- RelationGetNumberOfAttributes(relation));
-
+ Assert(BufferIsLockedByMe(buffer));
+ Assert(ItemIdIsNormal(lp));
AssertHasSnapshotForToast(relation);
- /*
- * Forbid this during a parallel operation, lest it allocate a combo CID.
- * Other workers might need that combo CID for visibility checks, and we
- * have no provision for broadcasting it to them.
- */
- if (IsInParallelMode())
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
- errmsg("cannot update tuples during a parallel operation")));
-
-#ifdef USE_ASSERT_CHECKING
- check_lock_if_inplace_updateable_rel(relation, otid, newtup);
-#endif
-
- /*
- * Fetch the list of attributes to be checked for various operations.
- *
- * For HOT considerations, this is wasted effort if we fail to update or
- * have to put the new tuple on a different page. But we must compute the
- * list before obtaining buffer lock --- in the worst case, if we are
- * doing an update on one of the relevant system catalogs, we could
- * deadlock if we try to fetch the list later. In any case, the relcache
- * caches the data so this is usually pretty cheap.
- *
- * We also need columns used by the replica identity and columns that are
- * considered the "key" of rows in the table.
- *
- * Note that we get copies of each bitmap, so we need not worry about
- * relcache flush happening midway through.
- */
- hot_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_HOT_BLOCKING);
- sum_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_SUMMARIZED);
- key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
- id_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
- interesting_attrs = NULL;
- interesting_attrs = bms_add_members(interesting_attrs, hot_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, sum_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, key_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, id_attrs);
-
- block = ItemPointerGetBlockNumber(otid);
- INJECTION_POINT("heap_update-before-pin", NULL);
- buffer = ReadBuffer(relation, block);
- page = BufferGetPage(buffer);
-
- /*
- * Before locking the buffer, pin the visibility map page if it appears to
- * be necessary. Since we haven't got the lock yet, someone else might be
- * in the middle of changing this, so we'll need to recheck after we have
- * the lock.
- */
- if (PageIsAllVisible(page))
- visibilitymap_pin(relation, block, &vmbuffer);
-
- LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
-
- lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
-
- /*
- * Usually, a buffer pin and/or snapshot blocks pruning of otid, ensuring
- * we see LP_NORMAL here. When the otid origin is a syscache, we may have
- * neither a pin nor a snapshot. Hence, we may see other LP_ states, each
- * of which indicates concurrent pruning.
- *
- * Failing with TM_Updated would be most accurate. However, unlike other
- * TM_Updated scenarios, we don't know the successor ctid in LP_UNUSED and
- * LP_DEAD cases. While the distinction between TM_Updated and TM_Deleted
- * does matter to SQL statements UPDATE and MERGE, those SQL statements
- * hold a snapshot that ensures LP_NORMAL. Hence, the choice between
- * TM_Updated and TM_Deleted affects only the wording of error messages.
- * Settle on TM_Deleted, for two reasons. First, it avoids complicating
- * the specification of when tmfd->ctid is valid. Second, it creates
- * error log evidence that we took this branch.
- *
- * Since it's possible to see LP_UNUSED at otid, it's also possible to see
- * LP_NORMAL for a tuple that replaced LP_UNUSED. If it's a tuple for an
- * unrelated row, we'll fail with "duplicate key value violates unique".
- * XXX if otid is the live, newer version of the newtup row, we'll discard
- * changes originating in versions of this catalog row after the version
- * the caller got from syscache. See syscache-update-pruned.spec.
- */
- if (!ItemIdIsNormal(lp))
- {
- Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
-
- UnlockReleaseBuffer(buffer);
- Assert(!have_tuple_lock);
- if (vmbuffer != InvalidBuffer)
- ReleaseBuffer(vmbuffer);
- tmfd->ctid = *otid;
- tmfd->xmax = InvalidTransactionId;
- tmfd->cmax = InvalidCommandId;
- *update_indexes = TU_None;
-
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- /* modified_attrs not yet initialized */
- bms_free(interesting_attrs);
- return TM_Deleted;
- }
-
- /*
- * Fill in enough data in oldtup for HeapDetermineColumnsInfo to work
- * properly.
- */
- oldtup.t_tableOid = RelationGetRelid(relation);
- oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
- oldtup.t_len = ItemIdGetLength(lp);
- oldtup.t_self = *otid;
-
- /* the new tuple is ready, except for this: */
+ /* The new tuple is ready, except for this */
newtup->t_tableOid = RelationGetRelid(relation);
- /*
- * Determine columns modified by the update. Additionally, identify
- * whether any of the unmodified replica identity key attributes in the
- * old tuple is externally stored or not. This is required because for
- * such attributes the flattened value won't be WAL logged as part of the
- * new tuple so we must include it as part of the old_key_tuple. See
- * ExtractReplicaIdentity.
- */
- modified_attrs = HeapDetermineColumnsInfo(relation, interesting_attrs,
- id_attrs, &oldtup,
- newtup, &id_has_external);
-
/*
* If we're not updating any "key" column, we can grab a weaker lock type.
* This allows for more concurrency when we are running simultaneously
@@ -3449,7 +3316,7 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
* is updates that don't manipulate key columns, not those that
* serendipitously arrive at the same key values.
*/
- if (!bms_overlap(modified_attrs, key_attrs))
+ if (!bms_overlap(mix_attrs, pk_attrs))
{
*lockmode = LockTupleNoKeyExclusive;
mxact_status = MultiXactStatusNoKeyUpdate;
@@ -3473,17 +3340,10 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
key_intact = false;
}
- /*
- * Note: beyond this point, use oldtup not otid to refer to old tuple.
- * otid may very well point at newtup->t_self, which we will overwrite
- * with the new tuple's location, so there's great risk of confusion if we
- * use otid anymore.
- */
-
l2:
checked_lockers = false;
locker_remains = false;
- result = HeapTupleSatisfiesUpdate(&oldtup, cid, buffer);
+ result = HeapTupleSatisfiesUpdate(oldtup, cid, buffer);
/* see below about the "no wait" case */
Assert(result != TM_BeingModified || wait);
@@ -3515,8 +3375,8 @@ l2:
*/
/* must copy state data before unlocking buffer */
- xwait = HeapTupleHeaderGetRawXmax(oldtup.t_data);
- infomask = oldtup.t_data->t_infomask;
+ xwait = HeapTupleHeaderGetRawXmax(oldtup->t_data);
+ infomask = oldtup->t_data->t_infomask;
/*
* Now we have to do something about the existing locker. If it's a
@@ -3556,13 +3416,12 @@ l2:
* requesting a lock and already have one; avoids deadlock).
*/
if (!current_is_member)
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &oldtup->t_self, *lockmode,
LockWaitBlock, &have_tuple_lock);
/* wait for multixact */
MultiXactIdWait((MultiXactId) xwait, mxact_status, infomask,
- relation, &oldtup.t_self, XLTW_Update,
- &remain);
+ relation, &oldtup->t_self, XLTW_Update, &remain);
checked_lockers = true;
locker_remains = remain != 0;
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
@@ -3572,9 +3431,9 @@ l2:
* could update this tuple before we get to this point. Check
* for xmax change, and start over if so.
*/
- if (xmax_infomask_changed(oldtup.t_data->t_infomask,
+ if (xmax_infomask_changed(oldtup->t_data->t_infomask,
infomask) ||
- !TransactionIdEquals(HeapTupleHeaderGetRawXmax(oldtup.t_data),
+ !TransactionIdEquals(HeapTupleHeaderGetRawXmax(oldtup->t_data),
xwait))
goto l2;
}
@@ -3599,8 +3458,8 @@ l2:
* before this one, which are important to keep in case this
* subxact aborts.
*/
- if (!HEAP_XMAX_IS_LOCKED_ONLY(oldtup.t_data->t_infomask))
- update_xact = HeapTupleGetUpdateXid(oldtup.t_data);
+ if (!HEAP_XMAX_IS_LOCKED_ONLY(oldtup->t_data->t_infomask))
+ update_xact = HeapTupleGetUpdateXid(oldtup->t_data);
else
update_xact = InvalidTransactionId;
@@ -3641,9 +3500,9 @@ l2:
* lock.
*/
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &oldtup->t_self, *lockmode,
LockWaitBlock, &have_tuple_lock);
- XactLockTableWait(xwait, relation, &oldtup.t_self,
+ XactLockTableWait(xwait, relation, &oldtup->t_self,
XLTW_Update);
checked_lockers = true;
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
@@ -3653,20 +3512,20 @@ l2:
* other xact could update this tuple before we get to this point.
* Check for xmax change, and start over if so.
*/
- if (xmax_infomask_changed(oldtup.t_data->t_infomask, infomask) ||
+ if (xmax_infomask_changed(oldtup->t_data->t_infomask, infomask) ||
!TransactionIdEquals(xwait,
- HeapTupleHeaderGetRawXmax(oldtup.t_data)))
+ HeapTupleHeaderGetRawXmax(oldtup->t_data)))
goto l2;
/* Otherwise check if it committed or aborted */
- UpdateXmaxHintBits(oldtup.t_data, buffer, xwait);
- if (oldtup.t_data->t_infomask & HEAP_XMAX_INVALID)
+ UpdateXmaxHintBits(oldtup->t_data, buffer, xwait);
+ if (oldtup->t_data->t_infomask & HEAP_XMAX_INVALID)
can_continue = true;
}
if (can_continue)
result = TM_Ok;
- else if (!ItemPointerEquals(&oldtup.t_self, &oldtup.t_data->t_ctid))
+ else if (!ItemPointerEquals(&oldtup->t_self, &oldtup->t_data->t_ctid))
result = TM_Updated;
else
result = TM_Deleted;
@@ -3679,39 +3538,33 @@ l2:
result == TM_Updated ||
result == TM_Deleted ||
result == TM_BeingModified);
- Assert(!(oldtup.t_data->t_infomask & HEAP_XMAX_INVALID));
+ Assert(!(oldtup->t_data->t_infomask & HEAP_XMAX_INVALID));
Assert(result != TM_Updated ||
- !ItemPointerEquals(&oldtup.t_self, &oldtup.t_data->t_ctid));
+ !ItemPointerEquals(&oldtup->t_self, &oldtup->t_data->t_ctid));
}
if (crosscheck != InvalidSnapshot && result == TM_Ok)
{
/* Perform additional check for transaction-snapshot mode RI updates */
- if (!HeapTupleSatisfiesVisibility(&oldtup, crosscheck, buffer))
+ if (!HeapTupleSatisfiesVisibility(oldtup, crosscheck, buffer))
result = TM_Updated;
}
if (result != TM_Ok)
{
- tmfd->ctid = oldtup.t_data->t_ctid;
- tmfd->xmax = HeapTupleHeaderGetUpdateXid(oldtup.t_data);
+ tmfd->ctid = oldtup->t_data->t_ctid;
+ tmfd->xmax = HeapTupleHeaderGetUpdateXid(oldtup->t_data);
if (result == TM_SelfModified)
- tmfd->cmax = HeapTupleHeaderGetCmax(oldtup.t_data);
+ tmfd->cmax = HeapTupleHeaderGetCmax(oldtup->t_data);
else
tmfd->cmax = InvalidCommandId;
UnlockReleaseBuffer(buffer);
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
- if (vmbuffer != InvalidBuffer)
- ReleaseBuffer(vmbuffer);
+ UnlockTupleTuplock(relation, &oldtup->t_self, *lockmode);
+ if (*vmbuffer != InvalidBuffer)
+ ReleaseBuffer(*vmbuffer);
*update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
return result;
}
@@ -3724,10 +3577,10 @@ l2:
* tuple has been locked or updated under us, but hopefully it won't
* happen very often.
*/
- if (vmbuffer == InvalidBuffer && PageIsAllVisible(page))
+ if (*vmbuffer == InvalidBuffer && PageIsAllVisible(page))
{
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- visibilitymap_pin(relation, block, &vmbuffer);
+ visibilitymap_pin(relation, block, vmbuffer);
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
goto l2;
}
@@ -3738,9 +3591,9 @@ l2:
* If the tuple we're updating is locked, we need to preserve the locking
* info in the old tuple's Xmax. Prepare a new Xmax value for this.
*/
- compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
- oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2,
+ compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup->t_data),
+ oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2,
xid, *lockmode, true,
&xmax_old_tuple, &infomask_old_tuple,
&infomask2_old_tuple);
@@ -3752,12 +3605,12 @@ l2:
* tuple. (In rare cases that might also be InvalidTransactionId and yet
* not have the HEAP_XMAX_INVALID bit set; that's fine.)
*/
- if ((oldtup.t_data->t_infomask & HEAP_XMAX_INVALID) ||
- HEAP_LOCKED_UPGRADED(oldtup.t_data->t_infomask) ||
+ if ((oldtup->t_data->t_infomask & HEAP_XMAX_INVALID) ||
+ HEAP_LOCKED_UPGRADED(oldtup->t_data->t_infomask) ||
(checked_lockers && !locker_remains))
xmax_new_tuple = InvalidTransactionId;
else
- xmax_new_tuple = HeapTupleHeaderGetRawXmax(oldtup.t_data);
+ xmax_new_tuple = HeapTupleHeaderGetRawXmax(oldtup->t_data);
if (!TransactionIdIsValid(xmax_new_tuple))
{
@@ -3772,7 +3625,7 @@ l2:
* Note that since we're doing an update, the only possibility is that
* the lockers had FOR KEY SHARE lock.
*/
- if (oldtup.t_data->t_infomask & HEAP_XMAX_IS_MULTI)
+ if (oldtup->t_data->t_infomask & HEAP_XMAX_IS_MULTI)
{
GetMultiXactIdHintBits(xmax_new_tuple, &infomask_new_tuple,
&infomask2_new_tuple);
@@ -3800,7 +3653,7 @@ l2:
* Replace cid with a combo CID if necessary. Note that we already put
* the plain cid into the new tuple.
*/
- HeapTupleHeaderAdjustCmax(oldtup.t_data, &cid, &iscombo);
+ HeapTupleHeaderAdjustCmax(oldtup->t_data, &cid, &iscombo);
/*
* If the toaster needs to be activated, OR if the new tuple will not fit
@@ -3817,12 +3670,12 @@ l2:
relation->rd_rel->relkind != RELKIND_MATVIEW)
{
/* toast table entries should never be recursively toasted */
- Assert(!HeapTupleHasExternal(&oldtup));
+ Assert(!HeapTupleHasExternal(oldtup));
Assert(!HeapTupleHasExternal(newtup));
need_toast = false;
}
else
- need_toast = (HeapTupleHasExternal(&oldtup) ||
+ need_toast = (HeapTupleHasExternal(oldtup) ||
HeapTupleHasExternal(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
@@ -3855,9 +3708,9 @@ l2:
* updating, because the potentially created multixact would otherwise
* be wrong.
*/
- compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
- oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2,
+ compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup->t_data),
+ oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2,
xid, *lockmode, false,
&xmax_lock_old_tuple, &infomask_lock_old_tuple,
&infomask2_lock_old_tuple);
@@ -3867,18 +3720,18 @@ l2:
START_CRIT_SECTION();
/* Clear obsolete visibility flags ... */
- oldtup.t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
- oldtup.t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
- HeapTupleClearHotUpdated(&oldtup);
+ oldtup->t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
+ oldtup->t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
+ HeapTupleClearHotUpdated(oldtup);
/* ... and store info about transaction updating this tuple */
Assert(TransactionIdIsValid(xmax_lock_old_tuple));
- HeapTupleHeaderSetXmax(oldtup.t_data, xmax_lock_old_tuple);
- oldtup.t_data->t_infomask |= infomask_lock_old_tuple;
- oldtup.t_data->t_infomask2 |= infomask2_lock_old_tuple;
- HeapTupleHeaderSetCmax(oldtup.t_data, cid, iscombo);
+ HeapTupleHeaderSetXmax(oldtup->t_data, xmax_lock_old_tuple);
+ oldtup->t_data->t_infomask |= infomask_lock_old_tuple;
+ oldtup->t_data->t_infomask2 |= infomask2_lock_old_tuple;
+ HeapTupleHeaderSetCmax(oldtup->t_data, cid, iscombo);
/* temporarily make it look not-updated, but locked */
- oldtup.t_data->t_ctid = oldtup.t_self;
+ oldtup->t_data->t_ctid = oldtup->t_self;
/*
* Clear all-frozen bit on visibility map if needed. We could
@@ -3887,7 +3740,7 @@ l2:
* worthwhile.
*/
if (PageIsAllVisible(page) &&
- visibilitymap_clear(relation, block, vmbuffer,
+ visibilitymap_clear(relation, block, *vmbuffer,
VISIBILITYMAP_ALL_FROZEN))
cleared_all_frozen = true;
@@ -3901,10 +3754,10 @@ l2:
XLogBeginInsert();
XLogRegisterBuffer(0, buffer, REGBUF_STANDARD);
- xlrec.offnum = ItemPointerGetOffsetNumber(&oldtup.t_self);
+ xlrec.offnum = ItemPointerGetOffsetNumber(&oldtup->t_self);
xlrec.xmax = xmax_lock_old_tuple;
- xlrec.infobits_set = compute_infobits(oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2);
+ xlrec.infobits_set = compute_infobits(oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2);
xlrec.flags =
cleared_all_frozen ? XLH_LOCK_ALL_FROZEN_CLEARED : 0;
XLogRegisterData(&xlrec, SizeOfHeapLock);
@@ -3926,7 +3779,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = heap_toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = heap_toast_insert_or_update(relation, newtup, oldtup, 0);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
@@ -3962,20 +3815,20 @@ l2:
/* It doesn't fit, must use RelationGetBufferForTuple. */
newbuf = RelationGetBufferForTuple(relation, heaptup->t_len,
buffer, 0, NULL,
- &vmbuffer_new, &vmbuffer,
+ &vmbuffer_new, vmbuffer,
0);
/* We're all done. */
break;
}
/* Acquire VM page pin if needed and we don't have it. */
- if (vmbuffer == InvalidBuffer && PageIsAllVisible(page))
- visibilitymap_pin(relation, block, &vmbuffer);
+ if (*vmbuffer == InvalidBuffer && PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, vmbuffer);
/* Re-acquire the lock on the old tuple's page. */
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
/* Re-check using the up-to-date free space */
pagefree = PageGetHeapFreeSpace(page);
if (newtupsize > pagefree ||
- (vmbuffer == InvalidBuffer && PageIsAllVisible(page)))
+ (*vmbuffer == InvalidBuffer && PageIsAllVisible(page)))
{
/*
* Rats, it doesn't fit anymore, or somebody just now set the
@@ -4013,7 +3866,7 @@ l2:
* will include checking the relation level, there is no benefit to a
* separate check for the new tuple.
*/
- CheckForSerializableConflictIn(relation, &oldtup.t_self,
+ CheckForSerializableConflictIn(relation, &oldtup->t_self,
BufferGetBlockNumber(buffer));
/*
@@ -4021,7 +3874,6 @@ l2:
* has enough space for the new tuple. If they are the same buffer, only
* one pin is held.
*/
-
if (newbuf == buffer)
{
/*
@@ -4029,7 +3881,7 @@ l2:
* to do a HOT update. Check if any of the index columns have been
* changed.
*/
- if (!bms_overlap(modified_attrs, hot_attrs))
+ if (!bms_overlap(mix_attrs, hot_attrs))
{
use_hot_update = true;
@@ -4040,7 +3892,7 @@ l2:
* indexes if the columns were updated, or we may fail to detect
* e.g. value bound changes in BRIN minmax indexes.
*/
- if (bms_overlap(modified_attrs, sum_attrs))
+ if (bms_overlap(mix_attrs, sum_attrs))
summarized_update = true;
}
}
@@ -4057,10 +3909,8 @@ l2:
* logged. Pass old key required as true only if the replica identity key
* columns are modified or it has external data.
*/
- old_key_tuple = ExtractReplicaIdentity(relation, &oldtup,
- bms_overlap(modified_attrs, id_attrs) ||
- id_has_external,
- &old_key_copied);
+ old_key_tuple = ExtractReplicaIdentity(relation, oldtup, rid_attrs,
+ rep_id_key_required, &old_key_copied);
/* NO EREPORT(ERROR) from here till changes are logged */
START_CRIT_SECTION();
@@ -4082,7 +3932,7 @@ l2:
if (use_hot_update)
{
/* Mark the old tuple as HOT-updated */
- HeapTupleSetHotUpdated(&oldtup);
+ HeapTupleSetHotUpdated(oldtup);
/* And mark the new tuple as heap-only */
HeapTupleSetHeapOnly(heaptup);
/* Mark the caller's copy too, in case different from heaptup */
@@ -4091,7 +3941,7 @@ l2:
else
{
/* Make sure tuples are correctly marked as not-HOT */
- HeapTupleClearHotUpdated(&oldtup);
+ HeapTupleClearHotUpdated(oldtup);
HeapTupleClearHeapOnly(heaptup);
HeapTupleClearHeapOnly(newtup);
}
@@ -4100,17 +3950,17 @@ l2:
/* Clear obsolete visibility flags, possibly set by ourselves above... */
- oldtup.t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
- oldtup.t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
+ oldtup->t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
+ oldtup->t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
/* ... and store info about transaction updating this tuple */
Assert(TransactionIdIsValid(xmax_old_tuple));
- HeapTupleHeaderSetXmax(oldtup.t_data, xmax_old_tuple);
- oldtup.t_data->t_infomask |= infomask_old_tuple;
- oldtup.t_data->t_infomask2 |= infomask2_old_tuple;
- HeapTupleHeaderSetCmax(oldtup.t_data, cid, iscombo);
+ HeapTupleHeaderSetXmax(oldtup->t_data, xmax_old_tuple);
+ oldtup->t_data->t_infomask |= infomask_old_tuple;
+ oldtup->t_data->t_infomask2 |= infomask2_old_tuple;
+ HeapTupleHeaderSetCmax(oldtup->t_data, cid, iscombo);
/* record address of new tuple in t_ctid of old one */
- oldtup.t_data->t_ctid = heaptup->t_self;
+ oldtup->t_data->t_ctid = heaptup->t_self;
/* clear PD_ALL_VISIBLE flags, reset all visibilitymap bits */
if (PageIsAllVisible(BufferGetPage(buffer)))
@@ -4118,7 +3968,7 @@ l2:
all_visible_cleared = true;
PageClearAllVisible(BufferGetPage(buffer));
visibilitymap_clear(relation, BufferGetBlockNumber(buffer),
- vmbuffer, VISIBILITYMAP_VALID_BITS);
+ *vmbuffer, VISIBILITYMAP_VALID_BITS);
}
if (newbuf != buffer && PageIsAllVisible(BufferGetPage(newbuf)))
{
@@ -4143,12 +3993,12 @@ l2:
*/
if (RelationIsAccessibleInLogicalDecoding(relation))
{
- log_heap_new_cid(relation, &oldtup);
+ log_heap_new_cid(relation, oldtup);
log_heap_new_cid(relation, heaptup);
}
recptr = log_heap_update(relation, buffer,
- newbuf, &oldtup, heaptup,
+ newbuf, oldtup, heaptup,
old_key_tuple,
all_visible_cleared,
all_visible_cleared_new);
@@ -4173,7 +4023,7 @@ l2:
* both tuple versions in one call to inval.c so we can avoid redundant
* sinval messages.)
*/
- CacheInvalidateHeapTuple(relation, &oldtup, heaptup);
+ CacheInvalidateHeapTuple(relation, oldtup, heaptup);
/* Now we can release the buffer(s) */
if (newbuf != buffer)
@@ -4181,14 +4031,14 @@ l2:
ReleaseBuffer(buffer);
if (BufferIsValid(vmbuffer_new))
ReleaseBuffer(vmbuffer_new);
- if (BufferIsValid(vmbuffer))
- ReleaseBuffer(vmbuffer);
+ if (BufferIsValid(*vmbuffer))
+ ReleaseBuffer(*vmbuffer);
/*
* Release the lmgr tuple lock, if we had it.
*/
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &oldtup->t_self, *lockmode);
pgstat_count_heap_update(relation, use_hot_update, newbuf != buffer);
@@ -4221,13 +4071,6 @@ l2:
if (old_key_tuple != NULL && old_key_copied)
heap_freetuple(old_key_tuple);
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
-
return TM_Ok;
}
@@ -4236,7 +4079,7 @@ l2:
* Confirm adequate lock held during heap_update(), per rules from
* README.tuplock section "Locking to write inplace-updated tables".
*/
-static void
+void
check_lock_if_inplace_updateable_rel(Relation relation,
const ItemPointerData *otid,
HeapTuple newtup)
@@ -4408,7 +4251,7 @@ heap_attr_equals(TupleDesc tupdesc, int attrnum, Datum value1, Datum value2,
* listed as interesting) of the old tuple is a member of external_cols and is
* stored externally.
*/
-static Bitmapset *
+Bitmapset *
HeapDetermineColumnsInfo(Relation relation,
Bitmapset *interesting_cols,
Bitmapset *external_cols,
@@ -4491,25 +4334,175 @@ HeapDetermineColumnsInfo(Relation relation,
}
/*
- * simple_heap_update - replace a tuple
- *
- * This routine may be used to update a tuple when concurrent updates of
- * the target tuple are not expected (for example, because we have a lock
- * on the relation associated with the tuple). Any failure is reported
- * via ereport().
+ * This routine may be used to update a tuple when concurrent updates of the
+ * target tuple are not expected (for example, because we have a lock on the
+ * relation associated with the tuple). Any failure is reported via ereport().
*/
void
-simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup,
+simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TM_FailureData tmfd;
LockTupleMode lockmode;
+ Buffer buffer;
+ Buffer vmbuffer = InvalidBuffer;
+ Page page;
+ BlockNumber block;
+ Bitmapset *hot_attrs,
+ *sum_attrs,
+ *pk_attrs,
+ *rid_attrs,
+ *mix_attrs,
+ *idx_attrs;
+ ItemId lp;
+ HeapTupleData oldtup;
+ bool rep_id_key_required = false;
+
+ Assert(ItemPointerIsValid(otid));
+
+ /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
+ Assert(HeapTupleHeaderGetNatts(tuple->t_data) <=
+ RelationGetNumberOfAttributes(relation));
+
+ /*
+ * Forbid this during a parallel operation, lest it allocate a combo CID.
+ * Other workers might need that combo CID for visibility checks, and we
+ * have no provision for broadcasting it to them.
+ */
+ if (IsInParallelMode())
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
+ errmsg("cannot update tuples during a parallel operation")));
+
+#ifdef USE_ASSERT_CHECKING
+ check_lock_if_inplace_updateable_rel(relation, otid, tuple);
+#endif
+
+ /*
+ * Fetch the list of attributes to be checked for various operations.
+ *
+ * For HOT considerations, this is wasted effort if we fail to update or
+ * have to put the new tuple on a different page. But we must compute the
+ * list before obtaining buffer lock --- in the worst case, if we are
+ * doing an update on one of the relevant system catalogs, we could
+ * deadlock if we try to fetch the list later. In any case, the relcache
+ * caches the data so this is usually pretty cheap.
+ *
+ * We also need columns used by the replica identity and columns that are
+ * considered the "key" of rows in the table.
+ *
+ * Note that we get copies of each bitmap, so we need not worry about
+ * relcache flush happening midway through.
+ */
+ hot_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_HOT_BLOCKING);
+ sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+ pk_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
+ rid_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
+ idx_attrs = bms_copy(hot_attrs);
+ idx_attrs = bms_add_members(idx_attrs, sum_attrs);
+ idx_attrs = bms_add_members(idx_attrs, pk_attrs);
+ idx_attrs = bms_add_members(idx_attrs, rid_attrs);
+
+ block = ItemPointerGetBlockNumber(otid);
+ INJECTION_POINT("heap_update-before-pin", NULL);
+ buffer = ReadBuffer(relation, block);
+ page = BufferGetPage(buffer);
+
+ /*
+ * Before locking the buffer, pin the visibility map page if it appears to
+ * be necessary. Since we haven't got the lock yet, someone else might be
+ * in the middle of changing this, so we'll need to recheck after we have
+ * the lock.
+ */
+ if (PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, &vmbuffer);
+
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+
+ lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
+
+ /*
+ * Usually, a buffer pin and/or snapshot blocks pruning of otid, ensuring
+ * we see LP_NORMAL here. When the otid origin is a syscache, we may have
+ * neither a pin nor a snapshot. Hence, we may see other LP_ states, each
+ * of which indicates concurrent pruning.
+ *
+ * Failing with TM_Updated would be most accurate. However, unlike other
+ * TM_Updated scenarios, we don't know the successor ctid in LP_UNUSED and
+ * LP_DEAD cases. While the distinction between TM_Updated and TM_Deleted
+ * does matter to SQL statements UPDATE and MERGE, those SQL statements
+ * hold a snapshot that ensures LP_NORMAL. Hence, the choice between
+ * TM_Updated and TM_Deleted affects only the wording of error messages.
+ * Settle on TM_Deleted, for two reasons. First, it avoids complicating
+ * the specification of when tmfd->ctid is valid. Second, it creates
+ * error log evidence that we took this branch.
+ *
+ * Since it's possible to see LP_UNUSED at otid, it's also possible to see
+ * LP_NORMAL for a tuple that replaced LP_UNUSED. If it's a tuple for an
+ * unrelated row, we'll fail with "duplicate key value violates unique".
+ * XXX if otid is the live, newer version of the newtup row, we'll discard
+ * changes originating in versions of this catalog row after the version
+ * the caller got from syscache. See syscache-update-pruned.spec.
+ */
+ if (!ItemIdIsNormal(lp))
+ {
+ Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
+
+ UnlockReleaseBuffer(buffer);
+ if (vmbuffer != InvalidBuffer)
+ ReleaseBuffer(vmbuffer);
+ *update_indexes = TU_None;
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(idx_attrs);
+ /* mix_attrs not yet initialized */
+
+ elog(ERROR, "tuple concurrently deleted");
+
+ return;
+ }
+
+ /*
+ * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
+ * then pass that on to heap_update.
+ */
+ oldtup.t_tableOid = RelationGetRelid(relation);
+ oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
+ oldtup.t_len = ItemIdGetLength(lp);
+ oldtup.t_self = *otid;
+
+ mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
+ &oldtup, tuple, &rep_id_key_required);
+
+ /*
+ * We'll need to WAL log the replica identity attributes if either they
+ * overlap with the modified indexed attributes or, as we've checked for
+ * just now in HeapDetermineColumnsInfo, they were unmodified external
+ * indexed attributes.
+ */
+ rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+
+ result = heap_update(relation, &oldtup, tuple, GetCurrentCommandId(true),
+ InvalidSnapshot, true /* wait for commit */ , &tmfd, &lockmode,
+ buffer, page, block, lp, hot_attrs, sum_attrs, pk_attrs,
+ rid_attrs, mix_attrs, &vmbuffer, rep_id_key_required,
+ update_indexes);
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(mix_attrs);
+ bms_free(idx_attrs);
- result = heap_update(relation, otid, tup,
- GetCurrentCommandId(true), InvalidSnapshot,
- true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
switch (result)
{
case TM_SelfModified:
@@ -9149,12 +9142,11 @@ log_heap_new_cid(Relation relation, HeapTuple tup)
* the same tuple that was passed in.
*/
static HeapTuple
-ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
- bool *copy)
+ExtractReplicaIdentity(Relation relation, HeapTuple tp, Bitmapset *rid_attrs,
+ bool key_required, bool *copy)
{
TupleDesc desc = RelationGetDescr(relation);
char replident = relation->rd_rel->relreplident;
- Bitmapset *idattrs;
HeapTuple key_tuple;
bool nulls[MaxHeapAttributeNumber];
Datum values[MaxHeapAttributeNumber];
@@ -9185,17 +9177,13 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
if (!key_required)
return NULL;
- /* find out the replica identity columns */
- idattrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
-
/*
* If there's no defined replica identity columns, treat as !key_required.
* (This case should not be reachable from heap_update, since that should
* calculate key_required accurately. But heap_delete just passes
* constant true for key_required, so we can hit this case in deletes.)
*/
- if (bms_is_empty(idattrs))
+ if (bms_is_empty(rid_attrs))
return NULL;
/*
@@ -9208,7 +9196,7 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
for (int i = 0; i < desc->natts; i++)
{
if (bms_is_member(i + 1 - FirstLowInvalidHeapAttributeNumber,
- idattrs))
+ rid_attrs))
Assert(!nulls[i]);
else
nulls[i] = true;
@@ -9217,8 +9205,6 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
key_tuple = heap_form_tuple(desc, values, nulls);
*copy = true;
- bms_free(idattrs);
-
/*
* If the tuple, which by here only contains indexed columns, still has
* toasted columns, force them to be inlined. This is somewhat unlikely
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bcbac844bb6..1cf9a18775d 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -44,6 +44,7 @@
#include "storage/procarray.h"
#include "storage/smgr.h"
#include "utils/builtins.h"
+#include "utils/injection_point.h"
#include "utils/rel.h"
static void reform_and_rewrite_tuple(HeapTuple tuple,
@@ -312,23 +313,133 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
return heap_delete(relation, tid, cid, crosscheck, wait, tmfd, changingPart);
}
-
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd,
LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
{
+ bool rep_id_key_required = false;
bool shouldFree = true;
HeapTuple tuple = ExecFetchSlotHeapTuple(slot, true, &shouldFree);
+ HeapTupleData oldtup;
+ Buffer buffer;
+ Buffer vmbuffer = InvalidBuffer;
+ Page page;
+ BlockNumber block;
+ ItemId lp;
+ Bitmapset *hot_attrs,
+ *sum_attrs,
+ *pk_attrs,
+ *rid_attrs,
+ *mix_attrs,
+ *idx_attrs;
TM_Result result;
+ Assert(ItemPointerIsValid(otid));
+
+ /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
+ Assert(HeapTupleHeaderGetNatts(tuple->t_data) <=
+ RelationGetNumberOfAttributes(relation));
+
+ /*
+ * Forbid this during a parallel operation, lest it allocate a combo CID.
+ * Other workers might need that combo CID for visibility checks, and we
+ * have no provision for broadcasting it to them.
+ */
+ if (IsInParallelMode())
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
+ errmsg("cannot update tuples during a parallel operation")));
+
+#ifdef USE_ASSERT_CHECKING
+ check_lock_if_inplace_updateable_rel(relation, otid, tuple);
+#endif
+
+ /*
+ * Fetch the list of attributes to be checked for various operations.
+ *
+ * For HOT considerations, this is wasted effort if we fail to update or
+ * have to put the new tuple on a different page. But we must compute the
+ * list before obtaining buffer lock --- in the worst case, if we are
+ * doing an update on one of the relevant system catalogs, we could
+ * deadlock if we try to fetch the list later. In any case, the relcache
+ * caches the data so this is usually pretty cheap.
+ *
+ * We also need columns used by the replica identity and columns that are
+ * considered the "key" of rows in the table.
+ *
+ * Note that we get copies of each bitmap, so we need not worry about
+ * relcache flush happening midway through.
+ */
+ hot_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_HOT_BLOCKING);
+ sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+ pk_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
+ rid_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
+ idx_attrs = bms_copy(hot_attrs);
+ idx_attrs = bms_add_members(idx_attrs, sum_attrs);
+ idx_attrs = bms_add_members(idx_attrs, pk_attrs);
+ idx_attrs = bms_add_members(idx_attrs, rid_attrs);
+
+ block = ItemPointerGetBlockNumber(otid);
+ INJECTION_POINT("heap_update-before-pin", NULL);
+ buffer = ReadBuffer(relation, block);
+ page = BufferGetPage(buffer);
+
+ /*
+ * Before locking the buffer, pin the visibility map page if it appears to
+ * be necessary. Since we haven't got the lock yet, someone else might be
+ * in the middle of changing this, so we'll need to recheck after we have
+ * the lock.
+ */
+ if (PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, &vmbuffer);
+
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+
+ lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
+
+ Assert(ItemIdIsNormal(lp));
+
+ /*
+ * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
+ * then pass that on to heap_update.
+ */
+ oldtup.t_tableOid = RelationGetRelid(relation);
+ oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
+ oldtup.t_len = ItemIdGetLength(lp);
+ oldtup.t_self = *otid;
+
+ mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
+ &oldtup, tuple, &rep_id_key_required);
+
+ /*
+ * We'll need to WAL log the replica identity attributes if either they
+ * overlap with the modified indexed attributes or, as we've checked for
+ * just now in HeapDetermineColumnsInfo, they were unmodified external
+ * indexed attributes.
+ */
+ rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
- result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
- tmfd, lockmode, update_indexes);
+ result = heap_update(relation, &oldtup, tuple, cid, crosscheck, wait, tmfd, lockmode,
+ buffer, page, block, lp, hot_attrs, sum_attrs, pk_attrs,
+ rid_attrs, mix_attrs, &vmbuffer, rep_id_key_required, update_indexes);
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(mix_attrs);
+ bms_free(idx_attrs);
+
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 909db73b7bb..41d541aa6b2 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -321,11 +321,13 @@ extern TM_Result heap_delete(Relation relation, const ItemPointerData *tid,
TM_FailureData *tmfd, bool changingPart);
extern void heap_finish_speculative(Relation relation, const ItemPointerData *tid);
extern void heap_abort_speculative(Relation relation, const ItemPointerData *tid);
-extern TM_Result heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+extern TM_Result heap_update(Relation relation, HeapTupleData *oldtup,
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
+ Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
+ Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
+ Bitmapset *mix_attrs, Buffer *vmbuffer,
+ bool rep_id_key_required, TU_UpdateIndexes *update_indexes);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -391,6 +393,18 @@ extern void log_heap_prune_and_freeze(Relation relation, Buffer buffer,
OffsetNumber *dead, int ndead,
OffsetNumber *unused, int nunused);
+/* in heap/heapam.c */
+extern Bitmapset *HeapDetermineColumnsInfo(Relation relation,
+ Bitmapset *interesting_cols,
+ Bitmapset *external_cols,
+ HeapTuple oldtup, HeapTuple newtup,
+ bool *has_external);
+#ifdef USE_ASSERT_CHECKING
+extern void check_lock_if_inplace_updateable_rel(Relation relation,
+ const ItemPointerData *otid,
+ HeapTuple newtup);
+#endif
+
/* in heap/vacuumlazy.c */
extern void heap_vacuum_rel(Relation rel,
const VacuumParams params, BufferAccessStrategy bstrategy);
--
2.49.0
[application/octet-stream] v21-0002-Track-changed-indexed-columns-in-the-executor-du.patch (34.3K, 3-v21-0002-Track-changed-indexed-columns-in-the-executor-du.patch)
download | inline diff:
From edc170a3f61de2141b383134ae40f105ee90aebe Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 26 Oct 2025 10:49:25 -0400
Subject: [PATCH v21 2/4] Track changed indexed columns in the executor during
UPDATEs
Refactor executor update logic to determine which indexed columns have
actually changed during an UPDATE operation rather than leaving this up
to HeapDetermineColumnsInfo in heap_update. This enables the comparison
to happen without taking a lock on the page and opens the door to reuse
in other code paths.
Because heap_update now requires the caller to provide the modified
indexed columns simple_heap_update has become a tad more complex. It is
frequently called from CatalogTupleUpdate which either updates heap
tuples via their form or using heap_modify_tuple. In both cases the
caller does know the modified set of attributes, but sadly those
attributes are lost before being provided to simple_heap_update. Due to
that the "simple" path has to retain the HeapDetermineColumnsInfo logic
of old (for now). In order for that to work it was necessary to split
the (overly large) heap_update call itself up. This moves up into
simple_heap_update and heap_tuple_update a bit of what existed in
heap_update itself. Ideally this will be cleaned up once
CatalogTupleUpdate paths are all recording modified attributes
correctly, when that happens the "simple" path can be simplified again.
ExecCheckIndexedAttrsForChanges replaces HeapDeterminesColumnsInfo and
tts_attr_equal replaces heap_attr_equal changing the test for equality
when calling into heap_tuple_update (but not simple_heap_update). In
the past we used datumIsEqual(), essentially a binary comparison using
memcmp(), now the comparison code in tts_attr_equal uses type-specific
equality function when available and falls back to datumIsEqual() when
not. This change in equality testing has some intended implications and
opens the door for more HOT updates (foreshadowing). For instance,
indexes with collation information allowing more HOT updates when the
index is specified to be case insensitive.
This change forced some logic changes in execReplication on the update
paths is now it is required to have knowledge of the set of attributes
that are both changed and referenced by indexes. Luckilly, the this is
available within calls to slot_modify_data() where LogicalRepTupleData
is processed and has a set of updated attributes. In this case rather
than using ExecCheckIndexedAttrsForChanges we can preseve what
slot_modify_data() identifies as the modified set and then intersect
that with the set of indexes on the relation and get the correct set of
modified indexed attributes required on heap_update().
---
src/backend/access/heap/heapam.c | 12 +-
src/backend/access/heap/heapam_handler.c | 72 +++++--
src/backend/access/table/tableam.c | 5 +-
src/backend/executor/execMain.c | 1 +
src/backend/executor/execReplication.c | 7 +
src/backend/executor/nodeModifyTable.c | 247 ++++++++++++++++++++++-
src/backend/nodes/bitmapset.c | 4 +
src/backend/replication/logical/worker.c | 72 ++++++-
src/backend/utils/cache/relcache.c | 15 ++
src/include/access/tableam.h | 8 +-
src/include/executor/executor.h | 5 +
src/include/nodes/execnodes.h | 1 +
src/include/utils/rel.h | 1 +
src/include/utils/relcache.h | 1 +
14 files changed, 415 insertions(+), 36 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index aff47481345..1cdb72b3a7a 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -3263,12 +3263,12 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* generated by another transaction).
*/
TM_Result
-heap_update(Relation relation, HeapTupleData *oldtup,
- HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
- Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
- Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
- Bitmapset *mix_attrs, Buffer *vmbuffer,
+heap_update(Relation relation, HeapTupleData *oldtup, HeapTuple newtup,
+ CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode,
+ Buffer buffer, Page page, BlockNumber block, ItemId lp,
+ Bitmapset *hot_attrs, Bitmapset *sum_attrs, Bitmapset *pk_attrs,
+ Bitmapset *rid_attrs, Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 1cf9a18775d..ef08e1d3e10 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -315,9 +315,12 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
- CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd,
- LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
+ CommandId cid, Snapshot snapshot,
+ Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd,
+ LockTupleMode *lockmode,
+ Bitmapset *mix_attrs,
+ TU_UpdateIndexes *update_indexes)
{
bool rep_id_key_required = false;
bool shouldFree = true;
@@ -332,7 +335,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
*sum_attrs,
*pk_attrs,
*rid_attrs,
- *mix_attrs,
*idx_attrs;
TM_Result result;
@@ -414,16 +416,61 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
oldtup.t_len = ItemIdGetLength(lp);
oldtup.t_self = *otid;
- mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
- &oldtup, tuple, &rep_id_key_required);
-
/*
- * We'll need to WAL log the replica identity attributes if either they
- * overlap with the modified indexed attributes or, as we've checked for
- * just now in HeapDetermineColumnsInfo, they were unmodified external
- * indexed attributes.
+ * We'll need to include the replica identity key when either the identity
+ * key attributes overlap with the modified index attributes or when the
+ * replica identity attributes are stored externally. This is required
+ * because for such attributes the flattened value won't be WAL logged as
+ * part of the new tuple so we must determine if we need to extract and
+ * include them as part of the old_key_tuple (see ExtractReplicaIdentity).
*/
- rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+ rep_id_key_required = bms_overlap(mix_attrs, rid_attrs);
+ if (!rep_id_key_required)
+ {
+ Bitmapset *attrs;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ int attidx = -1;
+
+ /*
+ * We don't own idx_attrs so we'll copy it and remove the modified set
+ * to reduce the attributes we need to test in the while loop and
+ * avoid a two branches in the loop.
+ */
+ attrs = bms_difference(idx_attrs, mix_attrs);
+ attrs = bms_int_members(attrs, rid_attrs);
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /*
+ * attidx is zero-based, attrnum is the normal attribute number
+ */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value;
+ bool isnull;
+
+ /*
+ * System attributes are not added into interesting_attrs in
+ * relcache
+ */
+ Assert(attrnum > 0);
+
+ value = heap_getattr(&oldtup, attrnum, tupdesc, &isnull);
+
+ /* No need to check attributes that can't be stored externally */
+ if (isnull ||
+ TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
+ continue;
+
+ /* Check if the old tuple's attribute is stored externally */
+ if (VARATT_IS_EXTERNAL((struct varlena *) DatumGetPointer(value)))
+ {
+ rep_id_key_required = true;
+ break;
+ }
+ }
+
+ bms_free(attrs);
+ }
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
@@ -437,7 +484,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
bms_free(sum_attrs);
bms_free(pk_attrs);
bms_free(rid_attrs);
- bms_free(mix_attrs);
bms_free(idx_attrs);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 5e41404937e..dadcf03ed24 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -336,6 +336,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
+ Bitmapset *modified_indexed_cols,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -346,7 +347,9 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
GetCurrentCommandId(true),
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, &lockmode,
+ modified_indexed_cols,
+ update_indexes);
switch (result)
{
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 27c9eec697b..6b7b6bc8019 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1282,6 +1282,7 @@ InitResultRelInfo(ResultRelInfo *resultRelInfo,
/* The following fields are set later if needed */
resultRelInfo->ri_RowIdAttNo = 0;
resultRelInfo->ri_extraUpdatedCols = NULL;
+ resultRelInfo->ri_ChangedIndexedCols = NULL;
resultRelInfo->ri_projectNew = NULL;
resultRelInfo->ri_newTupleSlot = NULL;
resultRelInfo->ri_oldTupleSlot = NULL;
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index def32774c90..2709e2db0f2 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -32,6 +32,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
#include "utils/typcache.h"
@@ -936,7 +937,13 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ /*
+ * We're not going to call ExecCheckIndexedAttrsForChanges here
+ * because we've already identified the changes earlier on thanks to
+ * slot_modify_data.
+ */
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
+ resultRelInfo->ri_ChangedIndexedCols,
&update_indexes);
conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 00429326c34..34f86546fc9 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -17,6 +17,7 @@
* ExecModifyTable - retrieve the next tuple from the node
* ExecEndModifyTable - shut down the ModifyTable node
* ExecReScanModifyTable - rescan the ModifyTable node
+ * ExecCheckIndexedAttrsForChanges - find set of updated indexed columns
*
* NOTES
* The ModifyTable node receives input from its outerPlan, which is
@@ -54,11 +55,14 @@
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/tupconvert.h"
+#include "access/tupdesc.h"
#include "access/xact.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
#include "executor/nodeModifyTable.h"
+#include "executor/tuptable.h"
#include "foreign/fdwapi.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
@@ -68,6 +72,8 @@
#include "storage/lmgr.h"
#include "utils/builtins.h"
#include "utils/datum.h"
+#include "utils/float.h"
+#include "utils/lsyscache.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
@@ -176,6 +182,219 @@ static TupleTableSlot *ExecMergeNotMatched(ModifyTableContext *context,
bool canSetTag);
+/*
+ * Compare two datums using the type's default equality operator.
+ *
+ * Returns true if the values are equal according to the type's equality
+ * operator, false otherwise. Falls back to binary comparison if no
+ * type-specific operator is available.
+ *
+ * This function uses the TypeCache infrastructure which caches operator
+ * lookups for efficiency.
+ */
+bool
+tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
+ Datum value1, Datum value2)
+{
+ TypeCacheEntry *typentry;
+
+ LOCAL_FCINFO(fcinfo, 2);
+ Datum result;
+
+ /*
+ * Fast path for common types to avoid even the type cache lookup. These
+ * types have simple equality semantics.
+ */
+ switch (typid)
+ {
+ case INT2OID:
+ return DatumGetInt16(value1) == DatumGetInt16(value2);
+ case INT4OID:
+ return DatumGetInt32(value1) == DatumGetInt32(value2);
+ case INT8OID:
+ return DatumGetInt64(value1) == DatumGetInt64(value2);
+ case FLOAT4OID:
+ return !float4_cmp_internal(DatumGetFloat4(value1), DatumGetFloat4(value2));
+ case FLOAT8OID:
+ return !float8_cmp_internal(DatumGetFloat8(value1), DatumGetFloat8(value2));
+ case BOOLOID:
+ return DatumGetBool(value1) == DatumGetBool(value2);
+ case OIDOID:
+ case REGPROCOID:
+ case REGPROCEDUREOID:
+ case REGOPEROID:
+ case REGOPERATOROID:
+ case REGCLASSOID:
+ case REGTYPEOID:
+ case REGROLEOID:
+ case REGNAMESPACEOID:
+ case REGCONFIGOID:
+ case REGDICTIONARYOID:
+ return DatumGetObjectId(value1) == DatumGetObjectId(value2);
+ case CHAROID:
+ return DatumGetChar(value1) == DatumGetChar(value2);
+ default:
+ /* Continue to type cache lookup */
+ break;
+ }
+
+ /*
+ * Look up the type's equality operator using the type cache. Request both
+ * the operator OID and the function info for efficiency.
+ */
+ typentry = lookup_type_cache(typid,
+ TYPECACHE_EQ_OPR | TYPECACHE_EQ_OPR_FINFO);
+
+ /*
+ * If no equality operator is available, fall back to binary comparison.
+ * This handles types that don't have proper equality operators defined.
+ */
+ if (!OidIsValid(typentry->eq_opr))
+ return datumIsEqual(value1, value2, typbyval, typlen);
+
+ /*
+ * Use the cached function info if available, otherwise look it up. The
+ * type cache keeps this around so subsequent calls are fast.
+ */
+ if (typentry->eq_opr_finfo.fn_addr == NULL)
+ {
+ Oid eq_proc = get_opcode(typentry->eq_opr);
+
+ if (!OidIsValid(eq_proc))
+ /* Shouldn't happen, but fall back to binary comparison */
+ return datumIsEqual(value1, value2, typbyval, typlen);
+
+ fmgr_info_cxt(eq_proc, &typentry->eq_opr_finfo,
+ CacheMemoryContext);
+ }
+
+ /* Set up function call */
+ InitFunctionCallInfoData(*fcinfo, &typentry->eq_opr_finfo, 2,
+ collation, NULL, NULL);
+
+ fcinfo->args[0].value = value1;
+ fcinfo->args[0].isnull = false;
+ fcinfo->args[1].value = value2;
+ fcinfo->args[1].isnull = false;
+
+ /* Invoke the equality operator */
+ result = FunctionCallInvoke(fcinfo);
+
+ /*
+ * If the function returned NULL (shouldn't happen for equality ops),
+ * treat as not equal for safety.
+ */
+ if (fcinfo->isnull)
+ return false;
+
+ return DatumGetBool(result);
+}
+
+/*
+ * Determine which updated attributes actually changed values between old and
+ * new tuples and are referenced by indexes on the relation.
+ *
+ * Returns a Bitmapset of attribute offsets (0-based, adjusted by
+ * FirstLowInvalidHeapAttributeNumber) or NULL if no attributes changed.
+ */
+Bitmapset *
+ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
+ TupleTableSlot *tts_old,
+ TupleTableSlot *tts_new)
+{
+ Relation relation = relinfo->ri_RelationDesc;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ Bitmapset *indexed_attrs;
+ Bitmapset *modified = NULL;
+ int attidx;
+
+ /* If no indexes, we're done */
+ if (relinfo->ri_NumIndices == 0)
+ return NULL;
+
+ /*
+ * Get the set of index key attributes. This includes summarizing,
+ * expression indexes and attributes mentioned in the predicate of a
+ * partition but not those in INCLUDING.
+ */
+ indexed_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_INDEXED);
+ Assert(!bms_is_empty(indexed_attrs));
+
+ /*
+ * NOTE: It is important to scan all indexed attributes in the tuples
+ * because ExecGetAllUpdatedCols won't include columns that may have been
+ * modified via heap_modify_tuple_by_col which is the case in
+ * tsvector_update_trigger.
+ */
+ attidx = -1;
+ while ((attidx = bms_next_member(indexed_attrs, attidx)) >= 0)
+ {
+ /* attidx is zero-based, attrnum is the normal attribute number */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Form_pg_attribute attr;
+ bool oldnull,
+ newnull;
+ Datum oldval,
+ newval;
+
+ /*
+ * If it's a whole-tuple reference, record as modified. It's not
+ * really worth supporting this case, since it could only succeed
+ * after a no-op update, which is hardly a case worth optimizing for.
+ */
+ if (attrnum == 0)
+ {
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /*
+ * Likewise, include in the modified set any system attribute other
+ * than tableOID; we cannot expect these to be consistent in a HOT
+ * chain, or even to be set correctly yet in the new tuple.
+ */
+ if (attrnum < 0)
+ {
+ if (attrnum != TableOidAttributeNumber)
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /* Extract values from both slots */
+ oldval = slot_getattr(tts_old, attrnum, &oldnull);
+ newval = slot_getattr(tts_new, attrnum, &newnull);
+
+ /* If one value is NULL and the other is not, they are not equal */
+ if (oldnull != newnull)
+ {
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /* If both are NULL, consider them equal */
+ if (oldnull)
+ continue;
+
+ /* Get attribute metadata */
+ Assert(attrnum > 0 && attrnum <= tupdesc->natts);
+ attr = TupleDescAttr(tupdesc, attrnum - 1);
+
+ /* Compare using type-specific equality operator */
+ if (!tts_attr_equal(attr->atttypid,
+ attr->attcollation,
+ attr->attbyval,
+ attr->attlen,
+ oldval,
+ newval))
+ modified = bms_add_member(modified, attidx);
+ }
+
+ bms_free(indexed_attrs);
+
+ return modified;
+}
+
/*
* Verify that the tuples to be produced by INSERT match the
* target relation's rowtype
@@ -2168,8 +2387,8 @@ ExecUpdatePrepareSlot(ResultRelInfo *resultRelInfo,
*/
static TM_Result
ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
- ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *slot,
- bool canSetTag, UpdateContext *updateCxt)
+ ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *oldSlot,
+ TupleTableSlot *slot, bool canSetTag, UpdateContext *updateCxt)
{
EState *estate = context->estate;
Relation resultRelationDesc = resultRelInfo->ri_RelationDesc;
@@ -2291,6 +2510,16 @@ lreplace:
if (resultRelationDesc->rd_att->constr)
ExecConstraints(resultRelInfo, slot, estate);
+ /*
+ * Identify which, if any, indexed attributes were modified here so that
+ * we might reuse it in a few places.
+ */
+ bms_free(resultRelInfo->ri_ChangedIndexedCols);
+ resultRelInfo->ri_ChangedIndexedCols = NULL;
+
+ resultRelInfo->ri_ChangedIndexedCols =
+ ExecCheckIndexedAttrsForChanges(resultRelInfo, oldSlot, slot);
+
/*
* replace the heap tuple
*
@@ -2306,6 +2535,7 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
+ resultRelInfo->ri_ChangedIndexedCols,
&updateCxt->updateIndexes);
return result;
@@ -2524,8 +2754,9 @@ ExecUpdate(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
*/
redo_act:
lockedtid = *tupleid;
- result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, slot,
- canSetTag, &updateCxt);
+
+ result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, oldSlot,
+ slot, canSetTag, &updateCxt);
/*
* If ExecUpdateAct reports that a cross-partition update was done,
@@ -3222,8 +3453,8 @@ lmerge_matched:
Assert(oldtuple == NULL);
result = ExecUpdateAct(context, resultRelInfo, tupleid,
- NULL, newslot, canSetTag,
- &updateCxt);
+ NULL, resultRelInfo->ri_oldTupleSlot,
+ newslot, canSetTag, &updateCxt);
/*
* As in ExecUpdate(), if ExecUpdateAct() reports that a
@@ -3248,6 +3479,7 @@ lmerge_matched:
tupleid, NULL, newslot);
mtstate->mt_merge_updated += 1;
}
+
break;
case CMD_DELETE:
@@ -4354,7 +4586,7 @@ ExecModifyTable(PlanState *pstate)
* For UPDATE/DELETE/MERGE, fetch the row identity info for the tuple
* to be updated/deleted/merged. For a heap relation, that's a TID;
* otherwise we may have a wholerow junk attr that carries the old
- * tuple in toto. Keep this in step with the part of
+ * tuple in total. Keep this in step with the part of
* ExecInitModifyTable that sets up ri_RowIdAttNo.
*/
if (operation == CMD_UPDATE || operation == CMD_DELETE ||
@@ -4530,6 +4762,7 @@ ExecModifyTable(PlanState *pstate)
/* Now apply the update. */
slot = ExecUpdate(&context, resultRelInfo, tupleid, oldtuple,
oldSlot, slot, node->canSetTag);
+
if (tuplock)
UnlockTuple(resultRelInfo->ri_RelationDesc, tupleid,
InplaceUpdateTupleLock);
diff --git a/src/backend/nodes/bitmapset.c b/src/backend/nodes/bitmapset.c
index b4ecf0b0390..9014990267a 100644
--- a/src/backend/nodes/bitmapset.c
+++ b/src/backend/nodes/bitmapset.c
@@ -238,6 +238,10 @@ bms_make_singleton(int x)
void
bms_free(Bitmapset *a)
{
+#if USE_ASSERT_CHECKING
+ Assert(bms_is_valid_set(a));
+#endif
+
if (a)
pfree(a);
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 93970c6af29..b363eaa49cc 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -243,6 +243,8 @@
*/
#include "postgres.h"
+#include "access/sysattr.h"
+#include "nodes/bitmapset.h"
#include <sys/stat.h>
#include <unistd.h>
@@ -275,7 +277,6 @@
#include "replication/logicalrelation.h"
#include "replication/logicalworker.h"
#include "replication/origin.h"
-#include "replication/slot.h"
#include "replication/walreceiver.h"
#include "replication/worker_internal.h"
#include "rewrite/rewriteHandler.h"
@@ -291,6 +292,7 @@
#include "utils/memutils.h"
#include "utils/pg_lsn.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/rls.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
@@ -1110,15 +1112,18 @@ slot_store_data(TupleTableSlot *slot, LogicalRepRelMapEntry *rel,
* "slot" is filled with a copy of the tuple in "srcslot", replacing
* columns provided in "tupleData" and leaving others as-is.
*
+ * Returns a bitmap of the modified columns.
+ *
* Caution: unreplaced pass-by-ref columns in "slot" will point into the
* storage for "srcslot". This is OK for current usage, but someday we may
* need to materialize "slot" at the end to make it independent of "srcslot".
*/
-static void
+static Bitmapset *
slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
LogicalRepRelMapEntry *rel,
LogicalRepTupleData *tupleData)
{
+ Bitmapset *modified = NULL;
int natts = slot->tts_tupleDescriptor->natts;
int i;
@@ -1195,6 +1200,28 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
slot->tts_isnull[i] = true;
}
+ /*
+ * Determine if the replicated value changed the local value by
+ * comparing slots. This is a subset of
+ * ExecCheckIndexedAttrsForChanges.
+ */
+ if (srcslot->tts_isnull[i] != slot->tts_isnull[i])
+ {
+ /* One is NULL, the other is not so the value changed */
+ modified = bms_add_member(modified, i + 1 - FirstLowInvalidHeapAttributeNumber);
+ }
+ else if (!srcslot->tts_isnull[i])
+ {
+ /* Both are not NULL, compare their values */
+ if (!tts_attr_equal(att->atttypid,
+ att->attcollation,
+ att->attbyval,
+ att->attlen,
+ srcslot->tts_values[i],
+ slot->tts_values[i]))
+ modified = bms_add_member(modified, i + 1 - FirstLowInvalidHeapAttributeNumber);
+ }
+
/* Reset attnum for error callback */
apply_error_callback_arg.remote_attnum = -1;
}
@@ -1202,6 +1229,8 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
/* And finally, declare that "slot" contains a valid virtual tuple */
ExecStoreVirtualTuple(slot);
+
+ return modified;
}
/*
@@ -2918,6 +2947,7 @@ apply_handle_update_internal(ApplyExecutionData *edata,
ConflictTupleInfo conflicttuple = {0};
bool found;
MemoryContext oldctx;
+ Bitmapset *indexed = NULL;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
ExecOpenIndices(relinfo, false);
@@ -2934,6 +2964,8 @@ apply_handle_update_internal(ApplyExecutionData *edata,
*/
if (found)
{
+ Bitmapset *modified = NULL;
+
/*
* Report the conflict if the tuple was modified by a different
* origin.
@@ -2957,15 +2989,29 @@ apply_handle_update_internal(ApplyExecutionData *edata,
/* Process and store remote tuple in the slot */
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot, localslot, relmapentry, newtup);
+ modified = slot_modify_data(remoteslot, localslot, relmapentry, newtup);
MemoryContextSwitchTo(oldctx);
+ /*
+ * Normally we'd call ExecCheckIndexedAttrForChanges but here we have
+ * the record of changed columns in the replication state, so let's
+ * use that instead.
+ */
+ indexed = RelationGetIndexAttrBitmap(relinfo->ri_RelationDesc,
+ INDEX_ATTR_BITMAP_INDEXED);
+
+ bms_free(relinfo->ri_ChangedIndexedCols);
+ relinfo->ri_ChangedIndexedCols = bms_int_members(modified, indexed);
+ bms_free(indexed);
+
EvalPlanQualSetSlot(&epqstate, remoteslot);
InitConflictIndexes(relinfo);
- /* Do the actual update. */
+ /* First check privileges */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
+
+ /* Then do the actual update. */
ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
remoteslot);
}
@@ -3455,6 +3501,8 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
bool found;
EPQState epqstate;
ConflictTupleInfo conflicttuple = {0};
+ Bitmapset *modified = NULL;
+ Bitmapset *indexed;
/* Get the matching local tuple from the partition. */
found = FindReplTupleInLocalRel(edata, partrel,
@@ -3523,8 +3571,8 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
* remoteslot_part.
*/
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot_part, localslot, part_entry,
- newtup);
+ modified = slot_modify_data(remoteslot_part, localslot, part_entry,
+ newtup);
MemoryContextSwitchTo(oldctx);
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
@@ -3549,6 +3597,18 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
EvalPlanQualSetSlot(&epqstate, remoteslot_part);
TargetPrivilegesCheck(partrelinfo->ri_RelationDesc,
ACL_UPDATE);
+
+ /*
+ * Normally we'd call ExecCheckIndexedAttrForChanges but
+ * here we have the record of changed columns in the
+ * replication state, so let's use that instead.
+ */
+ indexed = RelationGetIndexAttrBitmap(partrelinfo->ri_RelationDesc,
+ INDEX_ATTR_BITMAP_INDEXED);
+ bms_free(partrelinfo->ri_ChangedIndexedCols);
+ partrelinfo->ri_ChangedIndexedCols = bms_int_members(modified, indexed);
+ bms_free(indexed);
+
ExecSimpleRelationUpdate(partrelinfo, estate, &epqstate,
localslot, remoteslot_part);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 915d0bc9084..32825596be1 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -2482,6 +2482,7 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
bms_free(relation->rd_idattr);
bms_free(relation->rd_hotblockingattr);
bms_free(relation->rd_summarizedattr);
+ bms_free(relation->rd_indexedattr);
if (relation->rd_pubdesc)
pfree(relation->rd_pubdesc);
if (relation->rd_options)
@@ -5283,6 +5284,7 @@ RelationGetIndexPredicate(Relation relation)
* index (empty if FULL)
* INDEX_ATTR_BITMAP_HOT_BLOCKING Columns that block updates from being HOT
* INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
+ * INDEX_ATTR_BITMAP_INDEXED Columns referenced by indexes
*
* Attribute numbers are offset by FirstLowInvalidHeapAttributeNumber so that
* we can include system attributes (e.g., OID) in the bitmap representation.
@@ -5307,6 +5309,7 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
Bitmapset *idindexattrs; /* columns in the replica identity */
Bitmapset *hotblockingattrs; /* columns with HOT blocking indexes */
Bitmapset *summarizedattrs; /* columns with summarizing indexes */
+ Bitmapset *indexedattrs; /* columns referenced by indexes */
List *indexoidlist;
List *newindexoidlist;
Oid relpkindex;
@@ -5329,6 +5332,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
return bms_copy(relation->rd_hotblockingattr);
case INDEX_ATTR_BITMAP_SUMMARIZED:
return bms_copy(relation->rd_summarizedattr);
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return bms_copy(relation->rd_indexedattr);
default:
elog(ERROR, "unknown attrKind %u", attrKind);
}
@@ -5373,6 +5378,7 @@ restart:
idindexattrs = NULL;
hotblockingattrs = NULL;
summarizedattrs = NULL;
+ indexedattrs = NULL;
foreach(l, indexoidlist)
{
Oid indexOid = lfirst_oid(l);
@@ -5505,10 +5511,14 @@ restart:
bms_free(idindexattrs);
bms_free(hotblockingattrs);
bms_free(summarizedattrs);
+ bms_free(indexedattrs);
goto restart;
}
+ /* Combine all index attributes */
+ indexedattrs = bms_union(hotblockingattrs, summarizedattrs);
+
/* Don't leak the old values of these bitmaps, if any */
relation->rd_attrsvalid = false;
bms_free(relation->rd_keyattr);
@@ -5521,6 +5531,8 @@ restart:
relation->rd_hotblockingattr = NULL;
bms_free(relation->rd_summarizedattr);
relation->rd_summarizedattr = NULL;
+ bms_free(relation->rd_indexedattr);
+ relation->rd_indexedattr = NULL;
/*
* Now save copies of the bitmaps in the relcache entry. We intentionally
@@ -5535,6 +5547,7 @@ restart:
relation->rd_idattr = bms_copy(idindexattrs);
relation->rd_hotblockingattr = bms_copy(hotblockingattrs);
relation->rd_summarizedattr = bms_copy(summarizedattrs);
+ relation->rd_indexedattr = bms_copy(indexedattrs);
relation->rd_attrsvalid = true;
MemoryContextSwitchTo(oldcxt);
@@ -5551,6 +5564,8 @@ restart:
return hotblockingattrs;
case INDEX_ATTR_BITMAP_SUMMARIZED:
return summarizedattrs;
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return indexedattrs;
default:
elog(ERROR, "unknown attrKind %u", attrKind);
return NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index e16bf025692..8a5931a3118 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -549,6 +549,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
+ Bitmapset *updated_cols,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1502,12 +1503,12 @@ static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ Bitmapset *updated_cols, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
- wait, tmfd,
- lockmode, update_indexes);
+ wait, tmfd, lockmode,
+ updated_cols, update_indexes);
}
/*
@@ -2010,6 +2011,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
+ Bitmapset *modified_indexe_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index fa2b657fb2f..993dc0e6ced 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -800,5 +800,10 @@ extern ResultRelInfo *ExecLookupResultRelByOid(ModifyTableState *node,
Oid resultoid,
bool missing_ok,
bool update_cache);
+extern Bitmapset *ExecCheckIndexedAttrsForChanges(ResultRelInfo *resultRelInfo,
+ TupleTableSlot *tts_old,
+ TupleTableSlot *tts_new);
+extern bool tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
+ Datum value1, Datum value2);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 18ae8f0d4bb..8b08e0045ba 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -498,6 +498,7 @@ typedef struct ResultRelInfo
Bitmapset *ri_extraUpdatedCols;
/* true if the above has been computed */
bool ri_extraUpdatedCols_valid;
+ Bitmapset *ri_ChangedIndexedCols;
/* Projection to generate new tuple in an INSERT/UPDATE */
ProjectionInfo *ri_projectNew;
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 80286076a11..b23a7306e69 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -164,6 +164,7 @@ typedef struct RelationData
Bitmapset *rd_idattr; /* included in replica identity index */
Bitmapset *rd_hotblockingattr; /* cols blocking HOT update */
Bitmapset *rd_summarizedattr; /* cols indexed by summarizing indexes */
+ Bitmapset *rd_indexedattr; /* all cols referenced by indexes */
PublicationDesc *rd_pubdesc; /* publication descriptor, or NULL */
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 3561c6bef0b..d3fbb8b093a 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -71,6 +71,7 @@ typedef enum IndexAttrBitmapKind
INDEX_ATTR_BITMAP_IDENTITY_KEY,
INDEX_ATTR_BITMAP_HOT_BLOCKING,
INDEX_ATTR_BITMAP_SUMMARIZED,
+ INDEX_ATTR_BITMAP_INDEXED,
} IndexAttrBitmapKind;
extern Bitmapset *RelationGetIndexAttrBitmap(Relation relation,
--
2.49.0
[application/octet-stream] v21-0003-Replace-index_unchanged_by_update-with-ri_Change.patch (8.3K, 4-v21-0003-Replace-index_unchanged_by_update-with-ri_Change.patch)
download | inline diff:
From cf41827dd2ed13e5fa02763bddec570125af621e Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Fri, 31 Oct 2025 14:55:25 -0400
Subject: [PATCH v21 3/4] Replace index_unchanged_by_update with
ri_ChangedIndexedCols
In execIndexing on updates we'd like to pass a hint to the indexing code
when the indexed attributes are unchanged. This commit replaces the now
redundant code in index_unchanged_by_update with the same information
found earlier in the update path.
---
src/backend/catalog/toasting.c | 2 -
src/backend/executor/execIndexing.c | 156 +---------------------------
src/backend/nodes/makefuncs.c | 2 -
src/include/nodes/execnodes.h | 4 -
4 files changed, 1 insertion(+), 163 deletions(-)
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 874a8fc89ad..5d819bda54a 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -300,8 +300,6 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
indexInfo->ii_Unique = true;
indexInfo->ii_NullsNotDistinct = false;
indexInfo->ii_ReadyForInserts = true;
- indexInfo->ii_CheckedUnchanged = false;
- indexInfo->ii_IndexUnchanged = false;
indexInfo->ii_Concurrent = false;
indexInfo->ii_BrokenHotChain = false;
indexInfo->ii_ParallelWorkers = 0;
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 401606f840a..fb1bc3a480d 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -138,11 +138,6 @@ static bool check_exclusion_or_unique_constraint(Relation heap, Relation index,
static bool index_recheck_constraint(Relation index, const Oid *constr_procs,
const Datum *existing_values, const bool *existing_isnull,
const Datum *new_values);
-static bool index_unchanged_by_update(ResultRelInfo *resultRelInfo,
- EState *estate, IndexInfo *indexInfo,
- Relation indexRelation);
-static bool index_expression_changed_walker(Node *node,
- Bitmapset *allUpdatedCols);
static void ExecWithoutOverlapsNotEmpty(Relation rel, NameData attname, Datum attval,
char typtype, Oid atttypid);
@@ -440,10 +435,7 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* index. If we're being called as part of an UPDATE statement,
* consider if the 'indexUnchanged' = true hint should be passed.
*/
- indexUnchanged = update && index_unchanged_by_update(resultRelInfo,
- estate,
- indexInfo,
- indexRelation);
+ indexUnchanged = update && bms_is_empty(resultRelInfo->ri_ChangedIndexedCols);
satisfiesConstraint =
index_insert(indexRelation, /* index relation */
@@ -993,152 +985,6 @@ index_recheck_constraint(Relation index, const Oid *constr_procs,
return true;
}
-/*
- * Check if ExecInsertIndexTuples() should pass indexUnchanged hint.
- *
- * When the executor performs an UPDATE that requires a new round of index
- * tuples, determine if we should pass 'indexUnchanged' = true hint for one
- * single index.
- */
-static bool
-index_unchanged_by_update(ResultRelInfo *resultRelInfo, EState *estate,
- IndexInfo *indexInfo, Relation indexRelation)
-{
- Bitmapset *updatedCols;
- Bitmapset *extraUpdatedCols;
- Bitmapset *allUpdatedCols;
- bool hasexpression = false;
- List *idxExprs;
-
- /*
- * Check cache first
- */
- if (indexInfo->ii_CheckedUnchanged)
- return indexInfo->ii_IndexUnchanged;
- indexInfo->ii_CheckedUnchanged = true;
-
- /*
- * Check for indexed attribute overlap with updated columns.
- *
- * Only do this for key columns. A change to a non-key column within an
- * INCLUDE index should not be counted here. Non-key column values are
- * opaque payload state to the index AM, a little like an extra table TID.
- *
- * Note that row-level BEFORE triggers won't affect our behavior, since
- * they don't affect the updatedCols bitmaps generally. It doesn't seem
- * worth the trouble of checking which attributes were changed directly.
- */
- updatedCols = ExecGetUpdatedCols(resultRelInfo, estate);
- extraUpdatedCols = ExecGetExtraUpdatedCols(resultRelInfo, estate);
- for (int attr = 0; attr < indexInfo->ii_NumIndexKeyAttrs; attr++)
- {
- int keycol = indexInfo->ii_IndexAttrNumbers[attr];
-
- if (keycol <= 0)
- {
- /*
- * Skip expressions for now, but remember to deal with them later
- * on
- */
- hasexpression = true;
- continue;
- }
-
- if (bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
- updatedCols) ||
- bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
- extraUpdatedCols))
- {
- /* Changed key column -- don't hint for this index */
- indexInfo->ii_IndexUnchanged = false;
- return false;
- }
- }
-
- /*
- * When we get this far and index has no expressions, return true so that
- * index_insert() call will go on to pass 'indexUnchanged' = true hint.
- *
- * The _absence_ of an indexed key attribute that overlaps with updated
- * attributes (in addition to the total absence of indexed expressions)
- * shows that the index as a whole is logically unchanged by UPDATE.
- */
- if (!hasexpression)
- {
- indexInfo->ii_IndexUnchanged = true;
- return true;
- }
-
- /*
- * Need to pass only one bms to expression_tree_walker helper function.
- * Avoid allocating memory in common case where there are no extra cols.
- */
- if (!extraUpdatedCols)
- allUpdatedCols = updatedCols;
- else
- allUpdatedCols = bms_union(updatedCols, extraUpdatedCols);
-
- /*
- * We have to work slightly harder in the event of indexed expressions,
- * but the principle is the same as before: try to find columns (Vars,
- * actually) that overlap with known-updated columns.
- *
- * If we find any matching Vars, don't pass hint for index. Otherwise
- * pass hint.
- */
- idxExprs = RelationGetIndexExpressions(indexRelation);
- hasexpression = index_expression_changed_walker((Node *) idxExprs,
- allUpdatedCols);
- list_free(idxExprs);
- if (extraUpdatedCols)
- bms_free(allUpdatedCols);
-
- if (hasexpression)
- {
- indexInfo->ii_IndexUnchanged = false;
- return false;
- }
-
- /*
- * Deliberately don't consider index predicates. We should even give the
- * hint when result rel's "updated tuple" has no corresponding index
- * tuple, which is possible with a partial index (provided the usual
- * conditions are met).
- */
- indexInfo->ii_IndexUnchanged = true;
- return true;
-}
-
-/*
- * Indexed expression helper for index_unchanged_by_update().
- *
- * Returns true when Var that appears within allUpdatedCols located.
- */
-static bool
-index_expression_changed_walker(Node *node, Bitmapset *allUpdatedCols)
-{
- if (node == NULL)
- return false;
-
- if (IsA(node, Var))
- {
- Var *var = (Var *) node;
-
- if (bms_is_member(var->varattno - FirstLowInvalidHeapAttributeNumber,
- allUpdatedCols))
- {
- /* Var was updated -- indicates that we should not hint */
- return true;
- }
-
- /* Still haven't found a reason to not pass the hint */
- return false;
- }
-
- return expression_tree_walker(node, index_expression_changed_walker,
- allUpdatedCols);
-}
-
/*
* ExecWithoutOverlapsNotEmpty - raise an error if the tuple has an empty
* range or multirange in the given attribute.
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index e2d9e9be41a..d69dc090aa4 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -845,8 +845,6 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
n->ii_Unique = unique;
n->ii_NullsNotDistinct = nulls_not_distinct;
n->ii_ReadyForInserts = isready;
- n->ii_CheckedUnchanged = false;
- n->ii_IndexUnchanged = false;
n->ii_Concurrent = concurrent;
n->ii_Summarizing = summarizing;
n->ii_WithoutOverlaps = withoutoverlaps;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 8b08e0045ba..898368fb8cb 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -202,10 +202,6 @@ typedef struct IndexInfo
bool ii_NullsNotDistinct;
/* is it valid for inserts? */
bool ii_ReadyForInserts;
- /* IndexUnchanged status determined yet? */
- bool ii_CheckedUnchanged;
- /* aminsert hint, cached for retail inserts */
- bool ii_IndexUnchanged;
/* are we doing a concurrent index build? */
bool ii_Concurrent;
/* did we detect any broken HOT chains? */
--
2.49.0
[application/octet-stream] v21-0004-Enable-HOT-updates-for-expression-and-partial-in.patch (130.2K, 5-v21-0004-Enable-HOT-updates-for-expression-and-partial-in.patch)
download | inline diff:
From 632179fcd6802348e9424bae4b419d13feb24c0f Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 26 Oct 2025 10:49:25 -0400
Subject: [PATCH v21 4/4] Enable HOT updates for expression and partial indexes
Currently, PostgreSQL conservatively prevents HOT (Heap-Only Tuple)
updates whenever any indexed column changes, even if the indexed
portion of that column remains identical. This is overly restrictive
for expression indexes (where f(column) might not change even when
column changes) and partial indexes (where both old and new tuples
might fall outside the predicate). Finally, index AMs play no role
in deciding when they need a new index entry on update, the rules
regarding that are based on binary equality and the HEAP's model for
MVCC and related HOT optimization. Here we open that door a bit so
as to enable more nuanced control over the process. This enables
index AMs that require binary equality (as is the case for nbtree)
to do that without disallowing type-specific equality checking for
other indexes.
This patch introduces several improvements to enable HOT updates in
these cases:
Add amcomparedatums() callback to IndexAmRoutine. This allows index
access methods like GIN to provide custom logic for comparing datums by
extracting and comparing index keys rather than comparing the raw
datums. GIN indexes now implement gincomparedatums() which extracts keys
from both datums and compares the resulting key sets. Also, as
mentioned earlier nbtree implements this API and uses datumIsEqual() for
equality so that the manner in which it deduplicates TIDs on page split
doesn't have to change. This is not a required API, when not
implemented the executor will compare TupleTableSlot datum for equality
using type-specific operators and take into account collation so that an
update from "Apple" to "APPLE" on a case insensitive index can now be
HOT.
ExecWhichIndexesRequireUpdates() is re-written to find the set of
modified indexed attributes that trigger new index tuples on updated.
For partial indexes, this checks whether both old and new tuples satisfy
or fail the predicate. For expression indexes, this uses type-specific
equality operators to compare computed values. For extraction-based
indexes (GIN/RUM) that implement amcomparedatums() it uses that.
Importantly, table access methods can still signal using TU_Update if
all, none, or only summarizing indexes should be updated. While the
executor layer now owns determining what has changed due to an update
and is interested in only updating the minimum number of indexes
possible, the table AM can override that while performing
table_tuple_update(), which is what heap does. While this signal is
very specific to how the heap implements MVCC and its HOT optimization,
we'll leave replacing that for another day.
This optimization trades off some new overhead for the potential for
more updates to use the HOT optimized path and avoid index and heap
bloat. This should significantly improve update performance for tables
with expression indexes, partial indexes, and GIN/GiST indexes on
complex data types like JSONB and tsvector, while maintaining correct
index semantics. Minimal additional overhead due to type-specific
equality checking should be washed out by the benefits of updating
indexes fewer times.
One notable trade-off is that there are more calls to FormIndexDatum()
as a result. Caching these might reduce some of that overhead, but not
all. This lead to the change in the frequency for expressions in the
spec update test to output notice messages, but does not impact
correctness.
---
src/backend/access/brin/brin.c | 1 +
src/backend/access/gin/ginutil.c | 94 +-
src/backend/access/heap/heapam.c | 10 +-
src/backend/access/heap/heapam_handler.c | 6 +-
src/backend/access/nbtree/nbtree.c | 38 +
src/backend/access/table/tableam.c | 4 +-
src/backend/bootstrap/bootstrap.c | 8 +
src/backend/catalog/index.c | 57 +
src/backend/catalog/indexing.c | 16 +-
src/backend/catalog/toasting.c | 4 +
src/backend/executor/execIndexing.c | 45 +-
src/backend/executor/nodeModifyTable.c | 437 +++++--
src/backend/nodes/makefuncs.c | 4 +
src/include/access/amapi.h | 28 +
src/include/access/gin.h | 3 +
src/include/access/heapam.h | 6 +-
src/include/access/nbtree.h | 4 +
src/include/access/tableam.h | 8 +-
src/include/catalog/index.h | 1 +
src/include/executor/executor.h | 12 +-
src/include/nodes/execnodes.h | 19 +
.../expected/insert-conflict-specconflict.out | 20 +
.../expected/hot_expression_indexes.out | 1007 +++++++++++++++++
src/test/regress/parallel_schedule | 6 +
.../regress/sql/hot_expression_indexes.sql | 747 ++++++++++++
src/tools/pgindent/typedefs.list | 1 +
26 files changed, 2461 insertions(+), 125 deletions(-)
create mode 100644 src/test/regress/expected/hot_expression_indexes.out
create mode 100644 src/test/regress/sql/hot_expression_indexes.sql
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index cb3331921cb..36e639552e6 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -290,6 +290,7 @@ brinhandler(PG_FUNCTION_ARGS)
amroutine->amproperty = NULL;
amroutine->ambuildphasename = NULL;
amroutine->amvalidate = brinvalidate;
+ amroutine->amcomparedatums = NULL;
amroutine->amadjustmembers = NULL;
amroutine->ambeginscan = brinbeginscan;
amroutine->amrescan = brinrescan;
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index 78f7b7a2495..85e25ed73e8 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -26,6 +26,7 @@
#include "storage/indexfsm.h"
#include "utils/builtins.h"
#include "utils/index_selfuncs.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/typcache.h"
@@ -78,6 +79,7 @@ ginhandler(PG_FUNCTION_ARGS)
amroutine->amproperty = NULL;
amroutine->ambuildphasename = ginbuildphasename;
amroutine->amvalidate = ginvalidate;
+ amroutine->amcomparedatums = gincomparedatums;
amroutine->amadjustmembers = ginadjustmembers;
amroutine->ambeginscan = ginbeginscan;
amroutine->amrescan = ginrescan;
@@ -477,13 +479,6 @@ cmpEntries(const void *a, const void *b, void *arg)
return res;
}
-
-/*
- * Extract the index key values from an indexable item
- *
- * The resulting key values are sorted, and any duplicates are removed.
- * This avoids generating redundant index entries.
- */
Datum *
ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
@@ -729,3 +724,88 @@ ginbuildphasename(int64 phasenum)
return NULL;
}
}
+
+/*
+ * gincomparedatums - Compare two datums to determine if they produce identical keys
+ *
+ * This function extracts keys from both old_datum and new_datum using the
+ * opclass's extractValue function, then compares the extracted key arrays.
+ * Returns true if the key sets are identical (same keys, same counts).
+ *
+ * This enables HOT updates for GIN indexes when the indexed portions of a
+ * value haven't changed, even if the value itself has changed.
+ *
+ * Example: JSONB column with GIN index. If an update changes a non-indexed
+ * key in the JSONB document, the extracted keys are identical and we can
+ * do a HOT update.
+ */
+bool
+gincomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull)
+{
+ GinState ginstate;
+ Datum *old_keys;
+ Datum *new_keys;
+ GinNullCategory *old_categories;
+ GinNullCategory *new_categories;
+ int32 old_nkeys;
+ int32 new_nkeys;
+ MemoryContext tmpcontext;
+ MemoryContext oldcontext;
+ bool result = true;
+
+ /* Handle NULL cases */
+ if (old_isnull != new_isnull)
+ return false;
+ if (old_isnull)
+ return true;
+
+ /* Create temporary context for extraction work */
+ tmpcontext = AllocSetContextCreate(CurrentMemoryContext,
+ "GIN datum comparison",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(tmpcontext);
+
+ initGinState(&ginstate, index);
+
+ /*
+ * Extract keys from both datums using existing GIN infrastructure.
+ */
+ old_keys = ginExtractEntries(&ginstate, attnum, old_datum, old_isnull,
+ &old_nkeys, &old_categories);
+ new_keys = ginExtractEntries(&ginstate, attnum, new_datum, new_isnull,
+ &new_nkeys, &new_categories);
+
+ /* Different number of keys → definitely different */
+ if (old_nkeys != new_nkeys)
+ {
+ result = false;
+ goto cleanup;
+ }
+
+ /*
+ * Compare the sorted key arrays element-by-element. Since both arrays are
+ * already sorted by ginExtractEntries, we can do a simple O(n)
+ * comparison.
+ */
+ for (int i = 0; i < old_nkeys; i++)
+ {
+ int cmp = ginCompareEntries(&ginstate, attnum,
+ old_keys[i], old_categories[i],
+ new_keys[i], new_categories[i]);
+
+ if (cmp != 0)
+ {
+ result = false;
+ break;
+ }
+ }
+
+cleanup:
+ /* Clean up */
+ MemoryContextSwitchTo(oldcontext);
+ MemoryContextDelete(tmpcontext);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 1cdb72b3a7a..5b0ff13b13d 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -3268,7 +3268,7 @@ heap_update(Relation relation, HeapTupleData *oldtup, HeapTuple newtup,
TM_FailureData *tmfd, LockTupleMode *lockmode,
Buffer buffer, Page page, BlockNumber block, ItemId lp,
Bitmapset *hot_attrs, Bitmapset *sum_attrs, Bitmapset *pk_attrs,
- Bitmapset *rid_attrs, Bitmapset *mix_attrs, Buffer *vmbuffer,
+ Bitmapset *rid_attrs, const Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -4337,8 +4337,9 @@ HeapDetermineColumnsInfo(Relation relation,
* This routine may be used to update a tuple when concurrent updates of the
* target tuple are not expected (for example, because we have a lock on the
* relation associated with the tuple). Any failure is reported via ereport().
+ * Returns the set of modified indexed attributes.
*/
-void
+Bitmapset *
simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
@@ -4467,7 +4468,7 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
elog(ERROR, "tuple concurrently deleted");
- return;
+ return NULL;
}
/*
@@ -4500,7 +4501,6 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
bms_free(sum_attrs);
bms_free(pk_attrs);
bms_free(rid_attrs);
- bms_free(mix_attrs);
bms_free(idx_attrs);
switch (result)
@@ -4526,6 +4526,8 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
elog(ERROR, "unrecognized heap_update status: %u", result);
break;
}
+
+ return mix_attrs;
}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index ef08e1d3e10..7527809ec08 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -319,7 +319,7 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
Snapshot crosscheck, bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
- Bitmapset *mix_attrs,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes)
{
bool rep_id_key_required = false;
@@ -407,10 +407,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
Assert(ItemIdIsNormal(lp));
- /*
- * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
- * then pass that on to heap_update.
- */
oldtup.t_tableOid = RelationGetRelid(relation);
oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
oldtup.t_len = ItemIdGetLength(lp);
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index fdff960c130..73cc3208757 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -155,6 +155,7 @@ bthandler(PG_FUNCTION_ARGS)
amroutine->amproperty = btproperty;
amroutine->ambuildphasename = btbuildphasename;
amroutine->amvalidate = btvalidate;
+ amroutine->amcomparedatums = btcomparedatums;
amroutine->amadjustmembers = btadjustmembers;
amroutine->ambeginscan = btbeginscan;
amroutine->amrescan = btrescan;
@@ -1795,3 +1796,40 @@ bttranslatecmptype(CompareType cmptype, Oid opfamily)
return InvalidStrategy;
}
}
+
+/*
+ * btcomparedatums - Compare two datums for equality
+ *
+ * This function is necessary because nbtree requires that keys that are not
+ * binary identical not be "equal". Other indexes might allow "A" and "a" to
+ * be "equal" when collation is case insensative, but not nbtree. Why? Well,
+ * nbtree deduplicates TIDs on page split and the way it accomplish that is by
+ * doing a binary comparison of the keys.
+ */
+
+bool
+btcomparedatums(Relation index, int attrnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull)
+{
+ TupleDesc desc = RelationGetDescr(index);
+ CompactAttribute *att;
+
+ /*
+ * If one value is NULL and other is not, then they are certainly not
+ * equal
+ */
+ if (old_isnull != new_isnull)
+ return false;
+
+ /*
+ * If both are NULL, they can be considered equal.
+ */
+ if (old_isnull)
+ return true;
+
+ /* We do simple binary comparison of the two datums */
+ Assert(attrnum <= desc->natts);
+ att = TupleDescCompactAttr(desc, attrnum - 1);
+ return datumIsEqual(old_datum, new_datum, att->attbyval, att->attlen);
+}
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index dadcf03ed24..ef7736bfa76 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -336,7 +336,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
- Bitmapset *modified_indexed_cols,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -348,7 +348,7 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
&tmfd, &lockmode,
- modified_indexed_cols,
+ mix_attrs,
update_indexes);
switch (result)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index fc8638c1b61..329c110d0bf 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -961,10 +961,18 @@ index_register(Oid heap,
newind->il_info->ii_Expressions =
copyObject(indexInfo->ii_Expressions);
newind->il_info->ii_ExpressionsState = NIL;
+ /* expression attrs will likely be null, but may as well copy it */
+ newind->il_info->ii_ExpressionsAttrs =
+ copyObject(indexInfo->ii_ExpressionsAttrs);
/* predicate will likely be null, but may as well copy it */
newind->il_info->ii_Predicate =
copyObject(indexInfo->ii_Predicate);
newind->il_info->ii_PredicateState = NULL;
+ /* predicate attrs will likely be null, but may as well copy it */
+ newind->il_info->ii_PredicateAttrs =
+ copyObject(indexInfo->ii_PredicateAttrs);
+ newind->il_info->ii_CheckedPredicate = false;
+ newind->il_info->ii_PredicateSatisfied = false;
/* no exclusion constraints at bootstrap time, so no need to copy */
Assert(indexInfo->ii_ExclusionOps == NULL);
Assert(indexInfo->ii_ExclusionProcs == NULL);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 5d9db167e59..29b8cc4badd 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -27,6 +27,7 @@
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/relscan.h"
+#include "access/sysattr.h"
#include "access/tableam.h"
#include "access/toast_compression.h"
#include "access/transam.h"
@@ -58,6 +59,7 @@
#include "commands/trigger.h"
#include "executor/executor.h"
#include "miscadmin.h"
+#include "nodes/execnodes.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/optimizer.h"
@@ -2414,6 +2416,61 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* ----------------------------------------------------------------
*/
+/* ----------------
+ * BuildUpdateIndexInfo
+ *
+ * For expression indexes updates may not change the indexed value allowing
+ * for a HOT update. Add information to the IndexInfo to allow for checking
+ * if the indexed value has changed.
+ *
+ * Do this processing here rather than in BuildIndexInfo() to not incur the
+ * overhead in the common non-expression cases.
+ * ----------------
+ */
+void
+BuildUpdateIndexInfo(ResultRelInfo *resultRelInfo)
+{
+ for (int j = 0; j < resultRelInfo->ri_NumIndices; j++)
+ {
+ int i;
+ int indnkeyatts;
+ Bitmapset *attrs = NULL;
+ IndexInfo *ii = resultRelInfo->ri_IndexRelationInfo[j];
+
+ /*
+ * Expressions are not allowed on non-key attributes, so we can skip
+ * them as they should show up in the index HOT-blocking attributes.
+ */
+ indnkeyatts = ii->ii_NumIndexKeyAttrs;
+
+ /* Collect key attributes used by the index */
+ for (i = 0; i < indnkeyatts; i++)
+ {
+ AttrNumber attnum = ii->ii_IndexAttrNumbers[i];
+
+ if (attnum != 0)
+ attrs = bms_add_member(attrs, attnum - FirstLowInvalidHeapAttributeNumber);
+ }
+
+ /* Collect attributes used in the expression */
+ if (ii->ii_Expressions)
+ pull_varattnos((Node *) ii->ii_Expressions,
+ resultRelInfo->ri_RangeTableIndex,
+ &ii->ii_ExpressionsAttrs);
+
+ /* Collect attributes used in the predicate */
+ if (ii->ii_Predicate)
+ pull_varattnos((Node *) ii->ii_Predicate,
+ resultRelInfo->ri_RangeTableIndex,
+ &ii->ii_PredicateAttrs);
+
+ ii->ii_IndexedAttrs = bms_union(attrs, ii->ii_ExpressionsAttrs);
+
+ /* All indexes should index *something*! */
+ Assert(!bms_is_empty(ii->ii_IndexedAttrs));
+ }
+}
+
/* ----------------
* BuildIndexInfo
* Construct an IndexInfo record for an open index
diff --git a/src/backend/catalog/indexing.c b/src/backend/catalog/indexing.c
index 004c5121000..a361c215490 100644
--- a/src/backend/catalog/indexing.c
+++ b/src/backend/catalog/indexing.c
@@ -102,7 +102,7 @@ CatalogIndexInsert(CatalogIndexState indstate, HeapTuple heapTuple,
* Get information from the state structure. Fall out if nothing to do.
*/
numIndexes = indstate->ri_NumIndices;
- if (numIndexes == 0)
+ if (numIndexes == 0 || updateIndexes == TU_None)
return;
relationDescs = indstate->ri_IndexRelationDescs;
indexInfoArray = indstate->ri_IndexRelationInfo;
@@ -314,15 +314,18 @@ CatalogTupleUpdate(Relation heapRel, const ItemPointerData *otid, HeapTuple tup)
{
CatalogIndexState indstate;
TU_UpdateIndexes updateIndexes = TU_All;
+ Bitmapset *updatedAttrs;
CatalogTupleCheckConstraints(heapRel, tup);
indstate = CatalogOpenIndexes(heapRel);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
-
+ updatedAttrs = simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = updatedAttrs;
CatalogIndexInsert(indstate, tup, updateIndexes);
+
CatalogCloseIndexes(indstate);
+ bms_free(updatedAttrs);
}
/*
@@ -338,12 +341,15 @@ CatalogTupleUpdateWithInfo(Relation heapRel, const ItemPointerData *otid, HeapTu
CatalogIndexState indstate)
{
TU_UpdateIndexes updateIndexes = TU_All;
+ Bitmapset *updatedAttrs;
CatalogTupleCheckConstraints(heapRel, tup);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
-
+ updatedAttrs = simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = updatedAttrs;
CatalogIndexInsert(indstate, tup, updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = NULL;
+ bms_free(updatedAttrs);
}
/*
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 5d819bda54a..c665aa744b3 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -292,8 +292,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
indexInfo->ii_IndexAttrNumbers[1] = 2;
indexInfo->ii_Expressions = NIL;
indexInfo->ii_ExpressionsState = NIL;
+ indexInfo->ii_ExpressionsAttrs = NULL;
indexInfo->ii_Predicate = NIL;
indexInfo->ii_PredicateState = NULL;
+ indexInfo->ii_PredicateAttrs = NULL;
+ indexInfo->ii_CheckedPredicate = false;
+ indexInfo->ii_PredicateSatisfied = false;
indexInfo->ii_ExclusionOps = NULL;
indexInfo->ii_ExclusionProcs = NULL;
indexInfo->ii_ExclusionStrats = NULL;
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index fb1bc3a480d..20968a814d6 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -109,11 +109,15 @@
#include "access/genam.h"
#include "access/relscan.h"
#include "access/tableam.h"
+#include "access/sysattr.h"
#include "access/xact.h"
#include "catalog/index.h"
#include "executor/executor.h"
+#include "nodes/bitmapset.h"
+#include "nodes/execnodes.h"
#include "nodes/nodeFuncs.h"
#include "storage/lmgr.h"
+#include "utils/datum.h"
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
@@ -318,8 +322,8 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
Relation heapRelation;
IndexInfo **indexInfoArray;
ExprContext *econtext;
- Datum values[INDEX_MAX_KEYS];
- bool isnull[INDEX_MAX_KEYS];
+ Datum loc_values[INDEX_MAX_KEYS];
+ bool loc_isnull[INDEX_MAX_KEYS];
Assert(ItemPointerIsValid(tupleid));
@@ -343,13 +347,13 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
/* Arrange for econtext's scan tuple to be the tuple under test */
econtext->ecxt_scantuple = slot;
- /*
- * for each index, form and insert the index tuple
- */
+ /* Insert into each index that needs updating */
for (i = 0; i < numIndices; i++)
{
Relation indexRelation = relationDescs[i];
IndexInfo *indexInfo;
+ Datum *values;
+ bool *isnull;
bool applyNoDupErr;
IndexUniqueCheck checkUnique;
bool indexUnchanged;
@@ -366,7 +370,7 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
/*
* Skip processing of non-summarizing indexes if we only update
- * summarizing indexes
+ * summarizing indexes or if this index is unchanged.
*/
if (onlySummarizing && !indexInfo->ii_Summarizing)
continue;
@@ -387,8 +391,15 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
indexInfo->ii_PredicateState = predicate;
}
+ /* Check the index predicate if we haven't done so earlier on */
+ if (!indexInfo->ii_CheckedPredicate)
+ {
+ indexInfo->ii_PredicateSatisfied = ExecQual(predicate, econtext);
+ indexInfo->ii_CheckedPredicate = true;
+ }
+
/* Skip this index-update if the predicate isn't satisfied */
- if (!ExecQual(predicate, econtext))
+ if (!indexInfo->ii_PredicateSatisfied)
continue;
}
@@ -396,11 +407,10 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* FormIndexDatum fills in its values and isnull parameters with the
* appropriate values for the column(s) of the index.
*/
- FormIndexDatum(indexInfo,
- slot,
- estate,
- values,
- isnull);
+ FormIndexDatum(indexInfo, slot, estate, loc_values, loc_isnull);
+
+ values = loc_values;
+ isnull = loc_isnull;
/* Check whether to apply noDupErr to this index */
applyNoDupErr = noDupErr &&
@@ -435,7 +445,9 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* index. If we're being called as part of an UPDATE statement,
* consider if the 'indexUnchanged' = true hint should be passed.
*/
- indexUnchanged = update && bms_is_empty(resultRelInfo->ri_ChangedIndexedCols);
+ indexUnchanged = update &&
+ !bms_overlap(indexInfo->ii_IndexedAttrs,
+ resultRelInfo->ri_ChangedIndexedCols);
satisfiesConstraint =
index_insert(indexRelation, /* index relation */
@@ -604,7 +616,12 @@ ExecCheckIndexConstraints(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
checkedIndex = true;
/* Check for partial index */
- if (indexInfo->ii_Predicate != NIL)
+ if (indexInfo->ii_CheckedPredicate && !indexInfo->ii_PredicateSatisfied)
+ {
+ /* We've already checked and the predicate wasn't satisfied. */
+ continue;
+ }
+ else if (indexInfo->ii_Predicate != NIL)
{
ExprState *predicate;
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 34f86546fc9..e4b2cd5a3e8 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -54,10 +54,13 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/attnum.h"
+#include "access/sysattr.h"
#include "access/tableam.h"
#include "access/tupconvert.h"
#include "access/tupdesc.h"
#include "access/xact.h"
+#include "catalog/index.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -75,6 +78,7 @@
#include "utils/float.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
@@ -245,6 +249,10 @@ tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
typentry = lookup_type_cache(typid,
TYPECACHE_EQ_OPR | TYPECACHE_EQ_OPR_FINFO);
+ /* Use the type's collation if none provided */
+ if (collation == -1)
+ collation = typentry->typcollation;
+
/*
* If no equality operator is available, fall back to binary comparison.
* This handles types that don't have proper equality operators defined.
@@ -291,108 +299,356 @@ tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
}
/*
- * Determine which updated attributes actually changed values between old and
- * new tuples and are referenced by indexes on the relation.
+ * ExecCheckIndexedAttrsForChanges
+ *
+ * Determine which indexes need updating by finding the set of modified
+ * indexed attributes.
+ *
+ * For expression indexes and indexes which implement the amcomparedatums()
+ * index AM API we'll need to form index datum and compare each attribute to
+ * see if any actually changed.
+ *
+ * For expression indexes the result of the expression might not change at all,
+ * this is common with JSONB columns which require expression indexes and where
+ * it is commonplace to index a field within a document and have updates that
+ * generally don't update that field.
+ *
+ * Partial indexes won't trigger index tuples when the old/new tuples are both
+ * outside of the predicate range.
+ *
+ * All other indexes require testing old/new datum for equality. We do this
+ * by calling the type-specific equality operator when possible, otherwise we
+ * fall back to binary equality with datumIsEqual().
+ *
+ * For nbtree the amcomparedatums() API is critical as it requires that key
+ * attributes are equal when they memcmp(), which might not be the case when
+ * using type-specific comparison or factoring in collation which might make
+ * an index case insensitive.
*
- * Returns a Bitmapset of attribute offsets (0-based, adjusted by
- * FirstLowInvalidHeapAttributeNumber) or NULL if no attributes changed.
+ * All of this is to say that the goal is for the executor to know, ahead of
+ * calling into the table AM to process the update and before calling into the
+ * index AM for inserting new index tuples, which attributes truely necessitate
+ * a new index tuple.
+ *
+ * Returns a refined Bitmapset of attributes that force index updates.
*/
Bitmapset *
ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
- TupleTableSlot *tts_old,
- TupleTableSlot *tts_new)
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts)
{
Relation relation = relinfo->ri_RelationDesc;
TupleDesc tupdesc = RelationGetDescr(relation);
- Bitmapset *indexed_attrs;
- Bitmapset *modified = NULL;
- int attidx;
+ Bitmapset *mix_attrs = NULL;
/* If no indexes, we're done */
if (relinfo->ri_NumIndices == 0)
return NULL;
/*
- * Get the set of index key attributes. This includes summarizing,
- * expression indexes and attributes mentioned in the predicate of a
- * partition but not those in INCLUDING.
+ * NOTE: Expression and predicates that are observed to change will have
+ * all their attributes added into the m_attrs set knowing that some of
+ * those might not have changed. Take for instance an index on (a + b)
+ * followed by an index on (b) with an update that changes only the value
+ * of 'a'. We'll add both 'a' and 'b' to the m_attrs set then later when
+ * reviewing the second index add 'b' to the u_attrs (unchanged) set. In
+ * the end, we'll remove all the unchanged from the m_attrs and get our
+ * desired result.
*/
- indexed_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_INDEXED);
- Assert(!bms_is_empty(indexed_attrs));
- /*
- * NOTE: It is important to scan all indexed attributes in the tuples
- * because ExecGetAllUpdatedCols won't include columns that may have been
- * modified via heap_modify_tuple_by_col which is the case in
- * tsvector_update_trigger.
- */
- attidx = -1;
- while ((attidx = bms_next_member(indexed_attrs, attidx)) >= 0)
+ /* Find the indexes that reference this attribute */
+ for (int i = 0; i < relinfo->ri_NumIndices; i++)
{
- /* attidx is zero-based, attrnum is the normal attribute number */
- AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
- Form_pg_attribute attr;
- bool oldnull,
- newnull;
- Datum oldval,
- newval;
+ Relation indexRel = relinfo->ri_IndexRelationDescs[i];
+ IndexAmRoutine *amroutine = indexRel->rd_indam;
+ IndexInfo *indexInfo = relinfo->ri_IndexRelationInfo[i];
+ Bitmapset *m_attrs = NULL; /* (possibly) modified key attributes */
+ Bitmapset *p_attrs = NULL; /* (possibly) modified predicate attributes */
+ Bitmapset *u_attrs = NULL; /* unmodified attributes */
+ Bitmapset *pre_attrs = indexInfo->ii_PredicateAttrs;
+ bool has_expressions = (indexInfo->ii_Expressions != NIL);
+ bool has_am_compare = (amroutine->amcomparedatums != NULL);
+ bool is_partial = (indexInfo->ii_Predicate != NIL);
+ TupleTableSlot *save_scantuple;
+ ExprContext *econtext = GetPerTupleExprContext(estate);
+ Datum old_values[INDEX_MAX_KEYS];
+ bool old_isnull[INDEX_MAX_KEYS];
+ Datum new_values[INDEX_MAX_KEYS];
+ bool new_isnull[INDEX_MAX_KEYS];
+
+ /* If we've reviewed all the attributes on this index, move on */
+ if (bms_is_subset(indexInfo->ii_IndexedAttrs, mix_attrs))
+ continue;
- /*
- * If it's a whole-tuple reference, record as modified. It's not
- * really worth supporting this case, since it could only succeed
- * after a no-op update, which is hardly a case worth optimizing for.
- */
- if (attrnum == 0)
+ /* Checking partial at this point isn't viable when we're serializable */
+ if (is_partial && IsolationIsSerializable())
{
- modified = bms_add_member(modified, attidx);
- continue;
+ p_attrs = bms_copy(pre_attrs);
+ }
+ /* Check partial index predicate */
+ else if (is_partial)
+ {
+ ExprState *pstate;
+ bool old_qualifies,
+ new_qualifies;
+
+ if (!indexInfo->ii_CheckedPredicate)
+ pstate = ExecPrepareQual(indexInfo->ii_Predicate, estate);
+ else
+ pstate = indexInfo->ii_PredicateState;
+
+ save_scantuple = econtext->ecxt_scantuple;
+
+ econtext->ecxt_scantuple = old_tts;
+ old_qualifies = ExecQual(pstate, econtext);
+
+ econtext->ecxt_scantuple = new_tts;
+ new_qualifies = ExecQual(pstate, econtext);
+
+ econtext->ecxt_scantuple = save_scantuple;
+
+ indexInfo->ii_CheckedPredicate = true;
+ indexInfo->ii_PredicateState = pstate;
+ indexInfo->ii_PredicateSatisfied = new_qualifies;
+
+ /* Both outside predicate, index doesn't need update */
+ if (!old_qualifies && !new_qualifies)
+ continue;
+
+ /* A transition means we need to update the index */
+ if (old_qualifies != new_qualifies)
+ p_attrs = bms_copy(pre_attrs);
+
+ /*
+ * When both are within the predicate we must update this index,
+ * but only if one of the index key attributes changed.
+ */
}
/*
- * Likewise, include in the modified set any system attribute other
- * than tableOID; we cannot expect these to be consistent in a HOT
- * chain, or even to be set correctly yet in the new tuple.
+ * Expression indexes or extraction-based index require us to form
+ * index datums and compare. We've done all we can to avoid this
+ * overhead, now it's time to bite the bullet and get it done.
+ *
+ * XXX: Caching the values/isnull might be a win and avoid one of the
+ * added calls to FormIndexDatum().
*/
- if (attrnum < 0)
+ if (has_expressions || has_am_compare)
{
- if (attrnum != TableOidAttributeNumber)
- modified = bms_add_member(modified, attidx);
- continue;
- }
+ save_scantuple = econtext->ecxt_scantuple;
- /* Extract values from both slots */
- oldval = slot_getattr(tts_old, attrnum, &oldnull);
- newval = slot_getattr(tts_new, attrnum, &newnull);
+ /* Evaluate expressions (if any) to get base datums */
+ econtext->ecxt_scantuple = old_tts;
+ FormIndexDatum(indexInfo, old_tts, estate, old_values, old_isnull);
- /* If one value is NULL and the other is not, they are not equal */
- if (oldnull != newnull)
- {
- modified = bms_add_member(modified, attidx);
- continue;
+ econtext->ecxt_scantuple = new_tts;
+ FormIndexDatum(indexInfo, new_tts, estate, new_values, new_isnull);
+
+ econtext->ecxt_scantuple = save_scantuple;
+
+ /* Compare the index key datums for equality */
+ for (int j = 0; j < indexInfo->ii_NumIndexKeyAttrs; j++)
+ {
+ AttrNumber idx_attrnum = indexInfo->ii_IndexAttrNumbers[j];
+ int idx_attridx = idx_attrnum - FirstLowInvalidHeapAttributeNumber;
+ int nth_expr = 0;
+ bool values_equal = false;
+
+ /*
+ * We can't skip attributes that we've already identified as
+ * triggering an index update because we may have added an
+ * attribute from an expression index that didn't change but
+ * the expression did and that unchanged attribute is
+ * referenced in a subsequent index where we will discover that
+ * fact.
+ */
+
+ /* A change to/from NULL, record this attribute */
+ if (old_isnull[j] != new_isnull[j])
+ {
+ /* Expressions will have idx_attrnum == 0 */
+ if (idx_attrnum == 0)
+ m_attrs = bms_add_members(m_attrs, indexInfo->ii_ExpressionsAttrs);
+ else
+ m_attrs = bms_add_member(m_attrs, idx_attridx);
+ continue;
+ }
+
+ /* Both NULL, no change */
+ if (old_isnull[j])
+ {
+ if (idx_attrnum != 0)
+ u_attrs = bms_add_member(u_attrs, idx_attridx);
+
+ continue;
+ }
+
+ /*
+ * Use index AM's comparison function if present when comparing
+ * the index datum formed when creating an index key.
+ */
+ if (has_am_compare)
+ {
+ /*
+ * For nbtree to properly deduplicate TIDs on page split it
+ * must treat equality as binary comparison. So it is
+ * vital that we call it's comparedatums() function.
+ *
+ * In the case of GIN/RUM indexes they too behave
+ * differently and can even extract one or more portions of
+ * the datum when forming index tuples. We'd like to know
+ * if this update needs to trigger one or more index
+ * tuples, so we let the index AM perform their extraction
+ * and compare the results.
+ *
+ * There may be other similar index AM implementation with
+ * extraction where indexes are built using only part(s) of
+ * the Datum and might even need to invoke type-specific
+ * equality operators.
+ *
+ * NOTE: For AM comparison, pass the 1-based index
+ * attribute number. The AM's compare function expects the
+ * same numbering as used internally by the AM.
+ */
+ values_equal = amroutine->amcomparedatums(indexRel, j + 1,
+ old_values[j], old_isnull[j],
+ new_values[j], new_isnull[j]);
+ }
+ else
+ {
+ /*
+ * Expression index without custom AM comparison. Compare
+ * the expression results using type-specific equality
+ * which at this point is the expression's type, not the
+ * index's type. It is in index_form_tuple() that index
+ * attributes are transformed, not FormIndexDatum().
+ */
+ Oid expr_type_oid;
+ int16 typlen; /* Output: type length */
+ bool typbyval;
+ Expr *expr = (Expr *) list_nth(indexInfo->ii_Expressions, nth_expr);
+
+ Assert(expr != NULL);
+
+ /* Get type OID from the expression */
+ expr_type_oid = exprType((Node *) expr);
+
+ /* Get type information from the OID */
+ get_typlenbyval(expr_type_oid, &typlen, &typbyval);
+
+ values_equal = tts_attr_equal(expr_type_oid,
+ -1, /* use TBD expr type */
+ typbyval,
+ typlen,
+ old_values[j],
+ new_values[j]);
+ }
+
+ if (!values_equal)
+ {
+ /* Expressions will have idx_attrnum == 0 */
+ if (idx_attrnum == 0)
+ m_attrs = bms_add_members(m_attrs, indexInfo->ii_ExpressionsAttrs);
+ else
+ m_attrs = bms_add_member(m_attrs, idx_attridx);
+ }
+ else
+ {
+ if (idx_attrnum != 0)
+ u_attrs = bms_add_member(u_attrs, idx_attridx);
+ }
+
+ if (idx_attrnum == 0)
+ nth_expr++;
+ }
}
+ else
+ {
+ /*
+ * Here we know that we're reviewing an index that doesn't have a
+ * partial predicate, doesn't use expressions, and doesn't have a
+ * amcomparedatums() implementation.
+ */
- /* If both are NULL, consider them equal */
- if (oldnull)
- continue;
+ /* Compare the index key datums for equality */
+ for (int j = 0; j < indexInfo->ii_NumIndexKeyAttrs; j++)
+ {
+ Form_pg_attribute attr;
+ AttrNumber rel_attrnum;
+ int rel_attridx;
+ bool values_equal = false;
+ bool old_null,
+ new_null;
+ Datum old_val,
+ new_val;
- /* Get attribute metadata */
- Assert(attrnum > 0 && attrnum <= tupdesc->natts);
- attr = TupleDescAttr(tupdesc, attrnum - 1);
-
- /* Compare using type-specific equality operator */
- if (!tts_attr_equal(attr->atttypid,
- attr->attcollation,
- attr->attbyval,
- attr->attlen,
- oldval,
- newval))
- modified = bms_add_member(modified, attidx);
- }
+ rel_attrnum = indexInfo->ii_IndexAttrNumbers[j];
+ rel_attridx = rel_attrnum - FirstLowInvalidHeapAttributeNumber;
- bms_free(indexed_attrs);
+ /* Zero would mean expression, something we don't expect here */
+ Assert(rel_attrnum > 0 && rel_attrnum <= tupdesc->natts);
- return modified;
+ /* Extract values from both slots for this attribute */
+ old_val = slot_getattr(old_tts, rel_attrnum, &old_null);
+ new_val = slot_getattr(new_tts, rel_attrnum, &new_null);
+
+ /*
+ * If one value is NULL and the other is not, they are not
+ * equal
+ */
+ if (old_null != new_null)
+ {
+ m_attrs = bms_add_member(m_attrs, rel_attridx);
+ continue;
+ }
+
+ /* If both are NULL, consider them equal */
+ if (old_null)
+ {
+ u_attrs = bms_add_member(u_attrs, rel_attridx);
+ continue;
+ }
+
+ attr = TupleDescAttr(tupdesc, rel_attrnum - 1);
+
+ /*
+ * Compare using type-specific equality which at this point is
+ * the relation's type because FormIndexDatum() will populate
+ * the values/nulls but won't transform them into the final
+ * values destined for the index tuple, that's left to
+ * index_form_tuple() which we don't call (on purpose).
+ */
+ values_equal = tts_attr_equal(attr->atttypid,
+ attr->attcollation,
+ attr->attbyval,
+ attr->attlen,
+ old_val,
+ new_val);
+
+ if (!values_equal)
+ m_attrs = bms_add_member(m_attrs, rel_attridx);
+ else
+ u_attrs = bms_add_member(u_attrs, rel_attridx);
+ }
+ }
+
+ /*
+ * Here we know all the attributes we thought might be modified and
+ * all those we know haven't been. Take the difference and add it to
+ * the modified indexed attributes set.
+ */
+ m_attrs = bms_del_members(m_attrs, u_attrs);
+ p_attrs = bms_del_members(p_attrs, u_attrs);
+ mix_attrs = bms_add_members(mix_attrs, m_attrs);
+ mix_attrs = bms_add_members(mix_attrs, p_attrs);
+
+ bms_free(m_attrs);
+ bms_free(u_attrs);
+ bms_free(p_attrs);
+ }
+
+ return mix_attrs;
}
/*
@@ -2395,6 +2651,9 @@ ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
bool partition_constraint_failed;
TM_Result result;
+ /* The set of modified indexed attributes that trigger new index entries */
+ Bitmapset *mix_attrs = NULL;
+
updateCxt->crossPartUpdate = false;
/*
@@ -2517,13 +2776,32 @@ lreplace:
bms_free(resultRelInfo->ri_ChangedIndexedCols);
resultRelInfo->ri_ChangedIndexedCols = NULL;
- resultRelInfo->ri_ChangedIndexedCols =
- ExecCheckIndexedAttrsForChanges(resultRelInfo, oldSlot, slot);
+ /*
+ * During updates we'll need a bit more information in IndexInfo but we've
+ * delayed adding it until here. We check to ensure that there are
+ * indexes, that something has changed that is indexed, and that the first
+ * index doesn't yet have ii_IndexedAttrs set as a way to ensure we only
+ * build this when needed and only once. We don't build this in
+ * ExecOpenIndicies() as it is unnecessary overhead when not performing an
+ * update.
+ */
+ if (resultRelInfo->ri_NumIndices > 0 &&
+ bms_is_empty(resultRelInfo->ri_IndexRelationInfo[0]->ii_IndexedAttrs))
+ BuildUpdateIndexInfo(resultRelInfo);
+
+ /*
+ * Next up we need to find out the set of indexed attributes that have
+ * changed in value and should trigger a new index tuple. We could start
+ * with the set of updated columns via ExecGetUpdatedCols(), but if we do
+ * we will overlook attributes directly modified by heap_modify_tuple()
+ * which are not known to ExecGetUpdatedCols().
+ */
+ mix_attrs = ExecCheckIndexedAttrsForChanges(resultRelInfo, estate, oldSlot, slot);
/*
- * replace the heap tuple
+ * Call into the table AM to update the heap tuple.
*
- * Note: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
+ * NOTE: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
* the row to be updated is visible to that snapshot, and throw a
* can't-serialize error if not. This is a special-case behavior needed
* for referential integrity updates in transaction-snapshot mode
@@ -2535,9 +2813,12 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
- resultRelInfo->ri_ChangedIndexedCols,
+ mix_attrs,
&updateCxt->updateIndexes);
+ Assert(bms_is_empty(resultRelInfo->ri_ChangedIndexedCols));
+ resultRelInfo->ri_ChangedIndexedCols = mix_attrs;
+
return result;
}
@@ -2555,7 +2836,7 @@ ExecUpdateEpilogue(ModifyTableContext *context, UpdateContext *updateCxt,
ModifyTableState *mtstate = context->mtstate;
List *recheckIndexes = NIL;
- /* insert index entries for tuple if necessary */
+ /* Insert index entries for tuple if necessary */
if (resultRelInfo->ri_NumIndices > 0 && (updateCxt->updateIndexes != TU_None))
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
slot, context->estate,
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index d69dc090aa4..e9a53b95caf 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -855,10 +855,14 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
/* expressions */
n->ii_Expressions = expressions;
n->ii_ExpressionsState = NIL;
+ n->ii_ExpressionsAttrs = NULL;
/* predicates */
n->ii_Predicate = predicates;
n->ii_PredicateState = NULL;
+ n->ii_PredicateAttrs = NULL;
+ n->ii_CheckedPredicate = false;
+ n->ii_PredicateSatisfied = false;
/* exclusion constraints */
n->ii_ExclusionOps = NULL;
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index 63dd41c1f21..9bdf73eda59 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -211,6 +211,33 @@ typedef void (*ammarkpos_function) (IndexScanDesc scan);
/* restore marked scan position */
typedef void (*amrestrpos_function) (IndexScanDesc scan);
+/*
+ * amcomparedatums - Compare datums to determine if index update is needed
+ *
+ * This function compares old_datum and new_datum to determine if they would
+ * produce different index entries. For extraction-based indexes (GIN, RUM),
+ * this should:
+ * 1. Extract keys from old_datum using the opclass's extractValue function
+ * 2. Extract keys from new_datum using the opclass's extractValue function
+ * 3. Compare the two sets of keys using appropriate equality operators
+ * 4. Return true if the sets are equal (no index update needed)
+ *
+ * The comparison should account for:
+ * - Different numbers of extracted keys
+ * - NULL values
+ * - Type-specific equality (not just binary equality)
+ * - Opclass parameters (e.g., path in bson_rum_single_path_ops)
+ *
+ * For the DocumentDB example with path='a', this would extract values at
+ * path 'a' from both old and new BSON documents and compare them using
+ * BSON's equality operator.
+ */
+/* identify if updated datums would produce one or more index entries */
+typedef bool (*amcomparedatums_function) (Relation indexRelation,
+ int attno,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* Callback function signatures - for parallel index scans.
*/
@@ -313,6 +340,7 @@ typedef struct IndexAmRoutine
amendscan_function amendscan;
ammarkpos_function ammarkpos; /* can be NULL */
amrestrpos_function amrestrpos; /* can be NULL */
+ amcomparedatums_function amcomparedatums; /* can be NULL */
/* interface functions to support parallel index scans */
amestimateparallelscan_function amestimateparallelscan; /* can be NULL */
diff --git a/src/include/access/gin.h b/src/include/access/gin.h
index 13ea91922ef..2f265f4816c 100644
--- a/src/include/access/gin.h
+++ b/src/include/access/gin.h
@@ -100,6 +100,9 @@ extern PGDLLIMPORT int gin_pending_list_limit;
extern void ginGetStats(Relation index, GinStatsData *stats);
extern void ginUpdateStats(Relation index, const GinStatsData *stats,
bool is_build);
+extern bool gincomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
extern void _gin_parallel_build_main(dsm_segment *seg, shm_toc *toc);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 41d541aa6b2..59db389a546 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -326,7 +326,7 @@ extern TM_Result heap_update(Relation relation, HeapTupleData *oldtup,
TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
- Bitmapset *mix_attrs, Buffer *vmbuffer,
+ const Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
@@ -361,8 +361,8 @@ extern bool heap_tuple_needs_eventual_freeze(HeapTupleHeader tuple);
extern void simple_heap_insert(Relation relation, HeapTuple tup);
extern void simple_heap_delete(Relation relation, const ItemPointerData *tid);
-extern void simple_heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple tup, TU_UpdateIndexes *update_indexes);
+extern Bitmapset *simple_heap_update(Relation relation, const ItemPointerData *otid,
+ HeapTuple tup, TU_UpdateIndexes *update_indexes);
extern TransactionId heap_index_delete_tuples(Relation rel,
TM_IndexDeleteOp *delstate);
diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h
index 16be5c7a9c1..42bd329eaad 100644
--- a/src/include/access/nbtree.h
+++ b/src/include/access/nbtree.h
@@ -1210,6 +1210,10 @@ extern int btgettreeheight(Relation rel);
extern CompareType bttranslatestrategy(StrategyNumber strategy, Oid opfamily);
extern StrategyNumber bttranslatecmptype(CompareType cmptype, Oid opfamily);
+extern bool btcomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* prototypes for internal functions in nbtree.c
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 8a5931a3118..2b9206ff24a 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -549,7 +549,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
- Bitmapset *updated_cols,
+ const Bitmapset *updated_cols,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1503,12 +1503,12 @@ static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- Bitmapset *updated_cols, TU_UpdateIndexes *update_indexes)
+ const Bitmapset *mix_cols, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
wait, tmfd, lockmode,
- updated_cols, update_indexes);
+ mix_cols, update_indexes);
}
/*
@@ -2011,7 +2011,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
- Bitmapset *modified_indexe_attrs,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index dda95e54903..8d364f8b30f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -132,6 +132,7 @@ extern bool CompareIndexInfo(const IndexInfo *info1, const IndexInfo *info2,
const AttrMap *attmap);
extern void BuildSpeculativeIndexInfo(Relation index, IndexInfo *ii);
+extern void BuildUpdateIndexInfo(ResultRelInfo *resultRelInfo);
extern void FormIndexDatum(IndexInfo *indexInfo,
TupleTableSlot *slot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 993dc0e6ced..a19585ba065 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -739,6 +739,11 @@ extern Bitmapset *ExecGetAllUpdatedCols(ResultRelInfo *relinfo, EState *estate);
*/
extern void ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative);
extern void ExecCloseIndices(ResultRelInfo *resultRelInfo);
+extern Bitmapset *ExecWhichIndexesRequireUpdates(ResultRelInfo *relinfo,
+ Bitmapset *mix_attrs,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
extern List *ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
TupleTableSlot *slot, EState *estate,
bool update,
@@ -800,9 +805,10 @@ extern ResultRelInfo *ExecLookupResultRelByOid(ModifyTableState *node,
Oid resultoid,
bool missing_ok,
bool update_cache);
-extern Bitmapset *ExecCheckIndexedAttrsForChanges(ResultRelInfo *resultRelInfo,
- TupleTableSlot *tts_old,
- TupleTableSlot *tts_new);
+extern Bitmapset *ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
extern bool tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
Datum value1, Datum value2);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 898368fb8cb..d8e88817206 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -174,15 +174,29 @@ typedef struct IndexInfo
*/
AttrNumber ii_IndexAttrNumbers[INDEX_MAX_KEYS];
+ /*
+ * All key, expression, sumarizing, and partition attributes referenced by
+ * this index
+ */
+ Bitmapset *ii_IndexedAttrs;
+
/* expr trees for expression entries, or NIL if none */
List *ii_Expressions; /* list of Expr */
/* exec state for expressions, or NIL if none */
List *ii_ExpressionsState; /* list of ExprState */
+ /* attributes exclusively referenced by expression indexes */
+ Bitmapset *ii_ExpressionsAttrs;
/* partial-index predicate, or NIL if none */
List *ii_Predicate; /* list of Expr */
/* exec state for expressions, or NIL if none */
ExprState *ii_PredicateState;
+ /* attributes referenced by the predicate */
+ Bitmapset *ii_PredicateAttrs;
+ /* partial index predicate determined yet? */
+ bool ii_CheckedPredicate;
+ /* amupdate hint used to avoid rechecking predicate */
+ bool ii_PredicateSatisfied;
/* Per-column exclusion operators, or NULL if none */
Oid *ii_ExclusionOps; /* array with one entry per column */
@@ -494,6 +508,11 @@ typedef struct ResultRelInfo
Bitmapset *ri_extraUpdatedCols;
/* true if the above has been computed */
bool ri_extraUpdatedCols_valid;
+
+ /*
+ * For UPDATE a Bitmapset of the attributes that are both indexed and have
+ * changed in value.
+ */
Bitmapset *ri_ChangedIndexedCols;
/* Projection to generate new tuple in an INSERT/UPDATE */
diff --git a/src/test/isolation/expected/insert-conflict-specconflict.out b/src/test/isolation/expected/insert-conflict-specconflict.out
index e34a821c403..54b3981918c 100644
--- a/src/test/isolation/expected/insert-conflict-specconflict.out
+++ b/src/test/isolation/expected/insert-conflict-specconflict.out
@@ -80,6 +80,10 @@ pg_advisory_unlock
t
(1 row)
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
s1: NOTICE: acquiring advisory lock on 2
s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
@@ -172,6 +176,10 @@ pg_advisory_unlock
t
(1 row)
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
s2: NOTICE: acquiring advisory lock on 2
s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
@@ -369,6 +377,10 @@ key|data
step s1_commit: COMMIT;
s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
s2: NOTICE: acquiring advisory lock on 2
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
step s2_upsert: <... completed>
step controller_show: SELECT * FROM upserttest;
key|data
@@ -530,6 +542,14 @@ isolation/insert-conflict-specconflict/s2|transactionid|ExclusiveLock|t
step s2_commit: COMMIT;
s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_4() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 4
+s1: NOTICE: blurt_and_lock_4() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 4
step s1_upsert: <... completed>
step s1_noop:
step controller_show: SELECT * FROM upserttest;
diff --git a/src/test/regress/expected/hot_expression_indexes.out b/src/test/regress/expected/hot_expression_indexes.out
new file mode 100644
index 00000000000..ea856b5db63
--- /dev/null
+++ b/src/test/regress/expected/hot_expression_indexes.out
@@ -0,0 +1,1007 @@
+-- ================================================================
+-- Test Suite for Heap-only (HOT) Updates
+-- ================================================================
+-- Setup: Create function to measure HOT updates
+CREATE OR REPLACE FUNCTION check_hot_updates(
+ expected INT,
+ p_table_name TEXT DEFAULT 't',
+ p_schema_name TEXT DEFAULT current_schema()
+)
+RETURNS TABLE (
+ table_name TEXT,
+ total_updates BIGINT,
+ hot_updates BIGINT,
+ hot_update_percentage NUMERIC,
+ matches_expected BOOLEAN
+)
+LANGUAGE plpgsql
+AS $$
+DECLARE
+ v_relid oid;
+ v_qualified_name TEXT;
+ v_hot_updates BIGINT;
+ v_updates BIGINT;
+ v_xact_hot_updates BIGINT;
+ v_xact_updates BIGINT;
+BEGIN
+ -- Force statistics update
+ PERFORM pg_stat_force_next_flush();
+
+ -- Get table OID
+ v_qualified_name := quote_ident(p_schema_name) || '.' || quote_ident(p_table_name);
+ v_relid := v_qualified_name::regclass;
+
+ IF v_relid IS NULL THEN
+ RAISE EXCEPTION 'Table %.% not found', p_schema_name, p_table_name;
+ END IF;
+
+ -- Get cumulative + transaction stats
+ v_hot_updates := COALESCE(pg_stat_get_tuples_hot_updated(v_relid), 0);
+ v_updates := COALESCE(pg_stat_get_tuples_updated(v_relid), 0);
+ v_xact_hot_updates := COALESCE(pg_stat_get_xact_tuples_hot_updated(v_relid), 0);
+ v_xact_updates := COALESCE(pg_stat_get_xact_tuples_updated(v_relid), 0);
+
+ v_hot_updates := v_hot_updates + v_xact_hot_updates;
+ v_updates := v_updates + v_xact_updates;
+
+ RETURN QUERY
+ SELECT
+ p_table_name::TEXT,
+ v_updates::BIGINT,
+ v_hot_updates::BIGINT,
+ CASE WHEN v_updates > 0
+ THEN ROUND((v_hot_updates::numeric / v_updates::numeric * 100)::numeric, 2)
+ ELSE 0
+ END,
+ (v_hot_updates = expected)::BOOLEAN;
+END;
+$$;
+-- ================================================================
+-- Basic JSONB Expression Index
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_name_idx ON t((docs->>'name'));
+INSERT INTO t VALUES (1, '{"name": "alice", "age": 30}');
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET docs = '{"name": "alice", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update indexed JSONB field - should NOT be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update non-indexed field again - should be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 32}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 2 | 66.67 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Partial Index with Predicate Transitions
+-- ================================================================
+CREATE TABLE t(id INT, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_value_idx ON t(value) WHERE value > 10;
+INSERT INTO t VALUES (1, 5);
+-- Both outside predicate - should be HOT
+UPDATE t SET value = 8 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET value = 15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Both inside predicate, value changes - should NOT be HOT
+UPDATE t SET value = 20 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Transition out of predicate - should NOT be HOT
+UPDATE t SET value = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+-- Both outside predicate again - should be HOT
+UPDATE t SET value = 3 WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 5 | 2 | 40.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Expression Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((docs->>'status'))
+ WHERE (docs->>'priority')::int > 5;
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3}');
+-- Both outside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 4}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 10}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Inside predicate, status changes - should NOT be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 10}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Inside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 8}';
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 2 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Index on JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data);
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "database"]}');
+-- Change tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Change tags again - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Add field without changing existing keys - GIN keys changed (added "note"), NOT HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "note": "test"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Index with Unchanged Keys
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create GIN index on specific path
+CREATE INDEX t_gin_idx ON t USING gin((data->'tags'));
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "sql"], "status": "active"}');
+-- Change non-indexed field - GIN keys on 'tags' unchanged, should be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change indexed tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN with jsonb_path_ops
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data jsonb_path_ops);
+INSERT INTO t VALUES (1, '{"user": {"name": "alice"}, "tags": ["a", "b"]}');
+-- Change value at different path - keys changed, NOT HOT
+UPDATE t SET data = '{"user": {"name": "bob"}, "tags": ["a", "b"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Multi-Column Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, a INT, b INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t(id, abs(a), abs(b));
+INSERT INTO t VALUES (1, -5, -10);
+-- Change sign but not abs value - should be HOT
+UPDATE t SET a = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change abs value - should NOT be HOT
+UPDATE t SET b = -15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Change id - should NOT be HOT
+UPDATE t SET id = 2 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Mixed Index Types (BRIN + Expression)
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_idx ON t USING brin(value);
+CREATE INDEX t_expr_idx ON t((data->>'status'));
+INSERT INTO t VALUES (1, 100, '{"status": "active"}');
+-- Update only BRIN column - should be HOT
+UPDATE t SET value = 200 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update only expression column - should NOT be HOT
+UPDATE t SET data = '{"status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update both - should NOT be HOT
+UPDATE t SET value = 300, data = '{"status": "pending"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Expression with COLLATION and BTREE (nbtree) index
+-- ================================================================
+CREATE COLLATION case_insensitive (
+ provider = icu,
+ locale = 'und-u-ks-level2',
+ deterministic = false
+);
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ name TEXT COLLATE case_insensitive
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_lower_idx ON t USING BTREE (name COLLATE case_insensitive);
+INSERT INTO t VALUES (1, 'ALICE');
+-- Change case but not value - should NOT be HOT in BTREE
+UPDATE t SET name = 'Alice' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Change to new value - should NOT be HOT
+UPDATE t SET name = 'BOB' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Array Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_array_len_idx ON t(array_length(tags, 1));
+INSERT INTO t VALUES (1, ARRAY['a', 'b', 'c']);
+-- Same length, different elements - should be HOT
+UPDATE t SET tags = ARRAY['d', 'e', 'f'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Different length - should NOT be HOT
+UPDATE t SET tags = ARRAY['d', 'e'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Nested JSONB Expression and JSONB equality '->' (not '->>')
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_nested_idx ON t((data->'user'->'name'));
+INSERT INTO t VALUES ('{"user": {"name": "alice", "age": 30}}');
+-- Change nested non-indexed field - should be HOT
+UPDATE t SET data = '{"user": {"name": "alice", "age": 31}}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change nested indexed field - should NOT be HOT
+UPDATE t SET data = '{"user": {"name": "bob", "age": 31}}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Complex Predicate on Multiple JSONB Fields
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((data->>'status'))
+ WHERE (data->>'priority')::int > 5
+ AND (data->>'active')::boolean = true;
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3, "active": true}');
+-- Outside predicate (priority too low) - should be HOT
+UPDATE t SET data = '{"status": "done", "priority": 3, "active": true}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": true}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Inside predicate, change to outside (active = false) - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": false}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Array Index - Order Insensitive Extraction
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ data JSONB
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+-- GIN index on JSONB array (extracts all elements)
+CREATE INDEX t_items_gin ON t USING GIN ((data->'items'));
+INSERT INTO t VALUES (1, '{"items": [1, 2, 3], "status": "active"}');
+-- Update: Reorder array elements
+-- JSONB equality: NOT equal (different arrays)
+-- GIN extraction: Same elements extracted (might allow HOT if not careful)
+UPDATE t SET data = '{"items": [3, 2, 1], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update: Add/remove element
+UPDATE t SET data = '{"items": [1, 2, 3, 4], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- TOASTed Values in Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, large_text TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_substr_idx ON t(substr(large_text, 1, 10));
+INSERT INTO t VALUES (1, repeat('x', 5000) || 'identifier');
+-- Change end of string, prefix unchanged - should be HOT
+UPDATE t SET large_text = repeat('x', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change prefix - should NOT be HOT
+UPDATE t SET large_text = repeat('y', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- TEST: GIN with TOASTed TEXT (tsvector)
+-- ================================================================
+CREATE TABLE t(id INT, content TEXT, search_vec tsvector)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create trigger to maintain tsvector
+CREATE TRIGGER tsvectorupdate_toast
+ BEFORE INSERT OR UPDATE ON t
+ FOR EACH ROW EXECUTE FUNCTION
+ tsvector_update_trigger(search_vec, 'pg_catalog.english', content);
+CREATE INDEX t_gin ON t USING gin(search_vec);
+-- Insert with large content (will be TOASTed)
+INSERT INTO t (id, content) VALUES
+ (1, repeat('important keyword ', 1000) || repeat('filler text ', 10000));
+-- Verify initial state
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('important');
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+-- IMPORTANT: The BEFORE UPDATE trigger modifies search_vec, so by the time
+-- ExecWhichIndexesRequireUpdates() runs, search_vec has already changed.
+-- This means the comparison sees old tsvector vs. trigger-modified tsvector,
+-- not the natural progression. HOT won't happen because the trigger changed
+-- the indexed column.
+-- Update: Even though content keywords unchanged, trigger still fires
+UPDATE t
+SET content = repeat('important keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (trigger modifies search_vec, blocking HOT)
+-- This is actually correct behavior - the trigger updated an indexed column
+-- Update: Change indexed keywords
+UPDATE t
+SET content = repeat('critical keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (index keys changed)
+-- Verify query correctness
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('critical');
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+DROP TABLE t CASCADE;
+-- ================================================================
+-- TEST: GIN with TOASTed JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin((data->'tags'));
+-- Insert with TOASTed JSONB
+INSERT INTO t (id, data) VALUES
+ (1, jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('x', 10000)
+ ));
+-- Update: Change large_field, tags unchanged - should be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT update
+-- Update: Change tags - should NOT be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "sql"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Expected: Still 1 HOT
+-- Verify correctness
+SELECT count(*) FROM t WHERE data->'tags' @> '["database"]'::jsonb;
+ count
+-------
+ 0
+(1 row)
+
+-- Expected: 0 rows
+SELECT count(*) FROM t WHERE data->'tags' @> '["sql"]'::jsonb;
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+DROP TABLE t CASCADE;
+-- ================================================================
+-- TEST: GIN with Array of Large Strings
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin(tags);
+-- Insert with large array elements (might be TOASTed)
+INSERT INTO t (id, tags) VALUES
+ (1, ARRAY[repeat('tag1', 1000), repeat('tag2', 1000)]);
+-- Update: Change to different large values - NOT HOT
+UPDATE t
+SET tags = ARRAY[repeat('tag3', 1000), repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (keys actually changed)
+-- Update: Keep same tag values, just reorder - SHOULD BE HOT
+-- (GIN is order-insensitive: both [tag3,tag4] and [tag4,tag3]
+-- extract to the same sorted key set ['tag3','tag4'])
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000), repeat('tag3', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Expected: 1 HOT (GIN keys semantically identical)
+-- Update: Remove an element - NOT HOT (keys changed)
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Expected: Still 1 HOT (not this one)
+DROP TABLE t CASCADE;
+-- ================================================================
+-- BRIN Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ value INT,
+ description TEXT
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_partial_idx ON t USING brin(value) WHERE value > 100;
+INSERT INTO t VALUES (1, 50, 'below range');
+-- Test 1: Outside predicate
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Validate: Predicate query returns 0 rows
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+ cnt
+-----
+ 0
+(1 row)
+
+-- Test 2: Transition into predicate
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Validate: Predicate query returns 1 row with correct value
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+ cnt | max_val
+-----+---------
+ 1 | 150
+(1 row)
+
+-- Test 3: Inside predicate, value changes
+UPDATE t SET value = 160, description = 'updated again' WHERE id = 1;
+SELECT * FROM check_hot_updates(3, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 3 | 100.00 | t
+(1 row)
+
+-- Validate: Updated value (160) is returned
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+ cnt | max_val
+-----+---------
+ 1 | 160
+(1 row)
+
+-- Test 4: Transition out of predicate
+UPDATE t SET value = 50 WHERE id = 1;
+SELECT * FROM check_hot_updates(4, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 4 | 100.00 | t
+(1 row)
+
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+ cnt
+-----
+ 0
+(1 row)
+
+SELECT id, value, description FROM t;
+ id | value | description
+----+-------+---------------
+ 1 | 50 | updated again
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- HASH Index (Simple Column)
+-- ================================================================
+CREATE TABLE t(id INT, code VARCHAR(20), description TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_idx ON t USING hash(code);
+INSERT INTO t VALUES (1, 'CODE001', 'initial');
+-- Update non-indexed column - should be HOT
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update indexed column - HASH index requires update, NOT HOT
+UPDATE t SET code = 'CODE002' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update both - NOT HOT
+UPDATE t SET code = 'CODE003', description = 'changed' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Back to original code - NOT HOT (different hash bucket location)
+UPDATE t SET code = 'CODE001' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- HASH Index on Expression
+-- ================================================================
+CREATE TABLE t(id INT, email TEXT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_lower_email_idx ON t USING HASH(lower(email));
+INSERT INTO t VALUES (1, '[email protected]', '{"status": "new"}');
+-- Update non-indexed field - should be HOT
+UPDATE t SET data = '{"status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update email with case change only (same lowercase) - should be HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Update email to different lowercase - NOT HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 2 | 66.67 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- HASH Index on JSONB Field
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash((data->>'category'));
+INSERT INTO t VALUES (1, '{"category": "books", "title": "PostgreSQL Guide"}');
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET data = '{"category": "books", "title": "PostgreSQL Handbook"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update indexed JSONB field - NOT HOT
+UPDATE t SET data = '{"category": "videos", "title": "PostgreSQL Handbook"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update both - NOT HOT
+UPDATE t SET data = '{"category": "courses", "title": "PostgreSQL Basics"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- Multiple HASH Indexes
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, status VARCHAR, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+CREATE INDEX t_hash_status_idx ON t USING hash(status);
+INSERT INTO t VALUES (1, 'electronics', 'active', 100);
+-- Update non-indexed column - should be HOT
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update one indexed column - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update other indexed column - NOT HOT
+UPDATE t SET status = 'inactive' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Update both indexed columns - NOT HOT
+UPDATE t SET category = 'videos', status = 'pending' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- BRIN vs HASH Comparison
+-- ================================================================
+CREATE TABLE t_brin(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE TABLE t_hash(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_value_idx ON t_brin USING brin(value);
+CREATE INDEX t_hash_value_idx ON t_hash USING hash(value);
+INSERT INTO t_brin VALUES (1, 100, 'initial');
+INSERT INTO t_hash VALUES (1, 100, 'initial');
+-- Same update on both - different HOT behavior expected
+-- BRIN: might allow HOT (range summary unchanged)
+-- HASH: blocks HOT (hash bucket changed)
+UPDATE t_brin SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't_brin');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t_brin | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT (BRIN allows it for single row)
+UPDATE t_hash SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't_hash');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t_hash | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (HASH blocks it)
+DROP TABLE t_brin CASCADE;
+DROP TABLE t_hash CASCADE;
+-- ================================================================
+-- HASH Index with NULL Values
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'electronics', 'initial');
+-- Update indexed column to NULL - NOT HOT (hash value changed)
+UPDATE t SET category = NULL WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT
+-- Update indexed column from NULL to value - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Expected: 1 HOT
+DROP TABLE t CASCADE;
+-- ================================================================
+-- BRIN on JSONB Field
+-- ================================================================
+CREATE TABLE t(id INT, metrics JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- BRIN doesn't directly support JSONB, but we can test on expression
+CREATE INDEX t_brin_count_idx ON t USING brin(
+ CAST(metrics->>'count' AS INTEGER)
+);
+INSERT INTO t VALUES (1, '{"count": "100", "timestamp": "2024-01-01"}');
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET metrics = '{"count": "100", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT
+-- Update indexed field - BRIN allows HOT for single row
+UPDATE t SET metrics = '{"count": "150", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Expected: 2 HOT (BRIN permits single-row updates)
+DROP TABLE t CASCADE;
+-- ================================================================
+-- Mixed BRIN + HASH on Same Table
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, timestamp TIMESTAMP, price NUMERIC, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_timestamp_idx ON t USING brin(timestamp);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'books', '2024-01-01 10:00:00', 29.99, 'initial');
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT
+-- Update BRIN indexed column - allows HOT
+UPDATE t SET timestamp = '2024-01-02 10:00:00' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Expected: 2 HOT
+-- Update HASH indexed column - blocks HOT
+UPDATE t SET category = 'videos' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 2 | 66.67 | t
+(1 row)
+
+-- Expected: 2 HOT (HASH blocks it)
+-- Update price (non-indexed) - should be HOT
+UPDATE t SET price = 39.99 WHERE id = 1;
+SELECT * FROM check_hot_updates(3, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 3 | 75.00 | t
+(1 row)
+
+-- Expected: 3 HOT
+DROP TABLE t CASCADE;
+-- Cleanup
+DROP FUNCTION check_hot_updates(int, text, text);
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index f56482fb9f1..4459625a59b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -125,6 +125,12 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression compression_lz4 memoize stats predicate numa eager_aggregate
+
+# ----------
+# Another group of parallel tests, these focused on heap HOT updates
+# ----------
+test: hot_expression_indexes
+
# event_trigger depends on create_am and cannot run concurrently with
# any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/sql/hot_expression_indexes.sql b/src/test/regress/sql/hot_expression_indexes.sql
new file mode 100644
index 00000000000..4929be144ae
--- /dev/null
+++ b/src/test/regress/sql/hot_expression_indexes.sql
@@ -0,0 +1,747 @@
+-- ================================================================
+-- Test Suite for Heap-only (HOT) Updates
+-- ================================================================
+
+-- Setup: Create function to measure HOT updates
+CREATE OR REPLACE FUNCTION check_hot_updates(
+ expected INT,
+ p_table_name TEXT DEFAULT 't',
+ p_schema_name TEXT DEFAULT current_schema()
+)
+RETURNS TABLE (
+ table_name TEXT,
+ total_updates BIGINT,
+ hot_updates BIGINT,
+ hot_update_percentage NUMERIC,
+ matches_expected BOOLEAN
+)
+LANGUAGE plpgsql
+AS $$
+DECLARE
+ v_relid oid;
+ v_qualified_name TEXT;
+ v_hot_updates BIGINT;
+ v_updates BIGINT;
+ v_xact_hot_updates BIGINT;
+ v_xact_updates BIGINT;
+BEGIN
+ -- Force statistics update
+ PERFORM pg_stat_force_next_flush();
+
+ -- Get table OID
+ v_qualified_name := quote_ident(p_schema_name) || '.' || quote_ident(p_table_name);
+ v_relid := v_qualified_name::regclass;
+
+ IF v_relid IS NULL THEN
+ RAISE EXCEPTION 'Table %.% not found', p_schema_name, p_table_name;
+ END IF;
+
+ -- Get cumulative + transaction stats
+ v_hot_updates := COALESCE(pg_stat_get_tuples_hot_updated(v_relid), 0);
+ v_updates := COALESCE(pg_stat_get_tuples_updated(v_relid), 0);
+ v_xact_hot_updates := COALESCE(pg_stat_get_xact_tuples_hot_updated(v_relid), 0);
+ v_xact_updates := COALESCE(pg_stat_get_xact_tuples_updated(v_relid), 0);
+
+ v_hot_updates := v_hot_updates + v_xact_hot_updates;
+ v_updates := v_updates + v_xact_updates;
+
+ RETURN QUERY
+ SELECT
+ p_table_name::TEXT,
+ v_updates::BIGINT,
+ v_hot_updates::BIGINT,
+ CASE WHEN v_updates > 0
+ THEN ROUND((v_hot_updates::numeric / v_updates::numeric * 100)::numeric, 2)
+ ELSE 0
+ END,
+ (v_hot_updates = expected)::BOOLEAN;
+END;
+$$;
+
+-- ================================================================
+-- Basic JSONB Expression Index
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_name_idx ON t((docs->>'name'));
+INSERT INTO t VALUES (1, '{"name": "alice", "age": 30}');
+
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET docs = '{"name": "alice", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update indexed JSONB field - should NOT be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update non-indexed field again - should be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 32}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Partial Index with Predicate Transitions
+-- ================================================================
+CREATE TABLE t(id INT, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_value_idx ON t(value) WHERE value > 10;
+INSERT INTO t VALUES (1, 5);
+
+-- Both outside predicate - should be HOT
+UPDATE t SET value = 8 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET value = 15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Both inside predicate, value changes - should NOT be HOT
+UPDATE t SET value = 20 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Transition out of predicate - should NOT be HOT
+UPDATE t SET value = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Both outside predicate again - should be HOT
+UPDATE t SET value = 3 WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Expression Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((docs->>'status'))
+ WHERE (docs->>'priority')::int > 5;
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3}');
+
+-- Both outside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 4}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 10}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Inside predicate, status changes - should NOT be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 10}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Inside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 8}';
+SELECT * FROM check_hot_updates(2, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN Index on JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data);
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "database"]}');
+
+-- Change tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+-- Change tags again - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+-- Add field without changing existing keys - GIN keys changed (added "note"), NOT HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "note": "test"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN Index with Unchanged Keys
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create GIN index on specific path
+CREATE INDEX t_gin_idx ON t USING gin((data->'tags'));
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "sql"], "status": "active"}');
+
+-- Change non-indexed field - GIN keys on 'tags' unchanged, should be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change indexed tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN with jsonb_path_ops
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data jsonb_path_ops);
+INSERT INTO t VALUES (1, '{"user": {"name": "alice"}, "tags": ["a", "b"]}');
+
+-- Change value at different path - keys changed, NOT HOT
+UPDATE t SET data = '{"user": {"name": "bob"}, "tags": ["a", "b"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Multi-Column Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, a INT, b INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t(id, abs(a), abs(b));
+INSERT INTO t VALUES (1, -5, -10);
+
+-- Change sign but not abs value - should be HOT
+UPDATE t SET a = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change abs value - should NOT be HOT
+UPDATE t SET b = -15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change id - should NOT be HOT
+UPDATE t SET id = 2 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Mixed Index Types (BRIN + Expression)
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_idx ON t USING brin(value);
+CREATE INDEX t_expr_idx ON t((data->>'status'));
+INSERT INTO t VALUES (1, 100, '{"status": "active"}');
+
+-- Update only BRIN column - should be HOT
+UPDATE t SET value = 200 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update only expression column - should NOT be HOT
+UPDATE t SET data = '{"status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update both - should NOT be HOT
+UPDATE t SET value = 300, data = '{"status": "pending"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Expression with COLLATION and BTREE (nbtree) index
+-- ================================================================
+CREATE COLLATION case_insensitive (
+ provider = libc,
+ locale = 'C'
+);
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ name TEXT COLLATE case_insensitive
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+
+CREATE INDEX t_lower_idx ON t USING BTREE (name COLLATE case_insensitive);
+
+INSERT INTO t VALUES (1, 'ALICE');
+
+-- Change case but not value - should NOT be HOT in BTREE
+UPDATE t SET name = 'Alice' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+-- Change to new value - should NOT be HOT
+UPDATE t SET name = 'BOB' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Array Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_array_len_idx ON t(array_length(tags, 1));
+INSERT INTO t VALUES (1, ARRAY['a', 'b', 'c']);
+
+-- Same length, different elements - should be HOT
+UPDATE t SET tags = ARRAY['d', 'e', 'f'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Different length - should NOT be HOT
+UPDATE t SET tags = ARRAY['d', 'e'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Nested JSONB Expression and JSONB equality '->' (not '->>')
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_nested_idx ON t((data->'user'->'name'));
+INSERT INTO t VALUES ('{"user": {"name": "alice", "age": 30}}');
+
+-- Change nested non-indexed field - should be HOT
+UPDATE t SET data = '{"user": {"name": "alice", "age": 31}}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change nested indexed field - should NOT be HOT
+UPDATE t SET data = '{"user": {"name": "bob", "age": 31}}';
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Complex Predicate on Multiple JSONB Fields
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((data->>'status'))
+ WHERE (data->>'priority')::int > 5
+ AND (data->>'active')::boolean = true;
+
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3, "active": true}');
+
+-- Outside predicate (priority too low) - should be HOT
+UPDATE t SET data = '{"status": "done", "priority": 3, "active": true}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": true}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Inside predicate, change to outside (active = false) - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": false}';
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN Array Index - Order Insensitive Extraction
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ data JSONB
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+
+-- GIN index on JSONB array (extracts all elements)
+CREATE INDEX t_items_gin ON t USING GIN ((data->'items'));
+
+INSERT INTO t VALUES (1, '{"items": [1, 2, 3], "status": "active"}');
+
+-- Update: Reorder array elements
+-- JSONB equality: NOT equal (different arrays)
+-- GIN extraction: Same elements extracted (might allow HOT if not careful)
+UPDATE t SET data = '{"items": [3, 2, 1], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update: Add/remove element
+UPDATE t SET data = '{"items": [1, 2, 3, 4], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- TOASTed Values in Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, large_text TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_substr_idx ON t(substr(large_text, 1, 10));
+
+INSERT INTO t VALUES (1, repeat('x', 5000) || 'identifier');
+
+-- Change end of string, prefix unchanged - should be HOT
+UPDATE t SET large_text = repeat('x', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change prefix - should NOT be HOT
+UPDATE t SET large_text = repeat('y', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- TEST: GIN with TOASTed TEXT (tsvector)
+-- ================================================================
+CREATE TABLE t(id INT, content TEXT, search_vec tsvector)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+-- Create trigger to maintain tsvector
+CREATE TRIGGER tsvectorupdate_toast
+ BEFORE INSERT OR UPDATE ON t
+ FOR EACH ROW EXECUTE FUNCTION
+ tsvector_update_trigger(search_vec, 'pg_catalog.english', content);
+
+CREATE INDEX t_gin ON t USING gin(search_vec);
+
+-- Insert with large content (will be TOASTed)
+INSERT INTO t (id, content) VALUES
+ (1, repeat('important keyword ', 1000) || repeat('filler text ', 10000));
+
+-- Verify initial state
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('important');
+-- Expected: 1 row
+
+-- IMPORTANT: The BEFORE UPDATE trigger modifies search_vec, so by the time
+-- ExecWhichIndexesRequireUpdates() runs, search_vec has already changed.
+-- This means the comparison sees old tsvector vs. trigger-modified tsvector,
+-- not the natural progression. HOT won't happen because the trigger changed
+-- the indexed column.
+
+-- Update: Even though content keywords unchanged, trigger still fires
+UPDATE t
+SET content = repeat('important keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+-- Expected: 0 HOT (trigger modifies search_vec, blocking HOT)
+-- This is actually correct behavior - the trigger updated an indexed column
+
+-- Update: Change indexed keywords
+UPDATE t
+SET content = repeat('critical keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+-- Expected: 0 HOT (index keys changed)
+
+-- Verify query correctness
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('critical');
+-- Expected: 1 row
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- TEST: GIN with TOASTed JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin((data->'tags'));
+
+-- Insert with TOASTed JSONB
+INSERT INTO t (id, data) VALUES
+ (1, jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('x', 10000)
+ ));
+
+-- Update: Change large_field, tags unchanged - should be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: 1 HOT update
+
+-- Update: Change tags - should NOT be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "sql"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: Still 1 HOT
+
+-- Verify correctness
+SELECT count(*) FROM t WHERE data->'tags' @> '["database"]'::jsonb;
+-- Expected: 0 rows
+SELECT count(*) FROM t WHERE data->'tags' @> '["sql"]'::jsonb;
+-- Expected: 1 row
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- TEST: GIN with Array of Large Strings
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin(tags);
+
+-- Insert with large array elements (might be TOASTed)
+INSERT INTO t (id, tags) VALUES
+ (1, ARRAY[repeat('tag1', 1000), repeat('tag2', 1000)]);
+
+-- Update: Change to different large values - NOT HOT
+UPDATE t
+SET tags = ARRAY[repeat('tag3', 1000), repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+-- Expected: 0 HOT (keys actually changed)
+
+-- Update: Keep same tag values, just reorder - SHOULD BE HOT
+-- (GIN is order-insensitive: both [tag3,tag4] and [tag4,tag3]
+-- extract to the same sorted key set ['tag3','tag4'])
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000), repeat('tag3', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: 1 HOT (GIN keys semantically identical)
+
+-- Update: Remove an element - NOT HOT (keys changed)
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: Still 1 HOT (not this one)
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- BRIN Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ value INT,
+ description TEXT
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+
+CREATE INDEX t_brin_partial_idx ON t USING brin(value) WHERE value > 100;
+
+INSERT INTO t VALUES (1, 50, 'below range');
+
+-- Test 1: Outside predicate
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Validate: Predicate query returns 0 rows
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+
+-- Test 2: Transition into predicate
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+
+-- Validate: Predicate query returns 1 row with correct value
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+
+-- Test 3: Inside predicate, value changes
+UPDATE t SET value = 160, description = 'updated again' WHERE id = 1;
+SELECT * FROM check_hot_updates(3, 't');
+
+-- Validate: Updated value (160) is returned
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+
+-- Test 4: Transition out of predicate
+UPDATE t SET value = 50 WHERE id = 1;
+SELECT * FROM check_hot_updates(4, 't');
+
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+
+SELECT id, value, description FROM t;
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- HASH Index (Simple Column)
+-- ================================================================
+CREATE TABLE t(id INT, code VARCHAR(20), description TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_idx ON t USING hash(code);
+INSERT INTO t VALUES (1, 'CODE001', 'initial');
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update indexed column - HASH index requires update, NOT HOT
+UPDATE t SET code = 'CODE002' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update both - NOT HOT
+UPDATE t SET code = 'CODE003', description = 'changed' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Back to original code - NOT HOT (different hash bucket location)
+UPDATE t SET code = 'CODE001' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- HASH Index on Expression
+-- ================================================================
+CREATE TABLE t(id INT, email TEXT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_lower_email_idx ON t USING HASH(lower(email));
+INSERT INTO t VALUES (1, '[email protected]', '{"status": "new"}');
+
+-- Update non-indexed field - should be HOT
+UPDATE t SET data = '{"status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update email with case change only (same lowercase) - should be HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+
+-- Update email to different lowercase - NOT HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- HASH Index on JSONB Field
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash((data->>'category'));
+INSERT INTO t VALUES (1, '{"category": "books", "title": "PostgreSQL Guide"}');
+
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET data = '{"category": "books", "title": "PostgreSQL Handbook"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update indexed JSONB field - NOT HOT
+UPDATE t SET data = '{"category": "videos", "title": "PostgreSQL Handbook"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update both - NOT HOT
+UPDATE t SET data = '{"category": "courses", "title": "PostgreSQL Basics"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- Multiple HASH Indexes
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, status VARCHAR, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+CREATE INDEX t_hash_status_idx ON t USING hash(status);
+INSERT INTO t VALUES (1, 'electronics', 'active', 100);
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update one indexed column - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update other indexed column - NOT HOT
+UPDATE t SET status = 'inactive' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update both indexed columns - NOT HOT
+UPDATE t SET category = 'videos', status = 'pending' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- BRIN vs HASH Comparison
+-- ================================================================
+CREATE TABLE t_brin(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE TABLE t_hash(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+CREATE INDEX t_brin_value_idx ON t_brin USING brin(value);
+CREATE INDEX t_hash_value_idx ON t_hash USING hash(value);
+
+INSERT INTO t_brin VALUES (1, 100, 'initial');
+INSERT INTO t_hash VALUES (1, 100, 'initial');
+
+-- Same update on both - different HOT behavior expected
+-- BRIN: might allow HOT (range summary unchanged)
+-- HASH: blocks HOT (hash bucket changed)
+UPDATE t_brin SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't_brin');
+-- Expected: 1 HOT (BRIN allows it for single row)
+
+UPDATE t_hash SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't_hash');
+-- Expected: 0 HOT (HASH blocks it)
+
+DROP TABLE t_brin CASCADE;
+DROP TABLE t_hash CASCADE;
+
+-- ================================================================
+-- HASH Index with NULL Values
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'electronics', 'initial');
+
+-- Update indexed column to NULL - NOT HOT (hash value changed)
+UPDATE t SET category = NULL WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+-- Expected: 0 HOT
+
+-- Update indexed column from NULL to value - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+-- Expected: 0 HOT
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: 1 HOT
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- BRIN on JSONB Field
+-- ================================================================
+CREATE TABLE t(id INT, metrics JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- BRIN doesn't directly support JSONB, but we can test on expression
+CREATE INDEX t_brin_count_idx ON t USING brin(
+ CAST(metrics->>'count' AS INTEGER)
+);
+INSERT INTO t VALUES (1, '{"count": "100", "timestamp": "2024-01-01"}');
+
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET metrics = '{"count": "100", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: 1 HOT
+
+-- Update indexed field - BRIN allows HOT for single row
+UPDATE t SET metrics = '{"count": "150", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+-- Expected: 2 HOT (BRIN permits single-row updates)
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- Mixed BRIN + HASH on Same Table
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, timestamp TIMESTAMP, price NUMERIC, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_timestamp_idx ON t USING brin(timestamp);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'books', '2024-01-01 10:00:00', 29.99, 'initial');
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: 1 HOT
+
+-- Update BRIN indexed column - allows HOT
+UPDATE t SET timestamp = '2024-01-02 10:00:00' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+-- Expected: 2 HOT
+
+-- Update HASH indexed column - blocks HOT
+UPDATE t SET category = 'videos' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+-- Expected: 2 HOT (HASH blocks it)
+
+-- Update price (non-indexed) - should be HOT
+UPDATE t SET price = 39.99 WHERE id = 1;
+SELECT * FROM check_hot_updates(3, 't');
+-- Expected: 3 HOT
+
+DROP TABLE t CASCADE;
+
+-- Cleanup
+DROP FUNCTION check_hot_updates(int, text, text);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 23bce72ae64..52ef8f10b35 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -390,6 +390,7 @@ CachedFunctionCompileCallback
CachedFunctionDeleteCallback
CachedFunctionHashEntry
CachedFunctionHashKey
+CachedIndexDatum
CachedPlan
CachedPlanSource
CallContext
--
2.49.0
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-07 21:36 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-09 19:27 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2025-11-16 18:53 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-11-19 18:00 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
@ 2025-11-19 18:21 ` Greg Burd <[email protected]>
1 sibling, 0 replies; 37+ messages in thread
From: Greg Burd @ 2025-11-19 18:21 UTC (permalink / raw)
To: pgsql-hackers
On Nov 19 2025, at 1:00 pm, Greg Burd <[email protected]> wrote:
>
> On Nov 16 2025, at 1:53 pm, Greg Burd <[email protected]> wrote:
>
>> 0004 - Enable HOT updates for expression and partial indexes
>>
>> This finally gets us back to where this project started, but on much
>> more firm ground than before because we're not going to
>> self-deadlock.
>> The idea has grown from a small function into something larger, but only
>> out of necessity.
>>
>> In this patch I add ExecWhichIndexesRequireUpdates() in execIndexing.c
>> which implements (c) finding the set of attributes that force new index
>> updates. This set can be very different from the modified indexed
>> attributes. We know that some attributes are not equal to their
>> previous versions, but does that mean that the index that references
>> that attribute needs a new index tuple? It may, or it may not. Here's
>> the comment on that function that explains:
>>
>> /*
>> * ExecWhichIndexesRequireUpdates
>> *
>> * Determine which indexes need updating given modified indexed attributes.
>> * This function is a companion to ExecCheckIndexedAttrsForChanges().
>> On the
>> * surface, they appear similar but they are doing two very different things.
>> *
>> * For a standard index on a set of attributes this is the
>> intersection of
>> * the mix_attrs and the index attrs (key, expression, but not predicate).
>> *
>> * For expression indexes and indexes which implement the amcomparedatums()
>> * index AM API we'll need to form index datum and compare each
>> attribute to
>> * see if any actually changed.
>> *
>> * For expression indexes the result of the expression might not change
>> at all,
>> * this is common with JSONB columns which require expression indexes
>> and where
>> * it is commonplace to index a field within a document and have
>> updates that
>> * generally don't update that field.
>> *
>> * Partial indexes won't trigger index tuples when the old/new tuples
>> are both
>> * outside of the predicate range.
>> *
>> * For nbtree the amcomparedatums() API is critical as it requires
>> that key
>> * attributes are equal when they memcmp(), which might not be the
>> case when
>> * using type-specific comparison or factoring in collation which
>> might make
>> * an index case insensitive.
>> *
>> * All of this is to say that the goal is for the executor to know,
>> ahead of
>> * calling into the table AM for the update and before calling into
>> the index
>> * AM for inserting new index tuples, which attributes at a minimum will
>> * necessitate a new index tuple.
>> *
>> ...
>> */
>
> Attached are rebased (d5b4f3a6d4e) patches with the only changes
> happening in the last patch in the series.
>
> 0004 - Enable HOT updates for expression and partial indexes
>
> I was never happy with the dual functions
> ExecCheckIndexedAttrsForChanges() and ExecWhichIndexesRequireUpdates(),
> it felt like too much overhead and duplication of effort. While
> updating my tests, adding a few cases, I found that there was also a
> flaw in the logic. So, time to rewrite and combine them.
>
> What did I discover? Before the logic was to find the set of modified
> indexed attributes then review all the indexes for changed attributes
> using FormIndexDatum() and comparing before/after to see if expressions
> really changed the value to be indexed or not. The first pass didn't
> take into account expressions, the second did. So, an expression index
> over JSONB data wouldn't extract and test the field within the document,
> it was just comparing the entire document before/after using the jsonb
> comparison function, no bueno.
>
> This approach wraps both functions into one somewhat simplified
> function. The logic is basically, iterate over the indexes reviewing
> indexed attributes for changes. Along the way we call into the new
> index AM's comparison function when present, otherwise we find and use
> the proper type-specific comparison function for the datum. At the end
> of the function we have our Bitmapset of attributes that should trigger
> new index tuples.
>
>> What's left undone?
>>
>> * I need to check code coverage so that I might
>
> I did this and it was quite good, I'll do it again for this new series
> but it's nice to see that the tests are exercising the vast majority of
> the code paths.
>
>> * create tests covering all the new cases
>
> I think the coverage is good, maybe even redundant or overly complex
> in places.
>
>> * update the README.HOT documentation, wiki, etc.
>
> Soon, I hope to have this approach solid and under review before
> solidifying the docs.
>
>> * performance...
>
> Still as yet unmeasured, I know that there is more work per-update to
> perform these checks, so some overhead, but I don't know if that
> overhead is more than before with HeapDetermineColumnsInfo() and
> index_unchanged_by_update(). Those two functions did essentially the
> same thing, only with binary comparison (datumIsEqual()). I need to
> measure that. What about doing all this work outside of the buffer lock
> in heap_update()? Surely that'll give back a bit or at least add to
> concurrency. Forming index tuples a few extra times and evaluating the
> expressions 3 times rather than 1 is going to hurt, I think I can come
> up with a way to cache the formed datum and use it later on, but is that
> worth it? Complex expressions, yes. Also, what about expressions that
> expect to be executed once... and now are 3x? That's what forced my
> update to the insert-conflict-specconflict.out test, but AFAICT there is
> no way to test if an expression's value is going to change on update
> without exercising it once for the old tuple and once for the new tuple.
> Even if it were possible for an index to provide the key it might have
> changed after the expression evaluation (as is the case in hash), so I
> don't think this is avoidable. Maybe that's reason enough to add a
> reloption to disable the expression evaluation piece of this? Given
> that it might create a logic or performance regression. The flip side
> is the potential to use the HOT path, that's a real savings.
>
> One concerning thing is that nbtree's assumption that key attributes for
> TIDs must use binary comparison for equality. This means that for our
> common case (heap/btree) there is more work per-update than before,
> which is why I need to measure. I could look into eliminating the
> nbtree requirement, I don't understand it too well as yet by I believe
> that on page split there is an attempt to deduplicate TIDs into a
> TIDBitmap and the test for when that's possible is datumIsEqual(). If
> that were the same as in this new code, possibly evening using
> tts_attr_equal(), then... I don't know, I'll have to investigate. Chime
> in here if you can educate me on this one. :)
>
> best.
>
> -greg
Doh!
I forgot to commit the fixed regression test expected output before
formatting the patch set, here it is.
-greg
Attachments:
[application/octet-stream] v22-0001-Reorganize-heap-update-logic.patch (47.6K, 2-v22-0001-Reorganize-heap-update-logic.patch)
download | inline diff:
From 0de059ae17042a76594610e1b7b35dbb2db9415c Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 2 Nov 2025 11:36:20 -0500
Subject: [PATCH v22 1/4] Reorganize heap update logic
This commit refactors the interaction between heap_tuple_update(),
heap_update(), and simple_heap_update() to improve code organization
and flexibility. The changes are functionally equivalent to the
previous implementation and have no performance impact.
The primary motivation is to prepare for upcoming modifications to
how and where modified attributes are identified during the update
path, particularly for catalog updates.
As part of this reorganization, the handling of replica identity key
attributes has been adjusted. Instead of fetching a second copy of
the bitmap during an update operation, the caller is now required to
provide it. This change applies to both heap_update() and
heap_delete().
No user-visible changes.
---
src/backend/access/heap/heapam.c | 568 +++++++++++------------
src/backend/access/heap/heapam_handler.c | 117 ++++-
src/include/access/heapam.h | 24 +-
3 files changed, 410 insertions(+), 299 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 4b0c49f4bb0..aff47481345 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -39,18 +39,24 @@
#include "access/syncscan.h"
#include "access/valid.h"
#include "access/visibilitymap.h"
+#include "access/xact.h"
#include "access/xloginsert.h"
+#include "catalog/catalog.h"
#include "catalog/pg_database.h"
#include "catalog/pg_database_d.h"
#include "commands/vacuum.h"
+#include "nodes/bitmapset.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
+#include "storage/bufmgr.h"
+#include "storage/itemptr.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
#include "storage/procarray.h"
#include "utils/datum.h"
#include "utils/injection_point.h"
#include "utils/inval.h"
+#include "utils/relcache.h"
#include "utils/spccache.h"
#include "utils/syscache.h"
@@ -62,16 +68,8 @@ static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
HeapTuple newtup, HeapTuple old_key_tuple,
bool all_visible_cleared, bool new_all_visible_cleared);
#ifdef USE_ASSERT_CHECKING
-static void check_lock_if_inplace_updateable_rel(Relation relation,
- const ItemPointerData *otid,
- HeapTuple newtup);
static void check_inplace_rel_lock(HeapTuple oldtup);
#endif
-static Bitmapset *HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external);
static bool heap_acquire_tuplock(Relation relation, const ItemPointerData *tid,
LockTupleMode mode, LockWaitPolicy wait_policy,
bool *have_tuple_lock);
@@ -103,10 +101,10 @@ static bool ConditionalMultiXactIdWait(MultiXactId multi, MultiXactStatus status
static void index_delete_sort(TM_IndexDeleteOp *delstate);
static int bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate);
static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
-static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
+static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp,
+ Bitmapset *rid_attrs, bool key_required,
bool *copy);
-
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
* corresponding MultiXactStatuses (one to merely lock tuples, another one to
@@ -2799,6 +2797,7 @@ heap_delete(Relation relation, const ItemPointerData *tid,
Buffer buffer;
Buffer vmbuffer = InvalidBuffer;
TransactionId new_xmax;
+ Bitmapset *rid_attrs;
uint16 new_infomask,
new_infomask2;
bool have_tuple_lock = false;
@@ -2811,6 +2810,8 @@ heap_delete(Relation relation, const ItemPointerData *tid,
AssertHasSnapshotForToast(relation);
+ rid_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
/*
* Forbid this during a parallel operation, lest it allocate a combo CID.
* Other workers might need that combo CID for visibility checks, and we
@@ -3014,6 +3015,7 @@ l1:
UnlockTupleTuplock(relation, &(tp.t_self), LockTupleExclusive);
if (vmbuffer != InvalidBuffer)
ReleaseBuffer(vmbuffer);
+ bms_free(rid_attrs);
return result;
}
@@ -3035,7 +3037,10 @@ l1:
* Compute replica identity tuple before entering the critical section so
* we don't PANIC upon a memory allocation failure.
*/
- old_key_tuple = ExtractReplicaIdentity(relation, &tp, true, &old_key_copied);
+ old_key_tuple = ExtractReplicaIdentity(relation, &tp, rid_attrs,
+ true, &old_key_copied);
+ bms_free(rid_attrs);
+ rid_attrs = NULL;
/*
* If this is the first possibly-multixact-able operation in the current
@@ -3247,7 +3252,10 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* heap_update - replace a tuple
*
* See table_tuple_update() for an explanation of the parameters, except that
- * this routine directly takes a tuple rather than a slot.
+ * this routine directly takes a heap tuple rather than a slot.
+ *
+ * It's required that the caller has acquired the pin and lock on the buffer.
+ * That lock and pin will be managed here, not in the caller.
*
* In the failure cases, the routine fills *tmfd with the tuple's t_ctid,
* t_xmax (resolving a possible MultiXact, if necessary), and t_cmax (the last
@@ -3255,30 +3263,21 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* generated by another transaction).
*/
TM_Result
-heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+heap_update(Relation relation, HeapTupleData *oldtup,
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
+ Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
+ Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
+ Bitmapset *mix_attrs, Buffer *vmbuffer,
+ bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
- Bitmapset *hot_attrs;
- Bitmapset *sum_attrs;
- Bitmapset *key_attrs;
- Bitmapset *id_attrs;
- Bitmapset *interesting_attrs;
- Bitmapset *modified_attrs;
- ItemId lp;
- HeapTupleData oldtup;
HeapTuple heaptup;
HeapTuple old_key_tuple = NULL;
bool old_key_copied = false;
- Page page;
- BlockNumber block;
MultiXactStatus mxact_status;
- Buffer buffer,
- newbuf,
- vmbuffer = InvalidBuffer,
+ Buffer newbuf,
vmbuffer_new = InvalidBuffer;
bool need_toast;
Size newtupsize,
@@ -3292,7 +3291,6 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
bool all_visible_cleared_new = false;
bool checked_lockers;
bool locker_remains;
- bool id_has_external = false;
TransactionId xmax_new_tuple,
xmax_old_tuple;
uint16 infomask_old_tuple,
@@ -3300,144 +3298,13 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
infomask_new_tuple,
infomask2_new_tuple;
- Assert(ItemPointerIsValid(otid));
-
- /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
- Assert(HeapTupleHeaderGetNatts(newtup->t_data) <=
- RelationGetNumberOfAttributes(relation));
-
+ Assert(BufferIsLockedByMe(buffer));
+ Assert(ItemIdIsNormal(lp));
AssertHasSnapshotForToast(relation);
- /*
- * Forbid this during a parallel operation, lest it allocate a combo CID.
- * Other workers might need that combo CID for visibility checks, and we
- * have no provision for broadcasting it to them.
- */
- if (IsInParallelMode())
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
- errmsg("cannot update tuples during a parallel operation")));
-
-#ifdef USE_ASSERT_CHECKING
- check_lock_if_inplace_updateable_rel(relation, otid, newtup);
-#endif
-
- /*
- * Fetch the list of attributes to be checked for various operations.
- *
- * For HOT considerations, this is wasted effort if we fail to update or
- * have to put the new tuple on a different page. But we must compute the
- * list before obtaining buffer lock --- in the worst case, if we are
- * doing an update on one of the relevant system catalogs, we could
- * deadlock if we try to fetch the list later. In any case, the relcache
- * caches the data so this is usually pretty cheap.
- *
- * We also need columns used by the replica identity and columns that are
- * considered the "key" of rows in the table.
- *
- * Note that we get copies of each bitmap, so we need not worry about
- * relcache flush happening midway through.
- */
- hot_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_HOT_BLOCKING);
- sum_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_SUMMARIZED);
- key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
- id_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
- interesting_attrs = NULL;
- interesting_attrs = bms_add_members(interesting_attrs, hot_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, sum_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, key_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, id_attrs);
-
- block = ItemPointerGetBlockNumber(otid);
- INJECTION_POINT("heap_update-before-pin", NULL);
- buffer = ReadBuffer(relation, block);
- page = BufferGetPage(buffer);
-
- /*
- * Before locking the buffer, pin the visibility map page if it appears to
- * be necessary. Since we haven't got the lock yet, someone else might be
- * in the middle of changing this, so we'll need to recheck after we have
- * the lock.
- */
- if (PageIsAllVisible(page))
- visibilitymap_pin(relation, block, &vmbuffer);
-
- LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
-
- lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
-
- /*
- * Usually, a buffer pin and/or snapshot blocks pruning of otid, ensuring
- * we see LP_NORMAL here. When the otid origin is a syscache, we may have
- * neither a pin nor a snapshot. Hence, we may see other LP_ states, each
- * of which indicates concurrent pruning.
- *
- * Failing with TM_Updated would be most accurate. However, unlike other
- * TM_Updated scenarios, we don't know the successor ctid in LP_UNUSED and
- * LP_DEAD cases. While the distinction between TM_Updated and TM_Deleted
- * does matter to SQL statements UPDATE and MERGE, those SQL statements
- * hold a snapshot that ensures LP_NORMAL. Hence, the choice between
- * TM_Updated and TM_Deleted affects only the wording of error messages.
- * Settle on TM_Deleted, for two reasons. First, it avoids complicating
- * the specification of when tmfd->ctid is valid. Second, it creates
- * error log evidence that we took this branch.
- *
- * Since it's possible to see LP_UNUSED at otid, it's also possible to see
- * LP_NORMAL for a tuple that replaced LP_UNUSED. If it's a tuple for an
- * unrelated row, we'll fail with "duplicate key value violates unique".
- * XXX if otid is the live, newer version of the newtup row, we'll discard
- * changes originating in versions of this catalog row after the version
- * the caller got from syscache. See syscache-update-pruned.spec.
- */
- if (!ItemIdIsNormal(lp))
- {
- Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
-
- UnlockReleaseBuffer(buffer);
- Assert(!have_tuple_lock);
- if (vmbuffer != InvalidBuffer)
- ReleaseBuffer(vmbuffer);
- tmfd->ctid = *otid;
- tmfd->xmax = InvalidTransactionId;
- tmfd->cmax = InvalidCommandId;
- *update_indexes = TU_None;
-
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- /* modified_attrs not yet initialized */
- bms_free(interesting_attrs);
- return TM_Deleted;
- }
-
- /*
- * Fill in enough data in oldtup for HeapDetermineColumnsInfo to work
- * properly.
- */
- oldtup.t_tableOid = RelationGetRelid(relation);
- oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
- oldtup.t_len = ItemIdGetLength(lp);
- oldtup.t_self = *otid;
-
- /* the new tuple is ready, except for this: */
+ /* The new tuple is ready, except for this */
newtup->t_tableOid = RelationGetRelid(relation);
- /*
- * Determine columns modified by the update. Additionally, identify
- * whether any of the unmodified replica identity key attributes in the
- * old tuple is externally stored or not. This is required because for
- * such attributes the flattened value won't be WAL logged as part of the
- * new tuple so we must include it as part of the old_key_tuple. See
- * ExtractReplicaIdentity.
- */
- modified_attrs = HeapDetermineColumnsInfo(relation, interesting_attrs,
- id_attrs, &oldtup,
- newtup, &id_has_external);
-
/*
* If we're not updating any "key" column, we can grab a weaker lock type.
* This allows for more concurrency when we are running simultaneously
@@ -3449,7 +3316,7 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
* is updates that don't manipulate key columns, not those that
* serendipitously arrive at the same key values.
*/
- if (!bms_overlap(modified_attrs, key_attrs))
+ if (!bms_overlap(mix_attrs, pk_attrs))
{
*lockmode = LockTupleNoKeyExclusive;
mxact_status = MultiXactStatusNoKeyUpdate;
@@ -3473,17 +3340,10 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
key_intact = false;
}
- /*
- * Note: beyond this point, use oldtup not otid to refer to old tuple.
- * otid may very well point at newtup->t_self, which we will overwrite
- * with the new tuple's location, so there's great risk of confusion if we
- * use otid anymore.
- */
-
l2:
checked_lockers = false;
locker_remains = false;
- result = HeapTupleSatisfiesUpdate(&oldtup, cid, buffer);
+ result = HeapTupleSatisfiesUpdate(oldtup, cid, buffer);
/* see below about the "no wait" case */
Assert(result != TM_BeingModified || wait);
@@ -3515,8 +3375,8 @@ l2:
*/
/* must copy state data before unlocking buffer */
- xwait = HeapTupleHeaderGetRawXmax(oldtup.t_data);
- infomask = oldtup.t_data->t_infomask;
+ xwait = HeapTupleHeaderGetRawXmax(oldtup->t_data);
+ infomask = oldtup->t_data->t_infomask;
/*
* Now we have to do something about the existing locker. If it's a
@@ -3556,13 +3416,12 @@ l2:
* requesting a lock and already have one; avoids deadlock).
*/
if (!current_is_member)
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &oldtup->t_self, *lockmode,
LockWaitBlock, &have_tuple_lock);
/* wait for multixact */
MultiXactIdWait((MultiXactId) xwait, mxact_status, infomask,
- relation, &oldtup.t_self, XLTW_Update,
- &remain);
+ relation, &oldtup->t_self, XLTW_Update, &remain);
checked_lockers = true;
locker_remains = remain != 0;
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
@@ -3572,9 +3431,9 @@ l2:
* could update this tuple before we get to this point. Check
* for xmax change, and start over if so.
*/
- if (xmax_infomask_changed(oldtup.t_data->t_infomask,
+ if (xmax_infomask_changed(oldtup->t_data->t_infomask,
infomask) ||
- !TransactionIdEquals(HeapTupleHeaderGetRawXmax(oldtup.t_data),
+ !TransactionIdEquals(HeapTupleHeaderGetRawXmax(oldtup->t_data),
xwait))
goto l2;
}
@@ -3599,8 +3458,8 @@ l2:
* before this one, which are important to keep in case this
* subxact aborts.
*/
- if (!HEAP_XMAX_IS_LOCKED_ONLY(oldtup.t_data->t_infomask))
- update_xact = HeapTupleGetUpdateXid(oldtup.t_data);
+ if (!HEAP_XMAX_IS_LOCKED_ONLY(oldtup->t_data->t_infomask))
+ update_xact = HeapTupleGetUpdateXid(oldtup->t_data);
else
update_xact = InvalidTransactionId;
@@ -3641,9 +3500,9 @@ l2:
* lock.
*/
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &oldtup->t_self, *lockmode,
LockWaitBlock, &have_tuple_lock);
- XactLockTableWait(xwait, relation, &oldtup.t_self,
+ XactLockTableWait(xwait, relation, &oldtup->t_self,
XLTW_Update);
checked_lockers = true;
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
@@ -3653,20 +3512,20 @@ l2:
* other xact could update this tuple before we get to this point.
* Check for xmax change, and start over if so.
*/
- if (xmax_infomask_changed(oldtup.t_data->t_infomask, infomask) ||
+ if (xmax_infomask_changed(oldtup->t_data->t_infomask, infomask) ||
!TransactionIdEquals(xwait,
- HeapTupleHeaderGetRawXmax(oldtup.t_data)))
+ HeapTupleHeaderGetRawXmax(oldtup->t_data)))
goto l2;
/* Otherwise check if it committed or aborted */
- UpdateXmaxHintBits(oldtup.t_data, buffer, xwait);
- if (oldtup.t_data->t_infomask & HEAP_XMAX_INVALID)
+ UpdateXmaxHintBits(oldtup->t_data, buffer, xwait);
+ if (oldtup->t_data->t_infomask & HEAP_XMAX_INVALID)
can_continue = true;
}
if (can_continue)
result = TM_Ok;
- else if (!ItemPointerEquals(&oldtup.t_self, &oldtup.t_data->t_ctid))
+ else if (!ItemPointerEquals(&oldtup->t_self, &oldtup->t_data->t_ctid))
result = TM_Updated;
else
result = TM_Deleted;
@@ -3679,39 +3538,33 @@ l2:
result == TM_Updated ||
result == TM_Deleted ||
result == TM_BeingModified);
- Assert(!(oldtup.t_data->t_infomask & HEAP_XMAX_INVALID));
+ Assert(!(oldtup->t_data->t_infomask & HEAP_XMAX_INVALID));
Assert(result != TM_Updated ||
- !ItemPointerEquals(&oldtup.t_self, &oldtup.t_data->t_ctid));
+ !ItemPointerEquals(&oldtup->t_self, &oldtup->t_data->t_ctid));
}
if (crosscheck != InvalidSnapshot && result == TM_Ok)
{
/* Perform additional check for transaction-snapshot mode RI updates */
- if (!HeapTupleSatisfiesVisibility(&oldtup, crosscheck, buffer))
+ if (!HeapTupleSatisfiesVisibility(oldtup, crosscheck, buffer))
result = TM_Updated;
}
if (result != TM_Ok)
{
- tmfd->ctid = oldtup.t_data->t_ctid;
- tmfd->xmax = HeapTupleHeaderGetUpdateXid(oldtup.t_data);
+ tmfd->ctid = oldtup->t_data->t_ctid;
+ tmfd->xmax = HeapTupleHeaderGetUpdateXid(oldtup->t_data);
if (result == TM_SelfModified)
- tmfd->cmax = HeapTupleHeaderGetCmax(oldtup.t_data);
+ tmfd->cmax = HeapTupleHeaderGetCmax(oldtup->t_data);
else
tmfd->cmax = InvalidCommandId;
UnlockReleaseBuffer(buffer);
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
- if (vmbuffer != InvalidBuffer)
- ReleaseBuffer(vmbuffer);
+ UnlockTupleTuplock(relation, &oldtup->t_self, *lockmode);
+ if (*vmbuffer != InvalidBuffer)
+ ReleaseBuffer(*vmbuffer);
*update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
return result;
}
@@ -3724,10 +3577,10 @@ l2:
* tuple has been locked or updated under us, but hopefully it won't
* happen very often.
*/
- if (vmbuffer == InvalidBuffer && PageIsAllVisible(page))
+ if (*vmbuffer == InvalidBuffer && PageIsAllVisible(page))
{
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- visibilitymap_pin(relation, block, &vmbuffer);
+ visibilitymap_pin(relation, block, vmbuffer);
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
goto l2;
}
@@ -3738,9 +3591,9 @@ l2:
* If the tuple we're updating is locked, we need to preserve the locking
* info in the old tuple's Xmax. Prepare a new Xmax value for this.
*/
- compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
- oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2,
+ compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup->t_data),
+ oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2,
xid, *lockmode, true,
&xmax_old_tuple, &infomask_old_tuple,
&infomask2_old_tuple);
@@ -3752,12 +3605,12 @@ l2:
* tuple. (In rare cases that might also be InvalidTransactionId and yet
* not have the HEAP_XMAX_INVALID bit set; that's fine.)
*/
- if ((oldtup.t_data->t_infomask & HEAP_XMAX_INVALID) ||
- HEAP_LOCKED_UPGRADED(oldtup.t_data->t_infomask) ||
+ if ((oldtup->t_data->t_infomask & HEAP_XMAX_INVALID) ||
+ HEAP_LOCKED_UPGRADED(oldtup->t_data->t_infomask) ||
(checked_lockers && !locker_remains))
xmax_new_tuple = InvalidTransactionId;
else
- xmax_new_tuple = HeapTupleHeaderGetRawXmax(oldtup.t_data);
+ xmax_new_tuple = HeapTupleHeaderGetRawXmax(oldtup->t_data);
if (!TransactionIdIsValid(xmax_new_tuple))
{
@@ -3772,7 +3625,7 @@ l2:
* Note that since we're doing an update, the only possibility is that
* the lockers had FOR KEY SHARE lock.
*/
- if (oldtup.t_data->t_infomask & HEAP_XMAX_IS_MULTI)
+ if (oldtup->t_data->t_infomask & HEAP_XMAX_IS_MULTI)
{
GetMultiXactIdHintBits(xmax_new_tuple, &infomask_new_tuple,
&infomask2_new_tuple);
@@ -3800,7 +3653,7 @@ l2:
* Replace cid with a combo CID if necessary. Note that we already put
* the plain cid into the new tuple.
*/
- HeapTupleHeaderAdjustCmax(oldtup.t_data, &cid, &iscombo);
+ HeapTupleHeaderAdjustCmax(oldtup->t_data, &cid, &iscombo);
/*
* If the toaster needs to be activated, OR if the new tuple will not fit
@@ -3817,12 +3670,12 @@ l2:
relation->rd_rel->relkind != RELKIND_MATVIEW)
{
/* toast table entries should never be recursively toasted */
- Assert(!HeapTupleHasExternal(&oldtup));
+ Assert(!HeapTupleHasExternal(oldtup));
Assert(!HeapTupleHasExternal(newtup));
need_toast = false;
}
else
- need_toast = (HeapTupleHasExternal(&oldtup) ||
+ need_toast = (HeapTupleHasExternal(oldtup) ||
HeapTupleHasExternal(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
@@ -3855,9 +3708,9 @@ l2:
* updating, because the potentially created multixact would otherwise
* be wrong.
*/
- compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
- oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2,
+ compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup->t_data),
+ oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2,
xid, *lockmode, false,
&xmax_lock_old_tuple, &infomask_lock_old_tuple,
&infomask2_lock_old_tuple);
@@ -3867,18 +3720,18 @@ l2:
START_CRIT_SECTION();
/* Clear obsolete visibility flags ... */
- oldtup.t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
- oldtup.t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
- HeapTupleClearHotUpdated(&oldtup);
+ oldtup->t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
+ oldtup->t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
+ HeapTupleClearHotUpdated(oldtup);
/* ... and store info about transaction updating this tuple */
Assert(TransactionIdIsValid(xmax_lock_old_tuple));
- HeapTupleHeaderSetXmax(oldtup.t_data, xmax_lock_old_tuple);
- oldtup.t_data->t_infomask |= infomask_lock_old_tuple;
- oldtup.t_data->t_infomask2 |= infomask2_lock_old_tuple;
- HeapTupleHeaderSetCmax(oldtup.t_data, cid, iscombo);
+ HeapTupleHeaderSetXmax(oldtup->t_data, xmax_lock_old_tuple);
+ oldtup->t_data->t_infomask |= infomask_lock_old_tuple;
+ oldtup->t_data->t_infomask2 |= infomask2_lock_old_tuple;
+ HeapTupleHeaderSetCmax(oldtup->t_data, cid, iscombo);
/* temporarily make it look not-updated, but locked */
- oldtup.t_data->t_ctid = oldtup.t_self;
+ oldtup->t_data->t_ctid = oldtup->t_self;
/*
* Clear all-frozen bit on visibility map if needed. We could
@@ -3887,7 +3740,7 @@ l2:
* worthwhile.
*/
if (PageIsAllVisible(page) &&
- visibilitymap_clear(relation, block, vmbuffer,
+ visibilitymap_clear(relation, block, *vmbuffer,
VISIBILITYMAP_ALL_FROZEN))
cleared_all_frozen = true;
@@ -3901,10 +3754,10 @@ l2:
XLogBeginInsert();
XLogRegisterBuffer(0, buffer, REGBUF_STANDARD);
- xlrec.offnum = ItemPointerGetOffsetNumber(&oldtup.t_self);
+ xlrec.offnum = ItemPointerGetOffsetNumber(&oldtup->t_self);
xlrec.xmax = xmax_lock_old_tuple;
- xlrec.infobits_set = compute_infobits(oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2);
+ xlrec.infobits_set = compute_infobits(oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2);
xlrec.flags =
cleared_all_frozen ? XLH_LOCK_ALL_FROZEN_CLEARED : 0;
XLogRegisterData(&xlrec, SizeOfHeapLock);
@@ -3926,7 +3779,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = heap_toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = heap_toast_insert_or_update(relation, newtup, oldtup, 0);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
@@ -3962,20 +3815,20 @@ l2:
/* It doesn't fit, must use RelationGetBufferForTuple. */
newbuf = RelationGetBufferForTuple(relation, heaptup->t_len,
buffer, 0, NULL,
- &vmbuffer_new, &vmbuffer,
+ &vmbuffer_new, vmbuffer,
0);
/* We're all done. */
break;
}
/* Acquire VM page pin if needed and we don't have it. */
- if (vmbuffer == InvalidBuffer && PageIsAllVisible(page))
- visibilitymap_pin(relation, block, &vmbuffer);
+ if (*vmbuffer == InvalidBuffer && PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, vmbuffer);
/* Re-acquire the lock on the old tuple's page. */
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
/* Re-check using the up-to-date free space */
pagefree = PageGetHeapFreeSpace(page);
if (newtupsize > pagefree ||
- (vmbuffer == InvalidBuffer && PageIsAllVisible(page)))
+ (*vmbuffer == InvalidBuffer && PageIsAllVisible(page)))
{
/*
* Rats, it doesn't fit anymore, or somebody just now set the
@@ -4013,7 +3866,7 @@ l2:
* will include checking the relation level, there is no benefit to a
* separate check for the new tuple.
*/
- CheckForSerializableConflictIn(relation, &oldtup.t_self,
+ CheckForSerializableConflictIn(relation, &oldtup->t_self,
BufferGetBlockNumber(buffer));
/*
@@ -4021,7 +3874,6 @@ l2:
* has enough space for the new tuple. If they are the same buffer, only
* one pin is held.
*/
-
if (newbuf == buffer)
{
/*
@@ -4029,7 +3881,7 @@ l2:
* to do a HOT update. Check if any of the index columns have been
* changed.
*/
- if (!bms_overlap(modified_attrs, hot_attrs))
+ if (!bms_overlap(mix_attrs, hot_attrs))
{
use_hot_update = true;
@@ -4040,7 +3892,7 @@ l2:
* indexes if the columns were updated, or we may fail to detect
* e.g. value bound changes in BRIN minmax indexes.
*/
- if (bms_overlap(modified_attrs, sum_attrs))
+ if (bms_overlap(mix_attrs, sum_attrs))
summarized_update = true;
}
}
@@ -4057,10 +3909,8 @@ l2:
* logged. Pass old key required as true only if the replica identity key
* columns are modified or it has external data.
*/
- old_key_tuple = ExtractReplicaIdentity(relation, &oldtup,
- bms_overlap(modified_attrs, id_attrs) ||
- id_has_external,
- &old_key_copied);
+ old_key_tuple = ExtractReplicaIdentity(relation, oldtup, rid_attrs,
+ rep_id_key_required, &old_key_copied);
/* NO EREPORT(ERROR) from here till changes are logged */
START_CRIT_SECTION();
@@ -4082,7 +3932,7 @@ l2:
if (use_hot_update)
{
/* Mark the old tuple as HOT-updated */
- HeapTupleSetHotUpdated(&oldtup);
+ HeapTupleSetHotUpdated(oldtup);
/* And mark the new tuple as heap-only */
HeapTupleSetHeapOnly(heaptup);
/* Mark the caller's copy too, in case different from heaptup */
@@ -4091,7 +3941,7 @@ l2:
else
{
/* Make sure tuples are correctly marked as not-HOT */
- HeapTupleClearHotUpdated(&oldtup);
+ HeapTupleClearHotUpdated(oldtup);
HeapTupleClearHeapOnly(heaptup);
HeapTupleClearHeapOnly(newtup);
}
@@ -4100,17 +3950,17 @@ l2:
/* Clear obsolete visibility flags, possibly set by ourselves above... */
- oldtup.t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
- oldtup.t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
+ oldtup->t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
+ oldtup->t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
/* ... and store info about transaction updating this tuple */
Assert(TransactionIdIsValid(xmax_old_tuple));
- HeapTupleHeaderSetXmax(oldtup.t_data, xmax_old_tuple);
- oldtup.t_data->t_infomask |= infomask_old_tuple;
- oldtup.t_data->t_infomask2 |= infomask2_old_tuple;
- HeapTupleHeaderSetCmax(oldtup.t_data, cid, iscombo);
+ HeapTupleHeaderSetXmax(oldtup->t_data, xmax_old_tuple);
+ oldtup->t_data->t_infomask |= infomask_old_tuple;
+ oldtup->t_data->t_infomask2 |= infomask2_old_tuple;
+ HeapTupleHeaderSetCmax(oldtup->t_data, cid, iscombo);
/* record address of new tuple in t_ctid of old one */
- oldtup.t_data->t_ctid = heaptup->t_self;
+ oldtup->t_data->t_ctid = heaptup->t_self;
/* clear PD_ALL_VISIBLE flags, reset all visibilitymap bits */
if (PageIsAllVisible(BufferGetPage(buffer)))
@@ -4118,7 +3968,7 @@ l2:
all_visible_cleared = true;
PageClearAllVisible(BufferGetPage(buffer));
visibilitymap_clear(relation, BufferGetBlockNumber(buffer),
- vmbuffer, VISIBILITYMAP_VALID_BITS);
+ *vmbuffer, VISIBILITYMAP_VALID_BITS);
}
if (newbuf != buffer && PageIsAllVisible(BufferGetPage(newbuf)))
{
@@ -4143,12 +3993,12 @@ l2:
*/
if (RelationIsAccessibleInLogicalDecoding(relation))
{
- log_heap_new_cid(relation, &oldtup);
+ log_heap_new_cid(relation, oldtup);
log_heap_new_cid(relation, heaptup);
}
recptr = log_heap_update(relation, buffer,
- newbuf, &oldtup, heaptup,
+ newbuf, oldtup, heaptup,
old_key_tuple,
all_visible_cleared,
all_visible_cleared_new);
@@ -4173,7 +4023,7 @@ l2:
* both tuple versions in one call to inval.c so we can avoid redundant
* sinval messages.)
*/
- CacheInvalidateHeapTuple(relation, &oldtup, heaptup);
+ CacheInvalidateHeapTuple(relation, oldtup, heaptup);
/* Now we can release the buffer(s) */
if (newbuf != buffer)
@@ -4181,14 +4031,14 @@ l2:
ReleaseBuffer(buffer);
if (BufferIsValid(vmbuffer_new))
ReleaseBuffer(vmbuffer_new);
- if (BufferIsValid(vmbuffer))
- ReleaseBuffer(vmbuffer);
+ if (BufferIsValid(*vmbuffer))
+ ReleaseBuffer(*vmbuffer);
/*
* Release the lmgr tuple lock, if we had it.
*/
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &oldtup->t_self, *lockmode);
pgstat_count_heap_update(relation, use_hot_update, newbuf != buffer);
@@ -4221,13 +4071,6 @@ l2:
if (old_key_tuple != NULL && old_key_copied)
heap_freetuple(old_key_tuple);
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
-
return TM_Ok;
}
@@ -4236,7 +4079,7 @@ l2:
* Confirm adequate lock held during heap_update(), per rules from
* README.tuplock section "Locking to write inplace-updated tables".
*/
-static void
+void
check_lock_if_inplace_updateable_rel(Relation relation,
const ItemPointerData *otid,
HeapTuple newtup)
@@ -4408,7 +4251,7 @@ heap_attr_equals(TupleDesc tupdesc, int attrnum, Datum value1, Datum value2,
* listed as interesting) of the old tuple is a member of external_cols and is
* stored externally.
*/
-static Bitmapset *
+Bitmapset *
HeapDetermineColumnsInfo(Relation relation,
Bitmapset *interesting_cols,
Bitmapset *external_cols,
@@ -4491,25 +4334,175 @@ HeapDetermineColumnsInfo(Relation relation,
}
/*
- * simple_heap_update - replace a tuple
- *
- * This routine may be used to update a tuple when concurrent updates of
- * the target tuple are not expected (for example, because we have a lock
- * on the relation associated with the tuple). Any failure is reported
- * via ereport().
+ * This routine may be used to update a tuple when concurrent updates of the
+ * target tuple are not expected (for example, because we have a lock on the
+ * relation associated with the tuple). Any failure is reported via ereport().
*/
void
-simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup,
+simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TM_FailureData tmfd;
LockTupleMode lockmode;
+ Buffer buffer;
+ Buffer vmbuffer = InvalidBuffer;
+ Page page;
+ BlockNumber block;
+ Bitmapset *hot_attrs,
+ *sum_attrs,
+ *pk_attrs,
+ *rid_attrs,
+ *mix_attrs,
+ *idx_attrs;
+ ItemId lp;
+ HeapTupleData oldtup;
+ bool rep_id_key_required = false;
+
+ Assert(ItemPointerIsValid(otid));
+
+ /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
+ Assert(HeapTupleHeaderGetNatts(tuple->t_data) <=
+ RelationGetNumberOfAttributes(relation));
+
+ /*
+ * Forbid this during a parallel operation, lest it allocate a combo CID.
+ * Other workers might need that combo CID for visibility checks, and we
+ * have no provision for broadcasting it to them.
+ */
+ if (IsInParallelMode())
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
+ errmsg("cannot update tuples during a parallel operation")));
+
+#ifdef USE_ASSERT_CHECKING
+ check_lock_if_inplace_updateable_rel(relation, otid, tuple);
+#endif
+
+ /*
+ * Fetch the list of attributes to be checked for various operations.
+ *
+ * For HOT considerations, this is wasted effort if we fail to update or
+ * have to put the new tuple on a different page. But we must compute the
+ * list before obtaining buffer lock --- in the worst case, if we are
+ * doing an update on one of the relevant system catalogs, we could
+ * deadlock if we try to fetch the list later. In any case, the relcache
+ * caches the data so this is usually pretty cheap.
+ *
+ * We also need columns used by the replica identity and columns that are
+ * considered the "key" of rows in the table.
+ *
+ * Note that we get copies of each bitmap, so we need not worry about
+ * relcache flush happening midway through.
+ */
+ hot_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_HOT_BLOCKING);
+ sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+ pk_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
+ rid_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
+ idx_attrs = bms_copy(hot_attrs);
+ idx_attrs = bms_add_members(idx_attrs, sum_attrs);
+ idx_attrs = bms_add_members(idx_attrs, pk_attrs);
+ idx_attrs = bms_add_members(idx_attrs, rid_attrs);
+
+ block = ItemPointerGetBlockNumber(otid);
+ INJECTION_POINT("heap_update-before-pin", NULL);
+ buffer = ReadBuffer(relation, block);
+ page = BufferGetPage(buffer);
+
+ /*
+ * Before locking the buffer, pin the visibility map page if it appears to
+ * be necessary. Since we haven't got the lock yet, someone else might be
+ * in the middle of changing this, so we'll need to recheck after we have
+ * the lock.
+ */
+ if (PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, &vmbuffer);
+
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+
+ lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
+
+ /*
+ * Usually, a buffer pin and/or snapshot blocks pruning of otid, ensuring
+ * we see LP_NORMAL here. When the otid origin is a syscache, we may have
+ * neither a pin nor a snapshot. Hence, we may see other LP_ states, each
+ * of which indicates concurrent pruning.
+ *
+ * Failing with TM_Updated would be most accurate. However, unlike other
+ * TM_Updated scenarios, we don't know the successor ctid in LP_UNUSED and
+ * LP_DEAD cases. While the distinction between TM_Updated and TM_Deleted
+ * does matter to SQL statements UPDATE and MERGE, those SQL statements
+ * hold a snapshot that ensures LP_NORMAL. Hence, the choice between
+ * TM_Updated and TM_Deleted affects only the wording of error messages.
+ * Settle on TM_Deleted, for two reasons. First, it avoids complicating
+ * the specification of when tmfd->ctid is valid. Second, it creates
+ * error log evidence that we took this branch.
+ *
+ * Since it's possible to see LP_UNUSED at otid, it's also possible to see
+ * LP_NORMAL for a tuple that replaced LP_UNUSED. If it's a tuple for an
+ * unrelated row, we'll fail with "duplicate key value violates unique".
+ * XXX if otid is the live, newer version of the newtup row, we'll discard
+ * changes originating in versions of this catalog row after the version
+ * the caller got from syscache. See syscache-update-pruned.spec.
+ */
+ if (!ItemIdIsNormal(lp))
+ {
+ Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
+
+ UnlockReleaseBuffer(buffer);
+ if (vmbuffer != InvalidBuffer)
+ ReleaseBuffer(vmbuffer);
+ *update_indexes = TU_None;
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(idx_attrs);
+ /* mix_attrs not yet initialized */
+
+ elog(ERROR, "tuple concurrently deleted");
+
+ return;
+ }
+
+ /*
+ * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
+ * then pass that on to heap_update.
+ */
+ oldtup.t_tableOid = RelationGetRelid(relation);
+ oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
+ oldtup.t_len = ItemIdGetLength(lp);
+ oldtup.t_self = *otid;
+
+ mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
+ &oldtup, tuple, &rep_id_key_required);
+
+ /*
+ * We'll need to WAL log the replica identity attributes if either they
+ * overlap with the modified indexed attributes or, as we've checked for
+ * just now in HeapDetermineColumnsInfo, they were unmodified external
+ * indexed attributes.
+ */
+ rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+
+ result = heap_update(relation, &oldtup, tuple, GetCurrentCommandId(true),
+ InvalidSnapshot, true /* wait for commit */ , &tmfd, &lockmode,
+ buffer, page, block, lp, hot_attrs, sum_attrs, pk_attrs,
+ rid_attrs, mix_attrs, &vmbuffer, rep_id_key_required,
+ update_indexes);
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(mix_attrs);
+ bms_free(idx_attrs);
- result = heap_update(relation, otid, tup,
- GetCurrentCommandId(true), InvalidSnapshot,
- true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
switch (result)
{
case TM_SelfModified:
@@ -9149,12 +9142,11 @@ log_heap_new_cid(Relation relation, HeapTuple tup)
* the same tuple that was passed in.
*/
static HeapTuple
-ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
- bool *copy)
+ExtractReplicaIdentity(Relation relation, HeapTuple tp, Bitmapset *rid_attrs,
+ bool key_required, bool *copy)
{
TupleDesc desc = RelationGetDescr(relation);
char replident = relation->rd_rel->relreplident;
- Bitmapset *idattrs;
HeapTuple key_tuple;
bool nulls[MaxHeapAttributeNumber];
Datum values[MaxHeapAttributeNumber];
@@ -9185,17 +9177,13 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
if (!key_required)
return NULL;
- /* find out the replica identity columns */
- idattrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
-
/*
* If there's no defined replica identity columns, treat as !key_required.
* (This case should not be reachable from heap_update, since that should
* calculate key_required accurately. But heap_delete just passes
* constant true for key_required, so we can hit this case in deletes.)
*/
- if (bms_is_empty(idattrs))
+ if (bms_is_empty(rid_attrs))
return NULL;
/*
@@ -9208,7 +9196,7 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
for (int i = 0; i < desc->natts; i++)
{
if (bms_is_member(i + 1 - FirstLowInvalidHeapAttributeNumber,
- idattrs))
+ rid_attrs))
Assert(!nulls[i]);
else
nulls[i] = true;
@@ -9217,8 +9205,6 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
key_tuple = heap_form_tuple(desc, values, nulls);
*copy = true;
- bms_free(idattrs);
-
/*
* If the tuple, which by here only contains indexed columns, still has
* toasted columns, force them to be inlined. This is somewhat unlikely
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bcbac844bb6..1cf9a18775d 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -44,6 +44,7 @@
#include "storage/procarray.h"
#include "storage/smgr.h"
#include "utils/builtins.h"
+#include "utils/injection_point.h"
#include "utils/rel.h"
static void reform_and_rewrite_tuple(HeapTuple tuple,
@@ -312,23 +313,133 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
return heap_delete(relation, tid, cid, crosscheck, wait, tmfd, changingPart);
}
-
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd,
LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
{
+ bool rep_id_key_required = false;
bool shouldFree = true;
HeapTuple tuple = ExecFetchSlotHeapTuple(slot, true, &shouldFree);
+ HeapTupleData oldtup;
+ Buffer buffer;
+ Buffer vmbuffer = InvalidBuffer;
+ Page page;
+ BlockNumber block;
+ ItemId lp;
+ Bitmapset *hot_attrs,
+ *sum_attrs,
+ *pk_attrs,
+ *rid_attrs,
+ *mix_attrs,
+ *idx_attrs;
TM_Result result;
+ Assert(ItemPointerIsValid(otid));
+
+ /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
+ Assert(HeapTupleHeaderGetNatts(tuple->t_data) <=
+ RelationGetNumberOfAttributes(relation));
+
+ /*
+ * Forbid this during a parallel operation, lest it allocate a combo CID.
+ * Other workers might need that combo CID for visibility checks, and we
+ * have no provision for broadcasting it to them.
+ */
+ if (IsInParallelMode())
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
+ errmsg("cannot update tuples during a parallel operation")));
+
+#ifdef USE_ASSERT_CHECKING
+ check_lock_if_inplace_updateable_rel(relation, otid, tuple);
+#endif
+
+ /*
+ * Fetch the list of attributes to be checked for various operations.
+ *
+ * For HOT considerations, this is wasted effort if we fail to update or
+ * have to put the new tuple on a different page. But we must compute the
+ * list before obtaining buffer lock --- in the worst case, if we are
+ * doing an update on one of the relevant system catalogs, we could
+ * deadlock if we try to fetch the list later. In any case, the relcache
+ * caches the data so this is usually pretty cheap.
+ *
+ * We also need columns used by the replica identity and columns that are
+ * considered the "key" of rows in the table.
+ *
+ * Note that we get copies of each bitmap, so we need not worry about
+ * relcache flush happening midway through.
+ */
+ hot_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_HOT_BLOCKING);
+ sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+ pk_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
+ rid_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
+ idx_attrs = bms_copy(hot_attrs);
+ idx_attrs = bms_add_members(idx_attrs, sum_attrs);
+ idx_attrs = bms_add_members(idx_attrs, pk_attrs);
+ idx_attrs = bms_add_members(idx_attrs, rid_attrs);
+
+ block = ItemPointerGetBlockNumber(otid);
+ INJECTION_POINT("heap_update-before-pin", NULL);
+ buffer = ReadBuffer(relation, block);
+ page = BufferGetPage(buffer);
+
+ /*
+ * Before locking the buffer, pin the visibility map page if it appears to
+ * be necessary. Since we haven't got the lock yet, someone else might be
+ * in the middle of changing this, so we'll need to recheck after we have
+ * the lock.
+ */
+ if (PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, &vmbuffer);
+
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+
+ lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
+
+ Assert(ItemIdIsNormal(lp));
+
+ /*
+ * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
+ * then pass that on to heap_update.
+ */
+ oldtup.t_tableOid = RelationGetRelid(relation);
+ oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
+ oldtup.t_len = ItemIdGetLength(lp);
+ oldtup.t_self = *otid;
+
+ mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
+ &oldtup, tuple, &rep_id_key_required);
+
+ /*
+ * We'll need to WAL log the replica identity attributes if either they
+ * overlap with the modified indexed attributes or, as we've checked for
+ * just now in HeapDetermineColumnsInfo, they were unmodified external
+ * indexed attributes.
+ */
+ rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
- result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
- tmfd, lockmode, update_indexes);
+ result = heap_update(relation, &oldtup, tuple, cid, crosscheck, wait, tmfd, lockmode,
+ buffer, page, block, lp, hot_attrs, sum_attrs, pk_attrs,
+ rid_attrs, mix_attrs, &vmbuffer, rep_id_key_required, update_indexes);
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(mix_attrs);
+ bms_free(idx_attrs);
+
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 909db73b7bb..41d541aa6b2 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -321,11 +321,13 @@ extern TM_Result heap_delete(Relation relation, const ItemPointerData *tid,
TM_FailureData *tmfd, bool changingPart);
extern void heap_finish_speculative(Relation relation, const ItemPointerData *tid);
extern void heap_abort_speculative(Relation relation, const ItemPointerData *tid);
-extern TM_Result heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+extern TM_Result heap_update(Relation relation, HeapTupleData *oldtup,
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
+ Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
+ Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
+ Bitmapset *mix_attrs, Buffer *vmbuffer,
+ bool rep_id_key_required, TU_UpdateIndexes *update_indexes);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -391,6 +393,18 @@ extern void log_heap_prune_and_freeze(Relation relation, Buffer buffer,
OffsetNumber *dead, int ndead,
OffsetNumber *unused, int nunused);
+/* in heap/heapam.c */
+extern Bitmapset *HeapDetermineColumnsInfo(Relation relation,
+ Bitmapset *interesting_cols,
+ Bitmapset *external_cols,
+ HeapTuple oldtup, HeapTuple newtup,
+ bool *has_external);
+#ifdef USE_ASSERT_CHECKING
+extern void check_lock_if_inplace_updateable_rel(Relation relation,
+ const ItemPointerData *otid,
+ HeapTuple newtup);
+#endif
+
/* in heap/vacuumlazy.c */
extern void heap_vacuum_rel(Relation rel,
const VacuumParams params, BufferAccessStrategy bstrategy);
--
2.49.0
[application/octet-stream] v22-0002-Track-changed-indexed-columns-in-the-executor-du.patch (34.3K, 3-v22-0002-Track-changed-indexed-columns-in-the-executor-du.patch)
download | inline diff:
From edc170a3f61de2141b383134ae40f105ee90aebe Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 26 Oct 2025 10:49:25 -0400
Subject: [PATCH v22 2/4] Track changed indexed columns in the executor during
UPDATEs
Refactor executor update logic to determine which indexed columns have
actually changed during an UPDATE operation rather than leaving this up
to HeapDetermineColumnsInfo in heap_update. This enables the comparison
to happen without taking a lock on the page and opens the door to reuse
in other code paths.
Because heap_update now requires the caller to provide the modified
indexed columns simple_heap_update has become a tad more complex. It is
frequently called from CatalogTupleUpdate which either updates heap
tuples via their form or using heap_modify_tuple. In both cases the
caller does know the modified set of attributes, but sadly those
attributes are lost before being provided to simple_heap_update. Due to
that the "simple" path has to retain the HeapDetermineColumnsInfo logic
of old (for now). In order for that to work it was necessary to split
the (overly large) heap_update call itself up. This moves up into
simple_heap_update and heap_tuple_update a bit of what existed in
heap_update itself. Ideally this will be cleaned up once
CatalogTupleUpdate paths are all recording modified attributes
correctly, when that happens the "simple" path can be simplified again.
ExecCheckIndexedAttrsForChanges replaces HeapDeterminesColumnsInfo and
tts_attr_equal replaces heap_attr_equal changing the test for equality
when calling into heap_tuple_update (but not simple_heap_update). In
the past we used datumIsEqual(), essentially a binary comparison using
memcmp(), now the comparison code in tts_attr_equal uses type-specific
equality function when available and falls back to datumIsEqual() when
not. This change in equality testing has some intended implications and
opens the door for more HOT updates (foreshadowing). For instance,
indexes with collation information allowing more HOT updates when the
index is specified to be case insensitive.
This change forced some logic changes in execReplication on the update
paths is now it is required to have knowledge of the set of attributes
that are both changed and referenced by indexes. Luckilly, the this is
available within calls to slot_modify_data() where LogicalRepTupleData
is processed and has a set of updated attributes. In this case rather
than using ExecCheckIndexedAttrsForChanges we can preseve what
slot_modify_data() identifies as the modified set and then intersect
that with the set of indexes on the relation and get the correct set of
modified indexed attributes required on heap_update().
---
src/backend/access/heap/heapam.c | 12 +-
src/backend/access/heap/heapam_handler.c | 72 +++++--
src/backend/access/table/tableam.c | 5 +-
src/backend/executor/execMain.c | 1 +
src/backend/executor/execReplication.c | 7 +
src/backend/executor/nodeModifyTable.c | 247 ++++++++++++++++++++++-
src/backend/nodes/bitmapset.c | 4 +
src/backend/replication/logical/worker.c | 72 ++++++-
src/backend/utils/cache/relcache.c | 15 ++
src/include/access/tableam.h | 8 +-
src/include/executor/executor.h | 5 +
src/include/nodes/execnodes.h | 1 +
src/include/utils/rel.h | 1 +
src/include/utils/relcache.h | 1 +
14 files changed, 415 insertions(+), 36 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index aff47481345..1cdb72b3a7a 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -3263,12 +3263,12 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* generated by another transaction).
*/
TM_Result
-heap_update(Relation relation, HeapTupleData *oldtup,
- HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
- Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
- Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
- Bitmapset *mix_attrs, Buffer *vmbuffer,
+heap_update(Relation relation, HeapTupleData *oldtup, HeapTuple newtup,
+ CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode,
+ Buffer buffer, Page page, BlockNumber block, ItemId lp,
+ Bitmapset *hot_attrs, Bitmapset *sum_attrs, Bitmapset *pk_attrs,
+ Bitmapset *rid_attrs, Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 1cf9a18775d..ef08e1d3e10 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -315,9 +315,12 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
- CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd,
- LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
+ CommandId cid, Snapshot snapshot,
+ Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd,
+ LockTupleMode *lockmode,
+ Bitmapset *mix_attrs,
+ TU_UpdateIndexes *update_indexes)
{
bool rep_id_key_required = false;
bool shouldFree = true;
@@ -332,7 +335,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
*sum_attrs,
*pk_attrs,
*rid_attrs,
- *mix_attrs,
*idx_attrs;
TM_Result result;
@@ -414,16 +416,61 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
oldtup.t_len = ItemIdGetLength(lp);
oldtup.t_self = *otid;
- mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
- &oldtup, tuple, &rep_id_key_required);
-
/*
- * We'll need to WAL log the replica identity attributes if either they
- * overlap with the modified indexed attributes or, as we've checked for
- * just now in HeapDetermineColumnsInfo, they were unmodified external
- * indexed attributes.
+ * We'll need to include the replica identity key when either the identity
+ * key attributes overlap with the modified index attributes or when the
+ * replica identity attributes are stored externally. This is required
+ * because for such attributes the flattened value won't be WAL logged as
+ * part of the new tuple so we must determine if we need to extract and
+ * include them as part of the old_key_tuple (see ExtractReplicaIdentity).
*/
- rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+ rep_id_key_required = bms_overlap(mix_attrs, rid_attrs);
+ if (!rep_id_key_required)
+ {
+ Bitmapset *attrs;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ int attidx = -1;
+
+ /*
+ * We don't own idx_attrs so we'll copy it and remove the modified set
+ * to reduce the attributes we need to test in the while loop and
+ * avoid a two branches in the loop.
+ */
+ attrs = bms_difference(idx_attrs, mix_attrs);
+ attrs = bms_int_members(attrs, rid_attrs);
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /*
+ * attidx is zero-based, attrnum is the normal attribute number
+ */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value;
+ bool isnull;
+
+ /*
+ * System attributes are not added into interesting_attrs in
+ * relcache
+ */
+ Assert(attrnum > 0);
+
+ value = heap_getattr(&oldtup, attrnum, tupdesc, &isnull);
+
+ /* No need to check attributes that can't be stored externally */
+ if (isnull ||
+ TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
+ continue;
+
+ /* Check if the old tuple's attribute is stored externally */
+ if (VARATT_IS_EXTERNAL((struct varlena *) DatumGetPointer(value)))
+ {
+ rep_id_key_required = true;
+ break;
+ }
+ }
+
+ bms_free(attrs);
+ }
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
@@ -437,7 +484,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
bms_free(sum_attrs);
bms_free(pk_attrs);
bms_free(rid_attrs);
- bms_free(mix_attrs);
bms_free(idx_attrs);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 5e41404937e..dadcf03ed24 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -336,6 +336,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
+ Bitmapset *modified_indexed_cols,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -346,7 +347,9 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
GetCurrentCommandId(true),
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, &lockmode,
+ modified_indexed_cols,
+ update_indexes);
switch (result)
{
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 27c9eec697b..6b7b6bc8019 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1282,6 +1282,7 @@ InitResultRelInfo(ResultRelInfo *resultRelInfo,
/* The following fields are set later if needed */
resultRelInfo->ri_RowIdAttNo = 0;
resultRelInfo->ri_extraUpdatedCols = NULL;
+ resultRelInfo->ri_ChangedIndexedCols = NULL;
resultRelInfo->ri_projectNew = NULL;
resultRelInfo->ri_newTupleSlot = NULL;
resultRelInfo->ri_oldTupleSlot = NULL;
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index def32774c90..2709e2db0f2 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -32,6 +32,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
#include "utils/typcache.h"
@@ -936,7 +937,13 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ /*
+ * We're not going to call ExecCheckIndexedAttrsForChanges here
+ * because we've already identified the changes earlier on thanks to
+ * slot_modify_data.
+ */
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
+ resultRelInfo->ri_ChangedIndexedCols,
&update_indexes);
conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 00429326c34..34f86546fc9 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -17,6 +17,7 @@
* ExecModifyTable - retrieve the next tuple from the node
* ExecEndModifyTable - shut down the ModifyTable node
* ExecReScanModifyTable - rescan the ModifyTable node
+ * ExecCheckIndexedAttrsForChanges - find set of updated indexed columns
*
* NOTES
* The ModifyTable node receives input from its outerPlan, which is
@@ -54,11 +55,14 @@
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/tupconvert.h"
+#include "access/tupdesc.h"
#include "access/xact.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
#include "executor/nodeModifyTable.h"
+#include "executor/tuptable.h"
#include "foreign/fdwapi.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
@@ -68,6 +72,8 @@
#include "storage/lmgr.h"
#include "utils/builtins.h"
#include "utils/datum.h"
+#include "utils/float.h"
+#include "utils/lsyscache.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
@@ -176,6 +182,219 @@ static TupleTableSlot *ExecMergeNotMatched(ModifyTableContext *context,
bool canSetTag);
+/*
+ * Compare two datums using the type's default equality operator.
+ *
+ * Returns true if the values are equal according to the type's equality
+ * operator, false otherwise. Falls back to binary comparison if no
+ * type-specific operator is available.
+ *
+ * This function uses the TypeCache infrastructure which caches operator
+ * lookups for efficiency.
+ */
+bool
+tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
+ Datum value1, Datum value2)
+{
+ TypeCacheEntry *typentry;
+
+ LOCAL_FCINFO(fcinfo, 2);
+ Datum result;
+
+ /*
+ * Fast path for common types to avoid even the type cache lookup. These
+ * types have simple equality semantics.
+ */
+ switch (typid)
+ {
+ case INT2OID:
+ return DatumGetInt16(value1) == DatumGetInt16(value2);
+ case INT4OID:
+ return DatumGetInt32(value1) == DatumGetInt32(value2);
+ case INT8OID:
+ return DatumGetInt64(value1) == DatumGetInt64(value2);
+ case FLOAT4OID:
+ return !float4_cmp_internal(DatumGetFloat4(value1), DatumGetFloat4(value2));
+ case FLOAT8OID:
+ return !float8_cmp_internal(DatumGetFloat8(value1), DatumGetFloat8(value2));
+ case BOOLOID:
+ return DatumGetBool(value1) == DatumGetBool(value2);
+ case OIDOID:
+ case REGPROCOID:
+ case REGPROCEDUREOID:
+ case REGOPEROID:
+ case REGOPERATOROID:
+ case REGCLASSOID:
+ case REGTYPEOID:
+ case REGROLEOID:
+ case REGNAMESPACEOID:
+ case REGCONFIGOID:
+ case REGDICTIONARYOID:
+ return DatumGetObjectId(value1) == DatumGetObjectId(value2);
+ case CHAROID:
+ return DatumGetChar(value1) == DatumGetChar(value2);
+ default:
+ /* Continue to type cache lookup */
+ break;
+ }
+
+ /*
+ * Look up the type's equality operator using the type cache. Request both
+ * the operator OID and the function info for efficiency.
+ */
+ typentry = lookup_type_cache(typid,
+ TYPECACHE_EQ_OPR | TYPECACHE_EQ_OPR_FINFO);
+
+ /*
+ * If no equality operator is available, fall back to binary comparison.
+ * This handles types that don't have proper equality operators defined.
+ */
+ if (!OidIsValid(typentry->eq_opr))
+ return datumIsEqual(value1, value2, typbyval, typlen);
+
+ /*
+ * Use the cached function info if available, otherwise look it up. The
+ * type cache keeps this around so subsequent calls are fast.
+ */
+ if (typentry->eq_opr_finfo.fn_addr == NULL)
+ {
+ Oid eq_proc = get_opcode(typentry->eq_opr);
+
+ if (!OidIsValid(eq_proc))
+ /* Shouldn't happen, but fall back to binary comparison */
+ return datumIsEqual(value1, value2, typbyval, typlen);
+
+ fmgr_info_cxt(eq_proc, &typentry->eq_opr_finfo,
+ CacheMemoryContext);
+ }
+
+ /* Set up function call */
+ InitFunctionCallInfoData(*fcinfo, &typentry->eq_opr_finfo, 2,
+ collation, NULL, NULL);
+
+ fcinfo->args[0].value = value1;
+ fcinfo->args[0].isnull = false;
+ fcinfo->args[1].value = value2;
+ fcinfo->args[1].isnull = false;
+
+ /* Invoke the equality operator */
+ result = FunctionCallInvoke(fcinfo);
+
+ /*
+ * If the function returned NULL (shouldn't happen for equality ops),
+ * treat as not equal for safety.
+ */
+ if (fcinfo->isnull)
+ return false;
+
+ return DatumGetBool(result);
+}
+
+/*
+ * Determine which updated attributes actually changed values between old and
+ * new tuples and are referenced by indexes on the relation.
+ *
+ * Returns a Bitmapset of attribute offsets (0-based, adjusted by
+ * FirstLowInvalidHeapAttributeNumber) or NULL if no attributes changed.
+ */
+Bitmapset *
+ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
+ TupleTableSlot *tts_old,
+ TupleTableSlot *tts_new)
+{
+ Relation relation = relinfo->ri_RelationDesc;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ Bitmapset *indexed_attrs;
+ Bitmapset *modified = NULL;
+ int attidx;
+
+ /* If no indexes, we're done */
+ if (relinfo->ri_NumIndices == 0)
+ return NULL;
+
+ /*
+ * Get the set of index key attributes. This includes summarizing,
+ * expression indexes and attributes mentioned in the predicate of a
+ * partition but not those in INCLUDING.
+ */
+ indexed_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_INDEXED);
+ Assert(!bms_is_empty(indexed_attrs));
+
+ /*
+ * NOTE: It is important to scan all indexed attributes in the tuples
+ * because ExecGetAllUpdatedCols won't include columns that may have been
+ * modified via heap_modify_tuple_by_col which is the case in
+ * tsvector_update_trigger.
+ */
+ attidx = -1;
+ while ((attidx = bms_next_member(indexed_attrs, attidx)) >= 0)
+ {
+ /* attidx is zero-based, attrnum is the normal attribute number */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Form_pg_attribute attr;
+ bool oldnull,
+ newnull;
+ Datum oldval,
+ newval;
+
+ /*
+ * If it's a whole-tuple reference, record as modified. It's not
+ * really worth supporting this case, since it could only succeed
+ * after a no-op update, which is hardly a case worth optimizing for.
+ */
+ if (attrnum == 0)
+ {
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /*
+ * Likewise, include in the modified set any system attribute other
+ * than tableOID; we cannot expect these to be consistent in a HOT
+ * chain, or even to be set correctly yet in the new tuple.
+ */
+ if (attrnum < 0)
+ {
+ if (attrnum != TableOidAttributeNumber)
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /* Extract values from both slots */
+ oldval = slot_getattr(tts_old, attrnum, &oldnull);
+ newval = slot_getattr(tts_new, attrnum, &newnull);
+
+ /* If one value is NULL and the other is not, they are not equal */
+ if (oldnull != newnull)
+ {
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /* If both are NULL, consider them equal */
+ if (oldnull)
+ continue;
+
+ /* Get attribute metadata */
+ Assert(attrnum > 0 && attrnum <= tupdesc->natts);
+ attr = TupleDescAttr(tupdesc, attrnum - 1);
+
+ /* Compare using type-specific equality operator */
+ if (!tts_attr_equal(attr->atttypid,
+ attr->attcollation,
+ attr->attbyval,
+ attr->attlen,
+ oldval,
+ newval))
+ modified = bms_add_member(modified, attidx);
+ }
+
+ bms_free(indexed_attrs);
+
+ return modified;
+}
+
/*
* Verify that the tuples to be produced by INSERT match the
* target relation's rowtype
@@ -2168,8 +2387,8 @@ ExecUpdatePrepareSlot(ResultRelInfo *resultRelInfo,
*/
static TM_Result
ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
- ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *slot,
- bool canSetTag, UpdateContext *updateCxt)
+ ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *oldSlot,
+ TupleTableSlot *slot, bool canSetTag, UpdateContext *updateCxt)
{
EState *estate = context->estate;
Relation resultRelationDesc = resultRelInfo->ri_RelationDesc;
@@ -2291,6 +2510,16 @@ lreplace:
if (resultRelationDesc->rd_att->constr)
ExecConstraints(resultRelInfo, slot, estate);
+ /*
+ * Identify which, if any, indexed attributes were modified here so that
+ * we might reuse it in a few places.
+ */
+ bms_free(resultRelInfo->ri_ChangedIndexedCols);
+ resultRelInfo->ri_ChangedIndexedCols = NULL;
+
+ resultRelInfo->ri_ChangedIndexedCols =
+ ExecCheckIndexedAttrsForChanges(resultRelInfo, oldSlot, slot);
+
/*
* replace the heap tuple
*
@@ -2306,6 +2535,7 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
+ resultRelInfo->ri_ChangedIndexedCols,
&updateCxt->updateIndexes);
return result;
@@ -2524,8 +2754,9 @@ ExecUpdate(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
*/
redo_act:
lockedtid = *tupleid;
- result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, slot,
- canSetTag, &updateCxt);
+
+ result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, oldSlot,
+ slot, canSetTag, &updateCxt);
/*
* If ExecUpdateAct reports that a cross-partition update was done,
@@ -3222,8 +3453,8 @@ lmerge_matched:
Assert(oldtuple == NULL);
result = ExecUpdateAct(context, resultRelInfo, tupleid,
- NULL, newslot, canSetTag,
- &updateCxt);
+ NULL, resultRelInfo->ri_oldTupleSlot,
+ newslot, canSetTag, &updateCxt);
/*
* As in ExecUpdate(), if ExecUpdateAct() reports that a
@@ -3248,6 +3479,7 @@ lmerge_matched:
tupleid, NULL, newslot);
mtstate->mt_merge_updated += 1;
}
+
break;
case CMD_DELETE:
@@ -4354,7 +4586,7 @@ ExecModifyTable(PlanState *pstate)
* For UPDATE/DELETE/MERGE, fetch the row identity info for the tuple
* to be updated/deleted/merged. For a heap relation, that's a TID;
* otherwise we may have a wholerow junk attr that carries the old
- * tuple in toto. Keep this in step with the part of
+ * tuple in total. Keep this in step with the part of
* ExecInitModifyTable that sets up ri_RowIdAttNo.
*/
if (operation == CMD_UPDATE || operation == CMD_DELETE ||
@@ -4530,6 +4762,7 @@ ExecModifyTable(PlanState *pstate)
/* Now apply the update. */
slot = ExecUpdate(&context, resultRelInfo, tupleid, oldtuple,
oldSlot, slot, node->canSetTag);
+
if (tuplock)
UnlockTuple(resultRelInfo->ri_RelationDesc, tupleid,
InplaceUpdateTupleLock);
diff --git a/src/backend/nodes/bitmapset.c b/src/backend/nodes/bitmapset.c
index b4ecf0b0390..9014990267a 100644
--- a/src/backend/nodes/bitmapset.c
+++ b/src/backend/nodes/bitmapset.c
@@ -238,6 +238,10 @@ bms_make_singleton(int x)
void
bms_free(Bitmapset *a)
{
+#if USE_ASSERT_CHECKING
+ Assert(bms_is_valid_set(a));
+#endif
+
if (a)
pfree(a);
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 93970c6af29..b363eaa49cc 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -243,6 +243,8 @@
*/
#include "postgres.h"
+#include "access/sysattr.h"
+#include "nodes/bitmapset.h"
#include <sys/stat.h>
#include <unistd.h>
@@ -275,7 +277,6 @@
#include "replication/logicalrelation.h"
#include "replication/logicalworker.h"
#include "replication/origin.h"
-#include "replication/slot.h"
#include "replication/walreceiver.h"
#include "replication/worker_internal.h"
#include "rewrite/rewriteHandler.h"
@@ -291,6 +292,7 @@
#include "utils/memutils.h"
#include "utils/pg_lsn.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/rls.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
@@ -1110,15 +1112,18 @@ slot_store_data(TupleTableSlot *slot, LogicalRepRelMapEntry *rel,
* "slot" is filled with a copy of the tuple in "srcslot", replacing
* columns provided in "tupleData" and leaving others as-is.
*
+ * Returns a bitmap of the modified columns.
+ *
* Caution: unreplaced pass-by-ref columns in "slot" will point into the
* storage for "srcslot". This is OK for current usage, but someday we may
* need to materialize "slot" at the end to make it independent of "srcslot".
*/
-static void
+static Bitmapset *
slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
LogicalRepRelMapEntry *rel,
LogicalRepTupleData *tupleData)
{
+ Bitmapset *modified = NULL;
int natts = slot->tts_tupleDescriptor->natts;
int i;
@@ -1195,6 +1200,28 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
slot->tts_isnull[i] = true;
}
+ /*
+ * Determine if the replicated value changed the local value by
+ * comparing slots. This is a subset of
+ * ExecCheckIndexedAttrsForChanges.
+ */
+ if (srcslot->tts_isnull[i] != slot->tts_isnull[i])
+ {
+ /* One is NULL, the other is not so the value changed */
+ modified = bms_add_member(modified, i + 1 - FirstLowInvalidHeapAttributeNumber);
+ }
+ else if (!srcslot->tts_isnull[i])
+ {
+ /* Both are not NULL, compare their values */
+ if (!tts_attr_equal(att->atttypid,
+ att->attcollation,
+ att->attbyval,
+ att->attlen,
+ srcslot->tts_values[i],
+ slot->tts_values[i]))
+ modified = bms_add_member(modified, i + 1 - FirstLowInvalidHeapAttributeNumber);
+ }
+
/* Reset attnum for error callback */
apply_error_callback_arg.remote_attnum = -1;
}
@@ -1202,6 +1229,8 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
/* And finally, declare that "slot" contains a valid virtual tuple */
ExecStoreVirtualTuple(slot);
+
+ return modified;
}
/*
@@ -2918,6 +2947,7 @@ apply_handle_update_internal(ApplyExecutionData *edata,
ConflictTupleInfo conflicttuple = {0};
bool found;
MemoryContext oldctx;
+ Bitmapset *indexed = NULL;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
ExecOpenIndices(relinfo, false);
@@ -2934,6 +2964,8 @@ apply_handle_update_internal(ApplyExecutionData *edata,
*/
if (found)
{
+ Bitmapset *modified = NULL;
+
/*
* Report the conflict if the tuple was modified by a different
* origin.
@@ -2957,15 +2989,29 @@ apply_handle_update_internal(ApplyExecutionData *edata,
/* Process and store remote tuple in the slot */
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot, localslot, relmapentry, newtup);
+ modified = slot_modify_data(remoteslot, localslot, relmapentry, newtup);
MemoryContextSwitchTo(oldctx);
+ /*
+ * Normally we'd call ExecCheckIndexedAttrForChanges but here we have
+ * the record of changed columns in the replication state, so let's
+ * use that instead.
+ */
+ indexed = RelationGetIndexAttrBitmap(relinfo->ri_RelationDesc,
+ INDEX_ATTR_BITMAP_INDEXED);
+
+ bms_free(relinfo->ri_ChangedIndexedCols);
+ relinfo->ri_ChangedIndexedCols = bms_int_members(modified, indexed);
+ bms_free(indexed);
+
EvalPlanQualSetSlot(&epqstate, remoteslot);
InitConflictIndexes(relinfo);
- /* Do the actual update. */
+ /* First check privileges */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
+
+ /* Then do the actual update. */
ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
remoteslot);
}
@@ -3455,6 +3501,8 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
bool found;
EPQState epqstate;
ConflictTupleInfo conflicttuple = {0};
+ Bitmapset *modified = NULL;
+ Bitmapset *indexed;
/* Get the matching local tuple from the partition. */
found = FindReplTupleInLocalRel(edata, partrel,
@@ -3523,8 +3571,8 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
* remoteslot_part.
*/
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot_part, localslot, part_entry,
- newtup);
+ modified = slot_modify_data(remoteslot_part, localslot, part_entry,
+ newtup);
MemoryContextSwitchTo(oldctx);
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
@@ -3549,6 +3597,18 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
EvalPlanQualSetSlot(&epqstate, remoteslot_part);
TargetPrivilegesCheck(partrelinfo->ri_RelationDesc,
ACL_UPDATE);
+
+ /*
+ * Normally we'd call ExecCheckIndexedAttrForChanges but
+ * here we have the record of changed columns in the
+ * replication state, so let's use that instead.
+ */
+ indexed = RelationGetIndexAttrBitmap(partrelinfo->ri_RelationDesc,
+ INDEX_ATTR_BITMAP_INDEXED);
+ bms_free(partrelinfo->ri_ChangedIndexedCols);
+ partrelinfo->ri_ChangedIndexedCols = bms_int_members(modified, indexed);
+ bms_free(indexed);
+
ExecSimpleRelationUpdate(partrelinfo, estate, &epqstate,
localslot, remoteslot_part);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 915d0bc9084..32825596be1 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -2482,6 +2482,7 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
bms_free(relation->rd_idattr);
bms_free(relation->rd_hotblockingattr);
bms_free(relation->rd_summarizedattr);
+ bms_free(relation->rd_indexedattr);
if (relation->rd_pubdesc)
pfree(relation->rd_pubdesc);
if (relation->rd_options)
@@ -5283,6 +5284,7 @@ RelationGetIndexPredicate(Relation relation)
* index (empty if FULL)
* INDEX_ATTR_BITMAP_HOT_BLOCKING Columns that block updates from being HOT
* INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
+ * INDEX_ATTR_BITMAP_INDEXED Columns referenced by indexes
*
* Attribute numbers are offset by FirstLowInvalidHeapAttributeNumber so that
* we can include system attributes (e.g., OID) in the bitmap representation.
@@ -5307,6 +5309,7 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
Bitmapset *idindexattrs; /* columns in the replica identity */
Bitmapset *hotblockingattrs; /* columns with HOT blocking indexes */
Bitmapset *summarizedattrs; /* columns with summarizing indexes */
+ Bitmapset *indexedattrs; /* columns referenced by indexes */
List *indexoidlist;
List *newindexoidlist;
Oid relpkindex;
@@ -5329,6 +5332,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
return bms_copy(relation->rd_hotblockingattr);
case INDEX_ATTR_BITMAP_SUMMARIZED:
return bms_copy(relation->rd_summarizedattr);
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return bms_copy(relation->rd_indexedattr);
default:
elog(ERROR, "unknown attrKind %u", attrKind);
}
@@ -5373,6 +5378,7 @@ restart:
idindexattrs = NULL;
hotblockingattrs = NULL;
summarizedattrs = NULL;
+ indexedattrs = NULL;
foreach(l, indexoidlist)
{
Oid indexOid = lfirst_oid(l);
@@ -5505,10 +5511,14 @@ restart:
bms_free(idindexattrs);
bms_free(hotblockingattrs);
bms_free(summarizedattrs);
+ bms_free(indexedattrs);
goto restart;
}
+ /* Combine all index attributes */
+ indexedattrs = bms_union(hotblockingattrs, summarizedattrs);
+
/* Don't leak the old values of these bitmaps, if any */
relation->rd_attrsvalid = false;
bms_free(relation->rd_keyattr);
@@ -5521,6 +5531,8 @@ restart:
relation->rd_hotblockingattr = NULL;
bms_free(relation->rd_summarizedattr);
relation->rd_summarizedattr = NULL;
+ bms_free(relation->rd_indexedattr);
+ relation->rd_indexedattr = NULL;
/*
* Now save copies of the bitmaps in the relcache entry. We intentionally
@@ -5535,6 +5547,7 @@ restart:
relation->rd_idattr = bms_copy(idindexattrs);
relation->rd_hotblockingattr = bms_copy(hotblockingattrs);
relation->rd_summarizedattr = bms_copy(summarizedattrs);
+ relation->rd_indexedattr = bms_copy(indexedattrs);
relation->rd_attrsvalid = true;
MemoryContextSwitchTo(oldcxt);
@@ -5551,6 +5564,8 @@ restart:
return hotblockingattrs;
case INDEX_ATTR_BITMAP_SUMMARIZED:
return summarizedattrs;
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return indexedattrs;
default:
elog(ERROR, "unknown attrKind %u", attrKind);
return NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index e16bf025692..8a5931a3118 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -549,6 +549,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
+ Bitmapset *updated_cols,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1502,12 +1503,12 @@ static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ Bitmapset *updated_cols, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
- wait, tmfd,
- lockmode, update_indexes);
+ wait, tmfd, lockmode,
+ updated_cols, update_indexes);
}
/*
@@ -2010,6 +2011,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
+ Bitmapset *modified_indexe_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index fa2b657fb2f..993dc0e6ced 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -800,5 +800,10 @@ extern ResultRelInfo *ExecLookupResultRelByOid(ModifyTableState *node,
Oid resultoid,
bool missing_ok,
bool update_cache);
+extern Bitmapset *ExecCheckIndexedAttrsForChanges(ResultRelInfo *resultRelInfo,
+ TupleTableSlot *tts_old,
+ TupleTableSlot *tts_new);
+extern bool tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
+ Datum value1, Datum value2);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 18ae8f0d4bb..8b08e0045ba 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -498,6 +498,7 @@ typedef struct ResultRelInfo
Bitmapset *ri_extraUpdatedCols;
/* true if the above has been computed */
bool ri_extraUpdatedCols_valid;
+ Bitmapset *ri_ChangedIndexedCols;
/* Projection to generate new tuple in an INSERT/UPDATE */
ProjectionInfo *ri_projectNew;
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 80286076a11..b23a7306e69 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -164,6 +164,7 @@ typedef struct RelationData
Bitmapset *rd_idattr; /* included in replica identity index */
Bitmapset *rd_hotblockingattr; /* cols blocking HOT update */
Bitmapset *rd_summarizedattr; /* cols indexed by summarizing indexes */
+ Bitmapset *rd_indexedattr; /* all cols referenced by indexes */
PublicationDesc *rd_pubdesc; /* publication descriptor, or NULL */
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 3561c6bef0b..d3fbb8b093a 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -71,6 +71,7 @@ typedef enum IndexAttrBitmapKind
INDEX_ATTR_BITMAP_IDENTITY_KEY,
INDEX_ATTR_BITMAP_HOT_BLOCKING,
INDEX_ATTR_BITMAP_SUMMARIZED,
+ INDEX_ATTR_BITMAP_INDEXED,
} IndexAttrBitmapKind;
extern Bitmapset *RelationGetIndexAttrBitmap(Relation relation,
--
2.49.0
[application/octet-stream] v22-0003-Replace-index_unchanged_by_update-with-ri_Change.patch (8.3K, 4-v22-0003-Replace-index_unchanged_by_update-with-ri_Change.patch)
download | inline diff:
From cf41827dd2ed13e5fa02763bddec570125af621e Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Fri, 31 Oct 2025 14:55:25 -0400
Subject: [PATCH v22 3/4] Replace index_unchanged_by_update with
ri_ChangedIndexedCols
In execIndexing on updates we'd like to pass a hint to the indexing code
when the indexed attributes are unchanged. This commit replaces the now
redundant code in index_unchanged_by_update with the same information
found earlier in the update path.
---
src/backend/catalog/toasting.c | 2 -
src/backend/executor/execIndexing.c | 156 +---------------------------
src/backend/nodes/makefuncs.c | 2 -
src/include/nodes/execnodes.h | 4 -
4 files changed, 1 insertion(+), 163 deletions(-)
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 874a8fc89ad..5d819bda54a 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -300,8 +300,6 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
indexInfo->ii_Unique = true;
indexInfo->ii_NullsNotDistinct = false;
indexInfo->ii_ReadyForInserts = true;
- indexInfo->ii_CheckedUnchanged = false;
- indexInfo->ii_IndexUnchanged = false;
indexInfo->ii_Concurrent = false;
indexInfo->ii_BrokenHotChain = false;
indexInfo->ii_ParallelWorkers = 0;
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 401606f840a..fb1bc3a480d 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -138,11 +138,6 @@ static bool check_exclusion_or_unique_constraint(Relation heap, Relation index,
static bool index_recheck_constraint(Relation index, const Oid *constr_procs,
const Datum *existing_values, const bool *existing_isnull,
const Datum *new_values);
-static bool index_unchanged_by_update(ResultRelInfo *resultRelInfo,
- EState *estate, IndexInfo *indexInfo,
- Relation indexRelation);
-static bool index_expression_changed_walker(Node *node,
- Bitmapset *allUpdatedCols);
static void ExecWithoutOverlapsNotEmpty(Relation rel, NameData attname, Datum attval,
char typtype, Oid atttypid);
@@ -440,10 +435,7 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* index. If we're being called as part of an UPDATE statement,
* consider if the 'indexUnchanged' = true hint should be passed.
*/
- indexUnchanged = update && index_unchanged_by_update(resultRelInfo,
- estate,
- indexInfo,
- indexRelation);
+ indexUnchanged = update && bms_is_empty(resultRelInfo->ri_ChangedIndexedCols);
satisfiesConstraint =
index_insert(indexRelation, /* index relation */
@@ -993,152 +985,6 @@ index_recheck_constraint(Relation index, const Oid *constr_procs,
return true;
}
-/*
- * Check if ExecInsertIndexTuples() should pass indexUnchanged hint.
- *
- * When the executor performs an UPDATE that requires a new round of index
- * tuples, determine if we should pass 'indexUnchanged' = true hint for one
- * single index.
- */
-static bool
-index_unchanged_by_update(ResultRelInfo *resultRelInfo, EState *estate,
- IndexInfo *indexInfo, Relation indexRelation)
-{
- Bitmapset *updatedCols;
- Bitmapset *extraUpdatedCols;
- Bitmapset *allUpdatedCols;
- bool hasexpression = false;
- List *idxExprs;
-
- /*
- * Check cache first
- */
- if (indexInfo->ii_CheckedUnchanged)
- return indexInfo->ii_IndexUnchanged;
- indexInfo->ii_CheckedUnchanged = true;
-
- /*
- * Check for indexed attribute overlap with updated columns.
- *
- * Only do this for key columns. A change to a non-key column within an
- * INCLUDE index should not be counted here. Non-key column values are
- * opaque payload state to the index AM, a little like an extra table TID.
- *
- * Note that row-level BEFORE triggers won't affect our behavior, since
- * they don't affect the updatedCols bitmaps generally. It doesn't seem
- * worth the trouble of checking which attributes were changed directly.
- */
- updatedCols = ExecGetUpdatedCols(resultRelInfo, estate);
- extraUpdatedCols = ExecGetExtraUpdatedCols(resultRelInfo, estate);
- for (int attr = 0; attr < indexInfo->ii_NumIndexKeyAttrs; attr++)
- {
- int keycol = indexInfo->ii_IndexAttrNumbers[attr];
-
- if (keycol <= 0)
- {
- /*
- * Skip expressions for now, but remember to deal with them later
- * on
- */
- hasexpression = true;
- continue;
- }
-
- if (bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
- updatedCols) ||
- bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
- extraUpdatedCols))
- {
- /* Changed key column -- don't hint for this index */
- indexInfo->ii_IndexUnchanged = false;
- return false;
- }
- }
-
- /*
- * When we get this far and index has no expressions, return true so that
- * index_insert() call will go on to pass 'indexUnchanged' = true hint.
- *
- * The _absence_ of an indexed key attribute that overlaps with updated
- * attributes (in addition to the total absence of indexed expressions)
- * shows that the index as a whole is logically unchanged by UPDATE.
- */
- if (!hasexpression)
- {
- indexInfo->ii_IndexUnchanged = true;
- return true;
- }
-
- /*
- * Need to pass only one bms to expression_tree_walker helper function.
- * Avoid allocating memory in common case where there are no extra cols.
- */
- if (!extraUpdatedCols)
- allUpdatedCols = updatedCols;
- else
- allUpdatedCols = bms_union(updatedCols, extraUpdatedCols);
-
- /*
- * We have to work slightly harder in the event of indexed expressions,
- * but the principle is the same as before: try to find columns (Vars,
- * actually) that overlap with known-updated columns.
- *
- * If we find any matching Vars, don't pass hint for index. Otherwise
- * pass hint.
- */
- idxExprs = RelationGetIndexExpressions(indexRelation);
- hasexpression = index_expression_changed_walker((Node *) idxExprs,
- allUpdatedCols);
- list_free(idxExprs);
- if (extraUpdatedCols)
- bms_free(allUpdatedCols);
-
- if (hasexpression)
- {
- indexInfo->ii_IndexUnchanged = false;
- return false;
- }
-
- /*
- * Deliberately don't consider index predicates. We should even give the
- * hint when result rel's "updated tuple" has no corresponding index
- * tuple, which is possible with a partial index (provided the usual
- * conditions are met).
- */
- indexInfo->ii_IndexUnchanged = true;
- return true;
-}
-
-/*
- * Indexed expression helper for index_unchanged_by_update().
- *
- * Returns true when Var that appears within allUpdatedCols located.
- */
-static bool
-index_expression_changed_walker(Node *node, Bitmapset *allUpdatedCols)
-{
- if (node == NULL)
- return false;
-
- if (IsA(node, Var))
- {
- Var *var = (Var *) node;
-
- if (bms_is_member(var->varattno - FirstLowInvalidHeapAttributeNumber,
- allUpdatedCols))
- {
- /* Var was updated -- indicates that we should not hint */
- return true;
- }
-
- /* Still haven't found a reason to not pass the hint */
- return false;
- }
-
- return expression_tree_walker(node, index_expression_changed_walker,
- allUpdatedCols);
-}
-
/*
* ExecWithoutOverlapsNotEmpty - raise an error if the tuple has an empty
* range or multirange in the given attribute.
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index e2d9e9be41a..d69dc090aa4 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -845,8 +845,6 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
n->ii_Unique = unique;
n->ii_NullsNotDistinct = nulls_not_distinct;
n->ii_ReadyForInserts = isready;
- n->ii_CheckedUnchanged = false;
- n->ii_IndexUnchanged = false;
n->ii_Concurrent = concurrent;
n->ii_Summarizing = summarizing;
n->ii_WithoutOverlaps = withoutoverlaps;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 8b08e0045ba..898368fb8cb 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -202,10 +202,6 @@ typedef struct IndexInfo
bool ii_NullsNotDistinct;
/* is it valid for inserts? */
bool ii_ReadyForInserts;
- /* IndexUnchanged status determined yet? */
- bool ii_CheckedUnchanged;
- /* aminsert hint, cached for retail inserts */
- bool ii_IndexUnchanged;
/* are we doing a concurrent index build? */
bool ii_Concurrent;
/* did we detect any broken HOT chains? */
--
2.49.0
[application/octet-stream] v22-0004-Enable-HOT-updates-for-expression-and-partial-in.patch (130.2K, 5-v22-0004-Enable-HOT-updates-for-expression-and-partial-in.patch)
download | inline diff:
From 9f584afc7f27ac8e93c6cf425248a24fba679a4e Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 26 Oct 2025 10:49:25 -0400
Subject: [PATCH v22 4/4] Enable HOT updates for expression and partial indexes
Currently, PostgreSQL conservatively prevents HOT (Heap-Only Tuple)
updates whenever any indexed column changes, even if the indexed
portion of that column remains identical. This is overly restrictive
for expression indexes (where f(column) might not change even when
column changes) and partial indexes (where both old and new tuples
might fall outside the predicate). Finally, index AMs play no role
in deciding when they need a new index entry on update, the rules
regarding that are based on binary equality and the HEAP's model for
MVCC and related HOT optimization. Here we open that door a bit so
as to enable more nuanced control over the process. This enables
index AMs that require binary equality (as is the case for nbtree)
to do that without disallowing type-specific equality checking for
other indexes.
This patch introduces several improvements to enable HOT updates in
these cases:
Add amcomparedatums() callback to IndexAmRoutine. This allows index
access methods like GIN to provide custom logic for comparing datums by
extracting and comparing index keys rather than comparing the raw
datums. GIN indexes now implement gincomparedatums() which extracts keys
from both datums and compares the resulting key sets. Also, as
mentioned earlier nbtree implements this API and uses datumIsEqual() for
equality so that the manner in which it deduplicates TIDs on page split
doesn't have to change. This is not a required API, when not
implemented the executor will compare TupleTableSlot datum for equality
using type-specific operators and take into account collation so that an
update from "Apple" to "APPLE" on a case insensitive index can now be
HOT.
ExecWhichIndexesRequireUpdates() is re-written to find the set of
modified indexed attributes that trigger new index tuples on updated.
For partial indexes, this checks whether both old and new tuples satisfy
or fail the predicate. For expression indexes, this uses type-specific
equality operators to compare computed values. For extraction-based
indexes (GIN/RUM) that implement amcomparedatums() it uses that.
Importantly, table access methods can still signal using TU_Update if
all, none, or only summarizing indexes should be updated. While the
executor layer now owns determining what has changed due to an update
and is interested in only updating the minimum number of indexes
possible, the table AM can override that while performing
table_tuple_update(), which is what heap does. While this signal is
very specific to how the heap implements MVCC and its HOT optimization,
we'll leave replacing that for another day.
This optimization trades off some new overhead for the potential for
more updates to use the HOT optimized path and avoid index and heap
bloat. This should significantly improve update performance for tables
with expression indexes, partial indexes, and GIN/GiST indexes on
complex data types like JSONB and tsvector, while maintaining correct
index semantics. Minimal additional overhead due to type-specific
equality checking should be washed out by the benefits of updating
indexes fewer times.
One notable trade-off is that there are more calls to FormIndexDatum()
as a result. Caching these might reduce some of that overhead, but not
all. This lead to the change in the frequency for expressions in the
spec update test to output notice messages, but does not impact
correctness.
---
src/backend/access/brin/brin.c | 1 +
src/backend/access/gin/ginutil.c | 94 +-
src/backend/access/heap/heapam.c | 10 +-
src/backend/access/heap/heapam_handler.c | 6 +-
src/backend/access/nbtree/nbtree.c | 38 +
src/backend/access/table/tableam.c | 4 +-
src/backend/bootstrap/bootstrap.c | 8 +
src/backend/catalog/index.c | 57 +
src/backend/catalog/indexing.c | 16 +-
src/backend/catalog/toasting.c | 4 +
src/backend/executor/execIndexing.c | 45 +-
src/backend/executor/nodeModifyTable.c | 437 +++++--
src/backend/nodes/makefuncs.c | 4 +
src/include/access/amapi.h | 28 +
src/include/access/gin.h | 3 +
src/include/access/heapam.h | 6 +-
src/include/access/nbtree.h | 4 +
src/include/access/tableam.h | 8 +-
src/include/catalog/index.h | 1 +
src/include/executor/executor.h | 12 +-
src/include/nodes/execnodes.h | 19 +
.../expected/insert-conflict-specconflict.out | 20 +
.../expected/hot_expression_indexes.out | 1006 +++++++++++++++++
src/test/regress/parallel_schedule | 6 +
.../regress/sql/hot_expression_indexes.sql | 747 ++++++++++++
src/tools/pgindent/typedefs.list | 1 +
26 files changed, 2460 insertions(+), 125 deletions(-)
create mode 100644 src/test/regress/expected/hot_expression_indexes.out
create mode 100644 src/test/regress/sql/hot_expression_indexes.sql
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index cb3331921cb..36e639552e6 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -290,6 +290,7 @@ brinhandler(PG_FUNCTION_ARGS)
amroutine->amproperty = NULL;
amroutine->ambuildphasename = NULL;
amroutine->amvalidate = brinvalidate;
+ amroutine->amcomparedatums = NULL;
amroutine->amadjustmembers = NULL;
amroutine->ambeginscan = brinbeginscan;
amroutine->amrescan = brinrescan;
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index 78f7b7a2495..85e25ed73e8 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -26,6 +26,7 @@
#include "storage/indexfsm.h"
#include "utils/builtins.h"
#include "utils/index_selfuncs.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/typcache.h"
@@ -78,6 +79,7 @@ ginhandler(PG_FUNCTION_ARGS)
amroutine->amproperty = NULL;
amroutine->ambuildphasename = ginbuildphasename;
amroutine->amvalidate = ginvalidate;
+ amroutine->amcomparedatums = gincomparedatums;
amroutine->amadjustmembers = ginadjustmembers;
amroutine->ambeginscan = ginbeginscan;
amroutine->amrescan = ginrescan;
@@ -477,13 +479,6 @@ cmpEntries(const void *a, const void *b, void *arg)
return res;
}
-
-/*
- * Extract the index key values from an indexable item
- *
- * The resulting key values are sorted, and any duplicates are removed.
- * This avoids generating redundant index entries.
- */
Datum *
ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
@@ -729,3 +724,88 @@ ginbuildphasename(int64 phasenum)
return NULL;
}
}
+
+/*
+ * gincomparedatums - Compare two datums to determine if they produce identical keys
+ *
+ * This function extracts keys from both old_datum and new_datum using the
+ * opclass's extractValue function, then compares the extracted key arrays.
+ * Returns true if the key sets are identical (same keys, same counts).
+ *
+ * This enables HOT updates for GIN indexes when the indexed portions of a
+ * value haven't changed, even if the value itself has changed.
+ *
+ * Example: JSONB column with GIN index. If an update changes a non-indexed
+ * key in the JSONB document, the extracted keys are identical and we can
+ * do a HOT update.
+ */
+bool
+gincomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull)
+{
+ GinState ginstate;
+ Datum *old_keys;
+ Datum *new_keys;
+ GinNullCategory *old_categories;
+ GinNullCategory *new_categories;
+ int32 old_nkeys;
+ int32 new_nkeys;
+ MemoryContext tmpcontext;
+ MemoryContext oldcontext;
+ bool result = true;
+
+ /* Handle NULL cases */
+ if (old_isnull != new_isnull)
+ return false;
+ if (old_isnull)
+ return true;
+
+ /* Create temporary context for extraction work */
+ tmpcontext = AllocSetContextCreate(CurrentMemoryContext,
+ "GIN datum comparison",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(tmpcontext);
+
+ initGinState(&ginstate, index);
+
+ /*
+ * Extract keys from both datums using existing GIN infrastructure.
+ */
+ old_keys = ginExtractEntries(&ginstate, attnum, old_datum, old_isnull,
+ &old_nkeys, &old_categories);
+ new_keys = ginExtractEntries(&ginstate, attnum, new_datum, new_isnull,
+ &new_nkeys, &new_categories);
+
+ /* Different number of keys → definitely different */
+ if (old_nkeys != new_nkeys)
+ {
+ result = false;
+ goto cleanup;
+ }
+
+ /*
+ * Compare the sorted key arrays element-by-element. Since both arrays are
+ * already sorted by ginExtractEntries, we can do a simple O(n)
+ * comparison.
+ */
+ for (int i = 0; i < old_nkeys; i++)
+ {
+ int cmp = ginCompareEntries(&ginstate, attnum,
+ old_keys[i], old_categories[i],
+ new_keys[i], new_categories[i]);
+
+ if (cmp != 0)
+ {
+ result = false;
+ break;
+ }
+ }
+
+cleanup:
+ /* Clean up */
+ MemoryContextSwitchTo(oldcontext);
+ MemoryContextDelete(tmpcontext);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 1cdb72b3a7a..5b0ff13b13d 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -3268,7 +3268,7 @@ heap_update(Relation relation, HeapTupleData *oldtup, HeapTuple newtup,
TM_FailureData *tmfd, LockTupleMode *lockmode,
Buffer buffer, Page page, BlockNumber block, ItemId lp,
Bitmapset *hot_attrs, Bitmapset *sum_attrs, Bitmapset *pk_attrs,
- Bitmapset *rid_attrs, Bitmapset *mix_attrs, Buffer *vmbuffer,
+ Bitmapset *rid_attrs, const Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -4337,8 +4337,9 @@ HeapDetermineColumnsInfo(Relation relation,
* This routine may be used to update a tuple when concurrent updates of the
* target tuple are not expected (for example, because we have a lock on the
* relation associated with the tuple). Any failure is reported via ereport().
+ * Returns the set of modified indexed attributes.
*/
-void
+Bitmapset *
simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
@@ -4467,7 +4468,7 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
elog(ERROR, "tuple concurrently deleted");
- return;
+ return NULL;
}
/*
@@ -4500,7 +4501,6 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
bms_free(sum_attrs);
bms_free(pk_attrs);
bms_free(rid_attrs);
- bms_free(mix_attrs);
bms_free(idx_attrs);
switch (result)
@@ -4526,6 +4526,8 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
elog(ERROR, "unrecognized heap_update status: %u", result);
break;
}
+
+ return mix_attrs;
}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index ef08e1d3e10..7527809ec08 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -319,7 +319,7 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
Snapshot crosscheck, bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
- Bitmapset *mix_attrs,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes)
{
bool rep_id_key_required = false;
@@ -407,10 +407,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
Assert(ItemIdIsNormal(lp));
- /*
- * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
- * then pass that on to heap_update.
- */
oldtup.t_tableOid = RelationGetRelid(relation);
oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
oldtup.t_len = ItemIdGetLength(lp);
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index fdff960c130..73cc3208757 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -155,6 +155,7 @@ bthandler(PG_FUNCTION_ARGS)
amroutine->amproperty = btproperty;
amroutine->ambuildphasename = btbuildphasename;
amroutine->amvalidate = btvalidate;
+ amroutine->amcomparedatums = btcomparedatums;
amroutine->amadjustmembers = btadjustmembers;
amroutine->ambeginscan = btbeginscan;
amroutine->amrescan = btrescan;
@@ -1795,3 +1796,40 @@ bttranslatecmptype(CompareType cmptype, Oid opfamily)
return InvalidStrategy;
}
}
+
+/*
+ * btcomparedatums - Compare two datums for equality
+ *
+ * This function is necessary because nbtree requires that keys that are not
+ * binary identical not be "equal". Other indexes might allow "A" and "a" to
+ * be "equal" when collation is case insensative, but not nbtree. Why? Well,
+ * nbtree deduplicates TIDs on page split and the way it accomplish that is by
+ * doing a binary comparison of the keys.
+ */
+
+bool
+btcomparedatums(Relation index, int attrnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull)
+{
+ TupleDesc desc = RelationGetDescr(index);
+ CompactAttribute *att;
+
+ /*
+ * If one value is NULL and other is not, then they are certainly not
+ * equal
+ */
+ if (old_isnull != new_isnull)
+ return false;
+
+ /*
+ * If both are NULL, they can be considered equal.
+ */
+ if (old_isnull)
+ return true;
+
+ /* We do simple binary comparison of the two datums */
+ Assert(attrnum <= desc->natts);
+ att = TupleDescCompactAttr(desc, attrnum - 1);
+ return datumIsEqual(old_datum, new_datum, att->attbyval, att->attlen);
+}
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index dadcf03ed24..ef7736bfa76 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -336,7 +336,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
- Bitmapset *modified_indexed_cols,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -348,7 +348,7 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
&tmfd, &lockmode,
- modified_indexed_cols,
+ mix_attrs,
update_indexes);
switch (result)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index fc8638c1b61..329c110d0bf 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -961,10 +961,18 @@ index_register(Oid heap,
newind->il_info->ii_Expressions =
copyObject(indexInfo->ii_Expressions);
newind->il_info->ii_ExpressionsState = NIL;
+ /* expression attrs will likely be null, but may as well copy it */
+ newind->il_info->ii_ExpressionsAttrs =
+ copyObject(indexInfo->ii_ExpressionsAttrs);
/* predicate will likely be null, but may as well copy it */
newind->il_info->ii_Predicate =
copyObject(indexInfo->ii_Predicate);
newind->il_info->ii_PredicateState = NULL;
+ /* predicate attrs will likely be null, but may as well copy it */
+ newind->il_info->ii_PredicateAttrs =
+ copyObject(indexInfo->ii_PredicateAttrs);
+ newind->il_info->ii_CheckedPredicate = false;
+ newind->il_info->ii_PredicateSatisfied = false;
/* no exclusion constraints at bootstrap time, so no need to copy */
Assert(indexInfo->ii_ExclusionOps == NULL);
Assert(indexInfo->ii_ExclusionProcs == NULL);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 5d9db167e59..29b8cc4badd 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -27,6 +27,7 @@
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/relscan.h"
+#include "access/sysattr.h"
#include "access/tableam.h"
#include "access/toast_compression.h"
#include "access/transam.h"
@@ -58,6 +59,7 @@
#include "commands/trigger.h"
#include "executor/executor.h"
#include "miscadmin.h"
+#include "nodes/execnodes.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/optimizer.h"
@@ -2414,6 +2416,61 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* ----------------------------------------------------------------
*/
+/* ----------------
+ * BuildUpdateIndexInfo
+ *
+ * For expression indexes updates may not change the indexed value allowing
+ * for a HOT update. Add information to the IndexInfo to allow for checking
+ * if the indexed value has changed.
+ *
+ * Do this processing here rather than in BuildIndexInfo() to not incur the
+ * overhead in the common non-expression cases.
+ * ----------------
+ */
+void
+BuildUpdateIndexInfo(ResultRelInfo *resultRelInfo)
+{
+ for (int j = 0; j < resultRelInfo->ri_NumIndices; j++)
+ {
+ int i;
+ int indnkeyatts;
+ Bitmapset *attrs = NULL;
+ IndexInfo *ii = resultRelInfo->ri_IndexRelationInfo[j];
+
+ /*
+ * Expressions are not allowed on non-key attributes, so we can skip
+ * them as they should show up in the index HOT-blocking attributes.
+ */
+ indnkeyatts = ii->ii_NumIndexKeyAttrs;
+
+ /* Collect key attributes used by the index */
+ for (i = 0; i < indnkeyatts; i++)
+ {
+ AttrNumber attnum = ii->ii_IndexAttrNumbers[i];
+
+ if (attnum != 0)
+ attrs = bms_add_member(attrs, attnum - FirstLowInvalidHeapAttributeNumber);
+ }
+
+ /* Collect attributes used in the expression */
+ if (ii->ii_Expressions)
+ pull_varattnos((Node *) ii->ii_Expressions,
+ resultRelInfo->ri_RangeTableIndex,
+ &ii->ii_ExpressionsAttrs);
+
+ /* Collect attributes used in the predicate */
+ if (ii->ii_Predicate)
+ pull_varattnos((Node *) ii->ii_Predicate,
+ resultRelInfo->ri_RangeTableIndex,
+ &ii->ii_PredicateAttrs);
+
+ ii->ii_IndexedAttrs = bms_union(attrs, ii->ii_ExpressionsAttrs);
+
+ /* All indexes should index *something*! */
+ Assert(!bms_is_empty(ii->ii_IndexedAttrs));
+ }
+}
+
/* ----------------
* BuildIndexInfo
* Construct an IndexInfo record for an open index
diff --git a/src/backend/catalog/indexing.c b/src/backend/catalog/indexing.c
index 004c5121000..a361c215490 100644
--- a/src/backend/catalog/indexing.c
+++ b/src/backend/catalog/indexing.c
@@ -102,7 +102,7 @@ CatalogIndexInsert(CatalogIndexState indstate, HeapTuple heapTuple,
* Get information from the state structure. Fall out if nothing to do.
*/
numIndexes = indstate->ri_NumIndices;
- if (numIndexes == 0)
+ if (numIndexes == 0 || updateIndexes == TU_None)
return;
relationDescs = indstate->ri_IndexRelationDescs;
indexInfoArray = indstate->ri_IndexRelationInfo;
@@ -314,15 +314,18 @@ CatalogTupleUpdate(Relation heapRel, const ItemPointerData *otid, HeapTuple tup)
{
CatalogIndexState indstate;
TU_UpdateIndexes updateIndexes = TU_All;
+ Bitmapset *updatedAttrs;
CatalogTupleCheckConstraints(heapRel, tup);
indstate = CatalogOpenIndexes(heapRel);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
-
+ updatedAttrs = simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = updatedAttrs;
CatalogIndexInsert(indstate, tup, updateIndexes);
+
CatalogCloseIndexes(indstate);
+ bms_free(updatedAttrs);
}
/*
@@ -338,12 +341,15 @@ CatalogTupleUpdateWithInfo(Relation heapRel, const ItemPointerData *otid, HeapTu
CatalogIndexState indstate)
{
TU_UpdateIndexes updateIndexes = TU_All;
+ Bitmapset *updatedAttrs;
CatalogTupleCheckConstraints(heapRel, tup);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
-
+ updatedAttrs = simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = updatedAttrs;
CatalogIndexInsert(indstate, tup, updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = NULL;
+ bms_free(updatedAttrs);
}
/*
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 5d819bda54a..c665aa744b3 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -292,8 +292,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
indexInfo->ii_IndexAttrNumbers[1] = 2;
indexInfo->ii_Expressions = NIL;
indexInfo->ii_ExpressionsState = NIL;
+ indexInfo->ii_ExpressionsAttrs = NULL;
indexInfo->ii_Predicate = NIL;
indexInfo->ii_PredicateState = NULL;
+ indexInfo->ii_PredicateAttrs = NULL;
+ indexInfo->ii_CheckedPredicate = false;
+ indexInfo->ii_PredicateSatisfied = false;
indexInfo->ii_ExclusionOps = NULL;
indexInfo->ii_ExclusionProcs = NULL;
indexInfo->ii_ExclusionStrats = NULL;
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index fb1bc3a480d..20968a814d6 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -109,11 +109,15 @@
#include "access/genam.h"
#include "access/relscan.h"
#include "access/tableam.h"
+#include "access/sysattr.h"
#include "access/xact.h"
#include "catalog/index.h"
#include "executor/executor.h"
+#include "nodes/bitmapset.h"
+#include "nodes/execnodes.h"
#include "nodes/nodeFuncs.h"
#include "storage/lmgr.h"
+#include "utils/datum.h"
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
@@ -318,8 +322,8 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
Relation heapRelation;
IndexInfo **indexInfoArray;
ExprContext *econtext;
- Datum values[INDEX_MAX_KEYS];
- bool isnull[INDEX_MAX_KEYS];
+ Datum loc_values[INDEX_MAX_KEYS];
+ bool loc_isnull[INDEX_MAX_KEYS];
Assert(ItemPointerIsValid(tupleid));
@@ -343,13 +347,13 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
/* Arrange for econtext's scan tuple to be the tuple under test */
econtext->ecxt_scantuple = slot;
- /*
- * for each index, form and insert the index tuple
- */
+ /* Insert into each index that needs updating */
for (i = 0; i < numIndices; i++)
{
Relation indexRelation = relationDescs[i];
IndexInfo *indexInfo;
+ Datum *values;
+ bool *isnull;
bool applyNoDupErr;
IndexUniqueCheck checkUnique;
bool indexUnchanged;
@@ -366,7 +370,7 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
/*
* Skip processing of non-summarizing indexes if we only update
- * summarizing indexes
+ * summarizing indexes or if this index is unchanged.
*/
if (onlySummarizing && !indexInfo->ii_Summarizing)
continue;
@@ -387,8 +391,15 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
indexInfo->ii_PredicateState = predicate;
}
+ /* Check the index predicate if we haven't done so earlier on */
+ if (!indexInfo->ii_CheckedPredicate)
+ {
+ indexInfo->ii_PredicateSatisfied = ExecQual(predicate, econtext);
+ indexInfo->ii_CheckedPredicate = true;
+ }
+
/* Skip this index-update if the predicate isn't satisfied */
- if (!ExecQual(predicate, econtext))
+ if (!indexInfo->ii_PredicateSatisfied)
continue;
}
@@ -396,11 +407,10 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* FormIndexDatum fills in its values and isnull parameters with the
* appropriate values for the column(s) of the index.
*/
- FormIndexDatum(indexInfo,
- slot,
- estate,
- values,
- isnull);
+ FormIndexDatum(indexInfo, slot, estate, loc_values, loc_isnull);
+
+ values = loc_values;
+ isnull = loc_isnull;
/* Check whether to apply noDupErr to this index */
applyNoDupErr = noDupErr &&
@@ -435,7 +445,9 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* index. If we're being called as part of an UPDATE statement,
* consider if the 'indexUnchanged' = true hint should be passed.
*/
- indexUnchanged = update && bms_is_empty(resultRelInfo->ri_ChangedIndexedCols);
+ indexUnchanged = update &&
+ !bms_overlap(indexInfo->ii_IndexedAttrs,
+ resultRelInfo->ri_ChangedIndexedCols);
satisfiesConstraint =
index_insert(indexRelation, /* index relation */
@@ -604,7 +616,12 @@ ExecCheckIndexConstraints(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
checkedIndex = true;
/* Check for partial index */
- if (indexInfo->ii_Predicate != NIL)
+ if (indexInfo->ii_CheckedPredicate && !indexInfo->ii_PredicateSatisfied)
+ {
+ /* We've already checked and the predicate wasn't satisfied. */
+ continue;
+ }
+ else if (indexInfo->ii_Predicate != NIL)
{
ExprState *predicate;
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 34f86546fc9..e4b2cd5a3e8 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -54,10 +54,13 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/attnum.h"
+#include "access/sysattr.h"
#include "access/tableam.h"
#include "access/tupconvert.h"
#include "access/tupdesc.h"
#include "access/xact.h"
+#include "catalog/index.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -75,6 +78,7 @@
#include "utils/float.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
@@ -245,6 +249,10 @@ tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
typentry = lookup_type_cache(typid,
TYPECACHE_EQ_OPR | TYPECACHE_EQ_OPR_FINFO);
+ /* Use the type's collation if none provided */
+ if (collation == -1)
+ collation = typentry->typcollation;
+
/*
* If no equality operator is available, fall back to binary comparison.
* This handles types that don't have proper equality operators defined.
@@ -291,108 +299,356 @@ tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
}
/*
- * Determine which updated attributes actually changed values between old and
- * new tuples and are referenced by indexes on the relation.
+ * ExecCheckIndexedAttrsForChanges
+ *
+ * Determine which indexes need updating by finding the set of modified
+ * indexed attributes.
+ *
+ * For expression indexes and indexes which implement the amcomparedatums()
+ * index AM API we'll need to form index datum and compare each attribute to
+ * see if any actually changed.
+ *
+ * For expression indexes the result of the expression might not change at all,
+ * this is common with JSONB columns which require expression indexes and where
+ * it is commonplace to index a field within a document and have updates that
+ * generally don't update that field.
+ *
+ * Partial indexes won't trigger index tuples when the old/new tuples are both
+ * outside of the predicate range.
+ *
+ * All other indexes require testing old/new datum for equality. We do this
+ * by calling the type-specific equality operator when possible, otherwise we
+ * fall back to binary equality with datumIsEqual().
+ *
+ * For nbtree the amcomparedatums() API is critical as it requires that key
+ * attributes are equal when they memcmp(), which might not be the case when
+ * using type-specific comparison or factoring in collation which might make
+ * an index case insensitive.
*
- * Returns a Bitmapset of attribute offsets (0-based, adjusted by
- * FirstLowInvalidHeapAttributeNumber) or NULL if no attributes changed.
+ * All of this is to say that the goal is for the executor to know, ahead of
+ * calling into the table AM to process the update and before calling into the
+ * index AM for inserting new index tuples, which attributes truely necessitate
+ * a new index tuple.
+ *
+ * Returns a refined Bitmapset of attributes that force index updates.
*/
Bitmapset *
ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
- TupleTableSlot *tts_old,
- TupleTableSlot *tts_new)
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts)
{
Relation relation = relinfo->ri_RelationDesc;
TupleDesc tupdesc = RelationGetDescr(relation);
- Bitmapset *indexed_attrs;
- Bitmapset *modified = NULL;
- int attidx;
+ Bitmapset *mix_attrs = NULL;
/* If no indexes, we're done */
if (relinfo->ri_NumIndices == 0)
return NULL;
/*
- * Get the set of index key attributes. This includes summarizing,
- * expression indexes and attributes mentioned in the predicate of a
- * partition but not those in INCLUDING.
+ * NOTE: Expression and predicates that are observed to change will have
+ * all their attributes added into the m_attrs set knowing that some of
+ * those might not have changed. Take for instance an index on (a + b)
+ * followed by an index on (b) with an update that changes only the value
+ * of 'a'. We'll add both 'a' and 'b' to the m_attrs set then later when
+ * reviewing the second index add 'b' to the u_attrs (unchanged) set. In
+ * the end, we'll remove all the unchanged from the m_attrs and get our
+ * desired result.
*/
- indexed_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_INDEXED);
- Assert(!bms_is_empty(indexed_attrs));
- /*
- * NOTE: It is important to scan all indexed attributes in the tuples
- * because ExecGetAllUpdatedCols won't include columns that may have been
- * modified via heap_modify_tuple_by_col which is the case in
- * tsvector_update_trigger.
- */
- attidx = -1;
- while ((attidx = bms_next_member(indexed_attrs, attidx)) >= 0)
+ /* Find the indexes that reference this attribute */
+ for (int i = 0; i < relinfo->ri_NumIndices; i++)
{
- /* attidx is zero-based, attrnum is the normal attribute number */
- AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
- Form_pg_attribute attr;
- bool oldnull,
- newnull;
- Datum oldval,
- newval;
+ Relation indexRel = relinfo->ri_IndexRelationDescs[i];
+ IndexAmRoutine *amroutine = indexRel->rd_indam;
+ IndexInfo *indexInfo = relinfo->ri_IndexRelationInfo[i];
+ Bitmapset *m_attrs = NULL; /* (possibly) modified key attributes */
+ Bitmapset *p_attrs = NULL; /* (possibly) modified predicate attributes */
+ Bitmapset *u_attrs = NULL; /* unmodified attributes */
+ Bitmapset *pre_attrs = indexInfo->ii_PredicateAttrs;
+ bool has_expressions = (indexInfo->ii_Expressions != NIL);
+ bool has_am_compare = (amroutine->amcomparedatums != NULL);
+ bool is_partial = (indexInfo->ii_Predicate != NIL);
+ TupleTableSlot *save_scantuple;
+ ExprContext *econtext = GetPerTupleExprContext(estate);
+ Datum old_values[INDEX_MAX_KEYS];
+ bool old_isnull[INDEX_MAX_KEYS];
+ Datum new_values[INDEX_MAX_KEYS];
+ bool new_isnull[INDEX_MAX_KEYS];
+
+ /* If we've reviewed all the attributes on this index, move on */
+ if (bms_is_subset(indexInfo->ii_IndexedAttrs, mix_attrs))
+ continue;
- /*
- * If it's a whole-tuple reference, record as modified. It's not
- * really worth supporting this case, since it could only succeed
- * after a no-op update, which is hardly a case worth optimizing for.
- */
- if (attrnum == 0)
+ /* Checking partial at this point isn't viable when we're serializable */
+ if (is_partial && IsolationIsSerializable())
{
- modified = bms_add_member(modified, attidx);
- continue;
+ p_attrs = bms_copy(pre_attrs);
+ }
+ /* Check partial index predicate */
+ else if (is_partial)
+ {
+ ExprState *pstate;
+ bool old_qualifies,
+ new_qualifies;
+
+ if (!indexInfo->ii_CheckedPredicate)
+ pstate = ExecPrepareQual(indexInfo->ii_Predicate, estate);
+ else
+ pstate = indexInfo->ii_PredicateState;
+
+ save_scantuple = econtext->ecxt_scantuple;
+
+ econtext->ecxt_scantuple = old_tts;
+ old_qualifies = ExecQual(pstate, econtext);
+
+ econtext->ecxt_scantuple = new_tts;
+ new_qualifies = ExecQual(pstate, econtext);
+
+ econtext->ecxt_scantuple = save_scantuple;
+
+ indexInfo->ii_CheckedPredicate = true;
+ indexInfo->ii_PredicateState = pstate;
+ indexInfo->ii_PredicateSatisfied = new_qualifies;
+
+ /* Both outside predicate, index doesn't need update */
+ if (!old_qualifies && !new_qualifies)
+ continue;
+
+ /* A transition means we need to update the index */
+ if (old_qualifies != new_qualifies)
+ p_attrs = bms_copy(pre_attrs);
+
+ /*
+ * When both are within the predicate we must update this index,
+ * but only if one of the index key attributes changed.
+ */
}
/*
- * Likewise, include in the modified set any system attribute other
- * than tableOID; we cannot expect these to be consistent in a HOT
- * chain, or even to be set correctly yet in the new tuple.
+ * Expression indexes or extraction-based index require us to form
+ * index datums and compare. We've done all we can to avoid this
+ * overhead, now it's time to bite the bullet and get it done.
+ *
+ * XXX: Caching the values/isnull might be a win and avoid one of the
+ * added calls to FormIndexDatum().
*/
- if (attrnum < 0)
+ if (has_expressions || has_am_compare)
{
- if (attrnum != TableOidAttributeNumber)
- modified = bms_add_member(modified, attidx);
- continue;
- }
+ save_scantuple = econtext->ecxt_scantuple;
- /* Extract values from both slots */
- oldval = slot_getattr(tts_old, attrnum, &oldnull);
- newval = slot_getattr(tts_new, attrnum, &newnull);
+ /* Evaluate expressions (if any) to get base datums */
+ econtext->ecxt_scantuple = old_tts;
+ FormIndexDatum(indexInfo, old_tts, estate, old_values, old_isnull);
- /* If one value is NULL and the other is not, they are not equal */
- if (oldnull != newnull)
- {
- modified = bms_add_member(modified, attidx);
- continue;
+ econtext->ecxt_scantuple = new_tts;
+ FormIndexDatum(indexInfo, new_tts, estate, new_values, new_isnull);
+
+ econtext->ecxt_scantuple = save_scantuple;
+
+ /* Compare the index key datums for equality */
+ for (int j = 0; j < indexInfo->ii_NumIndexKeyAttrs; j++)
+ {
+ AttrNumber idx_attrnum = indexInfo->ii_IndexAttrNumbers[j];
+ int idx_attridx = idx_attrnum - FirstLowInvalidHeapAttributeNumber;
+ int nth_expr = 0;
+ bool values_equal = false;
+
+ /*
+ * We can't skip attributes that we've already identified as
+ * triggering an index update because we may have added an
+ * attribute from an expression index that didn't change but
+ * the expression did and that unchanged attribute is
+ * referenced in a subsequent index where we will discover that
+ * fact.
+ */
+
+ /* A change to/from NULL, record this attribute */
+ if (old_isnull[j] != new_isnull[j])
+ {
+ /* Expressions will have idx_attrnum == 0 */
+ if (idx_attrnum == 0)
+ m_attrs = bms_add_members(m_attrs, indexInfo->ii_ExpressionsAttrs);
+ else
+ m_attrs = bms_add_member(m_attrs, idx_attridx);
+ continue;
+ }
+
+ /* Both NULL, no change */
+ if (old_isnull[j])
+ {
+ if (idx_attrnum != 0)
+ u_attrs = bms_add_member(u_attrs, idx_attridx);
+
+ continue;
+ }
+
+ /*
+ * Use index AM's comparison function if present when comparing
+ * the index datum formed when creating an index key.
+ */
+ if (has_am_compare)
+ {
+ /*
+ * For nbtree to properly deduplicate TIDs on page split it
+ * must treat equality as binary comparison. So it is
+ * vital that we call it's comparedatums() function.
+ *
+ * In the case of GIN/RUM indexes they too behave
+ * differently and can even extract one or more portions of
+ * the datum when forming index tuples. We'd like to know
+ * if this update needs to trigger one or more index
+ * tuples, so we let the index AM perform their extraction
+ * and compare the results.
+ *
+ * There may be other similar index AM implementation with
+ * extraction where indexes are built using only part(s) of
+ * the Datum and might even need to invoke type-specific
+ * equality operators.
+ *
+ * NOTE: For AM comparison, pass the 1-based index
+ * attribute number. The AM's compare function expects the
+ * same numbering as used internally by the AM.
+ */
+ values_equal = amroutine->amcomparedatums(indexRel, j + 1,
+ old_values[j], old_isnull[j],
+ new_values[j], new_isnull[j]);
+ }
+ else
+ {
+ /*
+ * Expression index without custom AM comparison. Compare
+ * the expression results using type-specific equality
+ * which at this point is the expression's type, not the
+ * index's type. It is in index_form_tuple() that index
+ * attributes are transformed, not FormIndexDatum().
+ */
+ Oid expr_type_oid;
+ int16 typlen; /* Output: type length */
+ bool typbyval;
+ Expr *expr = (Expr *) list_nth(indexInfo->ii_Expressions, nth_expr);
+
+ Assert(expr != NULL);
+
+ /* Get type OID from the expression */
+ expr_type_oid = exprType((Node *) expr);
+
+ /* Get type information from the OID */
+ get_typlenbyval(expr_type_oid, &typlen, &typbyval);
+
+ values_equal = tts_attr_equal(expr_type_oid,
+ -1, /* use TBD expr type */
+ typbyval,
+ typlen,
+ old_values[j],
+ new_values[j]);
+ }
+
+ if (!values_equal)
+ {
+ /* Expressions will have idx_attrnum == 0 */
+ if (idx_attrnum == 0)
+ m_attrs = bms_add_members(m_attrs, indexInfo->ii_ExpressionsAttrs);
+ else
+ m_attrs = bms_add_member(m_attrs, idx_attridx);
+ }
+ else
+ {
+ if (idx_attrnum != 0)
+ u_attrs = bms_add_member(u_attrs, idx_attridx);
+ }
+
+ if (idx_attrnum == 0)
+ nth_expr++;
+ }
}
+ else
+ {
+ /*
+ * Here we know that we're reviewing an index that doesn't have a
+ * partial predicate, doesn't use expressions, and doesn't have a
+ * amcomparedatums() implementation.
+ */
- /* If both are NULL, consider them equal */
- if (oldnull)
- continue;
+ /* Compare the index key datums for equality */
+ for (int j = 0; j < indexInfo->ii_NumIndexKeyAttrs; j++)
+ {
+ Form_pg_attribute attr;
+ AttrNumber rel_attrnum;
+ int rel_attridx;
+ bool values_equal = false;
+ bool old_null,
+ new_null;
+ Datum old_val,
+ new_val;
- /* Get attribute metadata */
- Assert(attrnum > 0 && attrnum <= tupdesc->natts);
- attr = TupleDescAttr(tupdesc, attrnum - 1);
-
- /* Compare using type-specific equality operator */
- if (!tts_attr_equal(attr->atttypid,
- attr->attcollation,
- attr->attbyval,
- attr->attlen,
- oldval,
- newval))
- modified = bms_add_member(modified, attidx);
- }
+ rel_attrnum = indexInfo->ii_IndexAttrNumbers[j];
+ rel_attridx = rel_attrnum - FirstLowInvalidHeapAttributeNumber;
- bms_free(indexed_attrs);
+ /* Zero would mean expression, something we don't expect here */
+ Assert(rel_attrnum > 0 && rel_attrnum <= tupdesc->natts);
- return modified;
+ /* Extract values from both slots for this attribute */
+ old_val = slot_getattr(old_tts, rel_attrnum, &old_null);
+ new_val = slot_getattr(new_tts, rel_attrnum, &new_null);
+
+ /*
+ * If one value is NULL and the other is not, they are not
+ * equal
+ */
+ if (old_null != new_null)
+ {
+ m_attrs = bms_add_member(m_attrs, rel_attridx);
+ continue;
+ }
+
+ /* If both are NULL, consider them equal */
+ if (old_null)
+ {
+ u_attrs = bms_add_member(u_attrs, rel_attridx);
+ continue;
+ }
+
+ attr = TupleDescAttr(tupdesc, rel_attrnum - 1);
+
+ /*
+ * Compare using type-specific equality which at this point is
+ * the relation's type because FormIndexDatum() will populate
+ * the values/nulls but won't transform them into the final
+ * values destined for the index tuple, that's left to
+ * index_form_tuple() which we don't call (on purpose).
+ */
+ values_equal = tts_attr_equal(attr->atttypid,
+ attr->attcollation,
+ attr->attbyval,
+ attr->attlen,
+ old_val,
+ new_val);
+
+ if (!values_equal)
+ m_attrs = bms_add_member(m_attrs, rel_attridx);
+ else
+ u_attrs = bms_add_member(u_attrs, rel_attridx);
+ }
+ }
+
+ /*
+ * Here we know all the attributes we thought might be modified and
+ * all those we know haven't been. Take the difference and add it to
+ * the modified indexed attributes set.
+ */
+ m_attrs = bms_del_members(m_attrs, u_attrs);
+ p_attrs = bms_del_members(p_attrs, u_attrs);
+ mix_attrs = bms_add_members(mix_attrs, m_attrs);
+ mix_attrs = bms_add_members(mix_attrs, p_attrs);
+
+ bms_free(m_attrs);
+ bms_free(u_attrs);
+ bms_free(p_attrs);
+ }
+
+ return mix_attrs;
}
/*
@@ -2395,6 +2651,9 @@ ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
bool partition_constraint_failed;
TM_Result result;
+ /* The set of modified indexed attributes that trigger new index entries */
+ Bitmapset *mix_attrs = NULL;
+
updateCxt->crossPartUpdate = false;
/*
@@ -2517,13 +2776,32 @@ lreplace:
bms_free(resultRelInfo->ri_ChangedIndexedCols);
resultRelInfo->ri_ChangedIndexedCols = NULL;
- resultRelInfo->ri_ChangedIndexedCols =
- ExecCheckIndexedAttrsForChanges(resultRelInfo, oldSlot, slot);
+ /*
+ * During updates we'll need a bit more information in IndexInfo but we've
+ * delayed adding it until here. We check to ensure that there are
+ * indexes, that something has changed that is indexed, and that the first
+ * index doesn't yet have ii_IndexedAttrs set as a way to ensure we only
+ * build this when needed and only once. We don't build this in
+ * ExecOpenIndicies() as it is unnecessary overhead when not performing an
+ * update.
+ */
+ if (resultRelInfo->ri_NumIndices > 0 &&
+ bms_is_empty(resultRelInfo->ri_IndexRelationInfo[0]->ii_IndexedAttrs))
+ BuildUpdateIndexInfo(resultRelInfo);
+
+ /*
+ * Next up we need to find out the set of indexed attributes that have
+ * changed in value and should trigger a new index tuple. We could start
+ * with the set of updated columns via ExecGetUpdatedCols(), but if we do
+ * we will overlook attributes directly modified by heap_modify_tuple()
+ * which are not known to ExecGetUpdatedCols().
+ */
+ mix_attrs = ExecCheckIndexedAttrsForChanges(resultRelInfo, estate, oldSlot, slot);
/*
- * replace the heap tuple
+ * Call into the table AM to update the heap tuple.
*
- * Note: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
+ * NOTE: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
* the row to be updated is visible to that snapshot, and throw a
* can't-serialize error if not. This is a special-case behavior needed
* for referential integrity updates in transaction-snapshot mode
@@ -2535,9 +2813,12 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
- resultRelInfo->ri_ChangedIndexedCols,
+ mix_attrs,
&updateCxt->updateIndexes);
+ Assert(bms_is_empty(resultRelInfo->ri_ChangedIndexedCols));
+ resultRelInfo->ri_ChangedIndexedCols = mix_attrs;
+
return result;
}
@@ -2555,7 +2836,7 @@ ExecUpdateEpilogue(ModifyTableContext *context, UpdateContext *updateCxt,
ModifyTableState *mtstate = context->mtstate;
List *recheckIndexes = NIL;
- /* insert index entries for tuple if necessary */
+ /* Insert index entries for tuple if necessary */
if (resultRelInfo->ri_NumIndices > 0 && (updateCxt->updateIndexes != TU_None))
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
slot, context->estate,
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index d69dc090aa4..e9a53b95caf 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -855,10 +855,14 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
/* expressions */
n->ii_Expressions = expressions;
n->ii_ExpressionsState = NIL;
+ n->ii_ExpressionsAttrs = NULL;
/* predicates */
n->ii_Predicate = predicates;
n->ii_PredicateState = NULL;
+ n->ii_PredicateAttrs = NULL;
+ n->ii_CheckedPredicate = false;
+ n->ii_PredicateSatisfied = false;
/* exclusion constraints */
n->ii_ExclusionOps = NULL;
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index 63dd41c1f21..9bdf73eda59 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -211,6 +211,33 @@ typedef void (*ammarkpos_function) (IndexScanDesc scan);
/* restore marked scan position */
typedef void (*amrestrpos_function) (IndexScanDesc scan);
+/*
+ * amcomparedatums - Compare datums to determine if index update is needed
+ *
+ * This function compares old_datum and new_datum to determine if they would
+ * produce different index entries. For extraction-based indexes (GIN, RUM),
+ * this should:
+ * 1. Extract keys from old_datum using the opclass's extractValue function
+ * 2. Extract keys from new_datum using the opclass's extractValue function
+ * 3. Compare the two sets of keys using appropriate equality operators
+ * 4. Return true if the sets are equal (no index update needed)
+ *
+ * The comparison should account for:
+ * - Different numbers of extracted keys
+ * - NULL values
+ * - Type-specific equality (not just binary equality)
+ * - Opclass parameters (e.g., path in bson_rum_single_path_ops)
+ *
+ * For the DocumentDB example with path='a', this would extract values at
+ * path 'a' from both old and new BSON documents and compare them using
+ * BSON's equality operator.
+ */
+/* identify if updated datums would produce one or more index entries */
+typedef bool (*amcomparedatums_function) (Relation indexRelation,
+ int attno,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* Callback function signatures - for parallel index scans.
*/
@@ -313,6 +340,7 @@ typedef struct IndexAmRoutine
amendscan_function amendscan;
ammarkpos_function ammarkpos; /* can be NULL */
amrestrpos_function amrestrpos; /* can be NULL */
+ amcomparedatums_function amcomparedatums; /* can be NULL */
/* interface functions to support parallel index scans */
amestimateparallelscan_function amestimateparallelscan; /* can be NULL */
diff --git a/src/include/access/gin.h b/src/include/access/gin.h
index 13ea91922ef..2f265f4816c 100644
--- a/src/include/access/gin.h
+++ b/src/include/access/gin.h
@@ -100,6 +100,9 @@ extern PGDLLIMPORT int gin_pending_list_limit;
extern void ginGetStats(Relation index, GinStatsData *stats);
extern void ginUpdateStats(Relation index, const GinStatsData *stats,
bool is_build);
+extern bool gincomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
extern void _gin_parallel_build_main(dsm_segment *seg, shm_toc *toc);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 41d541aa6b2..59db389a546 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -326,7 +326,7 @@ extern TM_Result heap_update(Relation relation, HeapTupleData *oldtup,
TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
- Bitmapset *mix_attrs, Buffer *vmbuffer,
+ const Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
@@ -361,8 +361,8 @@ extern bool heap_tuple_needs_eventual_freeze(HeapTupleHeader tuple);
extern void simple_heap_insert(Relation relation, HeapTuple tup);
extern void simple_heap_delete(Relation relation, const ItemPointerData *tid);
-extern void simple_heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple tup, TU_UpdateIndexes *update_indexes);
+extern Bitmapset *simple_heap_update(Relation relation, const ItemPointerData *otid,
+ HeapTuple tup, TU_UpdateIndexes *update_indexes);
extern TransactionId heap_index_delete_tuples(Relation rel,
TM_IndexDeleteOp *delstate);
diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h
index 16be5c7a9c1..42bd329eaad 100644
--- a/src/include/access/nbtree.h
+++ b/src/include/access/nbtree.h
@@ -1210,6 +1210,10 @@ extern int btgettreeheight(Relation rel);
extern CompareType bttranslatestrategy(StrategyNumber strategy, Oid opfamily);
extern StrategyNumber bttranslatecmptype(CompareType cmptype, Oid opfamily);
+extern bool btcomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* prototypes for internal functions in nbtree.c
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 8a5931a3118..2b9206ff24a 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -549,7 +549,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
- Bitmapset *updated_cols,
+ const Bitmapset *updated_cols,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1503,12 +1503,12 @@ static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- Bitmapset *updated_cols, TU_UpdateIndexes *update_indexes)
+ const Bitmapset *mix_cols, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
wait, tmfd, lockmode,
- updated_cols, update_indexes);
+ mix_cols, update_indexes);
}
/*
@@ -2011,7 +2011,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
- Bitmapset *modified_indexe_attrs,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index dda95e54903..8d364f8b30f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -132,6 +132,7 @@ extern bool CompareIndexInfo(const IndexInfo *info1, const IndexInfo *info2,
const AttrMap *attmap);
extern void BuildSpeculativeIndexInfo(Relation index, IndexInfo *ii);
+extern void BuildUpdateIndexInfo(ResultRelInfo *resultRelInfo);
extern void FormIndexDatum(IndexInfo *indexInfo,
TupleTableSlot *slot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 993dc0e6ced..a19585ba065 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -739,6 +739,11 @@ extern Bitmapset *ExecGetAllUpdatedCols(ResultRelInfo *relinfo, EState *estate);
*/
extern void ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative);
extern void ExecCloseIndices(ResultRelInfo *resultRelInfo);
+extern Bitmapset *ExecWhichIndexesRequireUpdates(ResultRelInfo *relinfo,
+ Bitmapset *mix_attrs,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
extern List *ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
TupleTableSlot *slot, EState *estate,
bool update,
@@ -800,9 +805,10 @@ extern ResultRelInfo *ExecLookupResultRelByOid(ModifyTableState *node,
Oid resultoid,
bool missing_ok,
bool update_cache);
-extern Bitmapset *ExecCheckIndexedAttrsForChanges(ResultRelInfo *resultRelInfo,
- TupleTableSlot *tts_old,
- TupleTableSlot *tts_new);
+extern Bitmapset *ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
extern bool tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
Datum value1, Datum value2);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 898368fb8cb..d8e88817206 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -174,15 +174,29 @@ typedef struct IndexInfo
*/
AttrNumber ii_IndexAttrNumbers[INDEX_MAX_KEYS];
+ /*
+ * All key, expression, sumarizing, and partition attributes referenced by
+ * this index
+ */
+ Bitmapset *ii_IndexedAttrs;
+
/* expr trees for expression entries, or NIL if none */
List *ii_Expressions; /* list of Expr */
/* exec state for expressions, or NIL if none */
List *ii_ExpressionsState; /* list of ExprState */
+ /* attributes exclusively referenced by expression indexes */
+ Bitmapset *ii_ExpressionsAttrs;
/* partial-index predicate, or NIL if none */
List *ii_Predicate; /* list of Expr */
/* exec state for expressions, or NIL if none */
ExprState *ii_PredicateState;
+ /* attributes referenced by the predicate */
+ Bitmapset *ii_PredicateAttrs;
+ /* partial index predicate determined yet? */
+ bool ii_CheckedPredicate;
+ /* amupdate hint used to avoid rechecking predicate */
+ bool ii_PredicateSatisfied;
/* Per-column exclusion operators, or NULL if none */
Oid *ii_ExclusionOps; /* array with one entry per column */
@@ -494,6 +508,11 @@ typedef struct ResultRelInfo
Bitmapset *ri_extraUpdatedCols;
/* true if the above has been computed */
bool ri_extraUpdatedCols_valid;
+
+ /*
+ * For UPDATE a Bitmapset of the attributes that are both indexed and have
+ * changed in value.
+ */
Bitmapset *ri_ChangedIndexedCols;
/* Projection to generate new tuple in an INSERT/UPDATE */
diff --git a/src/test/isolation/expected/insert-conflict-specconflict.out b/src/test/isolation/expected/insert-conflict-specconflict.out
index e34a821c403..54b3981918c 100644
--- a/src/test/isolation/expected/insert-conflict-specconflict.out
+++ b/src/test/isolation/expected/insert-conflict-specconflict.out
@@ -80,6 +80,10 @@ pg_advisory_unlock
t
(1 row)
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
s1: NOTICE: acquiring advisory lock on 2
s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
@@ -172,6 +176,10 @@ pg_advisory_unlock
t
(1 row)
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
s2: NOTICE: acquiring advisory lock on 2
s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
@@ -369,6 +377,10 @@ key|data
step s1_commit: COMMIT;
s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
s2: NOTICE: acquiring advisory lock on 2
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
step s2_upsert: <... completed>
step controller_show: SELECT * FROM upserttest;
key|data
@@ -530,6 +542,14 @@ isolation/insert-conflict-specconflict/s2|transactionid|ExclusiveLock|t
step s2_commit: COMMIT;
s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_4() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 4
+s1: NOTICE: blurt_and_lock_4() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 4
step s1_upsert: <... completed>
step s1_noop:
step controller_show: SELECT * FROM upserttest;
diff --git a/src/test/regress/expected/hot_expression_indexes.out b/src/test/regress/expected/hot_expression_indexes.out
new file mode 100644
index 00000000000..29aad70e2aa
--- /dev/null
+++ b/src/test/regress/expected/hot_expression_indexes.out
@@ -0,0 +1,1006 @@
+-- ================================================================
+-- Test Suite for Heap-only (HOT) Updates
+-- ================================================================
+-- Setup: Create function to measure HOT updates
+CREATE OR REPLACE FUNCTION check_hot_updates(
+ expected INT,
+ p_table_name TEXT DEFAULT 't',
+ p_schema_name TEXT DEFAULT current_schema()
+)
+RETURNS TABLE (
+ table_name TEXT,
+ total_updates BIGINT,
+ hot_updates BIGINT,
+ hot_update_percentage NUMERIC,
+ matches_expected BOOLEAN
+)
+LANGUAGE plpgsql
+AS $$
+DECLARE
+ v_relid oid;
+ v_qualified_name TEXT;
+ v_hot_updates BIGINT;
+ v_updates BIGINT;
+ v_xact_hot_updates BIGINT;
+ v_xact_updates BIGINT;
+BEGIN
+ -- Force statistics update
+ PERFORM pg_stat_force_next_flush();
+
+ -- Get table OID
+ v_qualified_name := quote_ident(p_schema_name) || '.' || quote_ident(p_table_name);
+ v_relid := v_qualified_name::regclass;
+
+ IF v_relid IS NULL THEN
+ RAISE EXCEPTION 'Table %.% not found', p_schema_name, p_table_name;
+ END IF;
+
+ -- Get cumulative + transaction stats
+ v_hot_updates := COALESCE(pg_stat_get_tuples_hot_updated(v_relid), 0);
+ v_updates := COALESCE(pg_stat_get_tuples_updated(v_relid), 0);
+ v_xact_hot_updates := COALESCE(pg_stat_get_xact_tuples_hot_updated(v_relid), 0);
+ v_xact_updates := COALESCE(pg_stat_get_xact_tuples_updated(v_relid), 0);
+
+ v_hot_updates := v_hot_updates + v_xact_hot_updates;
+ v_updates := v_updates + v_xact_updates;
+
+ RETURN QUERY
+ SELECT
+ p_table_name::TEXT,
+ v_updates::BIGINT,
+ v_hot_updates::BIGINT,
+ CASE WHEN v_updates > 0
+ THEN ROUND((v_hot_updates::numeric / v_updates::numeric * 100)::numeric, 2)
+ ELSE 0
+ END,
+ (v_hot_updates = expected)::BOOLEAN;
+END;
+$$;
+-- ================================================================
+-- Basic JSONB Expression Index
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_name_idx ON t((docs->>'name'));
+INSERT INTO t VALUES (1, '{"name": "alice", "age": 30}');
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET docs = '{"name": "alice", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update indexed JSONB field - should NOT be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update non-indexed field again - should be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 32}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 2 | 66.67 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Partial Index with Predicate Transitions
+-- ================================================================
+CREATE TABLE t(id INT, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_value_idx ON t(value) WHERE value > 10;
+INSERT INTO t VALUES (1, 5);
+-- Both outside predicate - should be HOT
+UPDATE t SET value = 8 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET value = 15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Both inside predicate, value changes - should NOT be HOT
+UPDATE t SET value = 20 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Transition out of predicate - should NOT be HOT
+UPDATE t SET value = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+-- Both outside predicate again - should be HOT
+UPDATE t SET value = 3 WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 5 | 2 | 40.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Expression Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((docs->>'status'))
+ WHERE (docs->>'priority')::int > 5;
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3}');
+-- Both outside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 4}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 10}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Inside predicate, status changes - should NOT be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 10}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Inside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 8}';
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 2 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Index on JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data);
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "database"]}');
+-- Change tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Change tags again - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Add field without changing existing keys - GIN keys changed (added "note"), NOT HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "note": "test"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Index with Unchanged Keys
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create GIN index on specific path
+CREATE INDEX t_gin_idx ON t USING gin((data->'tags'));
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "sql"], "status": "active"}');
+-- Change non-indexed field - GIN keys on 'tags' unchanged, should be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change indexed tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN with jsonb_path_ops
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data jsonb_path_ops);
+INSERT INTO t VALUES (1, '{"user": {"name": "alice"}, "tags": ["a", "b"]}');
+-- Change value at different path - keys changed, NOT HOT
+UPDATE t SET data = '{"user": {"name": "bob"}, "tags": ["a", "b"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Multi-Column Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, a INT, b INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t(id, abs(a), abs(b));
+INSERT INTO t VALUES (1, -5, -10);
+-- Change sign but not abs value - should be HOT
+UPDATE t SET a = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change abs value - should NOT be HOT
+UPDATE t SET b = -15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Change id - should NOT be HOT
+UPDATE t SET id = 2 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Mixed Index Types (BRIN + Expression)
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_idx ON t USING brin(value);
+CREATE INDEX t_expr_idx ON t((data->>'status'));
+INSERT INTO t VALUES (1, 100, '{"status": "active"}');
+-- Update only BRIN column - should be HOT
+UPDATE t SET value = 200 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update only expression column - should NOT be HOT
+UPDATE t SET data = '{"status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update both - should NOT be HOT
+UPDATE t SET value = 300, data = '{"status": "pending"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Expression with COLLATION and BTREE (nbtree) index
+-- ================================================================
+CREATE COLLATION case_insensitive (
+ provider = libc,
+ locale = 'C'
+);
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ name TEXT COLLATE case_insensitive
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_lower_idx ON t USING BTREE (name COLLATE case_insensitive);
+INSERT INTO t VALUES (1, 'ALICE');
+-- Change case but not value - should NOT be HOT in BTREE
+UPDATE t SET name = 'Alice' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Change to new value - should NOT be HOT
+UPDATE t SET name = 'BOB' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Array Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_array_len_idx ON t(array_length(tags, 1));
+INSERT INTO t VALUES (1, ARRAY['a', 'b', 'c']);
+-- Same length, different elements - should be HOT
+UPDATE t SET tags = ARRAY['d', 'e', 'f'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Different length - should NOT be HOT
+UPDATE t SET tags = ARRAY['d', 'e'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Nested JSONB Expression and JSONB equality '->' (not '->>')
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_nested_idx ON t((data->'user'->'name'));
+INSERT INTO t VALUES ('{"user": {"name": "alice", "age": 30}}');
+-- Change nested non-indexed field - should be HOT
+UPDATE t SET data = '{"user": {"name": "alice", "age": 31}}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change nested indexed field - should NOT be HOT
+UPDATE t SET data = '{"user": {"name": "bob", "age": 31}}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Complex Predicate on Multiple JSONB Fields
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((data->>'status'))
+ WHERE (data->>'priority')::int > 5
+ AND (data->>'active')::boolean = true;
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3, "active": true}');
+-- Outside predicate (priority too low) - should be HOT
+UPDATE t SET data = '{"status": "done", "priority": 3, "active": true}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": true}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Inside predicate, change to outside (active = false) - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": false}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Array Index - Order Insensitive Extraction
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ data JSONB
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+-- GIN index on JSONB array (extracts all elements)
+CREATE INDEX t_items_gin ON t USING GIN ((data->'items'));
+INSERT INTO t VALUES (1, '{"items": [1, 2, 3], "status": "active"}');
+-- Update: Reorder array elements
+-- JSONB equality: NOT equal (different arrays)
+-- GIN extraction: Same elements extracted (might allow HOT if not careful)
+UPDATE t SET data = '{"items": [3, 2, 1], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update: Add/remove element
+UPDATE t SET data = '{"items": [1, 2, 3, 4], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- TOASTed Values in Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, large_text TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_substr_idx ON t(substr(large_text, 1, 10));
+INSERT INTO t VALUES (1, repeat('x', 5000) || 'identifier');
+-- Change end of string, prefix unchanged - should be HOT
+UPDATE t SET large_text = repeat('x', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change prefix - should NOT be HOT
+UPDATE t SET large_text = repeat('y', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- TEST: GIN with TOASTed TEXT (tsvector)
+-- ================================================================
+CREATE TABLE t(id INT, content TEXT, search_vec tsvector)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create trigger to maintain tsvector
+CREATE TRIGGER tsvectorupdate_toast
+ BEFORE INSERT OR UPDATE ON t
+ FOR EACH ROW EXECUTE FUNCTION
+ tsvector_update_trigger(search_vec, 'pg_catalog.english', content);
+CREATE INDEX t_gin ON t USING gin(search_vec);
+-- Insert with large content (will be TOASTed)
+INSERT INTO t (id, content) VALUES
+ (1, repeat('important keyword ', 1000) || repeat('filler text ', 10000));
+-- Verify initial state
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('important');
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+-- IMPORTANT: The BEFORE UPDATE trigger modifies search_vec, so by the time
+-- ExecWhichIndexesRequireUpdates() runs, search_vec has already changed.
+-- This means the comparison sees old tsvector vs. trigger-modified tsvector,
+-- not the natural progression. HOT won't happen because the trigger changed
+-- the indexed column.
+-- Update: Even though content keywords unchanged, trigger still fires
+UPDATE t
+SET content = repeat('important keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (trigger modifies search_vec, blocking HOT)
+-- This is actually correct behavior - the trigger updated an indexed column
+-- Update: Change indexed keywords
+UPDATE t
+SET content = repeat('critical keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (index keys changed)
+-- Verify query correctness
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('critical');
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+DROP TABLE t CASCADE;
+-- ================================================================
+-- TEST: GIN with TOASTed JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin((data->'tags'));
+-- Insert with TOASTed JSONB
+INSERT INTO t (id, data) VALUES
+ (1, jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('x', 10000)
+ ));
+-- Update: Change large_field, tags unchanged - should be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT update
+-- Update: Change tags - should NOT be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "sql"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Expected: Still 1 HOT
+-- Verify correctness
+SELECT count(*) FROM t WHERE data->'tags' @> '["database"]'::jsonb;
+ count
+-------
+ 0
+(1 row)
+
+-- Expected: 0 rows
+SELECT count(*) FROM t WHERE data->'tags' @> '["sql"]'::jsonb;
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+DROP TABLE t CASCADE;
+-- ================================================================
+-- TEST: GIN with Array of Large Strings
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin(tags);
+-- Insert with large array elements (might be TOASTed)
+INSERT INTO t (id, tags) VALUES
+ (1, ARRAY[repeat('tag1', 1000), repeat('tag2', 1000)]);
+-- Update: Change to different large values - NOT HOT
+UPDATE t
+SET tags = ARRAY[repeat('tag3', 1000), repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (keys actually changed)
+-- Update: Keep same tag values, just reorder - SHOULD BE HOT
+-- (GIN is order-insensitive: both [tag3,tag4] and [tag4,tag3]
+-- extract to the same sorted key set ['tag3','tag4'])
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000), repeat('tag3', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Expected: 1 HOT (GIN keys semantically identical)
+-- Update: Remove an element - NOT HOT (keys changed)
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Expected: Still 1 HOT (not this one)
+DROP TABLE t CASCADE;
+-- ================================================================
+-- BRIN Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ value INT,
+ description TEXT
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_partial_idx ON t USING brin(value) WHERE value > 100;
+INSERT INTO t VALUES (1, 50, 'below range');
+-- Test 1: Outside predicate
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Validate: Predicate query returns 0 rows
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+ cnt
+-----
+ 0
+(1 row)
+
+-- Test 2: Transition into predicate
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Validate: Predicate query returns 1 row with correct value
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+ cnt | max_val
+-----+---------
+ 1 | 150
+(1 row)
+
+-- Test 3: Inside predicate, value changes
+UPDATE t SET value = 160, description = 'updated again' WHERE id = 1;
+SELECT * FROM check_hot_updates(3, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 3 | 100.00 | t
+(1 row)
+
+-- Validate: Updated value (160) is returned
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+ cnt | max_val
+-----+---------
+ 1 | 160
+(1 row)
+
+-- Test 4: Transition out of predicate
+UPDATE t SET value = 50 WHERE id = 1;
+SELECT * FROM check_hot_updates(4, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 4 | 100.00 | t
+(1 row)
+
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+ cnt
+-----
+ 0
+(1 row)
+
+SELECT id, value, description FROM t;
+ id | value | description
+----+-------+---------------
+ 1 | 50 | updated again
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- HASH Index (Simple Column)
+-- ================================================================
+CREATE TABLE t(id INT, code VARCHAR(20), description TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_idx ON t USING hash(code);
+INSERT INTO t VALUES (1, 'CODE001', 'initial');
+-- Update non-indexed column - should be HOT
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update indexed column - HASH index requires update, NOT HOT
+UPDATE t SET code = 'CODE002' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update both - NOT HOT
+UPDATE t SET code = 'CODE003', description = 'changed' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Back to original code - NOT HOT (different hash bucket location)
+UPDATE t SET code = 'CODE001' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- HASH Index on Expression
+-- ================================================================
+CREATE TABLE t(id INT, email TEXT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_lower_email_idx ON t USING HASH(lower(email));
+INSERT INTO t VALUES (1, '[email protected]', '{"status": "new"}');
+-- Update non-indexed field - should be HOT
+UPDATE t SET data = '{"status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update email with case change only (same lowercase) - should be HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Update email to different lowercase - NOT HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 2 | 66.67 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- HASH Index on JSONB Field
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash((data->>'category'));
+INSERT INTO t VALUES (1, '{"category": "books", "title": "PostgreSQL Guide"}');
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET data = '{"category": "books", "title": "PostgreSQL Handbook"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update indexed JSONB field - NOT HOT
+UPDATE t SET data = '{"category": "videos", "title": "PostgreSQL Handbook"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update both - NOT HOT
+UPDATE t SET data = '{"category": "courses", "title": "PostgreSQL Basics"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- Multiple HASH Indexes
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, status VARCHAR, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+CREATE INDEX t_hash_status_idx ON t USING hash(status);
+INSERT INTO t VALUES (1, 'electronics', 'active', 100);
+-- Update non-indexed column - should be HOT
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update one indexed column - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update other indexed column - NOT HOT
+UPDATE t SET status = 'inactive' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Update both indexed columns - NOT HOT
+UPDATE t SET category = 'videos', status = 'pending' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- BRIN vs HASH Comparison
+-- ================================================================
+CREATE TABLE t_brin(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE TABLE t_hash(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_value_idx ON t_brin USING brin(value);
+CREATE INDEX t_hash_value_idx ON t_hash USING hash(value);
+INSERT INTO t_brin VALUES (1, 100, 'initial');
+INSERT INTO t_hash VALUES (1, 100, 'initial');
+-- Same update on both - different HOT behavior expected
+-- BRIN: might allow HOT (range summary unchanged)
+-- HASH: blocks HOT (hash bucket changed)
+UPDATE t_brin SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't_brin');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t_brin | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT (BRIN allows it for single row)
+UPDATE t_hash SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't_hash');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t_hash | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (HASH blocks it)
+DROP TABLE t_brin CASCADE;
+DROP TABLE t_hash CASCADE;
+-- ================================================================
+-- HASH Index with NULL Values
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'electronics', 'initial');
+-- Update indexed column to NULL - NOT HOT (hash value changed)
+UPDATE t SET category = NULL WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT
+-- Update indexed column from NULL to value - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Expected: 1 HOT
+DROP TABLE t CASCADE;
+-- ================================================================
+-- BRIN on JSONB Field
+-- ================================================================
+CREATE TABLE t(id INT, metrics JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- BRIN doesn't directly support JSONB, but we can test on expression
+CREATE INDEX t_brin_count_idx ON t USING brin(
+ CAST(metrics->>'count' AS INTEGER)
+);
+INSERT INTO t VALUES (1, '{"count": "100", "timestamp": "2024-01-01"}');
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET metrics = '{"count": "100", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT
+-- Update indexed field - BRIN allows HOT for single row
+UPDATE t SET metrics = '{"count": "150", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Expected: 2 HOT (BRIN permits single-row updates)
+DROP TABLE t CASCADE;
+-- ================================================================
+-- Mixed BRIN + HASH on Same Table
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, timestamp TIMESTAMP, price NUMERIC, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_timestamp_idx ON t USING brin(timestamp);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'books', '2024-01-01 10:00:00', 29.99, 'initial');
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT
+-- Update BRIN indexed column - allows HOT
+UPDATE t SET timestamp = '2024-01-02 10:00:00' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Expected: 2 HOT
+-- Update HASH indexed column - blocks HOT
+UPDATE t SET category = 'videos' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 2 | 66.67 | t
+(1 row)
+
+-- Expected: 2 HOT (HASH blocks it)
+-- Update price (non-indexed) - should be HOT
+UPDATE t SET price = 39.99 WHERE id = 1;
+SELECT * FROM check_hot_updates(3, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 3 | 75.00 | t
+(1 row)
+
+-- Expected: 3 HOT
+DROP TABLE t CASCADE;
+-- Cleanup
+DROP FUNCTION check_hot_updates(int, text, text);
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index f56482fb9f1..4459625a59b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -125,6 +125,12 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression compression_lz4 memoize stats predicate numa eager_aggregate
+
+# ----------
+# Another group of parallel tests, these focused on heap HOT updates
+# ----------
+test: hot_expression_indexes
+
# event_trigger depends on create_am and cannot run concurrently with
# any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/sql/hot_expression_indexes.sql b/src/test/regress/sql/hot_expression_indexes.sql
new file mode 100644
index 00000000000..4929be144ae
--- /dev/null
+++ b/src/test/regress/sql/hot_expression_indexes.sql
@@ -0,0 +1,747 @@
+-- ================================================================
+-- Test Suite for Heap-only (HOT) Updates
+-- ================================================================
+
+-- Setup: Create function to measure HOT updates
+CREATE OR REPLACE FUNCTION check_hot_updates(
+ expected INT,
+ p_table_name TEXT DEFAULT 't',
+ p_schema_name TEXT DEFAULT current_schema()
+)
+RETURNS TABLE (
+ table_name TEXT,
+ total_updates BIGINT,
+ hot_updates BIGINT,
+ hot_update_percentage NUMERIC,
+ matches_expected BOOLEAN
+)
+LANGUAGE plpgsql
+AS $$
+DECLARE
+ v_relid oid;
+ v_qualified_name TEXT;
+ v_hot_updates BIGINT;
+ v_updates BIGINT;
+ v_xact_hot_updates BIGINT;
+ v_xact_updates BIGINT;
+BEGIN
+ -- Force statistics update
+ PERFORM pg_stat_force_next_flush();
+
+ -- Get table OID
+ v_qualified_name := quote_ident(p_schema_name) || '.' || quote_ident(p_table_name);
+ v_relid := v_qualified_name::regclass;
+
+ IF v_relid IS NULL THEN
+ RAISE EXCEPTION 'Table %.% not found', p_schema_name, p_table_name;
+ END IF;
+
+ -- Get cumulative + transaction stats
+ v_hot_updates := COALESCE(pg_stat_get_tuples_hot_updated(v_relid), 0);
+ v_updates := COALESCE(pg_stat_get_tuples_updated(v_relid), 0);
+ v_xact_hot_updates := COALESCE(pg_stat_get_xact_tuples_hot_updated(v_relid), 0);
+ v_xact_updates := COALESCE(pg_stat_get_xact_tuples_updated(v_relid), 0);
+
+ v_hot_updates := v_hot_updates + v_xact_hot_updates;
+ v_updates := v_updates + v_xact_updates;
+
+ RETURN QUERY
+ SELECT
+ p_table_name::TEXT,
+ v_updates::BIGINT,
+ v_hot_updates::BIGINT,
+ CASE WHEN v_updates > 0
+ THEN ROUND((v_hot_updates::numeric / v_updates::numeric * 100)::numeric, 2)
+ ELSE 0
+ END,
+ (v_hot_updates = expected)::BOOLEAN;
+END;
+$$;
+
+-- ================================================================
+-- Basic JSONB Expression Index
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_name_idx ON t((docs->>'name'));
+INSERT INTO t VALUES (1, '{"name": "alice", "age": 30}');
+
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET docs = '{"name": "alice", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update indexed JSONB field - should NOT be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update non-indexed field again - should be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 32}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Partial Index with Predicate Transitions
+-- ================================================================
+CREATE TABLE t(id INT, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_value_idx ON t(value) WHERE value > 10;
+INSERT INTO t VALUES (1, 5);
+
+-- Both outside predicate - should be HOT
+UPDATE t SET value = 8 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET value = 15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Both inside predicate, value changes - should NOT be HOT
+UPDATE t SET value = 20 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Transition out of predicate - should NOT be HOT
+UPDATE t SET value = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Both outside predicate again - should be HOT
+UPDATE t SET value = 3 WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Expression Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((docs->>'status'))
+ WHERE (docs->>'priority')::int > 5;
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3}');
+
+-- Both outside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 4}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 10}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Inside predicate, status changes - should NOT be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 10}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Inside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 8}';
+SELECT * FROM check_hot_updates(2, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN Index on JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data);
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "database"]}');
+
+-- Change tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+-- Change tags again - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+-- Add field without changing existing keys - GIN keys changed (added "note"), NOT HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "note": "test"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN Index with Unchanged Keys
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create GIN index on specific path
+CREATE INDEX t_gin_idx ON t USING gin((data->'tags'));
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "sql"], "status": "active"}');
+
+-- Change non-indexed field - GIN keys on 'tags' unchanged, should be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change indexed tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN with jsonb_path_ops
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data jsonb_path_ops);
+INSERT INTO t VALUES (1, '{"user": {"name": "alice"}, "tags": ["a", "b"]}');
+
+-- Change value at different path - keys changed, NOT HOT
+UPDATE t SET data = '{"user": {"name": "bob"}, "tags": ["a", "b"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Multi-Column Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, a INT, b INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t(id, abs(a), abs(b));
+INSERT INTO t VALUES (1, -5, -10);
+
+-- Change sign but not abs value - should be HOT
+UPDATE t SET a = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change abs value - should NOT be HOT
+UPDATE t SET b = -15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change id - should NOT be HOT
+UPDATE t SET id = 2 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Mixed Index Types (BRIN + Expression)
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_idx ON t USING brin(value);
+CREATE INDEX t_expr_idx ON t((data->>'status'));
+INSERT INTO t VALUES (1, 100, '{"status": "active"}');
+
+-- Update only BRIN column - should be HOT
+UPDATE t SET value = 200 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update only expression column - should NOT be HOT
+UPDATE t SET data = '{"status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update both - should NOT be HOT
+UPDATE t SET value = 300, data = '{"status": "pending"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Expression with COLLATION and BTREE (nbtree) index
+-- ================================================================
+CREATE COLLATION case_insensitive (
+ provider = libc,
+ locale = 'C'
+);
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ name TEXT COLLATE case_insensitive
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+
+CREATE INDEX t_lower_idx ON t USING BTREE (name COLLATE case_insensitive);
+
+INSERT INTO t VALUES (1, 'ALICE');
+
+-- Change case but not value - should NOT be HOT in BTREE
+UPDATE t SET name = 'Alice' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+-- Change to new value - should NOT be HOT
+UPDATE t SET name = 'BOB' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Array Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_array_len_idx ON t(array_length(tags, 1));
+INSERT INTO t VALUES (1, ARRAY['a', 'b', 'c']);
+
+-- Same length, different elements - should be HOT
+UPDATE t SET tags = ARRAY['d', 'e', 'f'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Different length - should NOT be HOT
+UPDATE t SET tags = ARRAY['d', 'e'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Nested JSONB Expression and JSONB equality '->' (not '->>')
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_nested_idx ON t((data->'user'->'name'));
+INSERT INTO t VALUES ('{"user": {"name": "alice", "age": 30}}');
+
+-- Change nested non-indexed field - should be HOT
+UPDATE t SET data = '{"user": {"name": "alice", "age": 31}}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change nested indexed field - should NOT be HOT
+UPDATE t SET data = '{"user": {"name": "bob", "age": 31}}';
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Complex Predicate on Multiple JSONB Fields
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((data->>'status'))
+ WHERE (data->>'priority')::int > 5
+ AND (data->>'active')::boolean = true;
+
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3, "active": true}');
+
+-- Outside predicate (priority too low) - should be HOT
+UPDATE t SET data = '{"status": "done", "priority": 3, "active": true}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": true}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Inside predicate, change to outside (active = false) - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": false}';
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN Array Index - Order Insensitive Extraction
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ data JSONB
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+
+-- GIN index on JSONB array (extracts all elements)
+CREATE INDEX t_items_gin ON t USING GIN ((data->'items'));
+
+INSERT INTO t VALUES (1, '{"items": [1, 2, 3], "status": "active"}');
+
+-- Update: Reorder array elements
+-- JSONB equality: NOT equal (different arrays)
+-- GIN extraction: Same elements extracted (might allow HOT if not careful)
+UPDATE t SET data = '{"items": [3, 2, 1], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update: Add/remove element
+UPDATE t SET data = '{"items": [1, 2, 3, 4], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- TOASTed Values in Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, large_text TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_substr_idx ON t(substr(large_text, 1, 10));
+
+INSERT INTO t VALUES (1, repeat('x', 5000) || 'identifier');
+
+-- Change end of string, prefix unchanged - should be HOT
+UPDATE t SET large_text = repeat('x', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change prefix - should NOT be HOT
+UPDATE t SET large_text = repeat('y', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- TEST: GIN with TOASTed TEXT (tsvector)
+-- ================================================================
+CREATE TABLE t(id INT, content TEXT, search_vec tsvector)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+-- Create trigger to maintain tsvector
+CREATE TRIGGER tsvectorupdate_toast
+ BEFORE INSERT OR UPDATE ON t
+ FOR EACH ROW EXECUTE FUNCTION
+ tsvector_update_trigger(search_vec, 'pg_catalog.english', content);
+
+CREATE INDEX t_gin ON t USING gin(search_vec);
+
+-- Insert with large content (will be TOASTed)
+INSERT INTO t (id, content) VALUES
+ (1, repeat('important keyword ', 1000) || repeat('filler text ', 10000));
+
+-- Verify initial state
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('important');
+-- Expected: 1 row
+
+-- IMPORTANT: The BEFORE UPDATE trigger modifies search_vec, so by the time
+-- ExecWhichIndexesRequireUpdates() runs, search_vec has already changed.
+-- This means the comparison sees old tsvector vs. trigger-modified tsvector,
+-- not the natural progression. HOT won't happen because the trigger changed
+-- the indexed column.
+
+-- Update: Even though content keywords unchanged, trigger still fires
+UPDATE t
+SET content = repeat('important keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+-- Expected: 0 HOT (trigger modifies search_vec, blocking HOT)
+-- This is actually correct behavior - the trigger updated an indexed column
+
+-- Update: Change indexed keywords
+UPDATE t
+SET content = repeat('critical keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+-- Expected: 0 HOT (index keys changed)
+
+-- Verify query correctness
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('critical');
+-- Expected: 1 row
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- TEST: GIN with TOASTed JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin((data->'tags'));
+
+-- Insert with TOASTed JSONB
+INSERT INTO t (id, data) VALUES
+ (1, jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('x', 10000)
+ ));
+
+-- Update: Change large_field, tags unchanged - should be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: 1 HOT update
+
+-- Update: Change tags - should NOT be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "sql"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: Still 1 HOT
+
+-- Verify correctness
+SELECT count(*) FROM t WHERE data->'tags' @> '["database"]'::jsonb;
+-- Expected: 0 rows
+SELECT count(*) FROM t WHERE data->'tags' @> '["sql"]'::jsonb;
+-- Expected: 1 row
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- TEST: GIN with Array of Large Strings
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin(tags);
+
+-- Insert with large array elements (might be TOASTed)
+INSERT INTO t (id, tags) VALUES
+ (1, ARRAY[repeat('tag1', 1000), repeat('tag2', 1000)]);
+
+-- Update: Change to different large values - NOT HOT
+UPDATE t
+SET tags = ARRAY[repeat('tag3', 1000), repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+-- Expected: 0 HOT (keys actually changed)
+
+-- Update: Keep same tag values, just reorder - SHOULD BE HOT
+-- (GIN is order-insensitive: both [tag3,tag4] and [tag4,tag3]
+-- extract to the same sorted key set ['tag3','tag4'])
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000), repeat('tag3', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: 1 HOT (GIN keys semantically identical)
+
+-- Update: Remove an element - NOT HOT (keys changed)
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: Still 1 HOT (not this one)
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- BRIN Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ value INT,
+ description TEXT
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+
+CREATE INDEX t_brin_partial_idx ON t USING brin(value) WHERE value > 100;
+
+INSERT INTO t VALUES (1, 50, 'below range');
+
+-- Test 1: Outside predicate
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Validate: Predicate query returns 0 rows
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+
+-- Test 2: Transition into predicate
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+
+-- Validate: Predicate query returns 1 row with correct value
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+
+-- Test 3: Inside predicate, value changes
+UPDATE t SET value = 160, description = 'updated again' WHERE id = 1;
+SELECT * FROM check_hot_updates(3, 't');
+
+-- Validate: Updated value (160) is returned
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+
+-- Test 4: Transition out of predicate
+UPDATE t SET value = 50 WHERE id = 1;
+SELECT * FROM check_hot_updates(4, 't');
+
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+
+SELECT id, value, description FROM t;
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- HASH Index (Simple Column)
+-- ================================================================
+CREATE TABLE t(id INT, code VARCHAR(20), description TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_idx ON t USING hash(code);
+INSERT INTO t VALUES (1, 'CODE001', 'initial');
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update indexed column - HASH index requires update, NOT HOT
+UPDATE t SET code = 'CODE002' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update both - NOT HOT
+UPDATE t SET code = 'CODE003', description = 'changed' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Back to original code - NOT HOT (different hash bucket location)
+UPDATE t SET code = 'CODE001' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- HASH Index on Expression
+-- ================================================================
+CREATE TABLE t(id INT, email TEXT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_lower_email_idx ON t USING HASH(lower(email));
+INSERT INTO t VALUES (1, '[email protected]', '{"status": "new"}');
+
+-- Update non-indexed field - should be HOT
+UPDATE t SET data = '{"status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update email with case change only (same lowercase) - should be HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+
+-- Update email to different lowercase - NOT HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- HASH Index on JSONB Field
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash((data->>'category'));
+INSERT INTO t VALUES (1, '{"category": "books", "title": "PostgreSQL Guide"}');
+
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET data = '{"category": "books", "title": "PostgreSQL Handbook"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update indexed JSONB field - NOT HOT
+UPDATE t SET data = '{"category": "videos", "title": "PostgreSQL Handbook"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update both - NOT HOT
+UPDATE t SET data = '{"category": "courses", "title": "PostgreSQL Basics"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- Multiple HASH Indexes
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, status VARCHAR, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+CREATE INDEX t_hash_status_idx ON t USING hash(status);
+INSERT INTO t VALUES (1, 'electronics', 'active', 100);
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update one indexed column - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update other indexed column - NOT HOT
+UPDATE t SET status = 'inactive' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update both indexed columns - NOT HOT
+UPDATE t SET category = 'videos', status = 'pending' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- BRIN vs HASH Comparison
+-- ================================================================
+CREATE TABLE t_brin(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE TABLE t_hash(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+CREATE INDEX t_brin_value_idx ON t_brin USING brin(value);
+CREATE INDEX t_hash_value_idx ON t_hash USING hash(value);
+
+INSERT INTO t_brin VALUES (1, 100, 'initial');
+INSERT INTO t_hash VALUES (1, 100, 'initial');
+
+-- Same update on both - different HOT behavior expected
+-- BRIN: might allow HOT (range summary unchanged)
+-- HASH: blocks HOT (hash bucket changed)
+UPDATE t_brin SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't_brin');
+-- Expected: 1 HOT (BRIN allows it for single row)
+
+UPDATE t_hash SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't_hash');
+-- Expected: 0 HOT (HASH blocks it)
+
+DROP TABLE t_brin CASCADE;
+DROP TABLE t_hash CASCADE;
+
+-- ================================================================
+-- HASH Index with NULL Values
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'electronics', 'initial');
+
+-- Update indexed column to NULL - NOT HOT (hash value changed)
+UPDATE t SET category = NULL WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+-- Expected: 0 HOT
+
+-- Update indexed column from NULL to value - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+-- Expected: 0 HOT
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: 1 HOT
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- BRIN on JSONB Field
+-- ================================================================
+CREATE TABLE t(id INT, metrics JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- BRIN doesn't directly support JSONB, but we can test on expression
+CREATE INDEX t_brin_count_idx ON t USING brin(
+ CAST(metrics->>'count' AS INTEGER)
+);
+INSERT INTO t VALUES (1, '{"count": "100", "timestamp": "2024-01-01"}');
+
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET metrics = '{"count": "100", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: 1 HOT
+
+-- Update indexed field - BRIN allows HOT for single row
+UPDATE t SET metrics = '{"count": "150", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+-- Expected: 2 HOT (BRIN permits single-row updates)
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- Mixed BRIN + HASH on Same Table
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, timestamp TIMESTAMP, price NUMERIC, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_timestamp_idx ON t USING brin(timestamp);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'books', '2024-01-01 10:00:00', 29.99, 'initial');
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: 1 HOT
+
+-- Update BRIN indexed column - allows HOT
+UPDATE t SET timestamp = '2024-01-02 10:00:00' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+-- Expected: 2 HOT
+
+-- Update HASH indexed column - blocks HOT
+UPDATE t SET category = 'videos' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+-- Expected: 2 HOT (HASH blocks it)
+
+-- Update price (non-indexed) - should be HOT
+UPDATE t SET price = 39.99 WHERE id = 1;
+SELECT * FROM check_hot_updates(3, 't');
+-- Expected: 3 HOT
+
+DROP TABLE t CASCADE;
+
+-- Cleanup
+DROP FUNCTION check_hot_updates(int, text, text);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 23bce72ae64..52ef8f10b35 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -390,6 +390,7 @@ CachedFunctionCompileCallback
CachedFunctionDeleteCallback
CachedFunctionHashEntry
CachedFunctionHashKey
+CachedIndexDatum
CachedPlan
CachedPlanSource
CallContext
--
2.49.0
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-07 21:36 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-09 19:27 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2025-11-16 18:53 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-11-19 18:00 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
@ 2025-11-21 15:25 ` Matthias van de Meent <[email protected]>
2025-11-22 21:30 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
1 sibling, 1 reply; 37+ messages in thread
From: Matthias van de Meent @ 2025-11-21 15:25 UTC (permalink / raw)
To: Greg Burd <[email protected]>; +Cc: pgsql-hackers
On Wed, 19 Nov 2025 at 19:00, Greg Burd <[email protected]> wrote:
>
> Attached are rebased (d5b4f3a6d4e) patches with the only changes
> happening in the last patch in the series.
Here's a high-level review of the patchset, extending on what I shared
offline. I haven't looked too closely at the code changes.
Re: Perf testing
Apart from the workload we've discussed offline, there's another
workload to consider: Right now, we only really consider HOT when we
know there's space on the page. This patch, however, will front-load a
lot more checks before we have access to the page, and that will
~always impact update performance.
I'm a bit worried that the cost of off-page updates (when the page of
the old tuple can't fit the new tuple) will be too significantly
increased, especially considering that we have a default fillfactor of
100 -- if a table's tuples only grow, it's quite likely the table
frequently can't apply HOT regardless of the updated columns. So, a
workload that's tuned to only update tuples in a way that excercises
'can we HOT or not' code for already-full pages would be appreciated.
A solution to the issue might be needed if we lose too much
performance on expensive checks; a solution like passing page space of
the old tuple to the checks, and short-circuiting the non-HOT path if
that page space is too small for the new tuple.
0001:
I'm not sure I understand why the code is near completely duplicated
here. Maybe you can rename the resulting heap_update to
heap_update_ext, and keep a heap_update around which wraps this
heap_update_ext, allowing old callers to keep their signature until
they need to use _ext's features? Then you can introduce the
duplication if and when needed in later patches -- though I don't
expect a lot of this duplication to be strictly necessary, though that
may need some new helper functions.
I also see that for every update we're now copying, passing 5
bitmapsets individually and then freeing those bitmaps just a moment
later. I'd like to avoid that overhead and duplication if possible.
Maybe we can store these in an 'update context' struct, passed by
reference down to table_tuple_update() from the calling code, and then
onward to heap_update? That might then also be a prime candidate to
contain the EState * of ExecCheckIndexedAttrsForChanges.
0002:
This patch seems to have some formatting updates to changes you made
in 0001, without actually changing the code (e.g. at heap_update's
definition). When updating code, please put it in the expected
formatting in the same patch.
---
In the patch subject:
> For instance,
> indexes with collation information allowing more HOT updates when the
> index is specified to be case insensitive.
It is incorrect to assume that indexed "btree-equal" datums allow HOT
updates. The user can not be returned an old datum in index-only
scans, even if it's sorted the same as the new datum -- after all, a
function on the datum may return different results even if the datums
are otherwise equal. Think: count_uppercase(string). See also below at
"HOT, datum compare, etc".
---
With the addition of rd_indexedattr we now have 6 bitmaps in
RelationData, which generally only get accessed through
RelationGetIndexAttrBitmap by an enum value. Maybe it's now time to
bite the bullet and change that to a more general approach with
`Bitmapset *rd_bitmaps[NUM_INDEX_ATTR_BITMAP]`? That way, the hot
path in RelationGetIndexAttrBitmap would not depend on the compiler to
determine that the fast path can do simple offset arithmatic to get
the requested bitmap.
0003:
This looks like it's a cleanup patch for 0002, and doesn't have much
standing on its own. Maybe the changes for ri_ChangedIndexedCols can
all be moved into 0003? I think that gives this patch more weight and
standing.
0004:
This does two things:
1. Add index expression evaluation to the toolset to determine which
indexes were unchanged, and
2. Allow index access methods to say "this value has not changed"
even if the datum itself may have changed.
Could that be split up into two different patches?
(aside: 2 makes a lot of sense in some cases, like trgm indexes
strings if no new trigrams are added/removed, so I really like the
idea behind this change)
HOT, datum compare, etc.:
Note that for index-only scans on an index to return correct results,
you _must_ update the index (and thus, do a non-HOT update) whenever a
value changes its binary datum, even if the value has the same btree
sort location as the old value. Even for non-IOS-supporting indexes,
the index may need more information than what's used in btree
comparisons when it has a btree opclass.
As SP/GIST have IOS support, they also need to compare the image of
the datum and not use ordinary equality as defined in nbtree's compare
function: the value must be exactly equal to what the table AM
would've provided.
Primary example: Btree compares the `numeric` datums of `1.0` and
`1.00` as equal, but for a user there is an observable difference; the
following SQL must return `1.0` in every valid plan:
BEGIN;
INSERT INTO mytab (mynumeric) VALUES ('1.00');
UPDATE mytab SET mynumeric = '1.0';
SELECT mynumeric FROM mytab;
So, IMO, the default datum compare in ExecCheckIndexedAttrsForChanges
and friends should just use datumIsEqual, and not this new
tts_attr_equal.
Indexes without IOS support might be able to opt into using a more lax
datum comparator, but 1.) it should never be the default, as it'd be a
loaded footgun for IndexAM implementers, and 2.) should not depend on
another AM's understanding of attributes, as that is a very leaky
abstraction.
Kind regards,
Matthias van de Meent
Databricks (https://www.databricks.com)
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-07 21:36 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-09 19:27 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2025-11-16 18:53 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-11-19 18:00 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-11-21 15:25 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
@ 2025-11-22 21:30 ` Greg Burd <[email protected]>
2025-11-24 18:59 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Greg Burd @ 2025-11-22 21:30 UTC (permalink / raw)
To: Matthias van de Meent <[email protected]>; +Cc: pgsql-hackers
On Nov 21 2025, at 10:25 am, Matthias van de Meent
<[email protected]> wrote:
> On Wed, 19 Nov 2025 at 19:00, Greg Burd <[email protected]> wrote:
>>
>> Attached are rebased (d5b4f3a6d4e) patches with the only changes
>> happening in the last patch in the series.
>
> Here's a high-level review of the patchset, extending on what I shared
> offline. I haven't looked too closely at the code changes.
Matthias, thanks for spending a bit of time writing up your thoughts and
for chatting a bit with me before doing so. I really appreciate your
point of view.
> Re: Perf testing
>
> Apart from the workload we've discussed offline, there's another
> workload to consider: Right now, we only really consider HOT when we
> know there's space on the page.
Yes, and no. In master every UPDATE triggers a call into
HeapDetermineColumnsInfo() in heap_update(). That function's job is to
examine all the indexed attributes checking each attribute for changes.
The set of indexed attributes is generated by combining a set of
Bitmapsets fetched (copied) from the relcache:
hot_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_HOT_BLOCKING);
sum_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_SUMMARIZED);
key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
id_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_IDENTITY_KEY);
interesting_attrs = NULL;
interesting_attrs = bms_add_members(interesting_attrs, hot_attrs);
interesting_attrs = bms_add_members(interesting_attrs, sum_attrs);
interesting_attrs = bms_add_members(interesting_attrs, key_attrs);
interesting_attrs = bms_add_members(interesting_attrs, id_attrs);
These "interesting attributes" are each checked comparing the newly
formed updated HeapTuple passed in from the executor against a HeapTuple
read from the page. This happens after taking the buffer lock on that
page. The comparison is done with heap_attr_compare() which calls
datumIsEqual() which boils down to a memcmp(). The function returns the
"modified_attrs".
This is the primary "work" that happens before considering HOT that I've
moved outside the buffer lock and into the executor. The rest of the
HOT decision is 1) will it fit (possibly after being TOASTed), and b) do
the HOT blocking attributes overlap with the modified_attrs? If they
do, unless they are all summarizing you can't go HOT.
Net new work with my approach is only related to the more complicated
checks that have to be done when there are expressions, partial indexes,
or an index implements the new index AM API for comparing datums. In
those cases there is a bit more work, especially when it comes to
expression indexes. Evaluating partial index predicate twice rather
than once is a bit more overhead. Using type-specific comparison when
the index doesn't support index-only scans is a tad more. Overall, not
a whole lot of new work. Certainly a much more complex code path, and
maybe that'll show up in performance tests. I don't know yet.
> This patch, however, will front-load a
> lot more checks before we have access to the page, and that will
> ~always impact update performance.
Disagree, 90% of that work happens today on that same path after the
page lock. Even in the case that the HeapTuple can't fit into the page
that work will happen, there's not much net new "wasted" effort.
Concurrency under load may be better because buffer locks will be held
for less time.
> I'm a bit worried that the cost of off-page updates (when the page of
> the old tuple can't fit the new tuple) will be too significantly
> increased, especially considering that we have a default fillfactor of
> 100 -- if a table's tuples only grow, it's quite likely the table
> frequently can't apply HOT regardless of the updated columns. So, a
> workload that's tuned to only update tuples in a way that excercises
> 'can we HOT or not' code for already-full pages would be appreciated.
> A solution to the issue might be needed if we lose too much
> performance on expensive checks; a solution like passing page space of
> the old tuple to the checks, and short-circuiting the non-HOT path if
> that page space is too small for the new tuple.
Here's the problem, there's not much that can be known about what
ultimate size the HeapTuple will take or if the page can hold it until
after the lock.
Second, I feel that this signal would be specific to the heap, it's
particular MVCC implementation, and it's optimization (HOT). I really
wanted this solution to be non-heap-specific, but heap-enabling.
For me that meant that in the general case the executor should be
concerned with updating only the set of indexes that it must and no
more. So performing this work in the executor ahead of calling into the
table AM or index AM makes sense.
It is only due to heap's "special" model that we have other concerns
related to HOT. Sure, everyone/everything today uses heap so we should
pay attention to this, but I set out not to create yet another thing
that depends on heap's specific operational model. I think I did that.
> 0001:
>
> I'm not sure I understand why the code is near completely duplicated
> here. Maybe you can rename the resulting heap_update to
> heap_update_ext, and keep a heap_update around which wraps this
> heap_update_ext, allowing old callers to keep their signature until
> they need to use _ext's features? Then you can introduce the
> duplication if and when needed in later patches -- though I don't
> expect a lot of this duplication to be strictly necessary, though that
> may need some new helper functions.
This is just a split where I move the top portion of heap_update() into
the two paths that use it. Sure I could have pulled that into another
function, but in this series the next step is to obliterate one half and
(if I get my other patch in cf-6221) then I can completely remove
HeapDetermineColumnsInfo() and vastly simplify simple_heap_update().
> I also see that for every update we're now copying, passing 5
> bitmapsets individually and then freeing those bitmaps just a moment
> later. I'd like to avoid that overhead and duplication if possible.
We do this today, nothing new here. They are passed by reference, not
value into heap_update().
> Maybe we can store these in an 'update context' struct, passed by
> reference down to table_tuple_update() from the calling code, and then
> onward to heap_update? That might then also be a prime candidate to
> contain the EState * of ExecCheckIndexedAttrsForChanges.
I've explored using UpdateContext before, not a bad idea but again this
is just a setup commit. It could be cleaner on it's own, but it doesn't
really take a step backward on any dimension.
> 0002:
>
> This patch seems to have some formatting updates to changes you made
> in 0001, without actually changing the code (e.g. at heap_update's
> definition). When updating code, please put it in the expected
> formatting in the same patch.
I'll find/fix those after v23 attached, sorry for the noise.
> ---
> In the patch subject:
>> For instance,
>> indexes with collation information allowing more HOT updates when the
>> index is specified to be case insensitive.
>
> It is incorrect to assume that indexed "btree-equal" datums allow HOT
> updates. The user can not be returned an old datum in index-only
> scans, even if it's sorted the same as the new datum -- after all, a
> function on the datum may return different results even if the datums
> are otherwise equal. Think: count_uppercase(string). See also below at
> "HOT, datum compare, etc".
Thanks for pointing out the oversight for index-oriented scans (IOS),
you're right that the code in v22 doesn't handle that correctly. I'll
fix that. I still think that indexes that don't support IOS can and
should use the type-specific equality checks. This opens the door to
HOT with custom types that have unusual equality rules (see BSON).
> With the addition of rd_indexedattr we now have 6 bitmaps in
> RelationData, which generally only get accessed through
> RelationGetIndexAttrBitmap by an enum value. Maybe it's now time to
> bite the bullet and change that to a more general approach with
> `Bitmapset *rd_bitmaps[NUM_INDEX_ATTR_BITMAP]`? That way, the hot
> path in RelationGetIndexAttrBitmap would not depend on the compiler to
> determine that the fast path can do simple offset arithmatic to get
> the requested bitmap.
More than a few of those bitmaps in RelationData are purely for the HOT
tests, yes. I'll review those and see if I can shrink the set
meaningfully. It was something that had occurred to me too. I'll
review and consider ways to consolidate them.
> 0003:
> This looks like it's a cleanup patch for 0002, and doesn't have much
> standing on its own. Maybe the changes for ri_ChangedIndexedCols can
> all be moved into 0003? I think that gives this patch more weight and
> standing.
The goal in this patch was to show that we could eliminate the redundant
set work of index_unchanged_by_update() in execIndexing's update path.
Separating it out was done to make it more easily reviewed and to prove
that before/after tests passed making the change safe.
> 0004:
> This does two things:
> 1. Add index expression evaluation to the toolset to determine which
> indexes were unchanged, and
> 2. Allow index access methods to say "this value has not changed"
> even if the datum itself may have changed.
Yes, that's what it does. :)
> Could that be split up into two different patches?
Maybe, I might be able to add the index AM piece first and then the
expression piece.
> (aside: 2 makes a lot of sense in some cases, like trgm indexes
> strings if no new trigrams are added/removed, so I really like the
> idea behind this change).
Nice, I appreciate that.
> HOT, datum compare, etc.:
>
> Note that for index-only scans on an index to return correct results,
> you _must_ update the index (and thus, do a non-HOT update) whenever a
> value changes its binary datum, even if the value has the same btree
> sort location as the old value.
Yes, I get this now. I also wasn't testing the INCLUDING (non-key)
columns. I've fixed both of those in v23.
> Even for non-IOS-supporting indexes,
> the index may need more information than what's used in btree
> comparisons when it has a btree opclass.
It's not always just the btree opclass for equality, but that is common.
> As SP/GIST have IOS support, they also need to compare the image of
> the datum and not use ordinary equality as defined in nbtree's compare
> function: the value must be exactly equal to what the table AM
> would've provided.
In v23 I've changed the logic to use datumIsEqual() for any index that
supports IOS and doesn't supply a custom amcomparedatums() function.
> Primary example: Btree compares the `numeric` datums of `1.0` and
> `1.00` as equal, but for a user there is an observable difference; the
> following SQL must return `1.0` in every valid plan:
>
> BEGIN;
> INSERT INTO mytab (mynumeric) VALUES ('1.00');
> UPDATE mytab SET mynumeric = '1.0';
> SELECT mynumeric FROM mytab;
I'll try to reproduce this and add a test if I can.
> So, IMO, the default datum compare in ExecCheckIndexedAttrsForChanges
> and friends should just use datumIsEqual, and not this new
> tts_attr_equal.
Sure, and that's the case in v23. tts_attr_equal() still has value in
other cases so it's not gone.
> Indexes without IOS support might be able to opt into using a more lax
> datum comparator, but 1.) it should never be the default, as it'd be a
> loaded footgun for IndexAM implementers, and 2.) should not depend on
> another AM's understanding of attributes, as that is a very leaky
> abstraction.
Agree, which is why the default for IOS indexes is datumIsEqual() now.
> Kind regards,
>
> Matthias van de Meent
> Databricks (https://www.databricks.com)
Thanks again for the time and renewed interest in the patch! I've also
added a lot more tests into the heap_hot_updates.sql regression suite,
likely too many, but for now it's good to be testing the corners.
v23 attached, changes are all in 0004, best.
-greg
Attachments:
[application/octet-stream] v23-0001-Reorganize-heap-update-logic.patch (47.6K, 2-v23-0001-Reorganize-heap-update-logic.patch)
download | inline diff:
From 1acc8057e389b32e55915336d88070958a0ddd8c Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 2 Nov 2025 11:36:20 -0500
Subject: [PATCH v23 1/4] Reorganize heap update logic
This commit refactors the interaction between heap_tuple_update(),
heap_update(), and simple_heap_update() to improve code organization
and flexibility. The changes are functionally equivalent to the
previous implementation and have no performance impact.
The primary motivation is to prepare for upcoming modifications to
how and where modified attributes are identified during the update
path, particularly for catalog updates.
As part of this reorganization, the handling of replica identity key
attributes has been adjusted. Instead of fetching a second copy of
the bitmap during an update operation, the caller is now required to
provide it. This change applies to both heap_update() and
heap_delete().
No user-visible changes.
---
src/backend/access/heap/heapam.c | 568 +++++++++++------------
src/backend/access/heap/heapam_handler.c | 117 ++++-
src/include/access/heapam.h | 24 +-
3 files changed, 410 insertions(+), 299 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 4b0c49f4bb0..aff47481345 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -39,18 +39,24 @@
#include "access/syncscan.h"
#include "access/valid.h"
#include "access/visibilitymap.h"
+#include "access/xact.h"
#include "access/xloginsert.h"
+#include "catalog/catalog.h"
#include "catalog/pg_database.h"
#include "catalog/pg_database_d.h"
#include "commands/vacuum.h"
+#include "nodes/bitmapset.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
+#include "storage/bufmgr.h"
+#include "storage/itemptr.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
#include "storage/procarray.h"
#include "utils/datum.h"
#include "utils/injection_point.h"
#include "utils/inval.h"
+#include "utils/relcache.h"
#include "utils/spccache.h"
#include "utils/syscache.h"
@@ -62,16 +68,8 @@ static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
HeapTuple newtup, HeapTuple old_key_tuple,
bool all_visible_cleared, bool new_all_visible_cleared);
#ifdef USE_ASSERT_CHECKING
-static void check_lock_if_inplace_updateable_rel(Relation relation,
- const ItemPointerData *otid,
- HeapTuple newtup);
static void check_inplace_rel_lock(HeapTuple oldtup);
#endif
-static Bitmapset *HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external);
static bool heap_acquire_tuplock(Relation relation, const ItemPointerData *tid,
LockTupleMode mode, LockWaitPolicy wait_policy,
bool *have_tuple_lock);
@@ -103,10 +101,10 @@ static bool ConditionalMultiXactIdWait(MultiXactId multi, MultiXactStatus status
static void index_delete_sort(TM_IndexDeleteOp *delstate);
static int bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate);
static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
-static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
+static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp,
+ Bitmapset *rid_attrs, bool key_required,
bool *copy);
-
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
* corresponding MultiXactStatuses (one to merely lock tuples, another one to
@@ -2799,6 +2797,7 @@ heap_delete(Relation relation, const ItemPointerData *tid,
Buffer buffer;
Buffer vmbuffer = InvalidBuffer;
TransactionId new_xmax;
+ Bitmapset *rid_attrs;
uint16 new_infomask,
new_infomask2;
bool have_tuple_lock = false;
@@ -2811,6 +2810,8 @@ heap_delete(Relation relation, const ItemPointerData *tid,
AssertHasSnapshotForToast(relation);
+ rid_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
/*
* Forbid this during a parallel operation, lest it allocate a combo CID.
* Other workers might need that combo CID for visibility checks, and we
@@ -3014,6 +3015,7 @@ l1:
UnlockTupleTuplock(relation, &(tp.t_self), LockTupleExclusive);
if (vmbuffer != InvalidBuffer)
ReleaseBuffer(vmbuffer);
+ bms_free(rid_attrs);
return result;
}
@@ -3035,7 +3037,10 @@ l1:
* Compute replica identity tuple before entering the critical section so
* we don't PANIC upon a memory allocation failure.
*/
- old_key_tuple = ExtractReplicaIdentity(relation, &tp, true, &old_key_copied);
+ old_key_tuple = ExtractReplicaIdentity(relation, &tp, rid_attrs,
+ true, &old_key_copied);
+ bms_free(rid_attrs);
+ rid_attrs = NULL;
/*
* If this is the first possibly-multixact-able operation in the current
@@ -3247,7 +3252,10 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* heap_update - replace a tuple
*
* See table_tuple_update() for an explanation of the parameters, except that
- * this routine directly takes a tuple rather than a slot.
+ * this routine directly takes a heap tuple rather than a slot.
+ *
+ * It's required that the caller has acquired the pin and lock on the buffer.
+ * That lock and pin will be managed here, not in the caller.
*
* In the failure cases, the routine fills *tmfd with the tuple's t_ctid,
* t_xmax (resolving a possible MultiXact, if necessary), and t_cmax (the last
@@ -3255,30 +3263,21 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* generated by another transaction).
*/
TM_Result
-heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+heap_update(Relation relation, HeapTupleData *oldtup,
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
+ Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
+ Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
+ Bitmapset *mix_attrs, Buffer *vmbuffer,
+ bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
- Bitmapset *hot_attrs;
- Bitmapset *sum_attrs;
- Bitmapset *key_attrs;
- Bitmapset *id_attrs;
- Bitmapset *interesting_attrs;
- Bitmapset *modified_attrs;
- ItemId lp;
- HeapTupleData oldtup;
HeapTuple heaptup;
HeapTuple old_key_tuple = NULL;
bool old_key_copied = false;
- Page page;
- BlockNumber block;
MultiXactStatus mxact_status;
- Buffer buffer,
- newbuf,
- vmbuffer = InvalidBuffer,
+ Buffer newbuf,
vmbuffer_new = InvalidBuffer;
bool need_toast;
Size newtupsize,
@@ -3292,7 +3291,6 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
bool all_visible_cleared_new = false;
bool checked_lockers;
bool locker_remains;
- bool id_has_external = false;
TransactionId xmax_new_tuple,
xmax_old_tuple;
uint16 infomask_old_tuple,
@@ -3300,144 +3298,13 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
infomask_new_tuple,
infomask2_new_tuple;
- Assert(ItemPointerIsValid(otid));
-
- /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
- Assert(HeapTupleHeaderGetNatts(newtup->t_data) <=
- RelationGetNumberOfAttributes(relation));
-
+ Assert(BufferIsLockedByMe(buffer));
+ Assert(ItemIdIsNormal(lp));
AssertHasSnapshotForToast(relation);
- /*
- * Forbid this during a parallel operation, lest it allocate a combo CID.
- * Other workers might need that combo CID for visibility checks, and we
- * have no provision for broadcasting it to them.
- */
- if (IsInParallelMode())
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
- errmsg("cannot update tuples during a parallel operation")));
-
-#ifdef USE_ASSERT_CHECKING
- check_lock_if_inplace_updateable_rel(relation, otid, newtup);
-#endif
-
- /*
- * Fetch the list of attributes to be checked for various operations.
- *
- * For HOT considerations, this is wasted effort if we fail to update or
- * have to put the new tuple on a different page. But we must compute the
- * list before obtaining buffer lock --- in the worst case, if we are
- * doing an update on one of the relevant system catalogs, we could
- * deadlock if we try to fetch the list later. In any case, the relcache
- * caches the data so this is usually pretty cheap.
- *
- * We also need columns used by the replica identity and columns that are
- * considered the "key" of rows in the table.
- *
- * Note that we get copies of each bitmap, so we need not worry about
- * relcache flush happening midway through.
- */
- hot_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_HOT_BLOCKING);
- sum_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_SUMMARIZED);
- key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
- id_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
- interesting_attrs = NULL;
- interesting_attrs = bms_add_members(interesting_attrs, hot_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, sum_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, key_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, id_attrs);
-
- block = ItemPointerGetBlockNumber(otid);
- INJECTION_POINT("heap_update-before-pin", NULL);
- buffer = ReadBuffer(relation, block);
- page = BufferGetPage(buffer);
-
- /*
- * Before locking the buffer, pin the visibility map page if it appears to
- * be necessary. Since we haven't got the lock yet, someone else might be
- * in the middle of changing this, so we'll need to recheck after we have
- * the lock.
- */
- if (PageIsAllVisible(page))
- visibilitymap_pin(relation, block, &vmbuffer);
-
- LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
-
- lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
-
- /*
- * Usually, a buffer pin and/or snapshot blocks pruning of otid, ensuring
- * we see LP_NORMAL here. When the otid origin is a syscache, we may have
- * neither a pin nor a snapshot. Hence, we may see other LP_ states, each
- * of which indicates concurrent pruning.
- *
- * Failing with TM_Updated would be most accurate. However, unlike other
- * TM_Updated scenarios, we don't know the successor ctid in LP_UNUSED and
- * LP_DEAD cases. While the distinction between TM_Updated and TM_Deleted
- * does matter to SQL statements UPDATE and MERGE, those SQL statements
- * hold a snapshot that ensures LP_NORMAL. Hence, the choice between
- * TM_Updated and TM_Deleted affects only the wording of error messages.
- * Settle on TM_Deleted, for two reasons. First, it avoids complicating
- * the specification of when tmfd->ctid is valid. Second, it creates
- * error log evidence that we took this branch.
- *
- * Since it's possible to see LP_UNUSED at otid, it's also possible to see
- * LP_NORMAL for a tuple that replaced LP_UNUSED. If it's a tuple for an
- * unrelated row, we'll fail with "duplicate key value violates unique".
- * XXX if otid is the live, newer version of the newtup row, we'll discard
- * changes originating in versions of this catalog row after the version
- * the caller got from syscache. See syscache-update-pruned.spec.
- */
- if (!ItemIdIsNormal(lp))
- {
- Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
-
- UnlockReleaseBuffer(buffer);
- Assert(!have_tuple_lock);
- if (vmbuffer != InvalidBuffer)
- ReleaseBuffer(vmbuffer);
- tmfd->ctid = *otid;
- tmfd->xmax = InvalidTransactionId;
- tmfd->cmax = InvalidCommandId;
- *update_indexes = TU_None;
-
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- /* modified_attrs not yet initialized */
- bms_free(interesting_attrs);
- return TM_Deleted;
- }
-
- /*
- * Fill in enough data in oldtup for HeapDetermineColumnsInfo to work
- * properly.
- */
- oldtup.t_tableOid = RelationGetRelid(relation);
- oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
- oldtup.t_len = ItemIdGetLength(lp);
- oldtup.t_self = *otid;
-
- /* the new tuple is ready, except for this: */
+ /* The new tuple is ready, except for this */
newtup->t_tableOid = RelationGetRelid(relation);
- /*
- * Determine columns modified by the update. Additionally, identify
- * whether any of the unmodified replica identity key attributes in the
- * old tuple is externally stored or not. This is required because for
- * such attributes the flattened value won't be WAL logged as part of the
- * new tuple so we must include it as part of the old_key_tuple. See
- * ExtractReplicaIdentity.
- */
- modified_attrs = HeapDetermineColumnsInfo(relation, interesting_attrs,
- id_attrs, &oldtup,
- newtup, &id_has_external);
-
/*
* If we're not updating any "key" column, we can grab a weaker lock type.
* This allows for more concurrency when we are running simultaneously
@@ -3449,7 +3316,7 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
* is updates that don't manipulate key columns, not those that
* serendipitously arrive at the same key values.
*/
- if (!bms_overlap(modified_attrs, key_attrs))
+ if (!bms_overlap(mix_attrs, pk_attrs))
{
*lockmode = LockTupleNoKeyExclusive;
mxact_status = MultiXactStatusNoKeyUpdate;
@@ -3473,17 +3340,10 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
key_intact = false;
}
- /*
- * Note: beyond this point, use oldtup not otid to refer to old tuple.
- * otid may very well point at newtup->t_self, which we will overwrite
- * with the new tuple's location, so there's great risk of confusion if we
- * use otid anymore.
- */
-
l2:
checked_lockers = false;
locker_remains = false;
- result = HeapTupleSatisfiesUpdate(&oldtup, cid, buffer);
+ result = HeapTupleSatisfiesUpdate(oldtup, cid, buffer);
/* see below about the "no wait" case */
Assert(result != TM_BeingModified || wait);
@@ -3515,8 +3375,8 @@ l2:
*/
/* must copy state data before unlocking buffer */
- xwait = HeapTupleHeaderGetRawXmax(oldtup.t_data);
- infomask = oldtup.t_data->t_infomask;
+ xwait = HeapTupleHeaderGetRawXmax(oldtup->t_data);
+ infomask = oldtup->t_data->t_infomask;
/*
* Now we have to do something about the existing locker. If it's a
@@ -3556,13 +3416,12 @@ l2:
* requesting a lock and already have one; avoids deadlock).
*/
if (!current_is_member)
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &oldtup->t_self, *lockmode,
LockWaitBlock, &have_tuple_lock);
/* wait for multixact */
MultiXactIdWait((MultiXactId) xwait, mxact_status, infomask,
- relation, &oldtup.t_self, XLTW_Update,
- &remain);
+ relation, &oldtup->t_self, XLTW_Update, &remain);
checked_lockers = true;
locker_remains = remain != 0;
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
@@ -3572,9 +3431,9 @@ l2:
* could update this tuple before we get to this point. Check
* for xmax change, and start over if so.
*/
- if (xmax_infomask_changed(oldtup.t_data->t_infomask,
+ if (xmax_infomask_changed(oldtup->t_data->t_infomask,
infomask) ||
- !TransactionIdEquals(HeapTupleHeaderGetRawXmax(oldtup.t_data),
+ !TransactionIdEquals(HeapTupleHeaderGetRawXmax(oldtup->t_data),
xwait))
goto l2;
}
@@ -3599,8 +3458,8 @@ l2:
* before this one, which are important to keep in case this
* subxact aborts.
*/
- if (!HEAP_XMAX_IS_LOCKED_ONLY(oldtup.t_data->t_infomask))
- update_xact = HeapTupleGetUpdateXid(oldtup.t_data);
+ if (!HEAP_XMAX_IS_LOCKED_ONLY(oldtup->t_data->t_infomask))
+ update_xact = HeapTupleGetUpdateXid(oldtup->t_data);
else
update_xact = InvalidTransactionId;
@@ -3641,9 +3500,9 @@ l2:
* lock.
*/
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &oldtup->t_self, *lockmode,
LockWaitBlock, &have_tuple_lock);
- XactLockTableWait(xwait, relation, &oldtup.t_self,
+ XactLockTableWait(xwait, relation, &oldtup->t_self,
XLTW_Update);
checked_lockers = true;
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
@@ -3653,20 +3512,20 @@ l2:
* other xact could update this tuple before we get to this point.
* Check for xmax change, and start over if so.
*/
- if (xmax_infomask_changed(oldtup.t_data->t_infomask, infomask) ||
+ if (xmax_infomask_changed(oldtup->t_data->t_infomask, infomask) ||
!TransactionIdEquals(xwait,
- HeapTupleHeaderGetRawXmax(oldtup.t_data)))
+ HeapTupleHeaderGetRawXmax(oldtup->t_data)))
goto l2;
/* Otherwise check if it committed or aborted */
- UpdateXmaxHintBits(oldtup.t_data, buffer, xwait);
- if (oldtup.t_data->t_infomask & HEAP_XMAX_INVALID)
+ UpdateXmaxHintBits(oldtup->t_data, buffer, xwait);
+ if (oldtup->t_data->t_infomask & HEAP_XMAX_INVALID)
can_continue = true;
}
if (can_continue)
result = TM_Ok;
- else if (!ItemPointerEquals(&oldtup.t_self, &oldtup.t_data->t_ctid))
+ else if (!ItemPointerEquals(&oldtup->t_self, &oldtup->t_data->t_ctid))
result = TM_Updated;
else
result = TM_Deleted;
@@ -3679,39 +3538,33 @@ l2:
result == TM_Updated ||
result == TM_Deleted ||
result == TM_BeingModified);
- Assert(!(oldtup.t_data->t_infomask & HEAP_XMAX_INVALID));
+ Assert(!(oldtup->t_data->t_infomask & HEAP_XMAX_INVALID));
Assert(result != TM_Updated ||
- !ItemPointerEquals(&oldtup.t_self, &oldtup.t_data->t_ctid));
+ !ItemPointerEquals(&oldtup->t_self, &oldtup->t_data->t_ctid));
}
if (crosscheck != InvalidSnapshot && result == TM_Ok)
{
/* Perform additional check for transaction-snapshot mode RI updates */
- if (!HeapTupleSatisfiesVisibility(&oldtup, crosscheck, buffer))
+ if (!HeapTupleSatisfiesVisibility(oldtup, crosscheck, buffer))
result = TM_Updated;
}
if (result != TM_Ok)
{
- tmfd->ctid = oldtup.t_data->t_ctid;
- tmfd->xmax = HeapTupleHeaderGetUpdateXid(oldtup.t_data);
+ tmfd->ctid = oldtup->t_data->t_ctid;
+ tmfd->xmax = HeapTupleHeaderGetUpdateXid(oldtup->t_data);
if (result == TM_SelfModified)
- tmfd->cmax = HeapTupleHeaderGetCmax(oldtup.t_data);
+ tmfd->cmax = HeapTupleHeaderGetCmax(oldtup->t_data);
else
tmfd->cmax = InvalidCommandId;
UnlockReleaseBuffer(buffer);
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
- if (vmbuffer != InvalidBuffer)
- ReleaseBuffer(vmbuffer);
+ UnlockTupleTuplock(relation, &oldtup->t_self, *lockmode);
+ if (*vmbuffer != InvalidBuffer)
+ ReleaseBuffer(*vmbuffer);
*update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
return result;
}
@@ -3724,10 +3577,10 @@ l2:
* tuple has been locked or updated under us, but hopefully it won't
* happen very often.
*/
- if (vmbuffer == InvalidBuffer && PageIsAllVisible(page))
+ if (*vmbuffer == InvalidBuffer && PageIsAllVisible(page))
{
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- visibilitymap_pin(relation, block, &vmbuffer);
+ visibilitymap_pin(relation, block, vmbuffer);
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
goto l2;
}
@@ -3738,9 +3591,9 @@ l2:
* If the tuple we're updating is locked, we need to preserve the locking
* info in the old tuple's Xmax. Prepare a new Xmax value for this.
*/
- compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
- oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2,
+ compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup->t_data),
+ oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2,
xid, *lockmode, true,
&xmax_old_tuple, &infomask_old_tuple,
&infomask2_old_tuple);
@@ -3752,12 +3605,12 @@ l2:
* tuple. (In rare cases that might also be InvalidTransactionId and yet
* not have the HEAP_XMAX_INVALID bit set; that's fine.)
*/
- if ((oldtup.t_data->t_infomask & HEAP_XMAX_INVALID) ||
- HEAP_LOCKED_UPGRADED(oldtup.t_data->t_infomask) ||
+ if ((oldtup->t_data->t_infomask & HEAP_XMAX_INVALID) ||
+ HEAP_LOCKED_UPGRADED(oldtup->t_data->t_infomask) ||
(checked_lockers && !locker_remains))
xmax_new_tuple = InvalidTransactionId;
else
- xmax_new_tuple = HeapTupleHeaderGetRawXmax(oldtup.t_data);
+ xmax_new_tuple = HeapTupleHeaderGetRawXmax(oldtup->t_data);
if (!TransactionIdIsValid(xmax_new_tuple))
{
@@ -3772,7 +3625,7 @@ l2:
* Note that since we're doing an update, the only possibility is that
* the lockers had FOR KEY SHARE lock.
*/
- if (oldtup.t_data->t_infomask & HEAP_XMAX_IS_MULTI)
+ if (oldtup->t_data->t_infomask & HEAP_XMAX_IS_MULTI)
{
GetMultiXactIdHintBits(xmax_new_tuple, &infomask_new_tuple,
&infomask2_new_tuple);
@@ -3800,7 +3653,7 @@ l2:
* Replace cid with a combo CID if necessary. Note that we already put
* the plain cid into the new tuple.
*/
- HeapTupleHeaderAdjustCmax(oldtup.t_data, &cid, &iscombo);
+ HeapTupleHeaderAdjustCmax(oldtup->t_data, &cid, &iscombo);
/*
* If the toaster needs to be activated, OR if the new tuple will not fit
@@ -3817,12 +3670,12 @@ l2:
relation->rd_rel->relkind != RELKIND_MATVIEW)
{
/* toast table entries should never be recursively toasted */
- Assert(!HeapTupleHasExternal(&oldtup));
+ Assert(!HeapTupleHasExternal(oldtup));
Assert(!HeapTupleHasExternal(newtup));
need_toast = false;
}
else
- need_toast = (HeapTupleHasExternal(&oldtup) ||
+ need_toast = (HeapTupleHasExternal(oldtup) ||
HeapTupleHasExternal(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
@@ -3855,9 +3708,9 @@ l2:
* updating, because the potentially created multixact would otherwise
* be wrong.
*/
- compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
- oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2,
+ compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup->t_data),
+ oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2,
xid, *lockmode, false,
&xmax_lock_old_tuple, &infomask_lock_old_tuple,
&infomask2_lock_old_tuple);
@@ -3867,18 +3720,18 @@ l2:
START_CRIT_SECTION();
/* Clear obsolete visibility flags ... */
- oldtup.t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
- oldtup.t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
- HeapTupleClearHotUpdated(&oldtup);
+ oldtup->t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
+ oldtup->t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
+ HeapTupleClearHotUpdated(oldtup);
/* ... and store info about transaction updating this tuple */
Assert(TransactionIdIsValid(xmax_lock_old_tuple));
- HeapTupleHeaderSetXmax(oldtup.t_data, xmax_lock_old_tuple);
- oldtup.t_data->t_infomask |= infomask_lock_old_tuple;
- oldtup.t_data->t_infomask2 |= infomask2_lock_old_tuple;
- HeapTupleHeaderSetCmax(oldtup.t_data, cid, iscombo);
+ HeapTupleHeaderSetXmax(oldtup->t_data, xmax_lock_old_tuple);
+ oldtup->t_data->t_infomask |= infomask_lock_old_tuple;
+ oldtup->t_data->t_infomask2 |= infomask2_lock_old_tuple;
+ HeapTupleHeaderSetCmax(oldtup->t_data, cid, iscombo);
/* temporarily make it look not-updated, but locked */
- oldtup.t_data->t_ctid = oldtup.t_self;
+ oldtup->t_data->t_ctid = oldtup->t_self;
/*
* Clear all-frozen bit on visibility map if needed. We could
@@ -3887,7 +3740,7 @@ l2:
* worthwhile.
*/
if (PageIsAllVisible(page) &&
- visibilitymap_clear(relation, block, vmbuffer,
+ visibilitymap_clear(relation, block, *vmbuffer,
VISIBILITYMAP_ALL_FROZEN))
cleared_all_frozen = true;
@@ -3901,10 +3754,10 @@ l2:
XLogBeginInsert();
XLogRegisterBuffer(0, buffer, REGBUF_STANDARD);
- xlrec.offnum = ItemPointerGetOffsetNumber(&oldtup.t_self);
+ xlrec.offnum = ItemPointerGetOffsetNumber(&oldtup->t_self);
xlrec.xmax = xmax_lock_old_tuple;
- xlrec.infobits_set = compute_infobits(oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2);
+ xlrec.infobits_set = compute_infobits(oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2);
xlrec.flags =
cleared_all_frozen ? XLH_LOCK_ALL_FROZEN_CLEARED : 0;
XLogRegisterData(&xlrec, SizeOfHeapLock);
@@ -3926,7 +3779,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = heap_toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = heap_toast_insert_or_update(relation, newtup, oldtup, 0);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
@@ -3962,20 +3815,20 @@ l2:
/* It doesn't fit, must use RelationGetBufferForTuple. */
newbuf = RelationGetBufferForTuple(relation, heaptup->t_len,
buffer, 0, NULL,
- &vmbuffer_new, &vmbuffer,
+ &vmbuffer_new, vmbuffer,
0);
/* We're all done. */
break;
}
/* Acquire VM page pin if needed and we don't have it. */
- if (vmbuffer == InvalidBuffer && PageIsAllVisible(page))
- visibilitymap_pin(relation, block, &vmbuffer);
+ if (*vmbuffer == InvalidBuffer && PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, vmbuffer);
/* Re-acquire the lock on the old tuple's page. */
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
/* Re-check using the up-to-date free space */
pagefree = PageGetHeapFreeSpace(page);
if (newtupsize > pagefree ||
- (vmbuffer == InvalidBuffer && PageIsAllVisible(page)))
+ (*vmbuffer == InvalidBuffer && PageIsAllVisible(page)))
{
/*
* Rats, it doesn't fit anymore, or somebody just now set the
@@ -4013,7 +3866,7 @@ l2:
* will include checking the relation level, there is no benefit to a
* separate check for the new tuple.
*/
- CheckForSerializableConflictIn(relation, &oldtup.t_self,
+ CheckForSerializableConflictIn(relation, &oldtup->t_self,
BufferGetBlockNumber(buffer));
/*
@@ -4021,7 +3874,6 @@ l2:
* has enough space for the new tuple. If they are the same buffer, only
* one pin is held.
*/
-
if (newbuf == buffer)
{
/*
@@ -4029,7 +3881,7 @@ l2:
* to do a HOT update. Check if any of the index columns have been
* changed.
*/
- if (!bms_overlap(modified_attrs, hot_attrs))
+ if (!bms_overlap(mix_attrs, hot_attrs))
{
use_hot_update = true;
@@ -4040,7 +3892,7 @@ l2:
* indexes if the columns were updated, or we may fail to detect
* e.g. value bound changes in BRIN minmax indexes.
*/
- if (bms_overlap(modified_attrs, sum_attrs))
+ if (bms_overlap(mix_attrs, sum_attrs))
summarized_update = true;
}
}
@@ -4057,10 +3909,8 @@ l2:
* logged. Pass old key required as true only if the replica identity key
* columns are modified or it has external data.
*/
- old_key_tuple = ExtractReplicaIdentity(relation, &oldtup,
- bms_overlap(modified_attrs, id_attrs) ||
- id_has_external,
- &old_key_copied);
+ old_key_tuple = ExtractReplicaIdentity(relation, oldtup, rid_attrs,
+ rep_id_key_required, &old_key_copied);
/* NO EREPORT(ERROR) from here till changes are logged */
START_CRIT_SECTION();
@@ -4082,7 +3932,7 @@ l2:
if (use_hot_update)
{
/* Mark the old tuple as HOT-updated */
- HeapTupleSetHotUpdated(&oldtup);
+ HeapTupleSetHotUpdated(oldtup);
/* And mark the new tuple as heap-only */
HeapTupleSetHeapOnly(heaptup);
/* Mark the caller's copy too, in case different from heaptup */
@@ -4091,7 +3941,7 @@ l2:
else
{
/* Make sure tuples are correctly marked as not-HOT */
- HeapTupleClearHotUpdated(&oldtup);
+ HeapTupleClearHotUpdated(oldtup);
HeapTupleClearHeapOnly(heaptup);
HeapTupleClearHeapOnly(newtup);
}
@@ -4100,17 +3950,17 @@ l2:
/* Clear obsolete visibility flags, possibly set by ourselves above... */
- oldtup.t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
- oldtup.t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
+ oldtup->t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
+ oldtup->t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
/* ... and store info about transaction updating this tuple */
Assert(TransactionIdIsValid(xmax_old_tuple));
- HeapTupleHeaderSetXmax(oldtup.t_data, xmax_old_tuple);
- oldtup.t_data->t_infomask |= infomask_old_tuple;
- oldtup.t_data->t_infomask2 |= infomask2_old_tuple;
- HeapTupleHeaderSetCmax(oldtup.t_data, cid, iscombo);
+ HeapTupleHeaderSetXmax(oldtup->t_data, xmax_old_tuple);
+ oldtup->t_data->t_infomask |= infomask_old_tuple;
+ oldtup->t_data->t_infomask2 |= infomask2_old_tuple;
+ HeapTupleHeaderSetCmax(oldtup->t_data, cid, iscombo);
/* record address of new tuple in t_ctid of old one */
- oldtup.t_data->t_ctid = heaptup->t_self;
+ oldtup->t_data->t_ctid = heaptup->t_self;
/* clear PD_ALL_VISIBLE flags, reset all visibilitymap bits */
if (PageIsAllVisible(BufferGetPage(buffer)))
@@ -4118,7 +3968,7 @@ l2:
all_visible_cleared = true;
PageClearAllVisible(BufferGetPage(buffer));
visibilitymap_clear(relation, BufferGetBlockNumber(buffer),
- vmbuffer, VISIBILITYMAP_VALID_BITS);
+ *vmbuffer, VISIBILITYMAP_VALID_BITS);
}
if (newbuf != buffer && PageIsAllVisible(BufferGetPage(newbuf)))
{
@@ -4143,12 +3993,12 @@ l2:
*/
if (RelationIsAccessibleInLogicalDecoding(relation))
{
- log_heap_new_cid(relation, &oldtup);
+ log_heap_new_cid(relation, oldtup);
log_heap_new_cid(relation, heaptup);
}
recptr = log_heap_update(relation, buffer,
- newbuf, &oldtup, heaptup,
+ newbuf, oldtup, heaptup,
old_key_tuple,
all_visible_cleared,
all_visible_cleared_new);
@@ -4173,7 +4023,7 @@ l2:
* both tuple versions in one call to inval.c so we can avoid redundant
* sinval messages.)
*/
- CacheInvalidateHeapTuple(relation, &oldtup, heaptup);
+ CacheInvalidateHeapTuple(relation, oldtup, heaptup);
/* Now we can release the buffer(s) */
if (newbuf != buffer)
@@ -4181,14 +4031,14 @@ l2:
ReleaseBuffer(buffer);
if (BufferIsValid(vmbuffer_new))
ReleaseBuffer(vmbuffer_new);
- if (BufferIsValid(vmbuffer))
- ReleaseBuffer(vmbuffer);
+ if (BufferIsValid(*vmbuffer))
+ ReleaseBuffer(*vmbuffer);
/*
* Release the lmgr tuple lock, if we had it.
*/
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &oldtup->t_self, *lockmode);
pgstat_count_heap_update(relation, use_hot_update, newbuf != buffer);
@@ -4221,13 +4071,6 @@ l2:
if (old_key_tuple != NULL && old_key_copied)
heap_freetuple(old_key_tuple);
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
-
return TM_Ok;
}
@@ -4236,7 +4079,7 @@ l2:
* Confirm adequate lock held during heap_update(), per rules from
* README.tuplock section "Locking to write inplace-updated tables".
*/
-static void
+void
check_lock_if_inplace_updateable_rel(Relation relation,
const ItemPointerData *otid,
HeapTuple newtup)
@@ -4408,7 +4251,7 @@ heap_attr_equals(TupleDesc tupdesc, int attrnum, Datum value1, Datum value2,
* listed as interesting) of the old tuple is a member of external_cols and is
* stored externally.
*/
-static Bitmapset *
+Bitmapset *
HeapDetermineColumnsInfo(Relation relation,
Bitmapset *interesting_cols,
Bitmapset *external_cols,
@@ -4491,25 +4334,175 @@ HeapDetermineColumnsInfo(Relation relation,
}
/*
- * simple_heap_update - replace a tuple
- *
- * This routine may be used to update a tuple when concurrent updates of
- * the target tuple are not expected (for example, because we have a lock
- * on the relation associated with the tuple). Any failure is reported
- * via ereport().
+ * This routine may be used to update a tuple when concurrent updates of the
+ * target tuple are not expected (for example, because we have a lock on the
+ * relation associated with the tuple). Any failure is reported via ereport().
*/
void
-simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup,
+simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TM_FailureData tmfd;
LockTupleMode lockmode;
+ Buffer buffer;
+ Buffer vmbuffer = InvalidBuffer;
+ Page page;
+ BlockNumber block;
+ Bitmapset *hot_attrs,
+ *sum_attrs,
+ *pk_attrs,
+ *rid_attrs,
+ *mix_attrs,
+ *idx_attrs;
+ ItemId lp;
+ HeapTupleData oldtup;
+ bool rep_id_key_required = false;
+
+ Assert(ItemPointerIsValid(otid));
+
+ /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
+ Assert(HeapTupleHeaderGetNatts(tuple->t_data) <=
+ RelationGetNumberOfAttributes(relation));
+
+ /*
+ * Forbid this during a parallel operation, lest it allocate a combo CID.
+ * Other workers might need that combo CID for visibility checks, and we
+ * have no provision for broadcasting it to them.
+ */
+ if (IsInParallelMode())
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
+ errmsg("cannot update tuples during a parallel operation")));
+
+#ifdef USE_ASSERT_CHECKING
+ check_lock_if_inplace_updateable_rel(relation, otid, tuple);
+#endif
+
+ /*
+ * Fetch the list of attributes to be checked for various operations.
+ *
+ * For HOT considerations, this is wasted effort if we fail to update or
+ * have to put the new tuple on a different page. But we must compute the
+ * list before obtaining buffer lock --- in the worst case, if we are
+ * doing an update on one of the relevant system catalogs, we could
+ * deadlock if we try to fetch the list later. In any case, the relcache
+ * caches the data so this is usually pretty cheap.
+ *
+ * We also need columns used by the replica identity and columns that are
+ * considered the "key" of rows in the table.
+ *
+ * Note that we get copies of each bitmap, so we need not worry about
+ * relcache flush happening midway through.
+ */
+ hot_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_HOT_BLOCKING);
+ sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+ pk_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
+ rid_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
+ idx_attrs = bms_copy(hot_attrs);
+ idx_attrs = bms_add_members(idx_attrs, sum_attrs);
+ idx_attrs = bms_add_members(idx_attrs, pk_attrs);
+ idx_attrs = bms_add_members(idx_attrs, rid_attrs);
+
+ block = ItemPointerGetBlockNumber(otid);
+ INJECTION_POINT("heap_update-before-pin", NULL);
+ buffer = ReadBuffer(relation, block);
+ page = BufferGetPage(buffer);
+
+ /*
+ * Before locking the buffer, pin the visibility map page if it appears to
+ * be necessary. Since we haven't got the lock yet, someone else might be
+ * in the middle of changing this, so we'll need to recheck after we have
+ * the lock.
+ */
+ if (PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, &vmbuffer);
+
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+
+ lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
+
+ /*
+ * Usually, a buffer pin and/or snapshot blocks pruning of otid, ensuring
+ * we see LP_NORMAL here. When the otid origin is a syscache, we may have
+ * neither a pin nor a snapshot. Hence, we may see other LP_ states, each
+ * of which indicates concurrent pruning.
+ *
+ * Failing with TM_Updated would be most accurate. However, unlike other
+ * TM_Updated scenarios, we don't know the successor ctid in LP_UNUSED and
+ * LP_DEAD cases. While the distinction between TM_Updated and TM_Deleted
+ * does matter to SQL statements UPDATE and MERGE, those SQL statements
+ * hold a snapshot that ensures LP_NORMAL. Hence, the choice between
+ * TM_Updated and TM_Deleted affects only the wording of error messages.
+ * Settle on TM_Deleted, for two reasons. First, it avoids complicating
+ * the specification of when tmfd->ctid is valid. Second, it creates
+ * error log evidence that we took this branch.
+ *
+ * Since it's possible to see LP_UNUSED at otid, it's also possible to see
+ * LP_NORMAL for a tuple that replaced LP_UNUSED. If it's a tuple for an
+ * unrelated row, we'll fail with "duplicate key value violates unique".
+ * XXX if otid is the live, newer version of the newtup row, we'll discard
+ * changes originating in versions of this catalog row after the version
+ * the caller got from syscache. See syscache-update-pruned.spec.
+ */
+ if (!ItemIdIsNormal(lp))
+ {
+ Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
+
+ UnlockReleaseBuffer(buffer);
+ if (vmbuffer != InvalidBuffer)
+ ReleaseBuffer(vmbuffer);
+ *update_indexes = TU_None;
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(idx_attrs);
+ /* mix_attrs not yet initialized */
+
+ elog(ERROR, "tuple concurrently deleted");
+
+ return;
+ }
+
+ /*
+ * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
+ * then pass that on to heap_update.
+ */
+ oldtup.t_tableOid = RelationGetRelid(relation);
+ oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
+ oldtup.t_len = ItemIdGetLength(lp);
+ oldtup.t_self = *otid;
+
+ mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
+ &oldtup, tuple, &rep_id_key_required);
+
+ /*
+ * We'll need to WAL log the replica identity attributes if either they
+ * overlap with the modified indexed attributes or, as we've checked for
+ * just now in HeapDetermineColumnsInfo, they were unmodified external
+ * indexed attributes.
+ */
+ rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+
+ result = heap_update(relation, &oldtup, tuple, GetCurrentCommandId(true),
+ InvalidSnapshot, true /* wait for commit */ , &tmfd, &lockmode,
+ buffer, page, block, lp, hot_attrs, sum_attrs, pk_attrs,
+ rid_attrs, mix_attrs, &vmbuffer, rep_id_key_required,
+ update_indexes);
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(mix_attrs);
+ bms_free(idx_attrs);
- result = heap_update(relation, otid, tup,
- GetCurrentCommandId(true), InvalidSnapshot,
- true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
switch (result)
{
case TM_SelfModified:
@@ -9149,12 +9142,11 @@ log_heap_new_cid(Relation relation, HeapTuple tup)
* the same tuple that was passed in.
*/
static HeapTuple
-ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
- bool *copy)
+ExtractReplicaIdentity(Relation relation, HeapTuple tp, Bitmapset *rid_attrs,
+ bool key_required, bool *copy)
{
TupleDesc desc = RelationGetDescr(relation);
char replident = relation->rd_rel->relreplident;
- Bitmapset *idattrs;
HeapTuple key_tuple;
bool nulls[MaxHeapAttributeNumber];
Datum values[MaxHeapAttributeNumber];
@@ -9185,17 +9177,13 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
if (!key_required)
return NULL;
- /* find out the replica identity columns */
- idattrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
-
/*
* If there's no defined replica identity columns, treat as !key_required.
* (This case should not be reachable from heap_update, since that should
* calculate key_required accurately. But heap_delete just passes
* constant true for key_required, so we can hit this case in deletes.)
*/
- if (bms_is_empty(idattrs))
+ if (bms_is_empty(rid_attrs))
return NULL;
/*
@@ -9208,7 +9196,7 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
for (int i = 0; i < desc->natts; i++)
{
if (bms_is_member(i + 1 - FirstLowInvalidHeapAttributeNumber,
- idattrs))
+ rid_attrs))
Assert(!nulls[i]);
else
nulls[i] = true;
@@ -9217,8 +9205,6 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
key_tuple = heap_form_tuple(desc, values, nulls);
*copy = true;
- bms_free(idattrs);
-
/*
* If the tuple, which by here only contains indexed columns, still has
* toasted columns, force them to be inlined. This is somewhat unlikely
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bcbac844bb6..1cf9a18775d 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -44,6 +44,7 @@
#include "storage/procarray.h"
#include "storage/smgr.h"
#include "utils/builtins.h"
+#include "utils/injection_point.h"
#include "utils/rel.h"
static void reform_and_rewrite_tuple(HeapTuple tuple,
@@ -312,23 +313,133 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
return heap_delete(relation, tid, cid, crosscheck, wait, tmfd, changingPart);
}
-
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd,
LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
{
+ bool rep_id_key_required = false;
bool shouldFree = true;
HeapTuple tuple = ExecFetchSlotHeapTuple(slot, true, &shouldFree);
+ HeapTupleData oldtup;
+ Buffer buffer;
+ Buffer vmbuffer = InvalidBuffer;
+ Page page;
+ BlockNumber block;
+ ItemId lp;
+ Bitmapset *hot_attrs,
+ *sum_attrs,
+ *pk_attrs,
+ *rid_attrs,
+ *mix_attrs,
+ *idx_attrs;
TM_Result result;
+ Assert(ItemPointerIsValid(otid));
+
+ /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
+ Assert(HeapTupleHeaderGetNatts(tuple->t_data) <=
+ RelationGetNumberOfAttributes(relation));
+
+ /*
+ * Forbid this during a parallel operation, lest it allocate a combo CID.
+ * Other workers might need that combo CID for visibility checks, and we
+ * have no provision for broadcasting it to them.
+ */
+ if (IsInParallelMode())
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
+ errmsg("cannot update tuples during a parallel operation")));
+
+#ifdef USE_ASSERT_CHECKING
+ check_lock_if_inplace_updateable_rel(relation, otid, tuple);
+#endif
+
+ /*
+ * Fetch the list of attributes to be checked for various operations.
+ *
+ * For HOT considerations, this is wasted effort if we fail to update or
+ * have to put the new tuple on a different page. But we must compute the
+ * list before obtaining buffer lock --- in the worst case, if we are
+ * doing an update on one of the relevant system catalogs, we could
+ * deadlock if we try to fetch the list later. In any case, the relcache
+ * caches the data so this is usually pretty cheap.
+ *
+ * We also need columns used by the replica identity and columns that are
+ * considered the "key" of rows in the table.
+ *
+ * Note that we get copies of each bitmap, so we need not worry about
+ * relcache flush happening midway through.
+ */
+ hot_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_HOT_BLOCKING);
+ sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+ pk_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
+ rid_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
+ idx_attrs = bms_copy(hot_attrs);
+ idx_attrs = bms_add_members(idx_attrs, sum_attrs);
+ idx_attrs = bms_add_members(idx_attrs, pk_attrs);
+ idx_attrs = bms_add_members(idx_attrs, rid_attrs);
+
+ block = ItemPointerGetBlockNumber(otid);
+ INJECTION_POINT("heap_update-before-pin", NULL);
+ buffer = ReadBuffer(relation, block);
+ page = BufferGetPage(buffer);
+
+ /*
+ * Before locking the buffer, pin the visibility map page if it appears to
+ * be necessary. Since we haven't got the lock yet, someone else might be
+ * in the middle of changing this, so we'll need to recheck after we have
+ * the lock.
+ */
+ if (PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, &vmbuffer);
+
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+
+ lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
+
+ Assert(ItemIdIsNormal(lp));
+
+ /*
+ * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
+ * then pass that on to heap_update.
+ */
+ oldtup.t_tableOid = RelationGetRelid(relation);
+ oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
+ oldtup.t_len = ItemIdGetLength(lp);
+ oldtup.t_self = *otid;
+
+ mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
+ &oldtup, tuple, &rep_id_key_required);
+
+ /*
+ * We'll need to WAL log the replica identity attributes if either they
+ * overlap with the modified indexed attributes or, as we've checked for
+ * just now in HeapDetermineColumnsInfo, they were unmodified external
+ * indexed attributes.
+ */
+ rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
- result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
- tmfd, lockmode, update_indexes);
+ result = heap_update(relation, &oldtup, tuple, cid, crosscheck, wait, tmfd, lockmode,
+ buffer, page, block, lp, hot_attrs, sum_attrs, pk_attrs,
+ rid_attrs, mix_attrs, &vmbuffer, rep_id_key_required, update_indexes);
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(mix_attrs);
+ bms_free(idx_attrs);
+
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 632c4332a8c..2f9a2b069cd 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -364,11 +364,13 @@ extern TM_Result heap_delete(Relation relation, const ItemPointerData *tid,
TM_FailureData *tmfd, bool changingPart);
extern void heap_finish_speculative(Relation relation, const ItemPointerData *tid);
extern void heap_abort_speculative(Relation relation, const ItemPointerData *tid);
-extern TM_Result heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+extern TM_Result heap_update(Relation relation, HeapTupleData *oldtup,
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
+ Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
+ Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
+ Bitmapset *mix_attrs, Buffer *vmbuffer,
+ bool rep_id_key_required, TU_UpdateIndexes *update_indexes);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -430,6 +432,18 @@ extern void log_heap_prune_and_freeze(Relation relation, Buffer buffer,
OffsetNumber *dead, int ndead,
OffsetNumber *unused, int nunused);
+/* in heap/heapam.c */
+extern Bitmapset *HeapDetermineColumnsInfo(Relation relation,
+ Bitmapset *interesting_cols,
+ Bitmapset *external_cols,
+ HeapTuple oldtup, HeapTuple newtup,
+ bool *has_external);
+#ifdef USE_ASSERT_CHECKING
+extern void check_lock_if_inplace_updateable_rel(Relation relation,
+ const ItemPointerData *otid,
+ HeapTuple newtup);
+#endif
+
/* in heap/vacuumlazy.c */
extern void heap_vacuum_rel(Relation rel,
const VacuumParams params, BufferAccessStrategy bstrategy);
--
2.49.0
[application/octet-stream] v23-0002-Track-changed-indexed-columns-in-the-executor-du.patch (34.3K, 3-v23-0002-Track-changed-indexed-columns-in-the-executor-du.patch)
download | inline diff:
From 6a6c28b8ee25a0ef04d65aa54c4792fa8ada03fc Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 26 Oct 2025 10:49:25 -0400
Subject: [PATCH v23 2/4] Track changed indexed columns in the executor during
UPDATEs
Refactor executor update logic to determine which indexed columns have
actually changed during an UPDATE operation rather than leaving this up
to HeapDetermineColumnsInfo in heap_update. This enables the comparison
to happen without taking a lock on the page and opens the door to reuse
in other code paths.
Because heap_update now requires the caller to provide the modified
indexed columns simple_heap_update has become a tad more complex. It is
frequently called from CatalogTupleUpdate which either updates heap
tuples via their form or using heap_modify_tuple. In both cases the
caller does know the modified set of attributes, but sadly those
attributes are lost before being provided to simple_heap_update. Due to
that the "simple" path has to retain the HeapDetermineColumnsInfo logic
of old (for now). In order for that to work it was necessary to split
the (overly large) heap_update call itself up. This moves up into
simple_heap_update and heap_tuple_update a bit of what existed in
heap_update itself. Ideally this will be cleaned up once
CatalogTupleUpdate paths are all recording modified attributes
correctly, when that happens the "simple" path can be simplified again.
ExecCheckIndexedAttrsForChanges replaces HeapDeterminesColumnsInfo and
tts_attr_equal replaces heap_attr_equal changing the test for equality
when calling into heap_tuple_update (but not simple_heap_update). In
the past we used datumIsEqual(), essentially a binary comparison using
memcmp(), now the comparison code in tts_attr_equal uses type-specific
equality function when available and falls back to datumIsEqual() when
not. This change in equality testing has some intended implications and
opens the door for more HOT updates (foreshadowing). For instance,
indexes with collation information allowing more HOT updates when the
index is specified to be case insensitive.
This change forced some logic changes in execReplication on the update
paths is now it is required to have knowledge of the set of attributes
that are both changed and referenced by indexes. Luckilly, the this is
available within calls to slot_modify_data() where LogicalRepTupleData
is processed and has a set of updated attributes. In this case rather
than using ExecCheckIndexedAttrsForChanges we can preseve what
slot_modify_data() identifies as the modified set and then intersect
that with the set of indexes on the relation and get the correct set of
modified indexed attributes required on heap_update().
---
src/backend/access/heap/heapam.c | 12 +-
src/backend/access/heap/heapam_handler.c | 72 +++++--
src/backend/access/table/tableam.c | 5 +-
src/backend/executor/execMain.c | 1 +
src/backend/executor/execReplication.c | 7 +
src/backend/executor/nodeModifyTable.c | 247 ++++++++++++++++++++++-
src/backend/nodes/bitmapset.c | 4 +
src/backend/replication/logical/worker.c | 72 ++++++-
src/backend/utils/cache/relcache.c | 15 ++
src/include/access/tableam.h | 8 +-
src/include/executor/executor.h | 5 +
src/include/nodes/execnodes.h | 1 +
src/include/utils/rel.h | 1 +
src/include/utils/relcache.h | 1 +
14 files changed, 415 insertions(+), 36 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index aff47481345..1cdb72b3a7a 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -3263,12 +3263,12 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* generated by another transaction).
*/
TM_Result
-heap_update(Relation relation, HeapTupleData *oldtup,
- HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
- Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
- Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
- Bitmapset *mix_attrs, Buffer *vmbuffer,
+heap_update(Relation relation, HeapTupleData *oldtup, HeapTuple newtup,
+ CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode,
+ Buffer buffer, Page page, BlockNumber block, ItemId lp,
+ Bitmapset *hot_attrs, Bitmapset *sum_attrs, Bitmapset *pk_attrs,
+ Bitmapset *rid_attrs, Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 1cf9a18775d..ef08e1d3e10 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -315,9 +315,12 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
- CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd,
- LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
+ CommandId cid, Snapshot snapshot,
+ Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd,
+ LockTupleMode *lockmode,
+ Bitmapset *mix_attrs,
+ TU_UpdateIndexes *update_indexes)
{
bool rep_id_key_required = false;
bool shouldFree = true;
@@ -332,7 +335,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
*sum_attrs,
*pk_attrs,
*rid_attrs,
- *mix_attrs,
*idx_attrs;
TM_Result result;
@@ -414,16 +416,61 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
oldtup.t_len = ItemIdGetLength(lp);
oldtup.t_self = *otid;
- mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
- &oldtup, tuple, &rep_id_key_required);
-
/*
- * We'll need to WAL log the replica identity attributes if either they
- * overlap with the modified indexed attributes or, as we've checked for
- * just now in HeapDetermineColumnsInfo, they were unmodified external
- * indexed attributes.
+ * We'll need to include the replica identity key when either the identity
+ * key attributes overlap with the modified index attributes or when the
+ * replica identity attributes are stored externally. This is required
+ * because for such attributes the flattened value won't be WAL logged as
+ * part of the new tuple so we must determine if we need to extract and
+ * include them as part of the old_key_tuple (see ExtractReplicaIdentity).
*/
- rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+ rep_id_key_required = bms_overlap(mix_attrs, rid_attrs);
+ if (!rep_id_key_required)
+ {
+ Bitmapset *attrs;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ int attidx = -1;
+
+ /*
+ * We don't own idx_attrs so we'll copy it and remove the modified set
+ * to reduce the attributes we need to test in the while loop and
+ * avoid a two branches in the loop.
+ */
+ attrs = bms_difference(idx_attrs, mix_attrs);
+ attrs = bms_int_members(attrs, rid_attrs);
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /*
+ * attidx is zero-based, attrnum is the normal attribute number
+ */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value;
+ bool isnull;
+
+ /*
+ * System attributes are not added into interesting_attrs in
+ * relcache
+ */
+ Assert(attrnum > 0);
+
+ value = heap_getattr(&oldtup, attrnum, tupdesc, &isnull);
+
+ /* No need to check attributes that can't be stored externally */
+ if (isnull ||
+ TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
+ continue;
+
+ /* Check if the old tuple's attribute is stored externally */
+ if (VARATT_IS_EXTERNAL((struct varlena *) DatumGetPointer(value)))
+ {
+ rep_id_key_required = true;
+ break;
+ }
+ }
+
+ bms_free(attrs);
+ }
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
@@ -437,7 +484,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
bms_free(sum_attrs);
bms_free(pk_attrs);
bms_free(rid_attrs);
- bms_free(mix_attrs);
bms_free(idx_attrs);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 5e41404937e..dadcf03ed24 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -336,6 +336,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
+ Bitmapset *modified_indexed_cols,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -346,7 +347,9 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
GetCurrentCommandId(true),
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, &lockmode,
+ modified_indexed_cols,
+ update_indexes);
switch (result)
{
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 27c9eec697b..6b7b6bc8019 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1282,6 +1282,7 @@ InitResultRelInfo(ResultRelInfo *resultRelInfo,
/* The following fields are set later if needed */
resultRelInfo->ri_RowIdAttNo = 0;
resultRelInfo->ri_extraUpdatedCols = NULL;
+ resultRelInfo->ri_ChangedIndexedCols = NULL;
resultRelInfo->ri_projectNew = NULL;
resultRelInfo->ri_newTupleSlot = NULL;
resultRelInfo->ri_oldTupleSlot = NULL;
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index def32774c90..2709e2db0f2 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -32,6 +32,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
#include "utils/typcache.h"
@@ -936,7 +937,13 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ /*
+ * We're not going to call ExecCheckIndexedAttrsForChanges here
+ * because we've already identified the changes earlier on thanks to
+ * slot_modify_data.
+ */
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
+ resultRelInfo->ri_ChangedIndexedCols,
&update_indexes);
conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 00429326c34..34f86546fc9 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -17,6 +17,7 @@
* ExecModifyTable - retrieve the next tuple from the node
* ExecEndModifyTable - shut down the ModifyTable node
* ExecReScanModifyTable - rescan the ModifyTable node
+ * ExecCheckIndexedAttrsForChanges - find set of updated indexed columns
*
* NOTES
* The ModifyTable node receives input from its outerPlan, which is
@@ -54,11 +55,14 @@
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/tupconvert.h"
+#include "access/tupdesc.h"
#include "access/xact.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
#include "executor/nodeModifyTable.h"
+#include "executor/tuptable.h"
#include "foreign/fdwapi.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
@@ -68,6 +72,8 @@
#include "storage/lmgr.h"
#include "utils/builtins.h"
#include "utils/datum.h"
+#include "utils/float.h"
+#include "utils/lsyscache.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
@@ -176,6 +182,219 @@ static TupleTableSlot *ExecMergeNotMatched(ModifyTableContext *context,
bool canSetTag);
+/*
+ * Compare two datums using the type's default equality operator.
+ *
+ * Returns true if the values are equal according to the type's equality
+ * operator, false otherwise. Falls back to binary comparison if no
+ * type-specific operator is available.
+ *
+ * This function uses the TypeCache infrastructure which caches operator
+ * lookups for efficiency.
+ */
+bool
+tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
+ Datum value1, Datum value2)
+{
+ TypeCacheEntry *typentry;
+
+ LOCAL_FCINFO(fcinfo, 2);
+ Datum result;
+
+ /*
+ * Fast path for common types to avoid even the type cache lookup. These
+ * types have simple equality semantics.
+ */
+ switch (typid)
+ {
+ case INT2OID:
+ return DatumGetInt16(value1) == DatumGetInt16(value2);
+ case INT4OID:
+ return DatumGetInt32(value1) == DatumGetInt32(value2);
+ case INT8OID:
+ return DatumGetInt64(value1) == DatumGetInt64(value2);
+ case FLOAT4OID:
+ return !float4_cmp_internal(DatumGetFloat4(value1), DatumGetFloat4(value2));
+ case FLOAT8OID:
+ return !float8_cmp_internal(DatumGetFloat8(value1), DatumGetFloat8(value2));
+ case BOOLOID:
+ return DatumGetBool(value1) == DatumGetBool(value2);
+ case OIDOID:
+ case REGPROCOID:
+ case REGPROCEDUREOID:
+ case REGOPEROID:
+ case REGOPERATOROID:
+ case REGCLASSOID:
+ case REGTYPEOID:
+ case REGROLEOID:
+ case REGNAMESPACEOID:
+ case REGCONFIGOID:
+ case REGDICTIONARYOID:
+ return DatumGetObjectId(value1) == DatumGetObjectId(value2);
+ case CHAROID:
+ return DatumGetChar(value1) == DatumGetChar(value2);
+ default:
+ /* Continue to type cache lookup */
+ break;
+ }
+
+ /*
+ * Look up the type's equality operator using the type cache. Request both
+ * the operator OID and the function info for efficiency.
+ */
+ typentry = lookup_type_cache(typid,
+ TYPECACHE_EQ_OPR | TYPECACHE_EQ_OPR_FINFO);
+
+ /*
+ * If no equality operator is available, fall back to binary comparison.
+ * This handles types that don't have proper equality operators defined.
+ */
+ if (!OidIsValid(typentry->eq_opr))
+ return datumIsEqual(value1, value2, typbyval, typlen);
+
+ /*
+ * Use the cached function info if available, otherwise look it up. The
+ * type cache keeps this around so subsequent calls are fast.
+ */
+ if (typentry->eq_opr_finfo.fn_addr == NULL)
+ {
+ Oid eq_proc = get_opcode(typentry->eq_opr);
+
+ if (!OidIsValid(eq_proc))
+ /* Shouldn't happen, but fall back to binary comparison */
+ return datumIsEqual(value1, value2, typbyval, typlen);
+
+ fmgr_info_cxt(eq_proc, &typentry->eq_opr_finfo,
+ CacheMemoryContext);
+ }
+
+ /* Set up function call */
+ InitFunctionCallInfoData(*fcinfo, &typentry->eq_opr_finfo, 2,
+ collation, NULL, NULL);
+
+ fcinfo->args[0].value = value1;
+ fcinfo->args[0].isnull = false;
+ fcinfo->args[1].value = value2;
+ fcinfo->args[1].isnull = false;
+
+ /* Invoke the equality operator */
+ result = FunctionCallInvoke(fcinfo);
+
+ /*
+ * If the function returned NULL (shouldn't happen for equality ops),
+ * treat as not equal for safety.
+ */
+ if (fcinfo->isnull)
+ return false;
+
+ return DatumGetBool(result);
+}
+
+/*
+ * Determine which updated attributes actually changed values between old and
+ * new tuples and are referenced by indexes on the relation.
+ *
+ * Returns a Bitmapset of attribute offsets (0-based, adjusted by
+ * FirstLowInvalidHeapAttributeNumber) or NULL if no attributes changed.
+ */
+Bitmapset *
+ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
+ TupleTableSlot *tts_old,
+ TupleTableSlot *tts_new)
+{
+ Relation relation = relinfo->ri_RelationDesc;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ Bitmapset *indexed_attrs;
+ Bitmapset *modified = NULL;
+ int attidx;
+
+ /* If no indexes, we're done */
+ if (relinfo->ri_NumIndices == 0)
+ return NULL;
+
+ /*
+ * Get the set of index key attributes. This includes summarizing,
+ * expression indexes and attributes mentioned in the predicate of a
+ * partition but not those in INCLUDING.
+ */
+ indexed_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_INDEXED);
+ Assert(!bms_is_empty(indexed_attrs));
+
+ /*
+ * NOTE: It is important to scan all indexed attributes in the tuples
+ * because ExecGetAllUpdatedCols won't include columns that may have been
+ * modified via heap_modify_tuple_by_col which is the case in
+ * tsvector_update_trigger.
+ */
+ attidx = -1;
+ while ((attidx = bms_next_member(indexed_attrs, attidx)) >= 0)
+ {
+ /* attidx is zero-based, attrnum is the normal attribute number */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Form_pg_attribute attr;
+ bool oldnull,
+ newnull;
+ Datum oldval,
+ newval;
+
+ /*
+ * If it's a whole-tuple reference, record as modified. It's not
+ * really worth supporting this case, since it could only succeed
+ * after a no-op update, which is hardly a case worth optimizing for.
+ */
+ if (attrnum == 0)
+ {
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /*
+ * Likewise, include in the modified set any system attribute other
+ * than tableOID; we cannot expect these to be consistent in a HOT
+ * chain, or even to be set correctly yet in the new tuple.
+ */
+ if (attrnum < 0)
+ {
+ if (attrnum != TableOidAttributeNumber)
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /* Extract values from both slots */
+ oldval = slot_getattr(tts_old, attrnum, &oldnull);
+ newval = slot_getattr(tts_new, attrnum, &newnull);
+
+ /* If one value is NULL and the other is not, they are not equal */
+ if (oldnull != newnull)
+ {
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /* If both are NULL, consider them equal */
+ if (oldnull)
+ continue;
+
+ /* Get attribute metadata */
+ Assert(attrnum > 0 && attrnum <= tupdesc->natts);
+ attr = TupleDescAttr(tupdesc, attrnum - 1);
+
+ /* Compare using type-specific equality operator */
+ if (!tts_attr_equal(attr->atttypid,
+ attr->attcollation,
+ attr->attbyval,
+ attr->attlen,
+ oldval,
+ newval))
+ modified = bms_add_member(modified, attidx);
+ }
+
+ bms_free(indexed_attrs);
+
+ return modified;
+}
+
/*
* Verify that the tuples to be produced by INSERT match the
* target relation's rowtype
@@ -2168,8 +2387,8 @@ ExecUpdatePrepareSlot(ResultRelInfo *resultRelInfo,
*/
static TM_Result
ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
- ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *slot,
- bool canSetTag, UpdateContext *updateCxt)
+ ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *oldSlot,
+ TupleTableSlot *slot, bool canSetTag, UpdateContext *updateCxt)
{
EState *estate = context->estate;
Relation resultRelationDesc = resultRelInfo->ri_RelationDesc;
@@ -2291,6 +2510,16 @@ lreplace:
if (resultRelationDesc->rd_att->constr)
ExecConstraints(resultRelInfo, slot, estate);
+ /*
+ * Identify which, if any, indexed attributes were modified here so that
+ * we might reuse it in a few places.
+ */
+ bms_free(resultRelInfo->ri_ChangedIndexedCols);
+ resultRelInfo->ri_ChangedIndexedCols = NULL;
+
+ resultRelInfo->ri_ChangedIndexedCols =
+ ExecCheckIndexedAttrsForChanges(resultRelInfo, oldSlot, slot);
+
/*
* replace the heap tuple
*
@@ -2306,6 +2535,7 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
+ resultRelInfo->ri_ChangedIndexedCols,
&updateCxt->updateIndexes);
return result;
@@ -2524,8 +2754,9 @@ ExecUpdate(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
*/
redo_act:
lockedtid = *tupleid;
- result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, slot,
- canSetTag, &updateCxt);
+
+ result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, oldSlot,
+ slot, canSetTag, &updateCxt);
/*
* If ExecUpdateAct reports that a cross-partition update was done,
@@ -3222,8 +3453,8 @@ lmerge_matched:
Assert(oldtuple == NULL);
result = ExecUpdateAct(context, resultRelInfo, tupleid,
- NULL, newslot, canSetTag,
- &updateCxt);
+ NULL, resultRelInfo->ri_oldTupleSlot,
+ newslot, canSetTag, &updateCxt);
/*
* As in ExecUpdate(), if ExecUpdateAct() reports that a
@@ -3248,6 +3479,7 @@ lmerge_matched:
tupleid, NULL, newslot);
mtstate->mt_merge_updated += 1;
}
+
break;
case CMD_DELETE:
@@ -4354,7 +4586,7 @@ ExecModifyTable(PlanState *pstate)
* For UPDATE/DELETE/MERGE, fetch the row identity info for the tuple
* to be updated/deleted/merged. For a heap relation, that's a TID;
* otherwise we may have a wholerow junk attr that carries the old
- * tuple in toto. Keep this in step with the part of
+ * tuple in total. Keep this in step with the part of
* ExecInitModifyTable that sets up ri_RowIdAttNo.
*/
if (operation == CMD_UPDATE || operation == CMD_DELETE ||
@@ -4530,6 +4762,7 @@ ExecModifyTable(PlanState *pstate)
/* Now apply the update. */
slot = ExecUpdate(&context, resultRelInfo, tupleid, oldtuple,
oldSlot, slot, node->canSetTag);
+
if (tuplock)
UnlockTuple(resultRelInfo->ri_RelationDesc, tupleid,
InplaceUpdateTupleLock);
diff --git a/src/backend/nodes/bitmapset.c b/src/backend/nodes/bitmapset.c
index b4ecf0b0390..9014990267a 100644
--- a/src/backend/nodes/bitmapset.c
+++ b/src/backend/nodes/bitmapset.c
@@ -238,6 +238,10 @@ bms_make_singleton(int x)
void
bms_free(Bitmapset *a)
{
+#if USE_ASSERT_CHECKING
+ Assert(bms_is_valid_set(a));
+#endif
+
if (a)
pfree(a);
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 93970c6af29..b363eaa49cc 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -243,6 +243,8 @@
*/
#include "postgres.h"
+#include "access/sysattr.h"
+#include "nodes/bitmapset.h"
#include <sys/stat.h>
#include <unistd.h>
@@ -275,7 +277,6 @@
#include "replication/logicalrelation.h"
#include "replication/logicalworker.h"
#include "replication/origin.h"
-#include "replication/slot.h"
#include "replication/walreceiver.h"
#include "replication/worker_internal.h"
#include "rewrite/rewriteHandler.h"
@@ -291,6 +292,7 @@
#include "utils/memutils.h"
#include "utils/pg_lsn.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/rls.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
@@ -1110,15 +1112,18 @@ slot_store_data(TupleTableSlot *slot, LogicalRepRelMapEntry *rel,
* "slot" is filled with a copy of the tuple in "srcslot", replacing
* columns provided in "tupleData" and leaving others as-is.
*
+ * Returns a bitmap of the modified columns.
+ *
* Caution: unreplaced pass-by-ref columns in "slot" will point into the
* storage for "srcslot". This is OK for current usage, but someday we may
* need to materialize "slot" at the end to make it independent of "srcslot".
*/
-static void
+static Bitmapset *
slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
LogicalRepRelMapEntry *rel,
LogicalRepTupleData *tupleData)
{
+ Bitmapset *modified = NULL;
int natts = slot->tts_tupleDescriptor->natts;
int i;
@@ -1195,6 +1200,28 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
slot->tts_isnull[i] = true;
}
+ /*
+ * Determine if the replicated value changed the local value by
+ * comparing slots. This is a subset of
+ * ExecCheckIndexedAttrsForChanges.
+ */
+ if (srcslot->tts_isnull[i] != slot->tts_isnull[i])
+ {
+ /* One is NULL, the other is not so the value changed */
+ modified = bms_add_member(modified, i + 1 - FirstLowInvalidHeapAttributeNumber);
+ }
+ else if (!srcslot->tts_isnull[i])
+ {
+ /* Both are not NULL, compare their values */
+ if (!tts_attr_equal(att->atttypid,
+ att->attcollation,
+ att->attbyval,
+ att->attlen,
+ srcslot->tts_values[i],
+ slot->tts_values[i]))
+ modified = bms_add_member(modified, i + 1 - FirstLowInvalidHeapAttributeNumber);
+ }
+
/* Reset attnum for error callback */
apply_error_callback_arg.remote_attnum = -1;
}
@@ -1202,6 +1229,8 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
/* And finally, declare that "slot" contains a valid virtual tuple */
ExecStoreVirtualTuple(slot);
+
+ return modified;
}
/*
@@ -2918,6 +2947,7 @@ apply_handle_update_internal(ApplyExecutionData *edata,
ConflictTupleInfo conflicttuple = {0};
bool found;
MemoryContext oldctx;
+ Bitmapset *indexed = NULL;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
ExecOpenIndices(relinfo, false);
@@ -2934,6 +2964,8 @@ apply_handle_update_internal(ApplyExecutionData *edata,
*/
if (found)
{
+ Bitmapset *modified = NULL;
+
/*
* Report the conflict if the tuple was modified by a different
* origin.
@@ -2957,15 +2989,29 @@ apply_handle_update_internal(ApplyExecutionData *edata,
/* Process and store remote tuple in the slot */
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot, localslot, relmapentry, newtup);
+ modified = slot_modify_data(remoteslot, localslot, relmapentry, newtup);
MemoryContextSwitchTo(oldctx);
+ /*
+ * Normally we'd call ExecCheckIndexedAttrForChanges but here we have
+ * the record of changed columns in the replication state, so let's
+ * use that instead.
+ */
+ indexed = RelationGetIndexAttrBitmap(relinfo->ri_RelationDesc,
+ INDEX_ATTR_BITMAP_INDEXED);
+
+ bms_free(relinfo->ri_ChangedIndexedCols);
+ relinfo->ri_ChangedIndexedCols = bms_int_members(modified, indexed);
+ bms_free(indexed);
+
EvalPlanQualSetSlot(&epqstate, remoteslot);
InitConflictIndexes(relinfo);
- /* Do the actual update. */
+ /* First check privileges */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
+
+ /* Then do the actual update. */
ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
remoteslot);
}
@@ -3455,6 +3501,8 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
bool found;
EPQState epqstate;
ConflictTupleInfo conflicttuple = {0};
+ Bitmapset *modified = NULL;
+ Bitmapset *indexed;
/* Get the matching local tuple from the partition. */
found = FindReplTupleInLocalRel(edata, partrel,
@@ -3523,8 +3571,8 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
* remoteslot_part.
*/
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot_part, localslot, part_entry,
- newtup);
+ modified = slot_modify_data(remoteslot_part, localslot, part_entry,
+ newtup);
MemoryContextSwitchTo(oldctx);
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
@@ -3549,6 +3597,18 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
EvalPlanQualSetSlot(&epqstate, remoteslot_part);
TargetPrivilegesCheck(partrelinfo->ri_RelationDesc,
ACL_UPDATE);
+
+ /*
+ * Normally we'd call ExecCheckIndexedAttrForChanges but
+ * here we have the record of changed columns in the
+ * replication state, so let's use that instead.
+ */
+ indexed = RelationGetIndexAttrBitmap(partrelinfo->ri_RelationDesc,
+ INDEX_ATTR_BITMAP_INDEXED);
+ bms_free(partrelinfo->ri_ChangedIndexedCols);
+ partrelinfo->ri_ChangedIndexedCols = bms_int_members(modified, indexed);
+ bms_free(indexed);
+
ExecSimpleRelationUpdate(partrelinfo, estate, &epqstate,
localslot, remoteslot_part);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 915d0bc9084..32825596be1 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -2482,6 +2482,7 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
bms_free(relation->rd_idattr);
bms_free(relation->rd_hotblockingattr);
bms_free(relation->rd_summarizedattr);
+ bms_free(relation->rd_indexedattr);
if (relation->rd_pubdesc)
pfree(relation->rd_pubdesc);
if (relation->rd_options)
@@ -5283,6 +5284,7 @@ RelationGetIndexPredicate(Relation relation)
* index (empty if FULL)
* INDEX_ATTR_BITMAP_HOT_BLOCKING Columns that block updates from being HOT
* INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
+ * INDEX_ATTR_BITMAP_INDEXED Columns referenced by indexes
*
* Attribute numbers are offset by FirstLowInvalidHeapAttributeNumber so that
* we can include system attributes (e.g., OID) in the bitmap representation.
@@ -5307,6 +5309,7 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
Bitmapset *idindexattrs; /* columns in the replica identity */
Bitmapset *hotblockingattrs; /* columns with HOT blocking indexes */
Bitmapset *summarizedattrs; /* columns with summarizing indexes */
+ Bitmapset *indexedattrs; /* columns referenced by indexes */
List *indexoidlist;
List *newindexoidlist;
Oid relpkindex;
@@ -5329,6 +5332,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
return bms_copy(relation->rd_hotblockingattr);
case INDEX_ATTR_BITMAP_SUMMARIZED:
return bms_copy(relation->rd_summarizedattr);
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return bms_copy(relation->rd_indexedattr);
default:
elog(ERROR, "unknown attrKind %u", attrKind);
}
@@ -5373,6 +5378,7 @@ restart:
idindexattrs = NULL;
hotblockingattrs = NULL;
summarizedattrs = NULL;
+ indexedattrs = NULL;
foreach(l, indexoidlist)
{
Oid indexOid = lfirst_oid(l);
@@ -5505,10 +5511,14 @@ restart:
bms_free(idindexattrs);
bms_free(hotblockingattrs);
bms_free(summarizedattrs);
+ bms_free(indexedattrs);
goto restart;
}
+ /* Combine all index attributes */
+ indexedattrs = bms_union(hotblockingattrs, summarizedattrs);
+
/* Don't leak the old values of these bitmaps, if any */
relation->rd_attrsvalid = false;
bms_free(relation->rd_keyattr);
@@ -5521,6 +5531,8 @@ restart:
relation->rd_hotblockingattr = NULL;
bms_free(relation->rd_summarizedattr);
relation->rd_summarizedattr = NULL;
+ bms_free(relation->rd_indexedattr);
+ relation->rd_indexedattr = NULL;
/*
* Now save copies of the bitmaps in the relcache entry. We intentionally
@@ -5535,6 +5547,7 @@ restart:
relation->rd_idattr = bms_copy(idindexattrs);
relation->rd_hotblockingattr = bms_copy(hotblockingattrs);
relation->rd_summarizedattr = bms_copy(summarizedattrs);
+ relation->rd_indexedattr = bms_copy(indexedattrs);
relation->rd_attrsvalid = true;
MemoryContextSwitchTo(oldcxt);
@@ -5551,6 +5564,8 @@ restart:
return hotblockingattrs;
case INDEX_ATTR_BITMAP_SUMMARIZED:
return summarizedattrs;
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return indexedattrs;
default:
elog(ERROR, "unknown attrKind %u", attrKind);
return NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index e16bf025692..8a5931a3118 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -549,6 +549,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
+ Bitmapset *updated_cols,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1502,12 +1503,12 @@ static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ Bitmapset *updated_cols, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
- wait, tmfd,
- lockmode, update_indexes);
+ wait, tmfd, lockmode,
+ updated_cols, update_indexes);
}
/*
@@ -2010,6 +2011,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
+ Bitmapset *modified_indexe_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index fa2b657fb2f..993dc0e6ced 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -800,5 +800,10 @@ extern ResultRelInfo *ExecLookupResultRelByOid(ModifyTableState *node,
Oid resultoid,
bool missing_ok,
bool update_cache);
+extern Bitmapset *ExecCheckIndexedAttrsForChanges(ResultRelInfo *resultRelInfo,
+ TupleTableSlot *tts_old,
+ TupleTableSlot *tts_new);
+extern bool tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
+ Datum value1, Datum value2);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 18ae8f0d4bb..8b08e0045ba 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -498,6 +498,7 @@ typedef struct ResultRelInfo
Bitmapset *ri_extraUpdatedCols;
/* true if the above has been computed */
bool ri_extraUpdatedCols_valid;
+ Bitmapset *ri_ChangedIndexedCols;
/* Projection to generate new tuple in an INSERT/UPDATE */
ProjectionInfo *ri_projectNew;
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 80286076a11..b23a7306e69 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -164,6 +164,7 @@ typedef struct RelationData
Bitmapset *rd_idattr; /* included in replica identity index */
Bitmapset *rd_hotblockingattr; /* cols blocking HOT update */
Bitmapset *rd_summarizedattr; /* cols indexed by summarizing indexes */
+ Bitmapset *rd_indexedattr; /* all cols referenced by indexes */
PublicationDesc *rd_pubdesc; /* publication descriptor, or NULL */
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 3561c6bef0b..d3fbb8b093a 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -71,6 +71,7 @@ typedef enum IndexAttrBitmapKind
INDEX_ATTR_BITMAP_IDENTITY_KEY,
INDEX_ATTR_BITMAP_HOT_BLOCKING,
INDEX_ATTR_BITMAP_SUMMARIZED,
+ INDEX_ATTR_BITMAP_INDEXED,
} IndexAttrBitmapKind;
extern Bitmapset *RelationGetIndexAttrBitmap(Relation relation,
--
2.49.0
[application/octet-stream] v23-0003-Replace-index_unchanged_by_update-with-ri_Change.patch (8.3K, 4-v23-0003-Replace-index_unchanged_by_update-with-ri_Change.patch)
download | inline diff:
From ce0e50aa8b14f838f04b8b45af8a47f5c4e73510 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Fri, 31 Oct 2025 14:55:25 -0400
Subject: [PATCH v23 3/4] Replace index_unchanged_by_update with
ri_ChangedIndexedCols
In execIndexing on updates we'd like to pass a hint to the indexing code
when the indexed attributes are unchanged. This commit replaces the now
redundant code in index_unchanged_by_update with the same information
found earlier in the update path.
---
src/backend/catalog/toasting.c | 2 -
src/backend/executor/execIndexing.c | 156 +---------------------------
src/backend/nodes/makefuncs.c | 2 -
src/include/nodes/execnodes.h | 4 -
4 files changed, 1 insertion(+), 163 deletions(-)
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 874a8fc89ad..5d819bda54a 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -300,8 +300,6 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
indexInfo->ii_Unique = true;
indexInfo->ii_NullsNotDistinct = false;
indexInfo->ii_ReadyForInserts = true;
- indexInfo->ii_CheckedUnchanged = false;
- indexInfo->ii_IndexUnchanged = false;
indexInfo->ii_Concurrent = false;
indexInfo->ii_BrokenHotChain = false;
indexInfo->ii_ParallelWorkers = 0;
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 401606f840a..fb1bc3a480d 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -138,11 +138,6 @@ static bool check_exclusion_or_unique_constraint(Relation heap, Relation index,
static bool index_recheck_constraint(Relation index, const Oid *constr_procs,
const Datum *existing_values, const bool *existing_isnull,
const Datum *new_values);
-static bool index_unchanged_by_update(ResultRelInfo *resultRelInfo,
- EState *estate, IndexInfo *indexInfo,
- Relation indexRelation);
-static bool index_expression_changed_walker(Node *node,
- Bitmapset *allUpdatedCols);
static void ExecWithoutOverlapsNotEmpty(Relation rel, NameData attname, Datum attval,
char typtype, Oid atttypid);
@@ -440,10 +435,7 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* index. If we're being called as part of an UPDATE statement,
* consider if the 'indexUnchanged' = true hint should be passed.
*/
- indexUnchanged = update && index_unchanged_by_update(resultRelInfo,
- estate,
- indexInfo,
- indexRelation);
+ indexUnchanged = update && bms_is_empty(resultRelInfo->ri_ChangedIndexedCols);
satisfiesConstraint =
index_insert(indexRelation, /* index relation */
@@ -993,152 +985,6 @@ index_recheck_constraint(Relation index, const Oid *constr_procs,
return true;
}
-/*
- * Check if ExecInsertIndexTuples() should pass indexUnchanged hint.
- *
- * When the executor performs an UPDATE that requires a new round of index
- * tuples, determine if we should pass 'indexUnchanged' = true hint for one
- * single index.
- */
-static bool
-index_unchanged_by_update(ResultRelInfo *resultRelInfo, EState *estate,
- IndexInfo *indexInfo, Relation indexRelation)
-{
- Bitmapset *updatedCols;
- Bitmapset *extraUpdatedCols;
- Bitmapset *allUpdatedCols;
- bool hasexpression = false;
- List *idxExprs;
-
- /*
- * Check cache first
- */
- if (indexInfo->ii_CheckedUnchanged)
- return indexInfo->ii_IndexUnchanged;
- indexInfo->ii_CheckedUnchanged = true;
-
- /*
- * Check for indexed attribute overlap with updated columns.
- *
- * Only do this for key columns. A change to a non-key column within an
- * INCLUDE index should not be counted here. Non-key column values are
- * opaque payload state to the index AM, a little like an extra table TID.
- *
- * Note that row-level BEFORE triggers won't affect our behavior, since
- * they don't affect the updatedCols bitmaps generally. It doesn't seem
- * worth the trouble of checking which attributes were changed directly.
- */
- updatedCols = ExecGetUpdatedCols(resultRelInfo, estate);
- extraUpdatedCols = ExecGetExtraUpdatedCols(resultRelInfo, estate);
- for (int attr = 0; attr < indexInfo->ii_NumIndexKeyAttrs; attr++)
- {
- int keycol = indexInfo->ii_IndexAttrNumbers[attr];
-
- if (keycol <= 0)
- {
- /*
- * Skip expressions for now, but remember to deal with them later
- * on
- */
- hasexpression = true;
- continue;
- }
-
- if (bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
- updatedCols) ||
- bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
- extraUpdatedCols))
- {
- /* Changed key column -- don't hint for this index */
- indexInfo->ii_IndexUnchanged = false;
- return false;
- }
- }
-
- /*
- * When we get this far and index has no expressions, return true so that
- * index_insert() call will go on to pass 'indexUnchanged' = true hint.
- *
- * The _absence_ of an indexed key attribute that overlaps with updated
- * attributes (in addition to the total absence of indexed expressions)
- * shows that the index as a whole is logically unchanged by UPDATE.
- */
- if (!hasexpression)
- {
- indexInfo->ii_IndexUnchanged = true;
- return true;
- }
-
- /*
- * Need to pass only one bms to expression_tree_walker helper function.
- * Avoid allocating memory in common case where there are no extra cols.
- */
- if (!extraUpdatedCols)
- allUpdatedCols = updatedCols;
- else
- allUpdatedCols = bms_union(updatedCols, extraUpdatedCols);
-
- /*
- * We have to work slightly harder in the event of indexed expressions,
- * but the principle is the same as before: try to find columns (Vars,
- * actually) that overlap with known-updated columns.
- *
- * If we find any matching Vars, don't pass hint for index. Otherwise
- * pass hint.
- */
- idxExprs = RelationGetIndexExpressions(indexRelation);
- hasexpression = index_expression_changed_walker((Node *) idxExprs,
- allUpdatedCols);
- list_free(idxExprs);
- if (extraUpdatedCols)
- bms_free(allUpdatedCols);
-
- if (hasexpression)
- {
- indexInfo->ii_IndexUnchanged = false;
- return false;
- }
-
- /*
- * Deliberately don't consider index predicates. We should even give the
- * hint when result rel's "updated tuple" has no corresponding index
- * tuple, which is possible with a partial index (provided the usual
- * conditions are met).
- */
- indexInfo->ii_IndexUnchanged = true;
- return true;
-}
-
-/*
- * Indexed expression helper for index_unchanged_by_update().
- *
- * Returns true when Var that appears within allUpdatedCols located.
- */
-static bool
-index_expression_changed_walker(Node *node, Bitmapset *allUpdatedCols)
-{
- if (node == NULL)
- return false;
-
- if (IsA(node, Var))
- {
- Var *var = (Var *) node;
-
- if (bms_is_member(var->varattno - FirstLowInvalidHeapAttributeNumber,
- allUpdatedCols))
- {
- /* Var was updated -- indicates that we should not hint */
- return true;
- }
-
- /* Still haven't found a reason to not pass the hint */
- return false;
- }
-
- return expression_tree_walker(node, index_expression_changed_walker,
- allUpdatedCols);
-}
-
/*
* ExecWithoutOverlapsNotEmpty - raise an error if the tuple has an empty
* range or multirange in the given attribute.
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index e2d9e9be41a..d69dc090aa4 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -845,8 +845,6 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
n->ii_Unique = unique;
n->ii_NullsNotDistinct = nulls_not_distinct;
n->ii_ReadyForInserts = isready;
- n->ii_CheckedUnchanged = false;
- n->ii_IndexUnchanged = false;
n->ii_Concurrent = concurrent;
n->ii_Summarizing = summarizing;
n->ii_WithoutOverlaps = withoutoverlaps;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 8b08e0045ba..898368fb8cb 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -202,10 +202,6 @@ typedef struct IndexInfo
bool ii_NullsNotDistinct;
/* is it valid for inserts? */
bool ii_ReadyForInserts;
- /* IndexUnchanged status determined yet? */
- bool ii_CheckedUnchanged;
- /* aminsert hint, cached for retail inserts */
- bool ii_IndexUnchanged;
/* are we doing a concurrent index build? */
bool ii_Concurrent;
/* did we detect any broken HOT chains? */
--
2.49.0
[application/octet-stream] v23-0004-Enable-HOT-updates-for-expression-and-partial-in.patch (191.5K, 5-v23-0004-Enable-HOT-updates-for-expression-and-partial-in.patch)
download | inline diff:
From cad236149af2d2a4f5762334d0d96ba13443b022 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 26 Oct 2025 10:49:25 -0400
Subject: [PATCH v23 4/4] Enable HOT updates for expression and partial indexes
Currently, PostgreSQL conservatively prevents HOT (Heap-Only Tuple)
updates whenever any indexed column changes, even if the indexed
portion of that column remains identical. This is overly restrictive
for expression indexes (where f(column) might not change even when
column changes) and partial indexes (where both old and new tuples
might fall outside the predicate). Finally, index AMs play no role
in deciding when they need a new index entry on update, the rules
regarding that are based on binary equality and the HEAP's model for
MVCC and related HOT optimization. Here we open that door a bit so
as to enable more nuanced control over the process. This enables
index AMs that require binary equality (as is the case for nbtree)
to do that without disallowing type-specific equality checking for
other indexes.
This patch introduces several improvements to enable HOT updates in
these cases:
Add amcomparedatums() callback to IndexAmRoutine. This allows index
access methods like GIN to provide custom logic for comparing datums by
extracting and comparing index keys rather than comparing the raw
datums. GIN indexes now implement gincomparedatums() which extracts keys
from both datums and compares the resulting key sets. Also, as
mentioned earlier nbtree implements this API and uses datumIsEqual() for
equality so that the manner in which it deduplicates TIDs on page split
doesn't have to change. This is not a required API, when not
implemented the executor will compare TupleTableSlot datum for equality
using type-specific operators and take into account collation so that an
update from "Apple" to "APPLE" on a case insensitive index can now be
HOT.
ExecWhichIndexesRequireUpdates() is re-written to find the set of
modified indexed attributes that trigger new index tuples on updated.
For partial indexes, this checks whether both old and new tuples satisfy
or fail the predicate. For expression indexes, this uses type-specific
equality operators to compare computed values. For extraction-based
indexes (GIN/RUM) that implement amcomparedatums() it uses that.
Importantly, table access methods can still signal using TU_Update if
all, none, or only summarizing indexes should be updated. While the
executor layer now owns determining what has changed due to an update
and is interested in only updating the minimum number of indexes
possible, the table AM can override that while performing
table_tuple_update(), which is what heap does. While this signal is
very specific to how the heap implements MVCC and its HOT optimization,
we'll leave replacing that for another day.
This optimization trades off some new overhead for the potential for
more updates to use the HOT optimized path and avoid index and heap
bloat. This should significantly improve update performance for tables
with expression indexes, partial indexes, and GIN/GiST indexes on
complex data types like JSONB and tsvector, while maintaining correct
index semantics. Minimal additional overhead due to type-specific
equality checking should be washed out by the benefits of updating
indexes fewer times.
One notable trade-off is that there are more calls to FormIndexDatum()
as a result. Caching these might reduce some of that overhead, but not
all. This lead to the change in the frequency for expressions in the
spec update test to output notice messages, but does not impact
correctness.
---
src/backend/access/brin/brin.c | 1 +
src/backend/access/gin/ginutil.c | 92 +-
src/backend/access/hash/hash.c | 44 +
src/backend/access/heap/heapam.c | 10 +-
src/backend/access/heap/heapam_handler.c | 6 +-
src/backend/access/nbtree/nbtree.c | 1 +
src/backend/access/table/tableam.c | 4 +-
src/backend/bootstrap/bootstrap.c | 8 +
src/backend/catalog/index.c | 54 +
src/backend/catalog/indexing.c | 16 +-
src/backend/catalog/toasting.c | 4 +
src/backend/executor/execIndexing.c | 45 +-
src/backend/executor/nodeModifyTable.c | 496 ++++-
src/backend/nodes/makefuncs.c | 4 +
src/include/access/amapi.h | 28 +
src/include/access/gin.h | 3 +
src/include/access/heapam.h | 6 +-
src/include/access/nbtree.h | 4 +
src/include/access/tableam.h | 8 +-
src/include/catalog/index.h | 1 +
src/include/executor/executor.h | 12 +-
src/include/nodes/execnodes.h | 19 +
.../expected/insert-conflict-specconflict.out | 20 +
.../regress/expected/heap_hot_updates.out | 1922 +++++++++++++++++
src/test/regress/parallel_schedule | 6 +
src/test/regress/sql/heap_hot_updates.sql | 1325 ++++++++++++
src/tools/pgindent/typedefs.list | 1 +
27 files changed, 4015 insertions(+), 125 deletions(-)
create mode 100644 src/test/regress/expected/heap_hot_updates.out
create mode 100644 src/test/regress/sql/heap_hot_updates.sql
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index cb3331921cb..36e639552e6 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -290,6 +290,7 @@ brinhandler(PG_FUNCTION_ARGS)
amroutine->amproperty = NULL;
amroutine->ambuildphasename = NULL;
amroutine->amvalidate = brinvalidate;
+ amroutine->amcomparedatums = NULL;
amroutine->amadjustmembers = NULL;
amroutine->ambeginscan = brinbeginscan;
amroutine->amrescan = brinrescan;
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index 78f7b7a2495..8e31ec21c1c 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -26,6 +26,7 @@
#include "storage/indexfsm.h"
#include "utils/builtins.h"
#include "utils/index_selfuncs.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/typcache.h"
@@ -78,6 +79,7 @@ ginhandler(PG_FUNCTION_ARGS)
amroutine->amproperty = NULL;
amroutine->ambuildphasename = ginbuildphasename;
amroutine->amvalidate = ginvalidate;
+ amroutine->amcomparedatums = gincomparedatums;
amroutine->amadjustmembers = ginadjustmembers;
amroutine->ambeginscan = ginbeginscan;
amroutine->amrescan = ginrescan;
@@ -477,13 +479,6 @@ cmpEntries(const void *a, const void *b, void *arg)
return res;
}
-
-/*
- * Extract the index key values from an indexable item
- *
- * The resulting key values are sorted, and any duplicates are removed.
- * This avoids generating redundant index entries.
- */
Datum *
ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
@@ -729,3 +724,86 @@ ginbuildphasename(int64 phasenum)
return NULL;
}
}
+
+/*
+ * gincomparedatums - Compare datums to determine if they produce identical keys
+ *
+ * This function extracts keys from both old_datum and new_datum using the
+ * opclass's extractValue function, then compares the extracted key arrays.
+ * Returns true if the key sets are identical (same keys, same counts).
+ *
+ * This enables HOT updates for GIN indexes when the indexed portions of a
+ * value haven't changed, even if the value itself has changed.
+ *
+ * Example: JSONB column with GIN index. If an update changes a non-indexed
+ * key in the JSONB document, the extracted keys are identical and we can
+ * do a HOT update.
+ */
+bool
+gincomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull)
+{
+ GinState ginstate;
+ Datum *old_keys;
+ Datum *new_keys;
+ GinNullCategory *old_categories;
+ GinNullCategory *new_categories;
+ int32 old_nkeys;
+ int32 new_nkeys;
+ MemoryContext tmpcontext;
+ MemoryContext oldcontext;
+ bool result = true;
+
+ /* Handle NULL cases */
+ if (old_isnull != new_isnull)
+ return false;
+ if (old_isnull)
+ return true;
+
+ /* Create temporary context for extraction work */
+ tmpcontext = AllocSetContextCreate(CurrentMemoryContext,
+ "GIN datum comparison",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(tmpcontext);
+
+ initGinState(&ginstate, index);
+
+ /* Extract keys from both datums using existing GIN infrastructure */
+ old_keys = ginExtractEntries(&ginstate, attnum, old_datum, old_isnull,
+ &old_nkeys, &old_categories);
+ new_keys = ginExtractEntries(&ginstate, attnum, new_datum, new_isnull,
+ &new_nkeys, &new_categories);
+
+ /* Different number of keys, definitely different */
+ if (old_nkeys != new_nkeys)
+ {
+ result = false;
+ goto cleanup;
+ }
+
+ /*
+ * Compare the sorted key arrays element-by-element. Since both arrays are
+ * already sorted by ginExtractEntries, we can do a simple O(n)
+ * comparison.
+ */
+ for (int i = 0; i < old_nkeys; i++)
+ {
+ int cmp = ginCompareEntries(&ginstate, attnum,
+ old_keys[i], old_categories[i],
+ new_keys[i], new_categories[i]);
+
+ if (cmp != 0)
+ {
+ result = false;
+ break;
+ }
+ }
+
+cleanup:
+ /* Clean up */
+ MemoryContextSwitchTo(oldcontext);
+ MemoryContextDelete(tmpcontext);
+
+ return result;
+}
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 53061c819fb..91371dfdacd 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -50,6 +50,10 @@ static void hashbuildCallback(Relation index,
void *state);
+static bool hashcomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* Hash handler function: return IndexAmRoutine with access method parameters
* and callbacks.
@@ -98,6 +102,7 @@ hashhandler(PG_FUNCTION_ARGS)
amroutine->amproperty = NULL;
amroutine->ambuildphasename = NULL;
amroutine->amvalidate = hashvalidate;
+ amroutine->amcomparedatums = hashcomparedatums;
amroutine->amadjustmembers = hashadjustmembers;
amroutine->ambeginscan = hashbeginscan;
amroutine->amrescan = hashrescan;
@@ -944,3 +949,42 @@ hashtranslatecmptype(CompareType cmptype, Oid opfamily)
return HTEqualStrategyNumber;
return InvalidStrategy;
}
+
+/*
+ * hashcomparedatums - Compare datums to determine if they produce identical keys
+ *
+ * Returns true if the hash values are identical (index doesn't need update).
+ */
+bool
+hashcomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull)
+{
+ uint32 old_hashkey;
+ uint32 new_hashkey;
+
+ /* If both are NULL, they're equal */
+ if (old_isnull && new_isnull)
+ return true;
+
+ /* If NULL status differs, they're not equal */
+ if (old_isnull != new_isnull)
+ return false;
+
+ /*
+ * _hash_datum2hashkey() is used because we know this can't be a cross
+ * type comparison.
+ */
+ old_hashkey = _hash_datum2hashkey(index, old_datum);
+ new_hashkey = _hash_datum2hashkey(index, new_datum);
+
+ /*
+ * If hash keys are identical, the index entry would be the same. Return
+ * true to indicate no index update needed.
+ *
+ * Note: Hash collisions are rare but possible. If hash(x) == hash(y) but
+ * x != y, the hash index still treats them identically, so we correctly
+ * return true.
+ */
+ return (old_hashkey == new_hashkey);
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 1cdb72b3a7a..5b0ff13b13d 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -3268,7 +3268,7 @@ heap_update(Relation relation, HeapTupleData *oldtup, HeapTuple newtup,
TM_FailureData *tmfd, LockTupleMode *lockmode,
Buffer buffer, Page page, BlockNumber block, ItemId lp,
Bitmapset *hot_attrs, Bitmapset *sum_attrs, Bitmapset *pk_attrs,
- Bitmapset *rid_attrs, Bitmapset *mix_attrs, Buffer *vmbuffer,
+ Bitmapset *rid_attrs, const Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -4337,8 +4337,9 @@ HeapDetermineColumnsInfo(Relation relation,
* This routine may be used to update a tuple when concurrent updates of the
* target tuple are not expected (for example, because we have a lock on the
* relation associated with the tuple). Any failure is reported via ereport().
+ * Returns the set of modified indexed attributes.
*/
-void
+Bitmapset *
simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
@@ -4467,7 +4468,7 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
elog(ERROR, "tuple concurrently deleted");
- return;
+ return NULL;
}
/*
@@ -4500,7 +4501,6 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
bms_free(sum_attrs);
bms_free(pk_attrs);
bms_free(rid_attrs);
- bms_free(mix_attrs);
bms_free(idx_attrs);
switch (result)
@@ -4526,6 +4526,8 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
elog(ERROR, "unrecognized heap_update status: %u", result);
break;
}
+
+ return mix_attrs;
}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index ef08e1d3e10..7527809ec08 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -319,7 +319,7 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
Snapshot crosscheck, bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
- Bitmapset *mix_attrs,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes)
{
bool rep_id_key_required = false;
@@ -407,10 +407,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
Assert(ItemIdIsNormal(lp));
- /*
- * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
- * then pass that on to heap_update.
- */
oldtup.t_tableOid = RelationGetRelid(relation);
oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
oldtup.t_len = ItemIdGetLength(lp);
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index fdff960c130..e435f0d5db4 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -155,6 +155,7 @@ bthandler(PG_FUNCTION_ARGS)
amroutine->amproperty = btproperty;
amroutine->ambuildphasename = btbuildphasename;
amroutine->amvalidate = btvalidate;
+ amroutine->amcomparedatums = NULL;
amroutine->amadjustmembers = btadjustmembers;
amroutine->ambeginscan = btbeginscan;
amroutine->amrescan = btrescan;
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index dadcf03ed24..ef7736bfa76 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -336,7 +336,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
- Bitmapset *modified_indexed_cols,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -348,7 +348,7 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
&tmfd, &lockmode,
- modified_indexed_cols,
+ mix_attrs,
update_indexes);
switch (result)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index fc8638c1b61..329c110d0bf 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -961,10 +961,18 @@ index_register(Oid heap,
newind->il_info->ii_Expressions =
copyObject(indexInfo->ii_Expressions);
newind->il_info->ii_ExpressionsState = NIL;
+ /* expression attrs will likely be null, but may as well copy it */
+ newind->il_info->ii_ExpressionsAttrs =
+ copyObject(indexInfo->ii_ExpressionsAttrs);
/* predicate will likely be null, but may as well copy it */
newind->il_info->ii_Predicate =
copyObject(indexInfo->ii_Predicate);
newind->il_info->ii_PredicateState = NULL;
+ /* predicate attrs will likely be null, but may as well copy it */
+ newind->il_info->ii_PredicateAttrs =
+ copyObject(indexInfo->ii_PredicateAttrs);
+ newind->il_info->ii_CheckedPredicate = false;
+ newind->il_info->ii_PredicateSatisfied = false;
/* no exclusion constraints at bootstrap time, so no need to copy */
Assert(indexInfo->ii_ExclusionOps == NULL);
Assert(indexInfo->ii_ExclusionProcs == NULL);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 5d9db167e59..e88db7e919b 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -27,6 +27,7 @@
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/relscan.h"
+#include "access/sysattr.h"
#include "access/tableam.h"
#include "access/toast_compression.h"
#include "access/transam.h"
@@ -58,6 +59,7 @@
#include "commands/trigger.h"
#include "executor/executor.h"
#include "miscadmin.h"
+#include "nodes/execnodes.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/optimizer.h"
@@ -2414,6 +2416,58 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* ----------------------------------------------------------------
*/
+/* ----------------
+ * BuildUpdateIndexInfo
+ *
+ * For expression indexes updates may not change the indexed value allowing
+ * for a HOT update. Add information to the IndexInfo to allow for checking
+ * if the indexed value has changed.
+ *
+ * Do this processing here rather than in BuildIndexInfo() to not incur the
+ * overhead in the common non-expression cases.
+ * ----------------
+ */
+void
+BuildUpdateIndexInfo(ResultRelInfo *resultRelInfo)
+{
+ for (int j = 0; j < resultRelInfo->ri_NumIndices; j++)
+ {
+ int i;
+ int indnatts;
+ Bitmapset *attrs = NULL;
+ IndexInfo *ii = resultRelInfo->ri_IndexRelationInfo[j];
+
+ indnatts = ii->ii_NumIndexAttrs;
+
+ /* Collect key attributes used by the index, key and including */
+ for (i = 0; i < indnatts; i++)
+ {
+ AttrNumber attnum = ii->ii_IndexAttrNumbers[i];
+
+ if (attnum != 0)
+ attrs = bms_add_member(attrs, attnum - FirstLowInvalidHeapAttributeNumber);
+ }
+
+ /* Collect attributes used in the expression */
+ if (ii->ii_Expressions)
+ pull_varattnos((Node *) ii->ii_Expressions,
+ resultRelInfo->ri_RangeTableIndex,
+ &ii->ii_ExpressionsAttrs);
+
+ /* Collect attributes used in the predicate */
+ if (ii->ii_Predicate)
+ pull_varattnos((Node *) ii->ii_Predicate,
+ resultRelInfo->ri_RangeTableIndex,
+ &ii->ii_PredicateAttrs);
+
+ /* Combine key, including, and expression attributes, but not predicate */
+ ii->ii_IndexedAttrs = bms_union(attrs, ii->ii_ExpressionsAttrs);
+
+ /* All indexes should index *something*! */
+ Assert(!bms_is_empty(ii->ii_IndexedAttrs));
+ }
+}
+
/* ----------------
* BuildIndexInfo
* Construct an IndexInfo record for an open index
diff --git a/src/backend/catalog/indexing.c b/src/backend/catalog/indexing.c
index 004c5121000..a361c215490 100644
--- a/src/backend/catalog/indexing.c
+++ b/src/backend/catalog/indexing.c
@@ -102,7 +102,7 @@ CatalogIndexInsert(CatalogIndexState indstate, HeapTuple heapTuple,
* Get information from the state structure. Fall out if nothing to do.
*/
numIndexes = indstate->ri_NumIndices;
- if (numIndexes == 0)
+ if (numIndexes == 0 || updateIndexes == TU_None)
return;
relationDescs = indstate->ri_IndexRelationDescs;
indexInfoArray = indstate->ri_IndexRelationInfo;
@@ -314,15 +314,18 @@ CatalogTupleUpdate(Relation heapRel, const ItemPointerData *otid, HeapTuple tup)
{
CatalogIndexState indstate;
TU_UpdateIndexes updateIndexes = TU_All;
+ Bitmapset *updatedAttrs;
CatalogTupleCheckConstraints(heapRel, tup);
indstate = CatalogOpenIndexes(heapRel);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
-
+ updatedAttrs = simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = updatedAttrs;
CatalogIndexInsert(indstate, tup, updateIndexes);
+
CatalogCloseIndexes(indstate);
+ bms_free(updatedAttrs);
}
/*
@@ -338,12 +341,15 @@ CatalogTupleUpdateWithInfo(Relation heapRel, const ItemPointerData *otid, HeapTu
CatalogIndexState indstate)
{
TU_UpdateIndexes updateIndexes = TU_All;
+ Bitmapset *updatedAttrs;
CatalogTupleCheckConstraints(heapRel, tup);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
-
+ updatedAttrs = simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = updatedAttrs;
CatalogIndexInsert(indstate, tup, updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = NULL;
+ bms_free(updatedAttrs);
}
/*
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 5d819bda54a..c665aa744b3 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -292,8 +292,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
indexInfo->ii_IndexAttrNumbers[1] = 2;
indexInfo->ii_Expressions = NIL;
indexInfo->ii_ExpressionsState = NIL;
+ indexInfo->ii_ExpressionsAttrs = NULL;
indexInfo->ii_Predicate = NIL;
indexInfo->ii_PredicateState = NULL;
+ indexInfo->ii_PredicateAttrs = NULL;
+ indexInfo->ii_CheckedPredicate = false;
+ indexInfo->ii_PredicateSatisfied = false;
indexInfo->ii_ExclusionOps = NULL;
indexInfo->ii_ExclusionProcs = NULL;
indexInfo->ii_ExclusionStrats = NULL;
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index fb1bc3a480d..20968a814d6 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -109,11 +109,15 @@
#include "access/genam.h"
#include "access/relscan.h"
#include "access/tableam.h"
+#include "access/sysattr.h"
#include "access/xact.h"
#include "catalog/index.h"
#include "executor/executor.h"
+#include "nodes/bitmapset.h"
+#include "nodes/execnodes.h"
#include "nodes/nodeFuncs.h"
#include "storage/lmgr.h"
+#include "utils/datum.h"
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
@@ -318,8 +322,8 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
Relation heapRelation;
IndexInfo **indexInfoArray;
ExprContext *econtext;
- Datum values[INDEX_MAX_KEYS];
- bool isnull[INDEX_MAX_KEYS];
+ Datum loc_values[INDEX_MAX_KEYS];
+ bool loc_isnull[INDEX_MAX_KEYS];
Assert(ItemPointerIsValid(tupleid));
@@ -343,13 +347,13 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
/* Arrange for econtext's scan tuple to be the tuple under test */
econtext->ecxt_scantuple = slot;
- /*
- * for each index, form and insert the index tuple
- */
+ /* Insert into each index that needs updating */
for (i = 0; i < numIndices; i++)
{
Relation indexRelation = relationDescs[i];
IndexInfo *indexInfo;
+ Datum *values;
+ bool *isnull;
bool applyNoDupErr;
IndexUniqueCheck checkUnique;
bool indexUnchanged;
@@ -366,7 +370,7 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
/*
* Skip processing of non-summarizing indexes if we only update
- * summarizing indexes
+ * summarizing indexes or if this index is unchanged.
*/
if (onlySummarizing && !indexInfo->ii_Summarizing)
continue;
@@ -387,8 +391,15 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
indexInfo->ii_PredicateState = predicate;
}
+ /* Check the index predicate if we haven't done so earlier on */
+ if (!indexInfo->ii_CheckedPredicate)
+ {
+ indexInfo->ii_PredicateSatisfied = ExecQual(predicate, econtext);
+ indexInfo->ii_CheckedPredicate = true;
+ }
+
/* Skip this index-update if the predicate isn't satisfied */
- if (!ExecQual(predicate, econtext))
+ if (!indexInfo->ii_PredicateSatisfied)
continue;
}
@@ -396,11 +407,10 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* FormIndexDatum fills in its values and isnull parameters with the
* appropriate values for the column(s) of the index.
*/
- FormIndexDatum(indexInfo,
- slot,
- estate,
- values,
- isnull);
+ FormIndexDatum(indexInfo, slot, estate, loc_values, loc_isnull);
+
+ values = loc_values;
+ isnull = loc_isnull;
/* Check whether to apply noDupErr to this index */
applyNoDupErr = noDupErr &&
@@ -435,7 +445,9 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* index. If we're being called as part of an UPDATE statement,
* consider if the 'indexUnchanged' = true hint should be passed.
*/
- indexUnchanged = update && bms_is_empty(resultRelInfo->ri_ChangedIndexedCols);
+ indexUnchanged = update &&
+ !bms_overlap(indexInfo->ii_IndexedAttrs,
+ resultRelInfo->ri_ChangedIndexedCols);
satisfiesConstraint =
index_insert(indexRelation, /* index relation */
@@ -604,7 +616,12 @@ ExecCheckIndexConstraints(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
checkedIndex = true;
/* Check for partial index */
- if (indexInfo->ii_Predicate != NIL)
+ if (indexInfo->ii_CheckedPredicate && !indexInfo->ii_PredicateSatisfied)
+ {
+ /* We've already checked and the predicate wasn't satisfied. */
+ continue;
+ }
+ else if (indexInfo->ii_Predicate != NIL)
{
ExprState *predicate;
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 34f86546fc9..ea50fcaf5dd 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -54,10 +54,13 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/attnum.h"
+#include "access/sysattr.h"
#include "access/tableam.h"
#include "access/tupconvert.h"
#include "access/tupdesc.h"
#include "access/xact.h"
+#include "catalog/index.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -75,6 +78,7 @@
#include "utils/float.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
@@ -245,6 +249,10 @@ tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
typentry = lookup_type_cache(typid,
TYPECACHE_EQ_OPR | TYPECACHE_EQ_OPR_FINFO);
+ /* Use the type's collation if none provided */
+ if (collation == -1)
+ collation = typentry->typcollation;
+
/*
* If no equality operator is available, fall back to binary comparison.
* This handles types that don't have proper equality operators defined.
@@ -291,108 +299,415 @@ tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
}
/*
- * Determine which updated attributes actually changed values between old and
- * new tuples and are referenced by indexes on the relation.
+ * ExecCheckIndexedAttrsForChanges
+ *
+ * Determine which indexes need updating by finding the set of modified
+ * indexed attributes.
+ *
+ * For expression indexes and indexes which implement the amcomparedatums()
+ * index AM API we'll need to form index datum and compare each attribute to
+ * see if any actually changed.
+ *
+ * For expression indexes the result of the expression might not change at all,
+ * this is common with JSONB columns, which require expression indexes. It is
+ * is commonplace to index one or more fields within a document and perform
+ * updates to the document while leaving the indexed fields unchanged. These
+ * updates don't necessitate index updates.
+ *
+ * Partial indexes won't trigger index updates when the old/new tuples are both
+ * outside of the predicate range. A transition into or out of the predicate
+ * does require an index update.
+ *
+ * Indexes that support index-only scans (IOS) should return the value that
+ * is the binary equavalent of what is in the table. For that reason we must
+ * use datumIsEqual() when deciding if an index update is required or not.
+ *
+ * All other indexes require testing old/new datum for equality, we now test
+ * with a type-specific equality operator and fall back to datumIsEqual()
+ * when that isn't possible.
+ *
+ * For a BTREE index (nbtree) their is an additional reason to use binary
+ * comparison for equality. TID deduplication on page split in nbtree uses
+ * binary comparison.
+ *
+ * The goal is for the executor to know, ahead of calling into the table AM to
+ * process the update and before calling into the index AM for inserting new
+ * index tuples, which attributes in the new TupleTableSlot, if any, truely
+ * necessitate a new index tuple.
*
- * Returns a Bitmapset of attribute offsets (0-based, adjusted by
- * FirstLowInvalidHeapAttributeNumber) or NULL if no attributes changed.
+ * Returns a Bitmapset of attributes that intersects with indexes which require
+ * a new index tuple.
*/
Bitmapset *
ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
- TupleTableSlot *tts_old,
- TupleTableSlot *tts_new)
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts)
{
Relation relation = relinfo->ri_RelationDesc;
TupleDesc tupdesc = RelationGetDescr(relation);
- Bitmapset *indexed_attrs;
- Bitmapset *modified = NULL;
- int attidx;
+ Bitmapset *mix_attrs = NULL;
/* If no indexes, we're done */
if (relinfo->ri_NumIndices == 0)
return NULL;
/*
- * Get the set of index key attributes. This includes summarizing,
- * expression indexes and attributes mentioned in the predicate of a
- * partition but not those in INCLUDING.
+ * NOTE: Expression and predicates that are observed to change will have
+ * all their attributes added into the m_attrs set knowing that some of
+ * those might not have changed. Take for instance an index on (a + b)
+ * followed by an index on (b) with an update that changes only the value
+ * of 'a'. We'll add both 'a' and 'b' to the m_attrs set then later when
+ * reviewing the second index add 'b' to the u_attrs (unchanged) set. In
+ * the end, we'll remove all the unchanged from the m_attrs and get our
+ * desired result.
*/
- indexed_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_INDEXED);
- Assert(!bms_is_empty(indexed_attrs));
- /*
- * NOTE: It is important to scan all indexed attributes in the tuples
- * because ExecGetAllUpdatedCols won't include columns that may have been
- * modified via heap_modify_tuple_by_col which is the case in
- * tsvector_update_trigger.
- */
- attidx = -1;
- while ((attidx = bms_next_member(indexed_attrs, attidx)) >= 0)
+ /* Find the indexes that reference this attribute */
+ for (int i = 0; i < relinfo->ri_NumIndices; i++)
{
- /* attidx is zero-based, attrnum is the normal attribute number */
- AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
- Form_pg_attribute attr;
- bool oldnull,
- newnull;
- Datum oldval,
- newval;
+ Relation index = relinfo->ri_IndexRelationDescs[i];
+ IndexAmRoutine *amroutine = index->rd_indam;
+ IndexInfo *indexInfo = relinfo->ri_IndexRelationInfo[i];
+ Bitmapset *m_attrs = NULL; /* (possibly) modified key attributes */
+ Bitmapset *p_attrs = NULL; /* (possibly) modified predicate
+ * attributes */
+ Bitmapset *u_attrs = NULL; /* unmodified attributes */
+ Bitmapset *pre_attrs = indexInfo->ii_PredicateAttrs;
+ bool has_expressions = (indexInfo->ii_Expressions != NIL);
+ bool has_am_compare = (amroutine->amcomparedatums != NULL);
+ bool supports_ios = (amroutine->amcanreturn != NULL);
+ bool is_partial = (indexInfo->ii_Predicate != NIL);
+ TupleTableSlot *save_scantuple;
+ ExprContext *econtext = GetPerTupleExprContext(estate);
+ Datum old_values[INDEX_MAX_KEYS];
+ bool old_isnull[INDEX_MAX_KEYS];
+ Datum new_values[INDEX_MAX_KEYS];
+ bool new_isnull[INDEX_MAX_KEYS];
+
+ /* If we've reviewed all the attributes on this index, move on */
+ if (bms_is_subset(indexInfo->ii_IndexedAttrs, mix_attrs))
+ continue;
- /*
- * If it's a whole-tuple reference, record as modified. It's not
- * really worth supporting this case, since it could only succeed
- * after a no-op update, which is hardly a case worth optimizing for.
- */
- if (attrnum == 0)
+ /* Checking partial at this point isn't viable when we're serializable */
+ if (is_partial && IsolationIsSerializable())
{
- modified = bms_add_member(modified, attidx);
- continue;
+ p_attrs = bms_add_members(p_attrs, pre_attrs);
+ }
+ /* Check partial index predicate */
+ else if (is_partial)
+ {
+ ExprState *pstate;
+ bool old_qualifies,
+ new_qualifies;
+
+ if (!indexInfo->ii_CheckedPredicate)
+ pstate = ExecPrepareQual(indexInfo->ii_Predicate, estate);
+ else
+ pstate = indexInfo->ii_PredicateState;
+
+ save_scantuple = econtext->ecxt_scantuple;
+
+ econtext->ecxt_scantuple = old_tts;
+ old_qualifies = ExecQual(pstate, econtext);
+
+ econtext->ecxt_scantuple = new_tts;
+ new_qualifies = ExecQual(pstate, econtext);
+
+ econtext->ecxt_scantuple = save_scantuple;
+
+ indexInfo->ii_CheckedPredicate = true;
+ indexInfo->ii_PredicateState = pstate;
+ indexInfo->ii_PredicateSatisfied = new_qualifies;
+
+ /* Both outside predicate, index doesn't need update */
+ if (!old_qualifies && !new_qualifies)
+ continue;
+
+ /* A transition means we need to update the index */
+ if (old_qualifies != new_qualifies)
+ p_attrs = bms_copy(pre_attrs);
+
+ /*
+ * When both are within the predicate we must update this index,
+ * but only if one of the index key attributes changed.
+ */
}
/*
- * Likewise, include in the modified set any system attribute other
- * than tableOID; we cannot expect these to be consistent in a HOT
- * chain, or even to be set correctly yet in the new tuple.
+ * Expression indexes, or an index that has a comparison function,
+ * requires us to form index datums and compare. We've done all we
+ * can to avoid this overhead, now it's time to bite the bullet and
+ * get it done.
+ *
+ * XXX: Caching the values/isnull might be a win and avoid one of the
+ * added calls to FormIndexDatum().
*/
- if (attrnum < 0)
+ if (has_expressions || has_am_compare)
{
- if (attrnum != TableOidAttributeNumber)
- modified = bms_add_member(modified, attidx);
- continue;
- }
+ save_scantuple = econtext->ecxt_scantuple;
- /* Extract values from both slots */
- oldval = slot_getattr(tts_old, attrnum, &oldnull);
- newval = slot_getattr(tts_new, attrnum, &newnull);
+ /* Evaluate expressions (if any) to get base datums */
+ econtext->ecxt_scantuple = old_tts;
+ FormIndexDatum(indexInfo, old_tts, estate, old_values, old_isnull);
- /* If one value is NULL and the other is not, they are not equal */
- if (oldnull != newnull)
- {
- modified = bms_add_member(modified, attidx);
- continue;
+ econtext->ecxt_scantuple = new_tts;
+ FormIndexDatum(indexInfo, new_tts, estate, new_values, new_isnull);
+
+ econtext->ecxt_scantuple = save_scantuple;
+
+ /* Compare the index key datums for equality */
+ for (int j = 0; j < indexInfo->ii_NumIndexKeyAttrs; j++)
+ {
+ AttrNumber rel_attrnum = indexInfo->ii_IndexAttrNumbers[j];
+ int rel_attridx = rel_attrnum - FirstLowInvalidHeapAttributeNumber;
+ int nth_expr = 0;
+ bool values_equal = false;
+
+ /*
+ * We can't skip attributes that we've already identified as
+ * triggering an index update because we may have added an
+ * attribute from an expression index that didn't change but
+ * the expression did and that unchanged attribute is
+ * referenced in a subsequent index where we will discover
+ * that fact.
+ */
+
+ /* A change to/from NULL, record this attribute */
+ if (old_isnull[j] != new_isnull[j])
+ {
+ /* Expressions will have rel_attrnum == 0 */
+ if (rel_attrnum == 0)
+ m_attrs = bms_add_members(m_attrs, indexInfo->ii_ExpressionsAttrs);
+ else
+ m_attrs = bms_add_member(m_attrs, rel_attridx);
+ continue;
+ }
+
+ /* Both NULL, no change */
+ if (old_isnull[j])
+ {
+ if (rel_attrnum != 0)
+ u_attrs = bms_add_member(u_attrs, rel_attridx);
+
+ continue;
+ }
+
+ /*
+ * Use index AM's comparison function if present when
+ * comparing the index datum formed when creating an index
+ * key.
+ */
+ if (has_am_compare)
+ {
+ /*
+ * NOTE: For AM comparison, pass the 1-based index
+ * attribute number. The AM's compare function expects the
+ * same numbering as used internally by the AM.
+ */
+ values_equal = amroutine->amcomparedatums(index, j + 1,
+ old_values[j], old_isnull[j],
+ new_values[j], new_isnull[j]);
+ }
+ else
+ {
+ /* Non-zero attribute means not an expression */
+ if (rel_attrnum != 0)
+ {
+ if (supports_ios)
+ {
+ CompactAttribute *attr = TupleDescCompactAttr(tupdesc, rel_attrnum - 1);
+
+ values_equal = datumIsEqual(old_values[j],
+ new_values[j],
+ attr->attbyval,
+ attr->attlen);
+ }
+ else
+ {
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, rel_attrnum - 1);
+
+ /*
+ * Compare using type-specific equality which at
+ * this point is the relation's type because
+ * FormIndexDatum() will populate the values/nulls
+ * but won't transform them into the final values
+ * destined for the index tuple, that's left to
+ * index_form_tuple() which we don't call (on
+ * purpose).
+ */
+ values_equal = tts_attr_equal(attr->atttypid,
+ attr->attcollation,
+ attr->attbyval,
+ attr->attlen,
+ old_values[j],
+ new_values[j]);
+ }
+ }
+ else
+ {
+ /*
+ * An expression on an indexed attribute without
+ * custom AM comparison function. In this case, becase
+ * indexes will store the result of the expression's
+ * evaluation, we can test for equality using the
+ * expression's result type. This allows for JSONB
+ * and custom type equality tests, which may not be
+ * the same as binary equality, to be in effect. The
+ * result stored in the index and used in index-only
+ * scans will be valid as it is the expressions
+ * result, which shouldn't change given the same
+ * input.
+ *
+ * At this point the expression's type is what is
+ * required when testing for equality, not the index's
+ * type, because the value created by FormIndexDatum()
+ * is the expression's result. Later on in
+ * index_form_tuple() an index may transform the value
+ * when forming it's key (as is the case with HASH),
+ * but at this point the Datum is the expression's
+ * result type.
+ */
+ Oid expr_type_oid;
+ int16 typlen;
+ bool typbyval;
+ Expr *expr = (Expr *) list_nth(indexInfo->ii_Expressions, nth_expr);
+
+ Assert(expr != NULL);
+
+ /* Get type OID from the expression */
+ expr_type_oid = exprType((Node *) expr);
+
+ /* Get type information from the OID */
+ get_typlenbyval(expr_type_oid, &typlen, &typbyval);
+
+ values_equal = tts_attr_equal(expr_type_oid,
+ -1, /* use TBD expr type */
+ typbyval,
+ typlen,
+ old_values[j],
+ new_values[j]);
+ }
+ }
+
+ if (!values_equal)
+ {
+ /* Expressions will have rel_attrnum == 0 */
+ if (rel_attrnum == 0)
+ m_attrs = bms_add_members(m_attrs, indexInfo->ii_ExpressionsAttrs);
+ else
+ m_attrs = bms_add_member(m_attrs, rel_attridx);
+ }
+ else
+ {
+ if (rel_attrnum != 0)
+ u_attrs = bms_add_member(u_attrs, rel_attridx);
+ }
+
+ if (rel_attrnum == 0)
+ nth_expr++;
+ }
}
+ else
+ {
+ /*
+ * Here we know that we're reviewing an index that doesn't have a
+ * partial predicate, doesn't use expressions, and doesn't have a
+ * amcomparedatums() implementation. If this index supports IOS
+ * we need to use binary comparison, if not then type-specific
+ * will provide a more accurate result.
+ */
- /* If both are NULL, consider them equal */
- if (oldnull)
- continue;
+ /* Compare the index key datums for equality */
+ for (int j = 0; j < indexInfo->ii_NumIndexKeyAttrs; j++)
+ {
+ AttrNumber rel_attrnum;
+ int rel_attridx;
+ bool values_equal = false;
+ bool old_null,
+ new_null;
+ Datum old_val,
+ new_val;
- /* Get attribute metadata */
- Assert(attrnum > 0 && attrnum <= tupdesc->natts);
- attr = TupleDescAttr(tupdesc, attrnum - 1);
-
- /* Compare using type-specific equality operator */
- if (!tts_attr_equal(attr->atttypid,
- attr->attcollation,
- attr->attbyval,
- attr->attlen,
- oldval,
- newval))
- modified = bms_add_member(modified, attidx);
- }
+ rel_attrnum = indexInfo->ii_IndexAttrNumbers[j];
+ rel_attridx = rel_attrnum - FirstLowInvalidHeapAttributeNumber;
+
+ /* Zero would mean expression, something we don't expect here */
+ Assert(rel_attrnum > 0 && rel_attrnum <= tupdesc->natts);
+
+ /* Extract values from both slots for this attribute */
+ old_val = slot_getattr(old_tts, rel_attrnum, &old_null);
+ new_val = slot_getattr(new_tts, rel_attrnum, &new_null);
- bms_free(indexed_attrs);
+ /*
+ * If one value is NULL and the other is not, they are not
+ * equal
+ */
+ if (old_null != new_null)
+ {
+ m_attrs = bms_add_member(m_attrs, rel_attridx);
+ continue;
+ }
+
+ /* If both are NULL, consider them equal */
+ if (old_null)
+ {
+ u_attrs = bms_add_member(u_attrs, rel_attridx);
+ continue;
+ }
+
+ if (supports_ios)
+ {
+ CompactAttribute *attr = TupleDescCompactAttr(tupdesc, rel_attrnum - 1);
+
+ values_equal = datumIsEqual(old_val,
+ new_val,
+ attr->attbyval,
+ attr->attlen);
+ }
+ else
+ {
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, rel_attrnum - 1);
+
+ /*
+ * Compare using type-specific equality which at this
+ * point is the relation's type because FormIndexDatum()
+ * will populate the values/nulls but won't transform them
+ * into the final values destined for the index tuple,
+ * that's left to index_form_tuple() which we don't call
+ * (on purpose).
+ */
+ values_equal = tts_attr_equal(attr->atttypid,
+ attr->attcollation,
+ attr->attbyval,
+ attr->attlen,
+ old_val,
+ new_val);
+ }
+
+ if (!values_equal)
+ m_attrs = bms_add_member(m_attrs, rel_attridx);
+ else
+ u_attrs = bms_add_member(u_attrs, rel_attridx);
+ }
+ }
+
+ /*
+ * Here we know all the attributes we thought might be modified and
+ * all those we know haven't been. Take the difference and add it to
+ * the modified indexed attributes set.
+ */
+ m_attrs = bms_del_members(m_attrs, u_attrs);
+ p_attrs = bms_del_members(p_attrs, u_attrs);
+ mix_attrs = bms_add_members(mix_attrs, m_attrs);
+ mix_attrs = bms_add_members(mix_attrs, p_attrs);
+
+ bms_free(m_attrs);
+ bms_free(u_attrs);
+ bms_free(p_attrs);
+ }
- return modified;
+ return mix_attrs;
}
/*
@@ -2395,6 +2710,9 @@ ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
bool partition_constraint_failed;
TM_Result result;
+ /* The set of modified indexed attributes that trigger new index entries */
+ Bitmapset *mix_attrs = NULL;
+
updateCxt->crossPartUpdate = false;
/*
@@ -2517,13 +2835,32 @@ lreplace:
bms_free(resultRelInfo->ri_ChangedIndexedCols);
resultRelInfo->ri_ChangedIndexedCols = NULL;
- resultRelInfo->ri_ChangedIndexedCols =
- ExecCheckIndexedAttrsForChanges(resultRelInfo, oldSlot, slot);
+ /*
+ * During updates we'll need a bit more information in IndexInfo but we've
+ * delayed adding it until here. We check to ensure that there are
+ * indexes, that something has changed that is indexed, and that the first
+ * index doesn't yet have ii_IndexedAttrs set as a way to ensure we only
+ * build this when needed and only once. We don't build this in
+ * ExecOpenIndicies() as it is unnecessary overhead when not performing an
+ * update.
+ */
+ if (resultRelInfo->ri_NumIndices > 0 &&
+ bms_is_empty(resultRelInfo->ri_IndexRelationInfo[0]->ii_IndexedAttrs))
+ BuildUpdateIndexInfo(resultRelInfo);
+
+ /*
+ * Next up we need to find out the set of indexed attributes that have
+ * changed in value and should trigger a new index tuple. We could start
+ * with the set of updated columns via ExecGetUpdatedCols(), but if we do
+ * we will overlook attributes directly modified by heap_modify_tuple()
+ * which are not known to ExecGetUpdatedCols().
+ */
+ mix_attrs = ExecCheckIndexedAttrsForChanges(resultRelInfo, estate, oldSlot, slot);
/*
- * replace the heap tuple
+ * Call into the table AM to update the heap tuple.
*
- * Note: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
+ * NOTE: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
* the row to be updated is visible to that snapshot, and throw a
* can't-serialize error if not. This is a special-case behavior needed
* for referential integrity updates in transaction-snapshot mode
@@ -2535,9 +2872,12 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
- resultRelInfo->ri_ChangedIndexedCols,
+ mix_attrs,
&updateCxt->updateIndexes);
+ Assert(bms_is_empty(resultRelInfo->ri_ChangedIndexedCols));
+ resultRelInfo->ri_ChangedIndexedCols = mix_attrs;
+
return result;
}
@@ -2555,7 +2895,7 @@ ExecUpdateEpilogue(ModifyTableContext *context, UpdateContext *updateCxt,
ModifyTableState *mtstate = context->mtstate;
List *recheckIndexes = NIL;
- /* insert index entries for tuple if necessary */
+ /* Insert index entries for tuple if necessary */
if (resultRelInfo->ri_NumIndices > 0 && (updateCxt->updateIndexes != TU_None))
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
slot, context->estate,
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index d69dc090aa4..e9a53b95caf 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -855,10 +855,14 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
/* expressions */
n->ii_Expressions = expressions;
n->ii_ExpressionsState = NIL;
+ n->ii_ExpressionsAttrs = NULL;
/* predicates */
n->ii_Predicate = predicates;
n->ii_PredicateState = NULL;
+ n->ii_PredicateAttrs = NULL;
+ n->ii_CheckedPredicate = false;
+ n->ii_PredicateSatisfied = false;
/* exclusion constraints */
n->ii_ExclusionOps = NULL;
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index 63dd41c1f21..9bdf73eda59 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -211,6 +211,33 @@ typedef void (*ammarkpos_function) (IndexScanDesc scan);
/* restore marked scan position */
typedef void (*amrestrpos_function) (IndexScanDesc scan);
+/*
+ * amcomparedatums - Compare datums to determine if index update is needed
+ *
+ * This function compares old_datum and new_datum to determine if they would
+ * produce different index entries. For extraction-based indexes (GIN, RUM),
+ * this should:
+ * 1. Extract keys from old_datum using the opclass's extractValue function
+ * 2. Extract keys from new_datum using the opclass's extractValue function
+ * 3. Compare the two sets of keys using appropriate equality operators
+ * 4. Return true if the sets are equal (no index update needed)
+ *
+ * The comparison should account for:
+ * - Different numbers of extracted keys
+ * - NULL values
+ * - Type-specific equality (not just binary equality)
+ * - Opclass parameters (e.g., path in bson_rum_single_path_ops)
+ *
+ * For the DocumentDB example with path='a', this would extract values at
+ * path 'a' from both old and new BSON documents and compare them using
+ * BSON's equality operator.
+ */
+/* identify if updated datums would produce one or more index entries */
+typedef bool (*amcomparedatums_function) (Relation indexRelation,
+ int attno,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* Callback function signatures - for parallel index scans.
*/
@@ -313,6 +340,7 @@ typedef struct IndexAmRoutine
amendscan_function amendscan;
ammarkpos_function ammarkpos; /* can be NULL */
amrestrpos_function amrestrpos; /* can be NULL */
+ amcomparedatums_function amcomparedatums; /* can be NULL */
/* interface functions to support parallel index scans */
amestimateparallelscan_function amestimateparallelscan; /* can be NULL */
diff --git a/src/include/access/gin.h b/src/include/access/gin.h
index 13ea91922ef..2f265f4816c 100644
--- a/src/include/access/gin.h
+++ b/src/include/access/gin.h
@@ -100,6 +100,9 @@ extern PGDLLIMPORT int gin_pending_list_limit;
extern void ginGetStats(Relation index, GinStatsData *stats);
extern void ginUpdateStats(Relation index, const GinStatsData *stats,
bool is_build);
+extern bool gincomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
extern void _gin_parallel_build_main(dsm_segment *seg, shm_toc *toc);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 2f9a2b069cd..5783dbebff0 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -369,7 +369,7 @@ extern TM_Result heap_update(Relation relation, HeapTupleData *oldtup,
TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
- Bitmapset *mix_attrs, Buffer *vmbuffer,
+ const Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
@@ -404,8 +404,8 @@ extern bool heap_tuple_needs_eventual_freeze(HeapTupleHeader tuple);
extern void simple_heap_insert(Relation relation, HeapTuple tup);
extern void simple_heap_delete(Relation relation, const ItemPointerData *tid);
-extern void simple_heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple tup, TU_UpdateIndexes *update_indexes);
+extern Bitmapset *simple_heap_update(Relation relation, const ItemPointerData *otid,
+ HeapTuple tup, TU_UpdateIndexes *update_indexes);
extern TransactionId heap_index_delete_tuples(Relation rel,
TM_IndexDeleteOp *delstate);
diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h
index 16be5c7a9c1..42bd329eaad 100644
--- a/src/include/access/nbtree.h
+++ b/src/include/access/nbtree.h
@@ -1210,6 +1210,10 @@ extern int btgettreeheight(Relation rel);
extern CompareType bttranslatestrategy(StrategyNumber strategy, Oid opfamily);
extern StrategyNumber bttranslatecmptype(CompareType cmptype, Oid opfamily);
+extern bool btcomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* prototypes for internal functions in nbtree.c
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 8a5931a3118..2b9206ff24a 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -549,7 +549,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
- Bitmapset *updated_cols,
+ const Bitmapset *updated_cols,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1503,12 +1503,12 @@ static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- Bitmapset *updated_cols, TU_UpdateIndexes *update_indexes)
+ const Bitmapset *mix_cols, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
wait, tmfd, lockmode,
- updated_cols, update_indexes);
+ mix_cols, update_indexes);
}
/*
@@ -2011,7 +2011,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
- Bitmapset *modified_indexe_attrs,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index dda95e54903..8d364f8b30f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -132,6 +132,7 @@ extern bool CompareIndexInfo(const IndexInfo *info1, const IndexInfo *info2,
const AttrMap *attmap);
extern void BuildSpeculativeIndexInfo(Relation index, IndexInfo *ii);
+extern void BuildUpdateIndexInfo(ResultRelInfo *resultRelInfo);
extern void FormIndexDatum(IndexInfo *indexInfo,
TupleTableSlot *slot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 993dc0e6ced..a19585ba065 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -739,6 +739,11 @@ extern Bitmapset *ExecGetAllUpdatedCols(ResultRelInfo *relinfo, EState *estate);
*/
extern void ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative);
extern void ExecCloseIndices(ResultRelInfo *resultRelInfo);
+extern Bitmapset *ExecWhichIndexesRequireUpdates(ResultRelInfo *relinfo,
+ Bitmapset *mix_attrs,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
extern List *ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
TupleTableSlot *slot, EState *estate,
bool update,
@@ -800,9 +805,10 @@ extern ResultRelInfo *ExecLookupResultRelByOid(ModifyTableState *node,
Oid resultoid,
bool missing_ok,
bool update_cache);
-extern Bitmapset *ExecCheckIndexedAttrsForChanges(ResultRelInfo *resultRelInfo,
- TupleTableSlot *tts_old,
- TupleTableSlot *tts_new);
+extern Bitmapset *ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
extern bool tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
Datum value1, Datum value2);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 898368fb8cb..d8e88817206 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -174,15 +174,29 @@ typedef struct IndexInfo
*/
AttrNumber ii_IndexAttrNumbers[INDEX_MAX_KEYS];
+ /*
+ * All key, expression, sumarizing, and partition attributes referenced by
+ * this index
+ */
+ Bitmapset *ii_IndexedAttrs;
+
/* expr trees for expression entries, or NIL if none */
List *ii_Expressions; /* list of Expr */
/* exec state for expressions, or NIL if none */
List *ii_ExpressionsState; /* list of ExprState */
+ /* attributes exclusively referenced by expression indexes */
+ Bitmapset *ii_ExpressionsAttrs;
/* partial-index predicate, or NIL if none */
List *ii_Predicate; /* list of Expr */
/* exec state for expressions, or NIL if none */
ExprState *ii_PredicateState;
+ /* attributes referenced by the predicate */
+ Bitmapset *ii_PredicateAttrs;
+ /* partial index predicate determined yet? */
+ bool ii_CheckedPredicate;
+ /* amupdate hint used to avoid rechecking predicate */
+ bool ii_PredicateSatisfied;
/* Per-column exclusion operators, or NULL if none */
Oid *ii_ExclusionOps; /* array with one entry per column */
@@ -494,6 +508,11 @@ typedef struct ResultRelInfo
Bitmapset *ri_extraUpdatedCols;
/* true if the above has been computed */
bool ri_extraUpdatedCols_valid;
+
+ /*
+ * For UPDATE a Bitmapset of the attributes that are both indexed and have
+ * changed in value.
+ */
Bitmapset *ri_ChangedIndexedCols;
/* Projection to generate new tuple in an INSERT/UPDATE */
diff --git a/src/test/isolation/expected/insert-conflict-specconflict.out b/src/test/isolation/expected/insert-conflict-specconflict.out
index e34a821c403..54b3981918c 100644
--- a/src/test/isolation/expected/insert-conflict-specconflict.out
+++ b/src/test/isolation/expected/insert-conflict-specconflict.out
@@ -80,6 +80,10 @@ pg_advisory_unlock
t
(1 row)
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
s1: NOTICE: acquiring advisory lock on 2
s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
@@ -172,6 +176,10 @@ pg_advisory_unlock
t
(1 row)
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
s2: NOTICE: acquiring advisory lock on 2
s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
@@ -369,6 +377,10 @@ key|data
step s1_commit: COMMIT;
s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
s2: NOTICE: acquiring advisory lock on 2
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
step s2_upsert: <... completed>
step controller_show: SELECT * FROM upserttest;
key|data
@@ -530,6 +542,14 @@ isolation/insert-conflict-specconflict/s2|transactionid|ExclusiveLock|t
step s2_commit: COMMIT;
s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_4() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 4
+s1: NOTICE: blurt_and_lock_4() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 4
step s1_upsert: <... completed>
step s1_noop:
step controller_show: SELECT * FROM upserttest;
diff --git a/src/test/regress/expected/heap_hot_updates.out b/src/test/regress/expected/heap_hot_updates.out
new file mode 100644
index 00000000000..f6bd8b18af8
--- /dev/null
+++ b/src/test/regress/expected/heap_hot_updates.out
@@ -0,0 +1,1922 @@
+-- ================================================================
+-- Test Suite for Heap-only (HOT) Updates
+-- ================================================================
+-- Setup: Create function to measure HOT updates
+CREATE OR REPLACE FUNCTION check_hot_updates(
+ expected INT,
+ p_table_name TEXT DEFAULT 't',
+ p_schema_name TEXT DEFAULT current_schema()
+)
+RETURNS TABLE (
+ table_name TEXT,
+ total_updates BIGINT,
+ hot_updates BIGINT,
+ hot_update_percentage NUMERIC,
+ matches_expected BOOLEAN
+)
+LANGUAGE plpgsql
+AS $$
+DECLARE
+ v_relid oid;
+ v_qualified_name TEXT;
+ v_hot_updates BIGINT;
+ v_updates BIGINT;
+ v_xact_hot_updates BIGINT;
+ v_xact_updates BIGINT;
+BEGIN
+ -- Force statistics update
+ PERFORM pg_stat_force_next_flush();
+
+ -- Get table OID
+ v_qualified_name := quote_ident(p_schema_name) || '.' || quote_ident(p_table_name);
+ v_relid := v_qualified_name::regclass;
+
+ IF v_relid IS NULL THEN
+ RAISE EXCEPTION 'Table %.% not found', p_schema_name, p_table_name;
+ END IF;
+
+ -- Get cumulative + transaction stats
+ v_hot_updates := COALESCE(pg_stat_get_tuples_hot_updated(v_relid), 0);
+ v_updates := COALESCE(pg_stat_get_tuples_updated(v_relid), 0);
+ v_xact_hot_updates := COALESCE(pg_stat_get_xact_tuples_hot_updated(v_relid), 0);
+ v_xact_updates := COALESCE(pg_stat_get_xact_tuples_updated(v_relid), 0);
+
+ v_hot_updates := v_hot_updates + v_xact_hot_updates;
+ v_updates := v_updates + v_xact_updates;
+
+ RETURN QUERY
+ SELECT
+ p_table_name::TEXT,
+ v_updates::BIGINT,
+ v_hot_updates::BIGINT,
+ CASE WHEN v_updates > 0
+ THEN ROUND((v_hot_updates::numeric / v_updates::numeric * 100)::numeric, 2)
+ ELSE 0
+ END,
+ (v_hot_updates = expected)::BOOLEAN;
+END;
+$$;
+CREATE COLLATION case_insensitive (
+ provider = libc,
+ locale = 'C'
+);
+-- ================================================================
+-- Basic JSONB Expression Index
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_name_idx ON t((docs->'name'));
+INSERT INTO t VALUES (1, '{"name": "alice", "age": 30}');
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET docs = '{"name": "alice", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update indexed JSONB field - should NOT be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update non-indexed field again - should be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 32}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 2 | 66.67 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- JSONB Expression Index an some including columns
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB, status TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_name_idx ON t((docs->'name'));
+INSERT INTO t VALUES (1, '{"name": "alice", "age": 30}', 'ok');
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET docs = '{"name": "alice", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET status = 'not ok' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Partial Index with Predicate Transitions
+-- ================================================================
+CREATE TABLE t(id INT, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_value_idx ON t(value) WHERE value > 10;
+INSERT INTO t VALUES (1, 5);
+-- Both outside predicate - should be HOT
+UPDATE t SET value = 8 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET value = 15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Both inside predicate, value changes - should NOT be HOT
+UPDATE t SET value = 20 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Transition out of predicate - should NOT be HOT
+UPDATE t SET value = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+-- Both outside predicate again - should be HOT
+UPDATE t SET value = 3 WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 5 | 2 | 40.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Expression Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((docs->'status'))
+ WHERE (docs->'priority')::int > 5;
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3}');
+-- Both outside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 4}';
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 10}';
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Inside predicate, status changes - should NOT be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 10}';
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Inside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 8}';
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 2 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Index on JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data);
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "database"]}');
+-- Change tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Change tags again - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Add field without changing existing keys - GIN keys changed (added "note"), NOT HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "note": "test"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Index with Unchanged Keys
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create GIN index on specific path
+CREATE INDEX t_gin_idx ON t USING gin((data->'tags'));
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "sql"], "status": "active"}');
+-- Change non-indexed field - GIN keys on 'tags' unchanged, should be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change indexed tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN with jsonb_path_ops
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data jsonb_path_ops);
+INSERT INTO t VALUES (1, '{"user": {"name": "alice"}, "tags": ["a", "b"]}');
+-- Change value at different path - keys changed, NOT HOT
+UPDATE t SET data = '{"user": {"name": "bob"}, "tags": ["a", "b"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Multi-Column Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, a INT, b INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t(id, abs(a), abs(b));
+INSERT INTO t VALUES (1, -5, -10);
+-- Change sign but not abs value - should be HOT
+UPDATE t SET a = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change abs value - should NOT be HOT
+UPDATE t SET b = -15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Change id - should NOT be HOT
+UPDATE t SET id = 2 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Mixed Index Types (BRIN + Expression)
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_idx ON t USING brin(value);
+CREATE INDEX t_expr_idx ON t((data->'status'));
+INSERT INTO t VALUES (1, 100, '{"status": "active"}');
+-- Update only BRIN column - should be HOT
+UPDATE t SET value = 200 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update only expression column - should NOT be HOT
+UPDATE t SET data = '{"status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update both - should NOT be HOT
+UPDATE t SET value = 300, data = '{"status": "pending"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Expression with COLLATION and BTREE (nbtree) index
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ name TEXT COLLATE case_insensitive
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_lower_idx ON t USING BTREE (name COLLATE case_insensitive);
+INSERT INTO t VALUES (1, 'ALICE');
+-- Change case but not value - should NOT be HOT in BTREE
+UPDATE t SET name = 'Alice' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Change to new value - should NOT be HOT
+UPDATE t SET name = 'BOB' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Array Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_array_len_idx ON t(array_length(tags, 1));
+INSERT INTO t VALUES (1, ARRAY['a', 'b', 'c']);
+-- Same length, different elements - should be HOT
+UPDATE t SET tags = ARRAY['d', 'e', 'f'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Different length - should NOT be HOT
+UPDATE t SET tags = ARRAY['d', 'e'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Nested JSONB Expression and JSONB equality '->' (not '->>')
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_nested_idx ON t((data->'user'->'name'));
+INSERT INTO t VALUES ('{"user": {"name": "alice", "age": 30}}');
+-- Change nested non-indexed field - should be HOT
+UPDATE t SET data = '{"user": {"name": "alice", "age": 31}}';
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change nested indexed field - should NOT be HOT
+UPDATE t SET data = '{"user": {"name": "bob", "age": 31}}';
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Complex Predicate on Multiple JSONB Fields
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((data->'status'))
+ WHERE (data->'priority')::int > 5
+ AND (data->'active')::boolean = true;
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3, "active": true}');
+-- Outside predicate (priority too low) - should be HOT
+UPDATE t SET data = '{"status": "done", "priority": 3, "active": true}';
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": true}';
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Inside predicate, change to outside (active = false) - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": false}';
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Array Index - Order Insensitive Extraction
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ data JSONB
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+-- GIN index on JSONB array (extracts all elements)
+CREATE INDEX t_items_gin ON t USING GIN ((data->'items'));
+INSERT INTO t VALUES (1, '{"items": [1, 2, 3], "status": "active"}');
+-- Update: Reorder array elements
+-- JSONB equality: NOT equal (different arrays)
+-- GIN extraction: Same elements extracted (might allow HOT if not careful)
+UPDATE t SET data = '{"items": [3, 2, 1], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update: Add/remove element
+UPDATE t SET data = '{"items": [1, 2, 3, 4], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- TOASTed Values in Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, large_text TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_substr_idx ON t(substr(large_text, 1, 10));
+INSERT INTO t VALUES (1, repeat('x', 5000) || 'identifier');
+-- Change end of string, prefix unchanged - should be HOT
+UPDATE t SET large_text = repeat('x', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change prefix - should NOT be HOT
+UPDATE t SET large_text = repeat('y', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- TEST: GIN with TOASTed TEXT (tsvector)
+-- ================================================================
+CREATE TABLE t(id INT, content TEXT, search_vec tsvector)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create trigger to maintain tsvector
+CREATE TRIGGER tsvectorupdate_toast
+ BEFORE INSERT OR UPDATE ON t
+ FOR EACH ROW EXECUTE FUNCTION
+ tsvector_update_trigger(search_vec, 'pg_catalog.english', content);
+CREATE INDEX t_gin ON t USING gin(search_vec);
+-- Insert with large content (will be TOASTed)
+INSERT INTO t (id, content) VALUES
+ (1, repeat('important keyword ', 1000) || repeat('filler text ', 10000));
+-- Verify initial state
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('important');
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+-- IMPORTANT: The BEFORE UPDATE trigger modifies search_vec, so by the time
+-- ExecWhichIndexesRequireUpdates() runs, search_vec has already changed.
+-- This means the comparison sees old tsvector vs. trigger-modified tsvector,
+-- not the natural progression. HOT won't happen because the trigger changed
+-- the indexed column.
+-- Update: Even though content keywords unchanged, trigger still fires
+UPDATE t
+SET content = repeat('important keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (trigger modifies search_vec, blocking HOT)
+-- This is actually correct behavior - the trigger updated an indexed column
+-- Update: Change indexed keywords
+UPDATE t
+SET content = repeat('critical keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (index keys changed)
+-- Verify query correctness
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('critical');
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+DROP TABLE t CASCADE;
+-- ================================================================
+-- TEST: GIN with TOASTed JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin((data->'tags'));
+-- Insert with TOASTed JSONB
+INSERT INTO t (id, data) VALUES
+ (1, jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('x', 10000)
+ ));
+-- Update: Change large_field, tags unchanged - should be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT update
+-- Update: Change tags - should NOT be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "sql"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Expected: Still 1 HOT
+-- Verify correctness
+SELECT count(*) FROM t WHERE data->'tags' @> '["database"]'::jsonb;
+ count
+-------
+ 0
+(1 row)
+
+-- Expected: 0 rows
+SELECT count(*) FROM t WHERE data->'tags' @> '["sql"]'::jsonb;
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+DROP TABLE t CASCADE;
+-- ================================================================
+-- TEST: GIN with Array of Large Strings
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin(tags);
+-- Insert with large array elements (might be TOASTed)
+INSERT INTO t (id, tags) VALUES
+ (1, ARRAY[repeat('tag1', 1000), repeat('tag2', 1000)]);
+-- Update: Change to different large values - NOT HOT
+UPDATE t
+SET tags = ARRAY[repeat('tag3', 1000), repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (keys actually changed)
+-- Update: Keep same tag values, just reorder - SHOULD BE HOT
+-- (GIN is order-insensitive: both [tag3,tag4] and [tag4,tag3]
+-- extract to the same sorted key set ['tag3','tag4'])
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000), repeat('tag3', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Expected: 1 HOT (GIN keys semantically identical)
+-- Update: Remove an element - NOT HOT (keys changed)
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Expected: Still 1 HOT (not this one)
+DROP TABLE t CASCADE;
+-- ================================================================
+-- BRIN Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ value INT,
+ description TEXT
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_partial_idx ON t USING brin(value) WHERE value > 100;
+INSERT INTO t VALUES (1, 50, 'below range');
+-- Test 1: Outside predicate
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Validate: Predicate query returns 0 rows
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+ cnt
+-----
+ 0
+(1 row)
+
+-- Test 2: Transition into predicate
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Validate: Predicate query returns 1 row with correct value
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+ cnt | max_val
+-----+---------
+ 1 | 150
+(1 row)
+
+-- Test 3: Inside predicate, value changes
+UPDATE t SET value = 160, description = 'updated again' WHERE id = 1;
+SELECT * FROM check_hot_updates(3);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 3 | 100.00 | t
+(1 row)
+
+-- Validate: Updated value (160) is returned
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+ cnt | max_val
+-----+---------
+ 1 | 160
+(1 row)
+
+-- Test 4: Transition out of predicate
+UPDATE t SET value = 50 WHERE id = 1;
+SELECT * FROM check_hot_updates(4);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 4 | 100.00 | t
+(1 row)
+
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+ cnt
+-----
+ 0
+(1 row)
+
+SELECT id, value, description FROM t;
+ id | value | description
+----+-------+---------------
+ 1 | 50 | updated again
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- HASH Index (Simple Column)
+-- ================================================================
+CREATE TABLE t(id INT, code VARCHAR(20), description TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_idx ON t USING hash(code);
+INSERT INTO t VALUES (1, 'CODE001', 'initial');
+-- Update non-indexed column - should be HOT
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update indexed column - HASH index requires update, NOT HOT
+UPDATE t SET code = 'CODE002' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update both - NOT HOT
+UPDATE t SET code = 'CODE003', description = 'changed' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Back to original code - NOT HOT (different hash bucket location)
+UPDATE t SET code = 'CODE001' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- HASH Index on Expression
+-- ================================================================
+CREATE TABLE t(id INT, email TEXT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_lower_email_idx ON t USING HASH(lower(email));
+INSERT INTO t VALUES (1, '[email protected]', '{"status": "new"}');
+-- Update non-indexed field - should be HOT
+UPDATE t SET data = '{"status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update email with case change only (same lowercase) - should be HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Update email to different lowercase - NOT HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 2 | 66.67 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- HASH Index on JSONB Field
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash((data->'category'));
+INSERT INTO t VALUES (1, '{"category": "books", "title": "PostgreSQL Guide"}');
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET data = '{"category": "books", "title": "PostgreSQL Handbook"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update indexed JSONB field - NOT HOT
+UPDATE t SET data = '{"category": "videos", "title": "PostgreSQL Handbook"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update both - NOT HOT
+UPDATE t SET data = '{"category": "courses", "title": "PostgreSQL Basics"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- Multiple HASH Indexes
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, status VARCHAR, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+CREATE INDEX t_hash_status_idx ON t USING hash(status);
+INSERT INTO t VALUES (1, 'electronics', 'active', 100);
+-- Update non-indexed column - should be HOT
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update one indexed column - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update other indexed column - NOT HOT
+UPDATE t SET status = 'inactive' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Update both indexed columns - NOT HOT
+UPDATE t SET category = 'videos', status = 'pending' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- BRIN vs HASH Comparison
+-- ================================================================
+CREATE TABLE t_brin(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE TABLE t_hash(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_value_idx ON t_brin USING brin(value);
+CREATE INDEX t_hash_value_idx ON t_hash USING hash(value);
+INSERT INTO t_brin VALUES (1, 100, 'initial');
+INSERT INTO t_hash VALUES (1, 100, 'initial');
+-- Same update on both - different HOT behavior expected
+-- BRIN: might allow HOT (range summary unchanged)
+-- HASH: blocks HOT (hash bucket changed)
+UPDATE t_brin SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't_brin');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t_brin | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT (BRIN allows it for single row)
+UPDATE t_hash SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't_hash');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t_hash | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (HASH blocks it)
+DROP TABLE t_brin CASCADE;
+DROP TABLE t_hash CASCADE;
+-- ================================================================
+-- HASH Index with NULL Values
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'electronics', 'initial');
+-- Update indexed column to NULL - NOT HOT (hash value changed)
+UPDATE t SET category = NULL WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT
+-- Update indexed column from NULL to value - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Expected: 1 HOT
+DROP TABLE t CASCADE;
+-- ================================================================
+-- BRIN on JSONB Field
+-- ================================================================
+CREATE TABLE t(id INT, metrics JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- BRIN doesn't directly support JSONB, but we can test on expression
+CREATE INDEX t_brin_count_idx ON t USING brin(
+ CAST(metrics->>'count' AS INTEGER)
+);
+INSERT INTO t VALUES (1, '{"count": "100", "timestamp": "2024-01-01"}');
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET metrics = '{"count": "100", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT
+-- Update indexed field - BRIN allows HOT for single row
+UPDATE t SET metrics = '{"count": "150", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Expected: 2 HOT (BRIN permits single-row updates)
+DROP TABLE t CASCADE;
+-- ================================================================
+-- Mixed BRIN + HASH on Same Table
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, timestamp TIMESTAMP, price NUMERIC, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_timestamp_idx ON t USING brin(timestamp);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'books', '2024-01-01 10:00:00', 29.99, 'initial');
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT
+-- Update BRIN indexed column - allows HOT
+UPDATE t SET timestamp = '2024-01-02 10:00:00' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Expected: 2 HOT
+-- Update HASH indexed column - blocks HOT
+UPDATE t SET category = 'videos' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 2 | 66.67 | t
+(1 row)
+
+-- Expected: 2 HOT (HASH blocks it)
+-- Update price (non-indexed) - should be HOT
+UPDATE t SET price = 39.99 WHERE id = 1;
+SELECT * FROM check_hot_updates(3);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 3 | 75.00 | t
+(1 row)
+
+-- Expected: 3 HOT
+DROP TABLE t CASCADE;
+-- ================================================================
+-- Index both on a field in a JSONB document, and the document
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t((docs->'name'));
+CREATE INDEX t_docs_col_idx ON t(docs);
+INSERT INTO t VALUES (1, '{"name": "john", "data": "some data"}');
+-- Update impacts index on whole docment attribute, can't go HOT
+UPDATE t SET docs='{"name": "john", "data": "some other data"}' WHERE id=1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- Two indexes on a JSONB document, one partial
+-- ================================================================
+CREATE TABLE t (docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+INSERT INTO t (docs) VALUES ('{"a": 0, "b": 0}');
+INSERT INTO t (docs) SELECT jsonb_build_object('b', n) FROM generate_series(100, 10000) as n;
+CREATE INDEX t_idx_a ON t ((docs->'a'));
+CREATE INDEX t_idx_b ON t ((docs->'b')) WHERE (docs->'b')::numeric > 9;
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+-- Leave 'a' unchanged but modify 'b' to a value outside of the index predicate.
+-- This should be a HOT update because neither index is changed.
+UPDATE t SET docs = jsonb_build_object('a', 0, 'b', 1) WHERE (docs->'a')::numeric = 0;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Check to make sure that the index does not contain a value for 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->'b')::numeric > 9 AND (docs->'b')::numeric < 100;
+ QUERY PLAN
+-------------------------------------------------------------
+ Index Scan using t_idx_b on t
+ Filter: (((docs -> 'b'::text))::numeric < '100'::numeric)
+(2 rows)
+
+SELECT * FROM t WHERE (docs->'b')::numeric > 9 AND (docs->'b')::numeric < 100;
+ docs
+------
+(0 rows)
+
+-- Leave 'a' unchanged but modify 'b' to a value within the index predicate.
+-- This represents a change for field 'b' from unindexed to indexed and so
+-- this should not take the HOT path.
+UPDATE t SET docs = jsonb_build_object('a', 0, 'b', 10) WHERE (docs->'a')::numeric = 0;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Check to make sure that the index contains the new value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->'b')::numeric > 9 AND (docs->'b')::numeric < 100;
+ QUERY PLAN
+-------------------------------------------------------------
+ Index Scan using t_idx_b on t
+ Filter: (((docs -> 'b'::text))::numeric < '100'::numeric)
+(2 rows)
+
+SELECT * FROM t WHERE (docs->'b')::numeric > 9 AND (docs->'b')::numeric < 100;
+ docs
+-------------------
+ {"a": 0, "b": 10}
+(1 row)
+
+-- This update modifies the value of 'a', an indexed field, so it also cannot
+-- be a HOT update.
+UPDATE t SET docs = jsonb_build_object('a', 1, 'b', 10) WHERE (docs->'b')::numeric = 10;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- This update changes both 'a' and 'b' to new values this cannot use the HOT path.
+UPDATE t SET docs = jsonb_build_object('a', 2, 'b', 12) WHERE (docs->'b')::numeric = 10;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+-- Check to make sure that the index contains the new value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->'b')::numeric > 9 AND (docs->'b')::numeric < 100;
+ QUERY PLAN
+-------------------------------------------------------------
+ Index Scan using t_idx_b on t
+ Filter: (((docs -> 'b'::text))::numeric < '100'::numeric)
+(2 rows)
+
+SELECT * FROM t WHERE (docs->'b')::numeric > 9 AND (docs->'b')::numeric < 100;
+ docs
+-------------------
+ {"a": 2, "b": 12}
+(1 row)
+
+-- This update changes 'b' to a value outside its predicate requiring that
+-- we remove it from the index. That's a transition that can't be done
+-- during a HOT update.
+UPDATE t SET docs = jsonb_build_object('a', 2, 'b', 1) WHERE (docs->'b')::numeric = 12;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 5 | 1 | 20.00 | t
+(1 row)
+
+-- Check to make sure that the index no longer contains the value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->'b')::numeric > 9 AND (docs->'b')::numeric < 100;
+ QUERY PLAN
+-------------------------------------------------------------
+ Index Scan using t_idx_b on t
+ Filter: (((docs -> 'b'::text))::numeric < '100'::numeric)
+(2 rows)
+
+SELECT * FROM t WHERE (docs->'b')::numeric > 9 AND (docs->'b')::numeric < 100;
+ docs
+------
+(0 rows)
+
+DROP TABLE t CASCADE;
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+-- ================================================================
+-- Tests to check expression indexes
+-- ================================================================
+CREATE TABLE t(a INT, b INT) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx_a ON t(abs(a)) WHERE abs(a) > 10;
+CREATE INDEX t_idx_b ON t(abs(b));
+INSERT INTO t VALUES (-1, -1), (-2, -2), (-3, -3), (-4, -4), (-5, -5);
+INSERT INTO t SELECT m, n FROM generate_series(-10000, -10) AS m, abs(m) AS n;
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+-- The indexed value of b hasn't changed, this should be a HOT update.
+-- (-5, -5) -> (-5, 1)
+UPDATE t SET b = 5 WHERE a = -5;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+EXPLAIN (COSTS OFF) SELECT b FROM t WHERE abs(b) < 10 AND abs(b) > 0;
+ QUERY PLAN
+------------------------------------------------
+ Index Scan using t_idx_b on t
+ Index Cond: ((abs(b) < 10) AND (abs(b) > 0))
+(2 rows)
+
+SELECT b FROM t WHERE abs(b) < 10 AND abs(b) > 0;
+ b
+----
+ -1
+ -2
+ -3
+ -4
+ 5
+(5 rows)
+
+-- Now that we're not checking the predicate of the partial index, this
+-- update of a from -5 to 5 should be HOT because we should ignore the
+-- predicate and check the expression and find it unchanged.
+-- (-5, 1) -> (5, 1)
+UPDATE t SET a = 5 WHERE a = -5;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- This update moves a into the partial index and should not
+-- be HOT. Let's make sure of that and check the index as well.
+-- (-4, -4) -> (-11, -4)
+UPDATE t SET a = -11 WHERE a = -4;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 2 | 66.67 | t
+(1 row)
+
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE abs(a) > 10 AND abs(a) < 15;
+ QUERY PLAN
+-------------------------------
+ Index Scan using t_idx_a on t
+ Index Cond: (abs(a) < 15)
+(2 rows)
+
+SELECT * FROM t WHERE abs(a) > 10 AND abs(a) < 15;
+ a | b
+-----+----
+ -10 | 10
+ -11 | -4
+ -11 | 11
+ -12 | 12
+ -13 | 13
+ -14 | 14
+(6 rows)
+
+-- (-11, -4) -> (11, -4)
+UPDATE t SET a = 11 WHERE b = -4;
+SELECT * FROM check_hot_updates(3);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 3 | 75.00 | t
+(1 row)
+
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE abs(a) > 10 AND abs(a) < 15;
+ QUERY PLAN
+-------------------------------
+ Index Scan using t_idx_a on t
+ Index Cond: (abs(a) < 15)
+(2 rows)
+
+SELECT * FROM t WHERE abs(a) > 10 AND abs(a) < 15;
+ a | b
+-----+----
+ -10 | 10
+ 11 | -4
+ -11 | 11
+ -12 | 12
+ -13 | 13
+ -14 | 14
+(6 rows)
+
+-- (11, -4) -> (-4, -4)
+UPDATE t SET a = -4 WHERE b = -4;
+SELECT * FROM check_hot_updates(3);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 5 | 3 | 60.00 | t
+(1 row)
+
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE abs(a) > 10 AND abs(a) < 15;
+ QUERY PLAN
+-------------------------------
+ Index Scan using t_idx_a on t
+ Index Cond: (abs(a) < 15)
+(2 rows)
+
+SELECT * FROM t WHERE abs(a) > 10 AND abs(a) < 15;
+ a | b
+-----+----
+ -10 | 10
+ -11 | 11
+ -12 | 12
+ -13 | 13
+ -14 | 14
+(5 rows)
+
+-- This update of a from 5 to -1 is HOT despite that attribute
+-- being indexed because the before and after values for the
+-- partial index predicate are outside the index definition.
+-- (5, 1) -> (-1, 1)
+UPDATE t SET a = -1 WHERE a = 5;
+SELECT * FROM check_hot_updates(4);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 6 | 4 | 66.67 | t
+(1 row)
+
+-- This update of a from -2 to -1 will be HOT because the before/after values
+-- of a are both outside the predicate of the partial index.
+-- (-1, 1) -> (-2, 1)
+UPDATE t SET a = -2 WHERE b = -2;
+SELECT * FROM check_hot_updates(5);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 7 | 5 | 71.43 | t
+(1 row)
+
+-- The indexed value for b isn't changing, this should be HOT.
+-- (-2, -2) -> (-2, 2)
+UPDATE t SET b = 2 WHERE b = -2;
+SELECT * FROM check_hot_updates(6);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 8 | 6 | 75.00 | t
+(1 row)
+
+EXPLAIN (COSTS OFF) SELECT b FROM t WHERE abs(b) < 10 AND abs(b) > 0;
+ QUERY PLAN
+------------------------------------------------
+ Index Scan using t_idx_b on t
+ Index Cond: ((abs(b) < 10) AND (abs(b) > 0))
+(2 rows)
+
+SELECT b FROM t WHERE abs(b) < 10 AND abs(b) > 0;
+ b
+----
+ -1
+ 2
+ -3
+ -4
+ 5
+(5 rows)
+
+SELECT * FROM t where a > -10 AND a < 10;
+ a | b
+----+----
+ -1 | -1
+ -3 | -3
+ -1 | 5
+ -4 | -4
+ -2 | 2
+(5 rows)
+
+-- Before and after values for a are outside the predicate of the index,
+-- and because we're checking this should be HOT.
+-- (-2, 1) -> (5, 1)
+-- (-2, -2) -> (5, -2)
+UPDATE t SET a = 5 WHERE a = -1;
+SELECT * FROM check_hot_updates(8);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 10 | 8 | 80.00 | t
+(1 row)
+
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE abs(a) > 10 AND abs(a) < 15;
+ QUERY PLAN
+-------------------------------
+ Index Scan using t_idx_a on t
+ Index Cond: (abs(a) < 15)
+(2 rows)
+
+SELECT * FROM t WHERE abs(a) > 10 AND abs(a) < 15;
+ a | b
+-----+----
+ -10 | 10
+ -11 | 11
+ -12 | 12
+ -13 | 13
+ -14 | 14
+(5 rows)
+
+DROP TABLE t CASCADE;
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+-- ================================================================
+-- JSONB with two indexes each on separate fields, one partial
+-- ================================================================
+CREATE TABLE t(docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t((docs->'a')) WHERE (docs->'b')::integer = 1;
+INSERT INTO t VALUES ('{"a": 1, "b": 1}');
+EXPLAIN (COSTS OFF) SELECT * FROM t;
+ QUERY PLAN
+---------------
+ Seq Scan on t
+(1 row)
+
+SELECT * FROM t;
+ docs
+------------------
+ {"a": 1, "b": 1}
+(1 row)
+
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->'b')::integer = 1;
+ QUERY PLAN
+----------------------------------
+ Index Scan using t_docs_idx on t
+(1 row)
+
+SELECT * FROM t WHERE (docs->'b')::integer = 1;
+ docs
+------------------
+ {"a": 1, "b": 1}
+(1 row)
+
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 0 | 0 | 0 | t
+(1 row)
+
+UPDATE t SET docs='{"a": 1, "b": 0}';
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+SELECT * FROM t WHERE (docs->'b')::integer = 1;
+ docs
+------
+(0 rows)
+
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+DROP TABLE t CASCADE;
+-- ================================================================
+-- Tests for multi-column indexes
+-- ================================================================
+CREATE TABLE t(id INT, docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t(id, (docs->'a'));
+INSERT INTO t VALUES (1, '{"a": 1, "b": 1}');
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer > 0;
+ QUERY PLAN
+------------------------------------------------
+ Index Scan using t_docs_idx on t
+ Index Cond: (id > 0)
+ Filter: (((docs -> 'a'::text))::integer > 0)
+(3 rows)
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer > 0;
+ id | docs
+----+------------------
+ 1 | {"a": 1, "b": 1}
+(1 row)
+
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 0 | 0 | 0 | t
+(1 row)
+
+-- Changing the id attribute which is an indexed attribute should
+-- prevent HOT updates.
+UPDATE t SET id = 2;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer > 0;
+ id | docs
+----+------------------
+ 2 | {"a": 1, "b": 1}
+(1 row)
+
+-- Changing the docs->'a' field in the indexed attribute 'docs'
+-- should prevent HOT updates.
+UPDATE t SET docs='{"a": -2, "b": 1}';
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer < 0;
+ id | docs
+----+-------------------
+ 2 | {"a": -2, "b": 1}
+(1 row)
+
+-- Leaving the docs->'a' attribute unchanged means that the expression
+-- is unchanged and because the 'id' attribute isn't in the modified
+-- set the indexed tuple is unchanged, this can go HOT.
+UPDATE t SET docs='{"a": -2, "b": 2}';
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer < 0;
+ id | docs
+----+-------------------
+ 2 | {"a": -2, "b": 2}
+(1 row)
+
+-- Here we change the 'id' attribute and the 'docs' attribute setting
+-- the expression docs->'a' to a new value, this cannot be a HOT update.
+UPDATE t SET id = 3, docs='{"a": 3, "b": 3}';
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer > 0;
+ id | docs
+----+------------------
+ 3 | {"a": 3, "b": 3}
+(1 row)
+
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+DROP TABLE t CASCADE;
+-- ================================================================
+-- Relation with unique constraint, partial index
+-- ================================================================
+CREATE TABLE users (
+ user_id serial primary key,
+ name VARCHAR(255) NOT NULL,
+ email VARCHAR(255) NOT NULL,
+ EXCLUDE USING btree (lower(email) WITH =)
+);
+-- Add some data to the table and then update it in ways that should and should
+-- not be HOT updates.
+INSERT INTO users (name, email) VALUES
+('user1', '[email protected]'),
+('user2', '[email protected]'),
+('taken', '[email protected]'),
+('you', '[email protected]'),
+('taken', '[email protected]');
+-- Should fail because of the unique constraint on the email column.
+UPDATE users SET email = '[email protected]' WHERE email = '[email protected]';
+ERROR: conflicting key value violates exclusion constraint "users_lower_excl"
+DETAIL: Key (lower(email::text))=([email protected]) conflicts with existing key (lower(email::text))=([email protected]).
+SELECT * FROM check_hot_updates(0, 'users');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ users | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Should succeed because the email column is not being updated and should go HOT.
+UPDATE users SET name = 'foo' WHERE email = '[email protected]';
+SELECT * FROM check_hot_updates(1, 'users');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ users | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Create a partial index on the email column, updates
+CREATE INDEX idx_users_email_no_example ON users (lower(email)) WHERE lower(email) LIKE '%@example.com%';
+-- An update that changes the email column but not the indexed portion of it and falls outside the constraint.
+-- Shouldn't be a HOT update because of the exclusion constraint.
+UPDATE users SET email = '[email protected]' WHERE name = 'you';
+SELECT * FROM check_hot_updates(1, 'users');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ users | 3 | 1 | 33.33 | t
+(1 row)
+
+-- An update that changes the email column but not the indexed portion of it and falls within the constraint.
+-- Again, should fail constraint and fail to be a HOT update.
+UPDATE users SET email = '[email protected]' WHERE name = 'you';
+ERROR: conflicting key value violates exclusion constraint "users_lower_excl"
+DETAIL: Key (lower(email::text))=([email protected]) conflicts with existing key (lower(email::text))=([email protected]).
+SELECT * FROM check_hot_updates(1, 'users');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ users | 4 | 1 | 25.00 | t
+(1 row)
+
+DROP TABLE users CASCADE;
+-- ================================================================
+-- Constraints spoiling HOT updates, this time with a range.
+-- ================================================================
+CREATE TABLE events (
+ id serial primary key,
+ name VARCHAR(255) NOT NULL,
+ event_time tstzrange,
+ constraint no_screening_time_overlap exclude using gist (
+ event_time WITH &&
+ )
+);
+-- Add two non-overlapping events.
+INSERT INTO events (id, event_time, name)
+VALUES
+ (1, '["2023-01-01 19:00:00", "2023-01-01 20:45:00"]', 'event1'),
+ (2, '["2023-01-01 21:00:00", "2023-01-01 21:45:00"]', 'event2');
+-- Update the first event to overlap with the second, should fail the constraint and not be HOT.
+UPDATE events SET event_time = '["2023-01-01 20:00:00", "2023-01-01 21:45:00"]' WHERE id = 1;
+ERROR: conflicting key value violates exclusion constraint "no_screening_time_overlap"
+DETAIL: Key (event_time)=(["Sun Jan 01 20:00:00 2023 PST","Sun Jan 01 21:45:00 2023 PST"]) conflicts with existing key (event_time)=(["Sun Jan 01 21:00:00 2023 PST","Sun Jan 01 21:45:00 2023 PST"]).
+SELECT * FROM check_hot_updates(0, 'events');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ events | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Update the first event to not overlap with the second, again not HOT due to the constraint.
+UPDATE events SET event_time = '["2023-01-01 22:00:00", "2023-01-01 22:45:00"]' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 'events');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ events | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Update the first event to not overlap with the second, this time we're HOT because we don't overlap with the constraint.
+UPDATE events SET name = 'new name here' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 'events');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ events | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE events CASCADE;
+-- ================================================================
+-- Ensure that only the modified summarizing indexes are updated.
+-- ================================================================
+CREATE TABLE ex (id SERIAL primary key, att1 JSONB, att2 text, att3 text, att4 text) WITH (fillfactor = 60);
+CREATE INDEX ex_expr1_idx ON ex USING btree((att1->'data'));
+CREATE INDEX ex_sumr1_idx ON ex USING BRIN(att2);
+CREATE INDEX ex_expr2_idx ON ex USING btree((att1->'a'));
+CREATE INDEX ex_expr3_idx ON ex USING btree((att1->'b'));
+CREATE INDEX ex_expr4_idx ON ex USING btree((att1->'c'));
+CREATE INDEX ex_sumr2_idx ON ex USING BRIN(att3);
+CREATE INDEX ex_sumr3_idx ON ex USING BRIN(att4);
+CREATE INDEX ex_expr5_idx ON ex USING btree((att1->'d'));
+INSERT INTO ex (att1, att2) VALUES ('{"data": []}'::json, 'nothing special');
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+--------------+-----------------+------+------
+ 1 | {"data": []} | nothing special | |
+(1 row)
+
+-- Update att2 and att4 both are BRIN/summarizing indexes, this should be a HOT update and
+-- only update two of the three summarizing indexes.
+UPDATE ex SET att2 = 'special indeed', att4 = 'whatever';
+SELECT * FROM check_hot_updates(1, 'ex');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ ex | 1 | 1 | 100.00 | t
+(1 row)
+
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+--------------+----------------+------+----------
+ 1 | {"data": []} | special indeed | | whatever
+(1 row)
+
+-- Update att1 and att2, only one is BRIN/summarizing, this should NOT be a HOT update.
+UPDATE ex SET att1 = att1 || '{"data": "howdy"}', att2 = 'special, so special';
+SELECT * FROM check_hot_updates(1, 'ex');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ ex | 2 | 1 | 50.00 | t
+(1 row)
+
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+-------------------+---------------------+------+----------
+ 1 | {"data": "howdy"} | special, so special | | whatever
+(1 row)
+
+-- Update att2, att3, and att4 all are BRIN/summarizing indexes, this should be a HOT update
+-- and yet still update all three summarizing indexes.
+UPDATE ex SET att2 = 'a', att3 = 'b', att4 = 'c';
+SELECT * FROM check_hot_updates(2, 'ex');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ ex | 3 | 2 | 66.67 | t
+(1 row)
+
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+-------------------+------+------+------
+ 1 | {"data": "howdy"} | a | b | c
+(1 row)
+
+-- Update att1, att2, and att3 all modified values are BRIN/summarizing indexes, this should be a HOT update
+-- and yet still update all three summarizing indexes.
+UPDATE ex SET att1 = '{"data": "howdy"}', att2 = 'd', att3 = 'e';
+SELECT * FROM check_hot_updates(3, 'ex');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ ex | 4 | 3 | 75.00 | t
+(1 row)
+
+SELECT * FROM ex;
+ id | att1 | att2 | att3 | att4
+----+-------------------+------+------+------
+ 1 | {"data": "howdy"} | d | e | c
+(1 row)
+
+DROP TABLE ex CASCADE;
+-- ================================================================
+-- Don't update unmodified summarizing indexes but do allow HOT
+-- ================================================================
+CREATE TABLE ex (att1 JSONB, att2 text) WITH (fillfactor = 60);
+CREATE INDEX ex_expr1_idx ON ex USING btree((att1->'data'));
+CREATE INDEX ex_sumr1_idx ON ex USING BRIN(att2);
+INSERT INTO ex VALUES ('{"data": []}', 'nothing special');
+-- Update the unindexed value of att1, this should be a HOT update and and should
+-- update the summarizing index.
+UPDATE ex SET att1 = att1 || '{"status": "stalemate"}';
+SELECT * FROM check_hot_updates(1, 'ex');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ ex | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update the indexed value of att2, a summarized value, this is a summarized
+-- only update and should use the HOT path while still triggering an update to
+-- the summarizing BRIN index.
+UPDATE ex SET att2 = 'special indeed';
+SELECT * FROM check_hot_updates(2, 'ex');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ ex | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Update to att1 doesn't change the indexed value while the update to att2 does,
+-- this again is a summarized only update and should use the HOT path as well as
+-- trigger an update to the BRIN index.
+UPDATE ex SET att1 = att1 || '{"status": "checkmate"}', att2 = 'special, so special';
+SELECT * FROM check_hot_updates(3, 'ex');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ ex | 3 | 3 | 100.00 | t
+(1 row)
+
+-- This updates both indexes, the expression index on att1 and the summarizing
+-- index on att2. This should not be a HOT update because there are modified
+-- indexes and only some are summarized, not all. This should force all
+-- indexes to be updated.
+UPDATE ex SET att1 = att1 || '{"data": [1,2,3]}', att2 = 'do you want to play a game?';
+SELECT * FROM check_hot_updates(3, 'ex');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ ex | 4 | 3 | 75.00 | t
+(1 row)
+
+DROP TABLE ex CASCADE;
+-- ================================================================
+-- Ensure custom type equality operators are used
+-- ================================================================
+CREATE TYPE my_custom_type AS (val int);
+-- Comparison functions (returns boolean)
+CREATE FUNCTION my_custom_lt(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val < b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_le(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val <= b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_eq(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val = b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_ge(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val >= b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_gt(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val > b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+CREATE FUNCTION my_custom_ne(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val != b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+-- Comparison function (returns -1, 0, 1)
+CREATE FUNCTION my_custom_cmp(a my_custom_type, b my_custom_type) RETURNS int AS $$
+BEGIN
+ IF a.val < b.val THEN
+ RETURN -1;
+ ELSIF a.val > b.val THEN
+ RETURN 1;
+ ELSE
+ RETURN 0;
+ END IF;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+-- Create the operators
+CREATE OPERATOR < (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_lt,
+ COMMUTATOR = >,
+ NEGATOR = >=
+);
+CREATE OPERATOR <= (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_le,
+ COMMUTATOR = >=,
+ NEGATOR = >
+);
+CREATE OPERATOR = (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_eq,
+ COMMUTATOR = =,
+ NEGATOR = <>
+);
+CREATE OPERATOR >= (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_ge,
+ COMMUTATOR = <=,
+ NEGATOR = <
+);
+CREATE OPERATOR > (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_gt,
+ COMMUTATOR = <,
+ NEGATOR = <=
+);
+CREATE OPERATOR <> (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_ne,
+ COMMUTATOR = <>,
+ NEGATOR = =
+);
+-- Create the operator class (including the support function)
+CREATE OPERATOR CLASS my_custom_ops
+ DEFAULT FOR TYPE my_custom_type USING btree AS
+ OPERATOR 1 <,
+ OPERATOR 2 <=,
+ OPERATOR 3 =,
+ OPERATOR 4 >=,
+ OPERATOR 5 >,
+ FUNCTION 1 my_custom_cmp(my_custom_type, my_custom_type);
+-- Create the table
+CREATE TABLE my_table (
+ id int,
+ custom_val my_custom_type
+);
+-- Insert some data
+INSERT INTO my_table (id, custom_val) VALUES
+(1, ROW(3)::my_custom_type),
+(2, ROW(1)::my_custom_type),
+(3, ROW(4)::my_custom_type),
+(4, ROW(2)::my_custom_type);
+-- Create a function to use when indexing
+CREATE OR REPLACE FUNCTION abs_val(val my_custom_type) RETURNS int AS $$
+BEGIN
+ RETURN abs(val.val);
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+-- Create the index
+CREATE INDEX idx_custom_val_abs ON my_table (abs_val(custom_val));
+-- Update 1
+UPDATE my_table SET custom_val = ROW(5)::my_custom_type WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 'my_table');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ my_table | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Update 2
+UPDATE my_table SET custom_val = ROW(0)::my_custom_type WHERE custom_val < ROW(3)::my_custom_type;
+SELECT * FROM check_hot_updates(0, 'my_table');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ my_table | 3 | 0 | 0.00 | t
+(1 row)
+
+-- Update 3
+UPDATE my_table SET custom_val = ROW(6)::my_custom_type WHERE id = 3;
+SELECT * FROM check_hot_updates(0, 'my_table');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ my_table | 4 | 0 | 0.00 | t
+(1 row)
+
+-- Update 4
+UPDATE my_table SET id = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 'my_table');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ my_table | 5 | 1 | 20.00 | t
+(1 row)
+
+-- Query using the index
+SELECT * FROM my_table WHERE abs_val(custom_val) = 6;
+ id | custom_val
+----+------------
+ 3 | (6)
+(1 row)
+
+-- Clean up test case
+DROP TABLE my_table CASCADE;
+DROP OPERATOR CLASS my_custom_ops USING btree CASCADE;
+DROP OPERATOR < (my_custom_type, my_custom_type);
+DROP OPERATOR <= (my_custom_type, my_custom_type);
+DROP OPERATOR = (my_custom_type, my_custom_type);
+DROP OPERATOR >= (my_custom_type, my_custom_type);
+DROP OPERATOR > (my_custom_type, my_custom_type);
+DROP OPERATOR <> (my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_lt(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_le(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_eq(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_ge(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_gt(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_ne(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_cmp(my_custom_type, my_custom_type);
+DROP FUNCTION abs_val(my_custom_type);
+DROP TYPE my_custom_type CASCADE;
+-- Cleanup
+DROP FUNCTION check_hot_updates(int, text, text);
+DROP COLLATION case_insensitive;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index f56482fb9f1..9520d0d4601 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -125,6 +125,12 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression compression_lz4 memoize stats predicate numa eager_aggregate
+
+# ----------
+# Another group of parallel tests, these focused on heap HOT updates
+# ----------
+test: heap_hot_updates
+
# event_trigger depends on create_am and cannot run concurrently with
# any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/sql/heap_hot_updates.sql b/src/test/regress/sql/heap_hot_updates.sql
new file mode 100644
index 00000000000..8d5510989df
--- /dev/null
+++ b/src/test/regress/sql/heap_hot_updates.sql
@@ -0,0 +1,1325 @@
+-- ================================================================
+-- Test Suite for Heap-only (HOT) Updates
+-- ================================================================
+
+-- Setup: Create function to measure HOT updates
+CREATE OR REPLACE FUNCTION check_hot_updates(
+ expected INT,
+ p_table_name TEXT DEFAULT 't',
+ p_schema_name TEXT DEFAULT current_schema()
+)
+RETURNS TABLE (
+ table_name TEXT,
+ total_updates BIGINT,
+ hot_updates BIGINT,
+ hot_update_percentage NUMERIC,
+ matches_expected BOOLEAN
+)
+LANGUAGE plpgsql
+AS $$
+DECLARE
+ v_relid oid;
+ v_qualified_name TEXT;
+ v_hot_updates BIGINT;
+ v_updates BIGINT;
+ v_xact_hot_updates BIGINT;
+ v_xact_updates BIGINT;
+BEGIN
+ -- Force statistics update
+ PERFORM pg_stat_force_next_flush();
+
+ -- Get table OID
+ v_qualified_name := quote_ident(p_schema_name) || '.' || quote_ident(p_table_name);
+ v_relid := v_qualified_name::regclass;
+
+ IF v_relid IS NULL THEN
+ RAISE EXCEPTION 'Table %.% not found', p_schema_name, p_table_name;
+ END IF;
+
+ -- Get cumulative + transaction stats
+ v_hot_updates := COALESCE(pg_stat_get_tuples_hot_updated(v_relid), 0);
+ v_updates := COALESCE(pg_stat_get_tuples_updated(v_relid), 0);
+ v_xact_hot_updates := COALESCE(pg_stat_get_xact_tuples_hot_updated(v_relid), 0);
+ v_xact_updates := COALESCE(pg_stat_get_xact_tuples_updated(v_relid), 0);
+
+ v_hot_updates := v_hot_updates + v_xact_hot_updates;
+ v_updates := v_updates + v_xact_updates;
+
+ RETURN QUERY
+ SELECT
+ p_table_name::TEXT,
+ v_updates::BIGINT,
+ v_hot_updates::BIGINT,
+ CASE WHEN v_updates > 0
+ THEN ROUND((v_hot_updates::numeric / v_updates::numeric * 100)::numeric, 2)
+ ELSE 0
+ END,
+ (v_hot_updates = expected)::BOOLEAN;
+END;
+$$;
+
+CREATE COLLATION case_insensitive (
+ provider = libc,
+ locale = 'C'
+);
+
+-- ================================================================
+-- Basic JSONB Expression Index
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_name_idx ON t((docs->'name'));
+INSERT INTO t VALUES (1, '{"name": "alice", "age": 30}');
+
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET docs = '{"name": "alice", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update indexed JSONB field - should NOT be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update non-indexed field again - should be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 32}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+
+DROP TABLE t;
+
+-- ================================================================
+-- JSONB Expression Index an some including columns
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB, status TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_name_idx ON t((docs->'name'));
+INSERT INTO t VALUES (1, '{"name": "alice", "age": 30}', 'ok');
+
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET docs = '{"name": "alice", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET status = 'not ok' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+
+DROP TABLE t;
+
+-- ================================================================
+-- Partial Index with Predicate Transitions
+-- ================================================================
+CREATE TABLE t(id INT, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_value_idx ON t(value) WHERE value > 10;
+INSERT INTO t VALUES (1, 5);
+
+-- Both outside predicate - should be HOT
+UPDATE t SET value = 8 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET value = 15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Both inside predicate, value changes - should NOT be HOT
+UPDATE t SET value = 20 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Transition out of predicate - should NOT be HOT
+UPDATE t SET value = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Both outside predicate again - should be HOT
+UPDATE t SET value = 3 WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+
+DROP TABLE t;
+
+-- ================================================================
+-- Expression Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((docs->'status'))
+ WHERE (docs->'priority')::int > 5;
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3}');
+
+-- Both outside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 4}';
+SELECT * FROM check_hot_updates(1);
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 10}';
+SELECT * FROM check_hot_updates(1);
+
+-- Inside predicate, status changes - should NOT be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 10}';
+SELECT * FROM check_hot_updates(1);
+
+-- Inside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 8}';
+SELECT * FROM check_hot_updates(2);
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN Index on JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data);
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "database"]}');
+
+-- Change tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+
+-- Change tags again - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+
+-- Add field without changing existing keys - GIN keys changed (added "note"), NOT HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "note": "test"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN Index with Unchanged Keys
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create GIN index on specific path
+CREATE INDEX t_gin_idx ON t USING gin((data->'tags'));
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "sql"], "status": "active"}');
+
+-- Change non-indexed field - GIN keys on 'tags' unchanged, should be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Change indexed tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN with jsonb_path_ops
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data jsonb_path_ops);
+INSERT INTO t VALUES (1, '{"user": {"name": "alice"}, "tags": ["a", "b"]}');
+
+-- Change value at different path - keys changed, NOT HOT
+UPDATE t SET data = '{"user": {"name": "bob"}, "tags": ["a", "b"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+
+DROP TABLE t;
+
+-- ================================================================
+-- Multi-Column Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, a INT, b INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t(id, abs(a), abs(b));
+INSERT INTO t VALUES (1, -5, -10);
+
+-- Change sign but not abs value - should be HOT
+UPDATE t SET a = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Change abs value - should NOT be HOT
+UPDATE t SET b = -15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Change id - should NOT be HOT
+UPDATE t SET id = 2 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t;
+
+-- ================================================================
+-- Mixed Index Types (BRIN + Expression)
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_idx ON t USING brin(value);
+CREATE INDEX t_expr_idx ON t((data->'status'));
+INSERT INTO t VALUES (1, 100, '{"status": "active"}');
+
+-- Update only BRIN column - should be HOT
+UPDATE t SET value = 200 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update only expression column - should NOT be HOT
+UPDATE t SET data = '{"status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update both - should NOT be HOT
+UPDATE t SET value = 300, data = '{"status": "pending"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t;
+
+-- ================================================================
+-- Expression with COLLATION and BTREE (nbtree) index
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ name TEXT COLLATE case_insensitive
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+
+CREATE INDEX t_lower_idx ON t USING BTREE (name COLLATE case_insensitive);
+
+INSERT INTO t VALUES (1, 'ALICE');
+
+-- Change case but not value - should NOT be HOT in BTREE
+UPDATE t SET name = 'Alice' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+
+-- Change to new value - should NOT be HOT
+UPDATE t SET name = 'BOB' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+
+DROP TABLE t;
+
+-- ================================================================
+-- Array Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_array_len_idx ON t(array_length(tags, 1));
+INSERT INTO t VALUES (1, ARRAY['a', 'b', 'c']);
+
+-- Same length, different elements - should be HOT
+UPDATE t SET tags = ARRAY['d', 'e', 'f'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Different length - should NOT be HOT
+UPDATE t SET tags = ARRAY['d', 'e'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t;
+
+-- ================================================================
+-- Nested JSONB Expression and JSONB equality '->' (not '->>')
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_nested_idx ON t((data->'user'->'name'));
+INSERT INTO t VALUES ('{"user": {"name": "alice", "age": 30}}');
+
+-- Change nested non-indexed field - should be HOT
+UPDATE t SET data = '{"user": {"name": "alice", "age": 31}}';
+SELECT * FROM check_hot_updates(1);
+
+-- Change nested indexed field - should NOT be HOT
+UPDATE t SET data = '{"user": {"name": "bob", "age": 31}}';
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t;
+
+-- ================================================================
+-- Complex Predicate on Multiple JSONB Fields
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((data->'status'))
+ WHERE (data->'priority')::int > 5
+ AND (data->'active')::boolean = true;
+
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3, "active": true}');
+
+-- Outside predicate (priority too low) - should be HOT
+UPDATE t SET data = '{"status": "done", "priority": 3, "active": true}';
+SELECT * FROM check_hot_updates(1);
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": true}';
+SELECT * FROM check_hot_updates(1);
+
+-- Inside predicate, change to outside (active = false) - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": false}';
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN Array Index - Order Insensitive Extraction
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ data JSONB
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+
+-- GIN index on JSONB array (extracts all elements)
+CREATE INDEX t_items_gin ON t USING GIN ((data->'items'));
+
+INSERT INTO t VALUES (1, '{"items": [1, 2, 3], "status": "active"}');
+
+-- Update: Reorder array elements
+-- JSONB equality: NOT equal (different arrays)
+-- GIN extraction: Same elements extracted (might allow HOT if not careful)
+UPDATE t SET data = '{"items": [3, 2, 1], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update: Add/remove element
+UPDATE t SET data = '{"items": [1, 2, 3, 4], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t;
+
+-- ================================================================
+-- TOASTed Values in Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, large_text TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_substr_idx ON t(substr(large_text, 1, 10));
+
+INSERT INTO t VALUES (1, repeat('x', 5000) || 'identifier');
+
+-- Change end of string, prefix unchanged - should be HOT
+UPDATE t SET large_text = repeat('x', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Change prefix - should NOT be HOT
+UPDATE t SET large_text = repeat('y', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t;
+
+-- ================================================================
+-- TEST: GIN with TOASTed TEXT (tsvector)
+-- ================================================================
+CREATE TABLE t(id INT, content TEXT, search_vec tsvector)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+-- Create trigger to maintain tsvector
+CREATE TRIGGER tsvectorupdate_toast
+ BEFORE INSERT OR UPDATE ON t
+ FOR EACH ROW EXECUTE FUNCTION
+ tsvector_update_trigger(search_vec, 'pg_catalog.english', content);
+
+CREATE INDEX t_gin ON t USING gin(search_vec);
+
+-- Insert with large content (will be TOASTed)
+INSERT INTO t (id, content) VALUES
+ (1, repeat('important keyword ', 1000) || repeat('filler text ', 10000));
+
+-- Verify initial state
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('important');
+-- Expected: 1 row
+
+-- IMPORTANT: The BEFORE UPDATE trigger modifies search_vec, so by the time
+-- ExecWhichIndexesRequireUpdates() runs, search_vec has already changed.
+-- This means the comparison sees old tsvector vs. trigger-modified tsvector,
+-- not the natural progression. HOT won't happen because the trigger changed
+-- the indexed column.
+
+-- Update: Even though content keywords unchanged, trigger still fires
+UPDATE t
+SET content = repeat('important keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+-- Expected: 0 HOT (trigger modifies search_vec, blocking HOT)
+-- This is actually correct behavior - the trigger updated an indexed column
+
+-- Update: Change indexed keywords
+UPDATE t
+SET content = repeat('critical keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+-- Expected: 0 HOT (index keys changed)
+
+-- Verify query correctness
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('critical');
+-- Expected: 1 row
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- TEST: GIN with TOASTed JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin((data->'tags'));
+
+-- Insert with TOASTed JSONB
+INSERT INTO t (id, data) VALUES
+ (1, jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('x', 10000)
+ ));
+
+-- Update: Change large_field, tags unchanged - should be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: 1 HOT update
+
+-- Update: Change tags - should NOT be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "sql"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: Still 1 HOT
+
+-- Verify correctness
+SELECT count(*) FROM t WHERE data->'tags' @> '["database"]'::jsonb;
+-- Expected: 0 rows
+SELECT count(*) FROM t WHERE data->'tags' @> '["sql"]'::jsonb;
+-- Expected: 1 row
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- TEST: GIN with Array of Large Strings
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin(tags);
+
+-- Insert with large array elements (might be TOASTed)
+INSERT INTO t (id, tags) VALUES
+ (1, ARRAY[repeat('tag1', 1000), repeat('tag2', 1000)]);
+
+-- Update: Change to different large values - NOT HOT
+UPDATE t
+SET tags = ARRAY[repeat('tag3', 1000), repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+-- Expected: 0 HOT (keys actually changed)
+
+-- Update: Keep same tag values, just reorder - SHOULD BE HOT
+-- (GIN is order-insensitive: both [tag3,tag4] and [tag4,tag3]
+-- extract to the same sorted key set ['tag3','tag4'])
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000), repeat('tag3', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: 1 HOT (GIN keys semantically identical)
+
+-- Update: Remove an element - NOT HOT (keys changed)
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: Still 1 HOT (not this one)
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- BRIN Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ value INT,
+ description TEXT
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+
+CREATE INDEX t_brin_partial_idx ON t USING brin(value) WHERE value > 100;
+
+INSERT INTO t VALUES (1, 50, 'below range');
+
+-- Test 1: Outside predicate
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Validate: Predicate query returns 0 rows
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+
+-- Test 2: Transition into predicate
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+
+-- Validate: Predicate query returns 1 row with correct value
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+
+-- Test 3: Inside predicate, value changes
+UPDATE t SET value = 160, description = 'updated again' WHERE id = 1;
+SELECT * FROM check_hot_updates(3);
+
+-- Validate: Updated value (160) is returned
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+
+-- Test 4: Transition out of predicate
+UPDATE t SET value = 50 WHERE id = 1;
+SELECT * FROM check_hot_updates(4);
+
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+
+SELECT id, value, description FROM t;
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- HASH Index (Simple Column)
+-- ================================================================
+CREATE TABLE t(id INT, code VARCHAR(20), description TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_idx ON t USING hash(code);
+INSERT INTO t VALUES (1, 'CODE001', 'initial');
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update indexed column - HASH index requires update, NOT HOT
+UPDATE t SET code = 'CODE002' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update both - NOT HOT
+UPDATE t SET code = 'CODE003', description = 'changed' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Back to original code - NOT HOT (different hash bucket location)
+UPDATE t SET code = 'CODE001' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- HASH Index on Expression
+-- ================================================================
+CREATE TABLE t(id INT, email TEXT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_lower_email_idx ON t USING HASH(lower(email));
+INSERT INTO t VALUES (1, '[email protected]', '{"status": "new"}');
+
+-- Update non-indexed field - should be HOT
+UPDATE t SET data = '{"status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update email with case change only (same lowercase) - should be HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+
+-- Update email to different lowercase - NOT HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- HASH Index on JSONB Field
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash((data->'category'));
+INSERT INTO t VALUES (1, '{"category": "books", "title": "PostgreSQL Guide"}');
+
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET data = '{"category": "books", "title": "PostgreSQL Handbook"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update indexed JSONB field - NOT HOT
+UPDATE t SET data = '{"category": "videos", "title": "PostgreSQL Handbook"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update both - NOT HOT
+UPDATE t SET data = '{"category": "courses", "title": "PostgreSQL Basics"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- Multiple HASH Indexes
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, status VARCHAR, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+CREATE INDEX t_hash_status_idx ON t USING hash(status);
+INSERT INTO t VALUES (1, 'electronics', 'active', 100);
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update one indexed column - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update other indexed column - NOT HOT
+UPDATE t SET status = 'inactive' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update both indexed columns - NOT HOT
+UPDATE t SET category = 'videos', status = 'pending' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- BRIN vs HASH Comparison
+-- ================================================================
+CREATE TABLE t_brin(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE TABLE t_hash(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+CREATE INDEX t_brin_value_idx ON t_brin USING brin(value);
+CREATE INDEX t_hash_value_idx ON t_hash USING hash(value);
+
+INSERT INTO t_brin VALUES (1, 100, 'initial');
+INSERT INTO t_hash VALUES (1, 100, 'initial');
+
+-- Same update on both - different HOT behavior expected
+-- BRIN: might allow HOT (range summary unchanged)
+-- HASH: blocks HOT (hash bucket changed)
+UPDATE t_brin SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't_brin');
+-- Expected: 1 HOT (BRIN allows it for single row)
+
+UPDATE t_hash SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't_hash');
+-- Expected: 0 HOT (HASH blocks it)
+
+DROP TABLE t_brin CASCADE;
+DROP TABLE t_hash CASCADE;
+
+-- ================================================================
+-- HASH Index with NULL Values
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'electronics', 'initial');
+
+-- Update indexed column to NULL - NOT HOT (hash value changed)
+UPDATE t SET category = NULL WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+-- Expected: 0 HOT
+
+-- Update indexed column from NULL to value - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+-- Expected: 0 HOT
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: 1 HOT
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- BRIN on JSONB Field
+-- ================================================================
+CREATE TABLE t(id INT, metrics JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- BRIN doesn't directly support JSONB, but we can test on expression
+CREATE INDEX t_brin_count_idx ON t USING brin(
+ CAST(metrics->>'count' AS INTEGER)
+);
+INSERT INTO t VALUES (1, '{"count": "100", "timestamp": "2024-01-01"}');
+
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET metrics = '{"count": "100", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: 1 HOT
+
+-- Update indexed field - BRIN allows HOT for single row
+UPDATE t SET metrics = '{"count": "150", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+-- Expected: 2 HOT (BRIN permits single-row updates)
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- Mixed BRIN + HASH on Same Table
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, timestamp TIMESTAMP, price NUMERIC, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_timestamp_idx ON t USING brin(timestamp);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'books', '2024-01-01 10:00:00', 29.99, 'initial');
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: 1 HOT
+
+-- Update BRIN indexed column - allows HOT
+UPDATE t SET timestamp = '2024-01-02 10:00:00' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+-- Expected: 2 HOT
+
+-- Update HASH indexed column - blocks HOT
+UPDATE t SET category = 'videos' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+-- Expected: 2 HOT (HASH blocks it)
+
+-- Update price (non-indexed) - should be HOT
+UPDATE t SET price = 39.99 WHERE id = 1;
+SELECT * FROM check_hot_updates(3);
+-- Expected: 3 HOT
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- Index both on a field in a JSONB document, and the document
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t((docs->'name'));
+CREATE INDEX t_docs_col_idx ON t(docs);
+INSERT INTO t VALUES (1, '{"name": "john", "data": "some data"}');
+
+-- Update impacts index on whole docment attribute, can't go HOT
+UPDATE t SET docs='{"name": "john", "data": "some other data"}' WHERE id=1;
+SELECT * FROM check_hot_updates(0);
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- Two indexes on a JSONB document, one partial
+-- ================================================================
+CREATE TABLE t (docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+INSERT INTO t (docs) VALUES ('{"a": 0, "b": 0}');
+INSERT INTO t (docs) SELECT jsonb_build_object('b', n) FROM generate_series(100, 10000) as n;
+CREATE INDEX t_idx_a ON t ((docs->'a'));
+CREATE INDEX t_idx_b ON t ((docs->'b')) WHERE (docs->'b')::numeric > 9;
+
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+
+-- Leave 'a' unchanged but modify 'b' to a value outside of the index predicate.
+-- This should be a HOT update because neither index is changed.
+UPDATE t SET docs = jsonb_build_object('a', 0, 'b', 1) WHERE (docs->'a')::numeric = 0;
+SELECT * FROM check_hot_updates(1);
+
+-- Check to make sure that the index does not contain a value for 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->'b')::numeric > 9 AND (docs->'b')::numeric < 100;
+SELECT * FROM t WHERE (docs->'b')::numeric > 9 AND (docs->'b')::numeric < 100;
+
+-- Leave 'a' unchanged but modify 'b' to a value within the index predicate.
+-- This represents a change for field 'b' from unindexed to indexed and so
+-- this should not take the HOT path.
+UPDATE t SET docs = jsonb_build_object('a', 0, 'b', 10) WHERE (docs->'a')::numeric = 0;
+SELECT * FROM check_hot_updates(1);
+
+-- Check to make sure that the index contains the new value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->'b')::numeric > 9 AND (docs->'b')::numeric < 100;
+SELECT * FROM t WHERE (docs->'b')::numeric > 9 AND (docs->'b')::numeric < 100;
+
+-- This update modifies the value of 'a', an indexed field, so it also cannot
+-- be a HOT update.
+UPDATE t SET docs = jsonb_build_object('a', 1, 'b', 10) WHERE (docs->'b')::numeric = 10;
+SELECT * FROM check_hot_updates(1);
+
+-- This update changes both 'a' and 'b' to new values this cannot use the HOT path.
+UPDATE t SET docs = jsonb_build_object('a', 2, 'b', 12) WHERE (docs->'b')::numeric = 10;
+SELECT * FROM check_hot_updates(1);
+
+-- Check to make sure that the index contains the new value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->'b')::numeric > 9 AND (docs->'b')::numeric < 100;
+SELECT * FROM t WHERE (docs->'b')::numeric > 9 AND (docs->'b')::numeric < 100;
+
+-- This update changes 'b' to a value outside its predicate requiring that
+-- we remove it from the index. That's a transition that can't be done
+-- during a HOT update.
+UPDATE t SET docs = jsonb_build_object('a', 2, 'b', 1) WHERE (docs->'b')::numeric = 12;
+SELECT * FROM check_hot_updates(1);
+
+-- Check to make sure that the index no longer contains the value of 'b'
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->'b')::numeric > 9 AND (docs->'b')::numeric < 100;
+SELECT * FROM t WHERE (docs->'b')::numeric > 9 AND (docs->'b')::numeric < 100;
+
+DROP TABLE t CASCADE;
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+
+-- ================================================================
+-- Tests to check expression indexes
+-- ================================================================
+CREATE TABLE t(a INT, b INT) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx_a ON t(abs(a)) WHERE abs(a) > 10;
+CREATE INDEX t_idx_b ON t(abs(b));
+INSERT INTO t VALUES (-1, -1), (-2, -2), (-3, -3), (-4, -4), (-5, -5);
+INSERT INTO t SELECT m, n FROM generate_series(-10000, -10) AS m, abs(m) AS n;
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+
+-- The indexed value of b hasn't changed, this should be a HOT update.
+-- (-5, -5) -> (-5, 1)
+UPDATE t SET b = 5 WHERE a = -5;
+SELECT * FROM check_hot_updates(1);
+EXPLAIN (COSTS OFF) SELECT b FROM t WHERE abs(b) < 10 AND abs(b) > 0;
+SELECT b FROM t WHERE abs(b) < 10 AND abs(b) > 0;
+
+-- Now that we're not checking the predicate of the partial index, this
+-- update of a from -5 to 5 should be HOT because we should ignore the
+-- predicate and check the expression and find it unchanged.
+-- (-5, 1) -> (5, 1)
+UPDATE t SET a = 5 WHERE a = -5;
+SELECT * FROM check_hot_updates(2);
+
+-- This update moves a into the partial index and should not
+-- be HOT. Let's make sure of that and check the index as well.
+-- (-4, -4) -> (-11, -4)
+UPDATE t SET a = -11 WHERE a = -4;
+SELECT * FROM check_hot_updates(2);
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE abs(a) > 10 AND abs(a) < 15;
+SELECT * FROM t WHERE abs(a) > 10 AND abs(a) < 15;
+
+-- (-11, -4) -> (11, -4)
+UPDATE t SET a = 11 WHERE b = -4;
+SELECT * FROM check_hot_updates(3);
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE abs(a) > 10 AND abs(a) < 15;
+SELECT * FROM t WHERE abs(a) > 10 AND abs(a) < 15;
+
+-- (11, -4) -> (-4, -4)
+UPDATE t SET a = -4 WHERE b = -4;
+SELECT * FROM check_hot_updates(3);
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE abs(a) > 10 AND abs(a) < 15;
+SELECT * FROM t WHERE abs(a) > 10 AND abs(a) < 15;
+
+-- This update of a from 5 to -1 is HOT despite that attribute
+-- being indexed because the before and after values for the
+-- partial index predicate are outside the index definition.
+-- (5, 1) -> (-1, 1)
+UPDATE t SET a = -1 WHERE a = 5;
+SELECT * FROM check_hot_updates(4);
+
+-- This update of a from -2 to -1 will be HOT because the before/after values
+-- of a are both outside the predicate of the partial index.
+-- (-1, 1) -> (-2, 1)
+UPDATE t SET a = -2 WHERE b = -2;
+SELECT * FROM check_hot_updates(5);
+
+-- The indexed value for b isn't changing, this should be HOT.
+-- (-2, -2) -> (-2, 2)
+UPDATE t SET b = 2 WHERE b = -2;
+SELECT * FROM check_hot_updates(6);
+EXPLAIN (COSTS OFF) SELECT b FROM t WHERE abs(b) < 10 AND abs(b) > 0;
+SELECT b FROM t WHERE abs(b) < 10 AND abs(b) > 0;
+
+SELECT * FROM t where a > -10 AND a < 10;
+
+-- Before and after values for a are outside the predicate of the index,
+-- and because we're checking this should be HOT.
+-- (-2, 1) -> (5, 1)
+-- (-2, -2) -> (5, -2)
+UPDATE t SET a = 5 WHERE a = -1;
+SELECT * FROM check_hot_updates(8);
+
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE abs(a) > 10 AND abs(a) < 15;
+SELECT * FROM t WHERE abs(a) > 10 AND abs(a) < 15;
+
+DROP TABLE t CASCADE;
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+
+
+-- ================================================================
+-- JSONB with two indexes each on separate fields, one partial
+-- ================================================================
+CREATE TABLE t(docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t((docs->'a')) WHERE (docs->'b')::integer = 1;
+INSERT INTO t VALUES ('{"a": 1, "b": 1}');
+
+EXPLAIN (COSTS OFF) SELECT * FROM t;
+SELECT * FROM t;
+
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE (docs->'b')::integer = 1;
+SELECT * FROM t WHERE (docs->'b')::integer = 1;
+
+SELECT * FROM check_hot_updates(0);
+
+UPDATE t SET docs='{"a": 1, "b": 0}';
+SELECT * FROM check_hot_updates(0);
+
+SELECT * FROM t WHERE (docs->'b')::integer = 1;
+
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- Tests for multi-column indexes
+-- ================================================================
+CREATE TABLE t(id INT, docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t(id, (docs->'a'));
+INSERT INTO t VALUES (1, '{"a": 1, "b": 1}');
+
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+
+EXPLAIN (COSTS OFF) SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer > 0;
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer > 0;
+
+SELECT * FROM check_hot_updates(0);
+
+-- Changing the id attribute which is an indexed attribute should
+-- prevent HOT updates.
+UPDATE t SET id = 2;
+SELECT * FROM check_hot_updates(0);
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer > 0;
+
+-- Changing the docs->'a' field in the indexed attribute 'docs'
+-- should prevent HOT updates.
+UPDATE t SET docs='{"a": -2, "b": 1}';
+SELECT * FROM check_hot_updates(0);
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer < 0;
+
+-- Leaving the docs->'a' attribute unchanged means that the expression
+-- is unchanged and because the 'id' attribute isn't in the modified
+-- set the indexed tuple is unchanged, this can go HOT.
+UPDATE t SET docs='{"a": -2, "b": 2}';
+SELECT * FROM check_hot_updates(1);
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer < 0;
+
+-- Here we change the 'id' attribute and the 'docs' attribute setting
+-- the expression docs->'a' to a new value, this cannot be a HOT update.
+UPDATE t SET id = 3, docs='{"a": 3, "b": 3}';
+SELECT * FROM check_hot_updates(1);
+
+SELECT * FROM t WHERE id > 0 AND (docs->'a')::integer > 0;
+
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- Relation with unique constraint, partial index
+-- ================================================================
+CREATE TABLE users (
+ user_id serial primary key,
+ name VARCHAR(255) NOT NULL,
+ email VARCHAR(255) NOT NULL,
+ EXCLUDE USING btree (lower(email) WITH =)
+);
+
+-- Add some data to the table and then update it in ways that should and should
+-- not be HOT updates.
+INSERT INTO users (name, email) VALUES
+('user1', '[email protected]'),
+('user2', '[email protected]'),
+('taken', '[email protected]'),
+('you', '[email protected]'),
+('taken', '[email protected]');
+
+-- Should fail because of the unique constraint on the email column.
+UPDATE users SET email = '[email protected]' WHERE email = '[email protected]';
+SELECT * FROM check_hot_updates(0, 'users');
+
+-- Should succeed because the email column is not being updated and should go HOT.
+UPDATE users SET name = 'foo' WHERE email = '[email protected]';
+SELECT * FROM check_hot_updates(1, 'users');
+
+-- Create a partial index on the email column, updates
+CREATE INDEX idx_users_email_no_example ON users (lower(email)) WHERE lower(email) LIKE '%@example.com%';
+
+-- An update that changes the email column but not the indexed portion of it and falls outside the constraint.
+-- Shouldn't be a HOT update because of the exclusion constraint.
+UPDATE users SET email = '[email protected]' WHERE name = 'you';
+SELECT * FROM check_hot_updates(1, 'users');
+
+-- An update that changes the email column but not the indexed portion of it and falls within the constraint.
+-- Again, should fail constraint and fail to be a HOT update.
+UPDATE users SET email = '[email protected]' WHERE name = 'you';
+SELECT * FROM check_hot_updates(1, 'users');
+
+DROP TABLE users CASCADE;
+
+-- ================================================================
+-- Constraints spoiling HOT updates, this time with a range.
+-- ================================================================
+CREATE TABLE events (
+ id serial primary key,
+ name VARCHAR(255) NOT NULL,
+ event_time tstzrange,
+ constraint no_screening_time_overlap exclude using gist (
+ event_time WITH &&
+ )
+);
+
+-- Add two non-overlapping events.
+INSERT INTO events (id, event_time, name)
+VALUES
+ (1, '["2023-01-01 19:00:00", "2023-01-01 20:45:00"]', 'event1'),
+ (2, '["2023-01-01 21:00:00", "2023-01-01 21:45:00"]', 'event2');
+
+-- Update the first event to overlap with the second, should fail the constraint and not be HOT.
+UPDATE events SET event_time = '["2023-01-01 20:00:00", "2023-01-01 21:45:00"]' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 'events');
+
+-- Update the first event to not overlap with the second, again not HOT due to the constraint.
+UPDATE events SET event_time = '["2023-01-01 22:00:00", "2023-01-01 22:45:00"]' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 'events');
+
+-- Update the first event to not overlap with the second, this time we're HOT because we don't overlap with the constraint.
+UPDATE events SET name = 'new name here' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 'events');
+
+DROP TABLE events CASCADE;
+
+-- ================================================================
+-- Ensure that only the modified summarizing indexes are updated.
+-- ================================================================
+CREATE TABLE ex (id SERIAL primary key, att1 JSONB, att2 text, att3 text, att4 text) WITH (fillfactor = 60);
+CREATE INDEX ex_expr1_idx ON ex USING btree((att1->'data'));
+CREATE INDEX ex_sumr1_idx ON ex USING BRIN(att2);
+CREATE INDEX ex_expr2_idx ON ex USING btree((att1->'a'));
+CREATE INDEX ex_expr3_idx ON ex USING btree((att1->'b'));
+CREATE INDEX ex_expr4_idx ON ex USING btree((att1->'c'));
+CREATE INDEX ex_sumr2_idx ON ex USING BRIN(att3);
+CREATE INDEX ex_sumr3_idx ON ex USING BRIN(att4);
+CREATE INDEX ex_expr5_idx ON ex USING btree((att1->'d'));
+INSERT INTO ex (att1, att2) VALUES ('{"data": []}'::json, 'nothing special');
+
+SELECT * FROM ex;
+
+-- Update att2 and att4 both are BRIN/summarizing indexes, this should be a HOT update and
+-- only update two of the three summarizing indexes.
+UPDATE ex SET att2 = 'special indeed', att4 = 'whatever';
+SELECT * FROM check_hot_updates(1, 'ex');
+SELECT * FROM ex;
+
+-- Update att1 and att2, only one is BRIN/summarizing, this should NOT be a HOT update.
+UPDATE ex SET att1 = att1 || '{"data": "howdy"}', att2 = 'special, so special';
+SELECT * FROM check_hot_updates(1, 'ex');
+SELECT * FROM ex;
+
+-- Update att2, att3, and att4 all are BRIN/summarizing indexes, this should be a HOT update
+-- and yet still update all three summarizing indexes.
+UPDATE ex SET att2 = 'a', att3 = 'b', att4 = 'c';
+SELECT * FROM check_hot_updates(2, 'ex');
+SELECT * FROM ex;
+
+-- Update att1, att2, and att3 all modified values are BRIN/summarizing indexes, this should be a HOT update
+-- and yet still update all three summarizing indexes.
+UPDATE ex SET att1 = '{"data": "howdy"}', att2 = 'd', att3 = 'e';
+SELECT * FROM check_hot_updates(3, 'ex');
+SELECT * FROM ex;
+
+DROP TABLE ex CASCADE;
+
+-- ================================================================
+-- Don't update unmodified summarizing indexes but do allow HOT
+-- ================================================================
+CREATE TABLE ex (att1 JSONB, att2 text) WITH (fillfactor = 60);
+CREATE INDEX ex_expr1_idx ON ex USING btree((att1->'data'));
+CREATE INDEX ex_sumr1_idx ON ex USING BRIN(att2);
+INSERT INTO ex VALUES ('{"data": []}', 'nothing special');
+
+-- Update the unindexed value of att1, this should be a HOT update and and should
+-- update the summarizing index.
+UPDATE ex SET att1 = att1 || '{"status": "stalemate"}';
+SELECT * FROM check_hot_updates(1, 'ex');
+
+-- Update the indexed value of att2, a summarized value, this is a summarized
+-- only update and should use the HOT path while still triggering an update to
+-- the summarizing BRIN index.
+UPDATE ex SET att2 = 'special indeed';
+SELECT * FROM check_hot_updates(2, 'ex');
+
+-- Update to att1 doesn't change the indexed value while the update to att2 does,
+-- this again is a summarized only update and should use the HOT path as well as
+-- trigger an update to the BRIN index.
+UPDATE ex SET att1 = att1 || '{"status": "checkmate"}', att2 = 'special, so special';
+SELECT * FROM check_hot_updates(3, 'ex');
+
+-- This updates both indexes, the expression index on att1 and the summarizing
+-- index on att2. This should not be a HOT update because there are modified
+-- indexes and only some are summarized, not all. This should force all
+-- indexes to be updated.
+UPDATE ex SET att1 = att1 || '{"data": [1,2,3]}', att2 = 'do you want to play a game?';
+SELECT * FROM check_hot_updates(3, 'ex');
+
+DROP TABLE ex CASCADE;
+
+-- ================================================================
+-- Ensure custom type equality operators are used
+-- ================================================================
+
+CREATE TYPE my_custom_type AS (val int);
+
+-- Comparison functions (returns boolean)
+CREATE FUNCTION my_custom_lt(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val < b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_le(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val <= b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_eq(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val = b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_ge(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val >= b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_gt(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val > b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+CREATE FUNCTION my_custom_ne(a my_custom_type, b my_custom_type) RETURNS boolean AS $$
+BEGIN
+ RETURN a.val != b.val;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+-- Comparison function (returns -1, 0, 1)
+CREATE FUNCTION my_custom_cmp(a my_custom_type, b my_custom_type) RETURNS int AS $$
+BEGIN
+ IF a.val < b.val THEN
+ RETURN -1;
+ ELSIF a.val > b.val THEN
+ RETURN 1;
+ ELSE
+ RETURN 0;
+ END IF;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+-- Create the operators
+CREATE OPERATOR < (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_lt,
+ COMMUTATOR = >,
+ NEGATOR = >=
+);
+
+CREATE OPERATOR <= (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_le,
+ COMMUTATOR = >=,
+ NEGATOR = >
+);
+
+CREATE OPERATOR = (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_eq,
+ COMMUTATOR = =,
+ NEGATOR = <>
+);
+
+CREATE OPERATOR >= (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_ge,
+ COMMUTATOR = <=,
+ NEGATOR = <
+);
+
+CREATE OPERATOR > (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_gt,
+ COMMUTATOR = <,
+ NEGATOR = <=
+);
+
+CREATE OPERATOR <> (
+ LEFTARG = my_custom_type,
+ RIGHTARG = my_custom_type,
+ PROCEDURE = my_custom_ne,
+ COMMUTATOR = <>,
+ NEGATOR = =
+);
+
+-- Create the operator class (including the support function)
+CREATE OPERATOR CLASS my_custom_ops
+ DEFAULT FOR TYPE my_custom_type USING btree AS
+ OPERATOR 1 <,
+ OPERATOR 2 <=,
+ OPERATOR 3 =,
+ OPERATOR 4 >=,
+ OPERATOR 5 >,
+ FUNCTION 1 my_custom_cmp(my_custom_type, my_custom_type);
+
+-- Create the table
+CREATE TABLE my_table (
+ id int,
+ custom_val my_custom_type
+);
+
+-- Insert some data
+INSERT INTO my_table (id, custom_val) VALUES
+(1, ROW(3)::my_custom_type),
+(2, ROW(1)::my_custom_type),
+(3, ROW(4)::my_custom_type),
+(4, ROW(2)::my_custom_type);
+
+-- Create a function to use when indexing
+CREATE OR REPLACE FUNCTION abs_val(val my_custom_type) RETURNS int AS $$
+BEGIN
+ RETURN abs(val.val);
+END;
+$$ LANGUAGE plpgsql IMMUTABLE STRICT;
+
+-- Create the index
+CREATE INDEX idx_custom_val_abs ON my_table (abs_val(custom_val));
+
+-- Update 1
+UPDATE my_table SET custom_val = ROW(5)::my_custom_type WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 'my_table');
+
+-- Update 2
+UPDATE my_table SET custom_val = ROW(0)::my_custom_type WHERE custom_val < ROW(3)::my_custom_type;
+SELECT * FROM check_hot_updates(0, 'my_table');
+
+-- Update 3
+UPDATE my_table SET custom_val = ROW(6)::my_custom_type WHERE id = 3;
+SELECT * FROM check_hot_updates(0, 'my_table');
+
+-- Update 4
+UPDATE my_table SET id = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 'my_table');
+
+-- Query using the index
+SELECT * FROM my_table WHERE abs_val(custom_val) = 6;
+
+-- Clean up test case
+DROP TABLE my_table CASCADE;
+DROP OPERATOR CLASS my_custom_ops USING btree CASCADE;
+DROP OPERATOR < (my_custom_type, my_custom_type);
+DROP OPERATOR <= (my_custom_type, my_custom_type);
+DROP OPERATOR = (my_custom_type, my_custom_type);
+DROP OPERATOR >= (my_custom_type, my_custom_type);
+DROP OPERATOR > (my_custom_type, my_custom_type);
+DROP OPERATOR <> (my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_lt(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_le(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_eq(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_ge(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_gt(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_ne(my_custom_type, my_custom_type);
+DROP FUNCTION my_custom_cmp(my_custom_type, my_custom_type);
+DROP FUNCTION abs_val(my_custom_type);
+DROP TYPE my_custom_type CASCADE;
+
+-- Cleanup
+DROP FUNCTION check_hot_updates(int, text, text);
+DROP COLLATION case_insensitive;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 27a4d131897..1eace574994 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -390,6 +390,7 @@ CachedFunctionCompileCallback
CachedFunctionDeleteCallback
CachedFunctionHashEntry
CachedFunctionHashKey
+CachedIndexDatum
CachedPlan
CachedPlanSource
CallContext
--
2.49.0
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-07 21:36 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-09 19:27 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2025-11-16 18:53 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-11-19 18:00 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-11-21 15:25 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-11-22 21:30 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
@ 2025-11-24 18:59 ` Matthias van de Meent <[email protected]>
2025-12-03 22:06 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Matthias van de Meent @ 2025-11-24 18:59 UTC (permalink / raw)
To: Greg Burd <[email protected]>; +Cc: pgsql-hackers
On Sat, 22 Nov 2025, 22:30 Greg Burd, <[email protected]> wrote:
> Thanks for pointing out the oversight for index-oriented scans (IOS),
> you're right that the code in v22 doesn't handle that correctly. I'll
> fix that. I still think that indexes that don't support IOS can and
> should use the type-specific equality checks. This opens the door to
> HOT with custom types that have unusual equality rules (see BSON).
Do you have specific examples why it would be safe to default to
"unusual equality rules" for generally any index's data ingestion
needs? Why e.g. BSON must always be compared with their special
equality test (and not datumIsEqual), and why IOS-less indexes in
general are never going to distinguish between binary distinct but
btree-equal values, and why exact equality is the special case here?
I understand that you want to maximize optimization for specific
workloads that you have in mind, but lacking evidence to the contrary
I am really not convinced that your workloads are sufficiently
generalizable that they can (and should) be the baseline for these new
HOT rules: I have not yet seen good arguments why we could relax
"datum equality" to "type equality" without potentially breaking
existing indexes.
HOT was implemented quite conservatively to make sure that there are
no issues where changed values are not reflected in indexes: each
indexed TID represents a specific and unchanging set of indexed
values, in both the key and non-key attributes of indexes. If a value
changes however so slightly, that may be a cause for indexes to treat
it differently, and thus HOT must not be used.
Aside: The amsummarizing optimization gets around that check by
realizing the TID itself isn't really indexed, so the rules can be
relaxed around that, but it still needs to go through the effort to
update the summarizing indexes if the relevant attributes were ever so
slightly updated.
This patch right now wants to change these rules and behaviour of HOT
in two ways:
1.) Instead of only testing attributes mentioned by indexed
expressions for changes, it wants to test the output of the indexed
expressions.
I would consider this to be generally safe, as long as the expressions
comply with the rules we have for indexed expressions. [Which, if not
held, would break IOS and various other things, too, so relying on
these rules isn't new or special].
2.) Instead of datumIsEqual, it (by default) wants to do equality
checks as provided by the type's default btree opclass' = operator.
I have not seen evidence that this is safe. I have even explained with
an example that IOS will return distinctly wrong results if only
btree's = operator is used to determine if HOT can be applied, and
that doesn't even begin to cover the issues related to indexes that
may handle data differently from Btree.
I also don't want indexes that at some point in the future invent
support for IOS to return subtly incorrect results due to HOT checks
that depended on the output of a previous version's amcanreturn
output.
So, IMV, tts_attr_equal is just a rather expensive version of
datumIsEqual: the fast path cases would've been handled by
datumIsEqual at least as fast (without a switch() statement with 18
specific cases and a default branch); if there is no btree operator
it'll still default to btree compare, and if not then if the slow path
uses correctly implemented compare operators (for HOT, and potentially
all other possible indexes), then these would have an output that is
indistinguishable from datumIsEqual, with the only difference the
address and performance of the called function and a lot of added
catalog lookups.
All together, I think it's best to remove the second component of the
changes to the HOT rules (changing the type of matching done for
indexed values with tts_attr_compare) from this patchset.
If you believe this should be added regardless, I think it's best
discussed separately in its own thread and patchset -- it should be
relatively easy to introduce in both current and future versions of
this code, and (if you're correct and this is safe) it would have some
benefits even when committed on its own.
Kind regards,
Matthias van de Meent
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-07 21:36 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-09 19:27 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2025-11-16 18:53 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-11-19 18:00 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-11-21 15:25 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-11-22 21:30 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-11-24 18:59 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
@ 2025-12-03 22:06 ` Greg Burd <[email protected]>
2025-12-15 21:46 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Greg Burd @ 2025-12-03 22:06 UTC (permalink / raw)
To: Matthias van de Meent <[email protected]>; +Cc: pgsql-hackers
On Nov 24 2025, at 1:59 pm, Matthias van de Meent
<[email protected]> wrote:
> ... <awesome thoughtful questions, insights, etc> ...
>
> Kind regards,
>
> Matthias van de Meent
Hey Matthias,
I've updated the patch set to v25 and taken your suggested approach of
minimizing changes in hopes of getting the majority of this patch
series committed (I hope) soon. By that I mean that $subject isn't
really accurate anymore. This patch uses datumIsEqual(), but still
introduces the new index AM API and still moves
HeapDetermineColumnsInfo() into a function in nodeModifyTable.c called
ExecWhichIndexesRequireUpdates(). The "big idea" is that before calling
into the table AM to update a tuple the executor should know what the
impact of that update will be for the indexes on the relation.
ExecWhichIndexesRequireUpdates() finds the set of attributes that were
a) modified and b) are referenced by an index and cause that index to
need a new index tuple. This is left up to heap now, but really should
be generic across all table AMs (IMO), so that's what I've done.
At some point in the future maybe there's a way to switch from the
heap-specific model of all/none/summarized-only to something where we
only update indexes that really require the updates (HOT, WARM, etc.)
and I think this is a step in that direction, but for now the logic
remains the same as does the signal (TU_Updated).
I'll re-introduce $subject as a layer on v24 next week at which point
I'll try to address all the good points you raised in your email. Those
additions (HOT expressions, HOT partial indexes, and type-specific
equality tests) are the most controversial.
I think it may be possible that these first few patches are less
controversial and could make the cut sooner while those other ideas
remain up for debate. I'm open to that, I think the work in the attached
set is good and valuable on it's own.
So, the attached patch set
Benefits of the patch:
* the tests for what changed move outside of the buffer lock
* the redundant index_unchanged_by_update() is removed
Downsides of the patch:
* a bit of new overhead in some cases
* a bit more complicated logic than before
Recall that this patch set combines with another one of mine on the list
[1] which covers the simple_heap_update() path, this one doesn't cover
that case and you'll see that simple_heap_update() still depends on HeapDetermineColumnsInfo().
* 0001 - Prepare heapam_tuple_update() and simple_heap_update() for divergence
This splits off the top of heap_update() and places that logic in both
heapam_tuple_update() and simple_heap_update(). This patch is also
present in the other thread [1] and essentially the same. That thread
addresses the changes to the catalog tuple updates. No real effort was
made to make this patch "pretty" or "stand-alone" as it really is a
precursor to the work in 0002 and in [1].
* 0002 - Track changed indexed columns in the executor during UPDATEs
This is where the meat is, as described in earlier emails and the commit
message. HeapDetermineColumnsInfo() logic moves up into the executor
into ExecWhichIndexesRequireUpdates(). Some heap-specific logic related
to replica identity remains in heapam_tuple_update().
* 0003 - Replace index_unchanged_by_update() with ri_ChangedIndexedCols
This removes the now redundant index_unchanged_by_update() function and
instead uses the information gathered in
ExecWhichIndexesRequireUpdates() and recorded in ri_ChangedIndexCols for
the same outcome.
best.
-greg
[1] https://www.postgresql.org/message-id/flat/[email protected]
Attachments:
[application/octet-stream] v25-0001-Prepare-heapam_tuple_update-and-simple_heap_upda.patch (47.8K, 2-v25-0001-Prepare-heapam_tuple_update-and-simple_heap_upda.patch)
download | inline diff:
From 88fbab1f79d82b1eb51be2054cab303ae856f3e2 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 2 Nov 2025 11:36:20 -0500
Subject: [PATCH v25 1/3] Prepare heapam_tuple_update() and
simple_heap_update() for divergence
This commit lays the foundation for larger changes to come by taking the
first portion of heap_update() through the HeapDeterminColumnsInfo() and
replicating that logic in both heapam_tuple_update() and
simple_heap_upate(). This is done so that these two paths might diverge
in implementation later on. The simple_heap_update() path deals solely
with updates to catalog tuples which could record their modified
attributes rather than relearn them. The remaining calls from the
executor into the table AM update API could include the set of updated
attributes. This is foreshadowing... of course, as that's what the next
commit will start to do.
As part of this reorganization, the handling of replica identity key
attributes has been adjusted. Instead of fetching a second copy of
the bitmap during an update operation, the caller is now required to
provide it. This change applies to both heap_update() and
heap_delete().
---
src/backend/access/heap/heapam.c | 568 +++++++++++------------
src/backend/access/heap/heapam_handler.c | 117 ++++-
src/include/access/heapam.h | 24 +-
3 files changed, 410 insertions(+), 299 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 4d382a04338..30847db1fe3 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -39,18 +39,24 @@
#include "access/syncscan.h"
#include "access/valid.h"
#include "access/visibilitymap.h"
+#include "access/xact.h"
#include "access/xloginsert.h"
+#include "catalog/catalog.h"
#include "catalog/pg_database.h"
#include "catalog/pg_database_d.h"
#include "commands/vacuum.h"
+#include "nodes/bitmapset.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
+#include "storage/bufmgr.h"
+#include "storage/itemptr.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
#include "storage/procarray.h"
#include "utils/datum.h"
#include "utils/injection_point.h"
#include "utils/inval.h"
+#include "utils/relcache.h"
#include "utils/spccache.h"
#include "utils/syscache.h"
@@ -62,16 +68,8 @@ static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
HeapTuple newtup, HeapTuple old_key_tuple,
bool all_visible_cleared, bool new_all_visible_cleared);
#ifdef USE_ASSERT_CHECKING
-static void check_lock_if_inplace_updateable_rel(Relation relation,
- const ItemPointerData *otid,
- HeapTuple newtup);
static void check_inplace_rel_lock(HeapTuple oldtup);
#endif
-static Bitmapset *HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external);
static bool heap_acquire_tuplock(Relation relation, const ItemPointerData *tid,
LockTupleMode mode, LockWaitPolicy wait_policy,
bool *have_tuple_lock);
@@ -103,10 +101,10 @@ static bool ConditionalMultiXactIdWait(MultiXactId multi, MultiXactStatus status
static void index_delete_sort(TM_IndexDeleteOp *delstate);
static int bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate);
static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
-static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
+static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp,
+ Bitmapset *rid_attrs, bool key_required,
bool *copy);
-
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
* corresponding MultiXactStatuses (one to merely lock tuples, another one to
@@ -2814,6 +2812,7 @@ heap_delete(Relation relation, const ItemPointerData *tid,
Buffer buffer;
Buffer vmbuffer = InvalidBuffer;
TransactionId new_xmax;
+ Bitmapset *rid_attrs;
uint16 new_infomask,
new_infomask2;
bool have_tuple_lock = false;
@@ -2826,6 +2825,8 @@ heap_delete(Relation relation, const ItemPointerData *tid,
AssertHasSnapshotForToast(relation);
+ rid_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
/*
* Forbid this during a parallel operation, lest it allocate a combo CID.
* Other workers might need that combo CID for visibility checks, and we
@@ -3029,6 +3030,7 @@ l1:
UnlockTupleTuplock(relation, &(tp.t_self), LockTupleExclusive);
if (vmbuffer != InvalidBuffer)
ReleaseBuffer(vmbuffer);
+ bms_free(rid_attrs);
return result;
}
@@ -3050,7 +3052,10 @@ l1:
* Compute replica identity tuple before entering the critical section so
* we don't PANIC upon a memory allocation failure.
*/
- old_key_tuple = ExtractReplicaIdentity(relation, &tp, true, &old_key_copied);
+ old_key_tuple = ExtractReplicaIdentity(relation, &tp, rid_attrs,
+ true, &old_key_copied);
+ bms_free(rid_attrs);
+ rid_attrs = NULL;
/*
* If this is the first possibly-multixact-able operation in the current
@@ -3262,7 +3267,10 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* heap_update - replace a tuple
*
* See table_tuple_update() for an explanation of the parameters, except that
- * this routine directly takes a tuple rather than a slot.
+ * this routine directly takes a heap tuple rather than a slot.
+ *
+ * It's required that the caller has acquired the pin and lock on the buffer.
+ * That lock and pin will be managed here, not in the caller.
*
* In the failure cases, the routine fills *tmfd with the tuple's t_ctid,
* t_xmax (resolving a possible MultiXact, if necessary), and t_cmax (the last
@@ -3270,30 +3278,21 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* generated by another transaction).
*/
TM_Result
-heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+heap_update(Relation relation, HeapTupleData *oldtup,
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
+ Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
+ Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
+ Bitmapset *mix_attrs, Buffer *vmbuffer,
+ bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
- Bitmapset *hot_attrs;
- Bitmapset *sum_attrs;
- Bitmapset *key_attrs;
- Bitmapset *id_attrs;
- Bitmapset *interesting_attrs;
- Bitmapset *modified_attrs;
- ItemId lp;
- HeapTupleData oldtup;
HeapTuple heaptup;
HeapTuple old_key_tuple = NULL;
bool old_key_copied = false;
- Page page;
- BlockNumber block;
MultiXactStatus mxact_status;
- Buffer buffer,
- newbuf,
- vmbuffer = InvalidBuffer,
+ Buffer newbuf,
vmbuffer_new = InvalidBuffer;
bool need_toast;
Size newtupsize,
@@ -3307,7 +3306,6 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
bool all_visible_cleared_new = false;
bool checked_lockers;
bool locker_remains;
- bool id_has_external = false;
TransactionId xmax_new_tuple,
xmax_old_tuple;
uint16 infomask_old_tuple,
@@ -3315,144 +3313,13 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
infomask_new_tuple,
infomask2_new_tuple;
- Assert(ItemPointerIsValid(otid));
-
- /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
- Assert(HeapTupleHeaderGetNatts(newtup->t_data) <=
- RelationGetNumberOfAttributes(relation));
-
+ Assert(BufferIsLockedByMe(buffer));
+ Assert(ItemIdIsNormal(lp));
AssertHasSnapshotForToast(relation);
- /*
- * Forbid this during a parallel operation, lest it allocate a combo CID.
- * Other workers might need that combo CID for visibility checks, and we
- * have no provision for broadcasting it to them.
- */
- if (IsInParallelMode())
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
- errmsg("cannot update tuples during a parallel operation")));
-
-#ifdef USE_ASSERT_CHECKING
- check_lock_if_inplace_updateable_rel(relation, otid, newtup);
-#endif
-
- /*
- * Fetch the list of attributes to be checked for various operations.
- *
- * For HOT considerations, this is wasted effort if we fail to update or
- * have to put the new tuple on a different page. But we must compute the
- * list before obtaining buffer lock --- in the worst case, if we are
- * doing an update on one of the relevant system catalogs, we could
- * deadlock if we try to fetch the list later. In any case, the relcache
- * caches the data so this is usually pretty cheap.
- *
- * We also need columns used by the replica identity and columns that are
- * considered the "key" of rows in the table.
- *
- * Note that we get copies of each bitmap, so we need not worry about
- * relcache flush happening midway through.
- */
- hot_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_HOT_BLOCKING);
- sum_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_SUMMARIZED);
- key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
- id_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
- interesting_attrs = NULL;
- interesting_attrs = bms_add_members(interesting_attrs, hot_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, sum_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, key_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, id_attrs);
-
- block = ItemPointerGetBlockNumber(otid);
- INJECTION_POINT("heap_update-before-pin", NULL);
- buffer = ReadBuffer(relation, block);
- page = BufferGetPage(buffer);
-
- /*
- * Before locking the buffer, pin the visibility map page if it appears to
- * be necessary. Since we haven't got the lock yet, someone else might be
- * in the middle of changing this, so we'll need to recheck after we have
- * the lock.
- */
- if (PageIsAllVisible(page))
- visibilitymap_pin(relation, block, &vmbuffer);
-
- LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
-
- lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
-
- /*
- * Usually, a buffer pin and/or snapshot blocks pruning of otid, ensuring
- * we see LP_NORMAL here. When the otid origin is a syscache, we may have
- * neither a pin nor a snapshot. Hence, we may see other LP_ states, each
- * of which indicates concurrent pruning.
- *
- * Failing with TM_Updated would be most accurate. However, unlike other
- * TM_Updated scenarios, we don't know the successor ctid in LP_UNUSED and
- * LP_DEAD cases. While the distinction between TM_Updated and TM_Deleted
- * does matter to SQL statements UPDATE and MERGE, those SQL statements
- * hold a snapshot that ensures LP_NORMAL. Hence, the choice between
- * TM_Updated and TM_Deleted affects only the wording of error messages.
- * Settle on TM_Deleted, for two reasons. First, it avoids complicating
- * the specification of when tmfd->ctid is valid. Second, it creates
- * error log evidence that we took this branch.
- *
- * Since it's possible to see LP_UNUSED at otid, it's also possible to see
- * LP_NORMAL for a tuple that replaced LP_UNUSED. If it's a tuple for an
- * unrelated row, we'll fail with "duplicate key value violates unique".
- * XXX if otid is the live, newer version of the newtup row, we'll discard
- * changes originating in versions of this catalog row after the version
- * the caller got from syscache. See syscache-update-pruned.spec.
- */
- if (!ItemIdIsNormal(lp))
- {
- Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
-
- UnlockReleaseBuffer(buffer);
- Assert(!have_tuple_lock);
- if (vmbuffer != InvalidBuffer)
- ReleaseBuffer(vmbuffer);
- tmfd->ctid = *otid;
- tmfd->xmax = InvalidTransactionId;
- tmfd->cmax = InvalidCommandId;
- *update_indexes = TU_None;
-
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- /* modified_attrs not yet initialized */
- bms_free(interesting_attrs);
- return TM_Deleted;
- }
-
- /*
- * Fill in enough data in oldtup for HeapDetermineColumnsInfo to work
- * properly.
- */
- oldtup.t_tableOid = RelationGetRelid(relation);
- oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
- oldtup.t_len = ItemIdGetLength(lp);
- oldtup.t_self = *otid;
-
- /* the new tuple is ready, except for this: */
+ /* The new tuple is ready, except for this */
newtup->t_tableOid = RelationGetRelid(relation);
- /*
- * Determine columns modified by the update. Additionally, identify
- * whether any of the unmodified replica identity key attributes in the
- * old tuple is externally stored or not. This is required because for
- * such attributes the flattened value won't be WAL logged as part of the
- * new tuple so we must include it as part of the old_key_tuple. See
- * ExtractReplicaIdentity.
- */
- modified_attrs = HeapDetermineColumnsInfo(relation, interesting_attrs,
- id_attrs, &oldtup,
- newtup, &id_has_external);
-
/*
* If we're not updating any "key" column, we can grab a weaker lock type.
* This allows for more concurrency when we are running simultaneously
@@ -3464,7 +3331,7 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
* is updates that don't manipulate key columns, not those that
* serendipitously arrive at the same key values.
*/
- if (!bms_overlap(modified_attrs, key_attrs))
+ if (!bms_overlap(mix_attrs, pk_attrs))
{
*lockmode = LockTupleNoKeyExclusive;
mxact_status = MultiXactStatusNoKeyUpdate;
@@ -3488,17 +3355,10 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
key_intact = false;
}
- /*
- * Note: beyond this point, use oldtup not otid to refer to old tuple.
- * otid may very well point at newtup->t_self, which we will overwrite
- * with the new tuple's location, so there's great risk of confusion if we
- * use otid anymore.
- */
-
l2:
checked_lockers = false;
locker_remains = false;
- result = HeapTupleSatisfiesUpdate(&oldtup, cid, buffer);
+ result = HeapTupleSatisfiesUpdate(oldtup, cid, buffer);
/* see below about the "no wait" case */
Assert(result != TM_BeingModified || wait);
@@ -3530,8 +3390,8 @@ l2:
*/
/* must copy state data before unlocking buffer */
- xwait = HeapTupleHeaderGetRawXmax(oldtup.t_data);
- infomask = oldtup.t_data->t_infomask;
+ xwait = HeapTupleHeaderGetRawXmax(oldtup->t_data);
+ infomask = oldtup->t_data->t_infomask;
/*
* Now we have to do something about the existing locker. If it's a
@@ -3571,13 +3431,12 @@ l2:
* requesting a lock and already have one; avoids deadlock).
*/
if (!current_is_member)
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &oldtup->t_self, *lockmode,
LockWaitBlock, &have_tuple_lock);
/* wait for multixact */
MultiXactIdWait((MultiXactId) xwait, mxact_status, infomask,
- relation, &oldtup.t_self, XLTW_Update,
- &remain);
+ relation, &oldtup->t_self, XLTW_Update, &remain);
checked_lockers = true;
locker_remains = remain != 0;
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
@@ -3587,9 +3446,9 @@ l2:
* could update this tuple before we get to this point. Check
* for xmax change, and start over if so.
*/
- if (xmax_infomask_changed(oldtup.t_data->t_infomask,
+ if (xmax_infomask_changed(oldtup->t_data->t_infomask,
infomask) ||
- !TransactionIdEquals(HeapTupleHeaderGetRawXmax(oldtup.t_data),
+ !TransactionIdEquals(HeapTupleHeaderGetRawXmax(oldtup->t_data),
xwait))
goto l2;
}
@@ -3614,8 +3473,8 @@ l2:
* before this one, which are important to keep in case this
* subxact aborts.
*/
- if (!HEAP_XMAX_IS_LOCKED_ONLY(oldtup.t_data->t_infomask))
- update_xact = HeapTupleGetUpdateXid(oldtup.t_data);
+ if (!HEAP_XMAX_IS_LOCKED_ONLY(oldtup->t_data->t_infomask))
+ update_xact = HeapTupleGetUpdateXid(oldtup->t_data);
else
update_xact = InvalidTransactionId;
@@ -3656,9 +3515,9 @@ l2:
* lock.
*/
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &oldtup->t_self, *lockmode,
LockWaitBlock, &have_tuple_lock);
- XactLockTableWait(xwait, relation, &oldtup.t_self,
+ XactLockTableWait(xwait, relation, &oldtup->t_self,
XLTW_Update);
checked_lockers = true;
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
@@ -3668,20 +3527,20 @@ l2:
* other xact could update this tuple before we get to this point.
* Check for xmax change, and start over if so.
*/
- if (xmax_infomask_changed(oldtup.t_data->t_infomask, infomask) ||
+ if (xmax_infomask_changed(oldtup->t_data->t_infomask, infomask) ||
!TransactionIdEquals(xwait,
- HeapTupleHeaderGetRawXmax(oldtup.t_data)))
+ HeapTupleHeaderGetRawXmax(oldtup->t_data)))
goto l2;
/* Otherwise check if it committed or aborted */
- UpdateXmaxHintBits(oldtup.t_data, buffer, xwait);
- if (oldtup.t_data->t_infomask & HEAP_XMAX_INVALID)
+ UpdateXmaxHintBits(oldtup->t_data, buffer, xwait);
+ if (oldtup->t_data->t_infomask & HEAP_XMAX_INVALID)
can_continue = true;
}
if (can_continue)
result = TM_Ok;
- else if (!ItemPointerEquals(&oldtup.t_self, &oldtup.t_data->t_ctid))
+ else if (!ItemPointerEquals(&oldtup->t_self, &oldtup->t_data->t_ctid))
result = TM_Updated;
else
result = TM_Deleted;
@@ -3694,39 +3553,33 @@ l2:
result == TM_Updated ||
result == TM_Deleted ||
result == TM_BeingModified);
- Assert(!(oldtup.t_data->t_infomask & HEAP_XMAX_INVALID));
+ Assert(!(oldtup->t_data->t_infomask & HEAP_XMAX_INVALID));
Assert(result != TM_Updated ||
- !ItemPointerEquals(&oldtup.t_self, &oldtup.t_data->t_ctid));
+ !ItemPointerEquals(&oldtup->t_self, &oldtup->t_data->t_ctid));
}
if (crosscheck != InvalidSnapshot && result == TM_Ok)
{
/* Perform additional check for transaction-snapshot mode RI updates */
- if (!HeapTupleSatisfiesVisibility(&oldtup, crosscheck, buffer))
+ if (!HeapTupleSatisfiesVisibility(oldtup, crosscheck, buffer))
result = TM_Updated;
}
if (result != TM_Ok)
{
- tmfd->ctid = oldtup.t_data->t_ctid;
- tmfd->xmax = HeapTupleHeaderGetUpdateXid(oldtup.t_data);
+ tmfd->ctid = oldtup->t_data->t_ctid;
+ tmfd->xmax = HeapTupleHeaderGetUpdateXid(oldtup->t_data);
if (result == TM_SelfModified)
- tmfd->cmax = HeapTupleHeaderGetCmax(oldtup.t_data);
+ tmfd->cmax = HeapTupleHeaderGetCmax(oldtup->t_data);
else
tmfd->cmax = InvalidCommandId;
UnlockReleaseBuffer(buffer);
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
- if (vmbuffer != InvalidBuffer)
- ReleaseBuffer(vmbuffer);
+ UnlockTupleTuplock(relation, &oldtup->t_self, *lockmode);
+ if (*vmbuffer != InvalidBuffer)
+ ReleaseBuffer(*vmbuffer);
*update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
return result;
}
@@ -3739,10 +3592,10 @@ l2:
* tuple has been locked or updated under us, but hopefully it won't
* happen very often.
*/
- if (vmbuffer == InvalidBuffer && PageIsAllVisible(page))
+ if (*vmbuffer == InvalidBuffer && PageIsAllVisible(page))
{
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- visibilitymap_pin(relation, block, &vmbuffer);
+ visibilitymap_pin(relation, block, vmbuffer);
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
goto l2;
}
@@ -3753,9 +3606,9 @@ l2:
* If the tuple we're updating is locked, we need to preserve the locking
* info in the old tuple's Xmax. Prepare a new Xmax value for this.
*/
- compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
- oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2,
+ compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup->t_data),
+ oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2,
xid, *lockmode, true,
&xmax_old_tuple, &infomask_old_tuple,
&infomask2_old_tuple);
@@ -3767,12 +3620,12 @@ l2:
* tuple. (In rare cases that might also be InvalidTransactionId and yet
* not have the HEAP_XMAX_INVALID bit set; that's fine.)
*/
- if ((oldtup.t_data->t_infomask & HEAP_XMAX_INVALID) ||
- HEAP_LOCKED_UPGRADED(oldtup.t_data->t_infomask) ||
+ if ((oldtup->t_data->t_infomask & HEAP_XMAX_INVALID) ||
+ HEAP_LOCKED_UPGRADED(oldtup->t_data->t_infomask) ||
(checked_lockers && !locker_remains))
xmax_new_tuple = InvalidTransactionId;
else
- xmax_new_tuple = HeapTupleHeaderGetRawXmax(oldtup.t_data);
+ xmax_new_tuple = HeapTupleHeaderGetRawXmax(oldtup->t_data);
if (!TransactionIdIsValid(xmax_new_tuple))
{
@@ -3787,7 +3640,7 @@ l2:
* Note that since we're doing an update, the only possibility is that
* the lockers had FOR KEY SHARE lock.
*/
- if (oldtup.t_data->t_infomask & HEAP_XMAX_IS_MULTI)
+ if (oldtup->t_data->t_infomask & HEAP_XMAX_IS_MULTI)
{
GetMultiXactIdHintBits(xmax_new_tuple, &infomask_new_tuple,
&infomask2_new_tuple);
@@ -3815,7 +3668,7 @@ l2:
* Replace cid with a combo CID if necessary. Note that we already put
* the plain cid into the new tuple.
*/
- HeapTupleHeaderAdjustCmax(oldtup.t_data, &cid, &iscombo);
+ HeapTupleHeaderAdjustCmax(oldtup->t_data, &cid, &iscombo);
/*
* If the toaster needs to be activated, OR if the new tuple will not fit
@@ -3832,12 +3685,12 @@ l2:
relation->rd_rel->relkind != RELKIND_MATVIEW)
{
/* toast table entries should never be recursively toasted */
- Assert(!HeapTupleHasExternal(&oldtup));
+ Assert(!HeapTupleHasExternal(oldtup));
Assert(!HeapTupleHasExternal(newtup));
need_toast = false;
}
else
- need_toast = (HeapTupleHasExternal(&oldtup) ||
+ need_toast = (HeapTupleHasExternal(oldtup) ||
HeapTupleHasExternal(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
@@ -3870,9 +3723,9 @@ l2:
* updating, because the potentially created multixact would otherwise
* be wrong.
*/
- compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
- oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2,
+ compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup->t_data),
+ oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2,
xid, *lockmode, false,
&xmax_lock_old_tuple, &infomask_lock_old_tuple,
&infomask2_lock_old_tuple);
@@ -3882,18 +3735,18 @@ l2:
START_CRIT_SECTION();
/* Clear obsolete visibility flags ... */
- oldtup.t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
- oldtup.t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
- HeapTupleClearHotUpdated(&oldtup);
+ oldtup->t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
+ oldtup->t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
+ HeapTupleClearHotUpdated(oldtup);
/* ... and store info about transaction updating this tuple */
Assert(TransactionIdIsValid(xmax_lock_old_tuple));
- HeapTupleHeaderSetXmax(oldtup.t_data, xmax_lock_old_tuple);
- oldtup.t_data->t_infomask |= infomask_lock_old_tuple;
- oldtup.t_data->t_infomask2 |= infomask2_lock_old_tuple;
- HeapTupleHeaderSetCmax(oldtup.t_data, cid, iscombo);
+ HeapTupleHeaderSetXmax(oldtup->t_data, xmax_lock_old_tuple);
+ oldtup->t_data->t_infomask |= infomask_lock_old_tuple;
+ oldtup->t_data->t_infomask2 |= infomask2_lock_old_tuple;
+ HeapTupleHeaderSetCmax(oldtup->t_data, cid, iscombo);
/* temporarily make it look not-updated, but locked */
- oldtup.t_data->t_ctid = oldtup.t_self;
+ oldtup->t_data->t_ctid = oldtup->t_self;
/*
* Clear all-frozen bit on visibility map if needed. We could
@@ -3902,7 +3755,7 @@ l2:
* worthwhile.
*/
if (PageIsAllVisible(page) &&
- visibilitymap_clear(relation, block, vmbuffer,
+ visibilitymap_clear(relation, block, *vmbuffer,
VISIBILITYMAP_ALL_FROZEN))
cleared_all_frozen = true;
@@ -3916,10 +3769,10 @@ l2:
XLogBeginInsert();
XLogRegisterBuffer(0, buffer, REGBUF_STANDARD);
- xlrec.offnum = ItemPointerGetOffsetNumber(&oldtup.t_self);
+ xlrec.offnum = ItemPointerGetOffsetNumber(&oldtup->t_self);
xlrec.xmax = xmax_lock_old_tuple;
- xlrec.infobits_set = compute_infobits(oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2);
+ xlrec.infobits_set = compute_infobits(oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2);
xlrec.flags =
cleared_all_frozen ? XLH_LOCK_ALL_FROZEN_CLEARED : 0;
XLogRegisterData(&xlrec, SizeOfHeapLock);
@@ -3941,7 +3794,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = heap_toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = heap_toast_insert_or_update(relation, newtup, oldtup, 0);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
@@ -3977,20 +3830,20 @@ l2:
/* It doesn't fit, must use RelationGetBufferForTuple. */
newbuf = RelationGetBufferForTuple(relation, heaptup->t_len,
buffer, 0, NULL,
- &vmbuffer_new, &vmbuffer,
+ &vmbuffer_new, vmbuffer,
0);
/* We're all done. */
break;
}
/* Acquire VM page pin if needed and we don't have it. */
- if (vmbuffer == InvalidBuffer && PageIsAllVisible(page))
- visibilitymap_pin(relation, block, &vmbuffer);
+ if (*vmbuffer == InvalidBuffer && PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, vmbuffer);
/* Re-acquire the lock on the old tuple's page. */
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
/* Re-check using the up-to-date free space */
pagefree = PageGetHeapFreeSpace(page);
if (newtupsize > pagefree ||
- (vmbuffer == InvalidBuffer && PageIsAllVisible(page)))
+ (*vmbuffer == InvalidBuffer && PageIsAllVisible(page)))
{
/*
* Rats, it doesn't fit anymore, or somebody just now set the
@@ -4028,7 +3881,7 @@ l2:
* will include checking the relation level, there is no benefit to a
* separate check for the new tuple.
*/
- CheckForSerializableConflictIn(relation, &oldtup.t_self,
+ CheckForSerializableConflictIn(relation, &oldtup->t_self,
BufferGetBlockNumber(buffer));
/*
@@ -4036,7 +3889,6 @@ l2:
* has enough space for the new tuple. If they are the same buffer, only
* one pin is held.
*/
-
if (newbuf == buffer)
{
/*
@@ -4044,7 +3896,7 @@ l2:
* to do a HOT update. Check if any of the index columns have been
* changed.
*/
- if (!bms_overlap(modified_attrs, hot_attrs))
+ if (!bms_overlap(mix_attrs, hot_attrs))
{
use_hot_update = true;
@@ -4055,7 +3907,7 @@ l2:
* indexes if the columns were updated, or we may fail to detect
* e.g. value bound changes in BRIN minmax indexes.
*/
- if (bms_overlap(modified_attrs, sum_attrs))
+ if (bms_overlap(mix_attrs, sum_attrs))
summarized_update = true;
}
}
@@ -4072,10 +3924,8 @@ l2:
* logged. Pass old key required as true only if the replica identity key
* columns are modified or it has external data.
*/
- old_key_tuple = ExtractReplicaIdentity(relation, &oldtup,
- bms_overlap(modified_attrs, id_attrs) ||
- id_has_external,
- &old_key_copied);
+ old_key_tuple = ExtractReplicaIdentity(relation, oldtup, rid_attrs,
+ rep_id_key_required, &old_key_copied);
/* NO EREPORT(ERROR) from here till changes are logged */
START_CRIT_SECTION();
@@ -4097,7 +3947,7 @@ l2:
if (use_hot_update)
{
/* Mark the old tuple as HOT-updated */
- HeapTupleSetHotUpdated(&oldtup);
+ HeapTupleSetHotUpdated(oldtup);
/* And mark the new tuple as heap-only */
HeapTupleSetHeapOnly(heaptup);
/* Mark the caller's copy too, in case different from heaptup */
@@ -4106,7 +3956,7 @@ l2:
else
{
/* Make sure tuples are correctly marked as not-HOT */
- HeapTupleClearHotUpdated(&oldtup);
+ HeapTupleClearHotUpdated(oldtup);
HeapTupleClearHeapOnly(heaptup);
HeapTupleClearHeapOnly(newtup);
}
@@ -4115,17 +3965,17 @@ l2:
/* Clear obsolete visibility flags, possibly set by ourselves above... */
- oldtup.t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
- oldtup.t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
+ oldtup->t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
+ oldtup->t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
/* ... and store info about transaction updating this tuple */
Assert(TransactionIdIsValid(xmax_old_tuple));
- HeapTupleHeaderSetXmax(oldtup.t_data, xmax_old_tuple);
- oldtup.t_data->t_infomask |= infomask_old_tuple;
- oldtup.t_data->t_infomask2 |= infomask2_old_tuple;
- HeapTupleHeaderSetCmax(oldtup.t_data, cid, iscombo);
+ HeapTupleHeaderSetXmax(oldtup->t_data, xmax_old_tuple);
+ oldtup->t_data->t_infomask |= infomask_old_tuple;
+ oldtup->t_data->t_infomask2 |= infomask2_old_tuple;
+ HeapTupleHeaderSetCmax(oldtup->t_data, cid, iscombo);
/* record address of new tuple in t_ctid of old one */
- oldtup.t_data->t_ctid = heaptup->t_self;
+ oldtup->t_data->t_ctid = heaptup->t_self;
/* clear PD_ALL_VISIBLE flags, reset all visibilitymap bits */
if (PageIsAllVisible(BufferGetPage(buffer)))
@@ -4133,7 +3983,7 @@ l2:
all_visible_cleared = true;
PageClearAllVisible(BufferGetPage(buffer));
visibilitymap_clear(relation, BufferGetBlockNumber(buffer),
- vmbuffer, VISIBILITYMAP_VALID_BITS);
+ *vmbuffer, VISIBILITYMAP_VALID_BITS);
}
if (newbuf != buffer && PageIsAllVisible(BufferGetPage(newbuf)))
{
@@ -4158,12 +4008,12 @@ l2:
*/
if (RelationIsAccessibleInLogicalDecoding(relation))
{
- log_heap_new_cid(relation, &oldtup);
+ log_heap_new_cid(relation, oldtup);
log_heap_new_cid(relation, heaptup);
}
recptr = log_heap_update(relation, buffer,
- newbuf, &oldtup, heaptup,
+ newbuf, oldtup, heaptup,
old_key_tuple,
all_visible_cleared,
all_visible_cleared_new);
@@ -4188,7 +4038,7 @@ l2:
* both tuple versions in one call to inval.c so we can avoid redundant
* sinval messages.)
*/
- CacheInvalidateHeapTuple(relation, &oldtup, heaptup);
+ CacheInvalidateHeapTuple(relation, oldtup, heaptup);
/* Now we can release the buffer(s) */
if (newbuf != buffer)
@@ -4196,14 +4046,14 @@ l2:
ReleaseBuffer(buffer);
if (BufferIsValid(vmbuffer_new))
ReleaseBuffer(vmbuffer_new);
- if (BufferIsValid(vmbuffer))
- ReleaseBuffer(vmbuffer);
+ if (BufferIsValid(*vmbuffer))
+ ReleaseBuffer(*vmbuffer);
/*
* Release the lmgr tuple lock, if we had it.
*/
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &oldtup->t_self, *lockmode);
pgstat_count_heap_update(relation, use_hot_update, newbuf != buffer);
@@ -4236,13 +4086,6 @@ l2:
if (old_key_tuple != NULL && old_key_copied)
heap_freetuple(old_key_tuple);
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
-
return TM_Ok;
}
@@ -4251,7 +4094,7 @@ l2:
* Confirm adequate lock held during heap_update(), per rules from
* README.tuplock section "Locking to write inplace-updated tables".
*/
-static void
+void
check_lock_if_inplace_updateable_rel(Relation relation,
const ItemPointerData *otid,
HeapTuple newtup)
@@ -4423,7 +4266,7 @@ heap_attr_equals(TupleDesc tupdesc, int attrnum, Datum value1, Datum value2,
* listed as interesting) of the old tuple is a member of external_cols and is
* stored externally.
*/
-static Bitmapset *
+Bitmapset *
HeapDetermineColumnsInfo(Relation relation,
Bitmapset *interesting_cols,
Bitmapset *external_cols,
@@ -4506,25 +4349,175 @@ HeapDetermineColumnsInfo(Relation relation,
}
/*
- * simple_heap_update - replace a tuple
- *
- * This routine may be used to update a tuple when concurrent updates of
- * the target tuple are not expected (for example, because we have a lock
- * on the relation associated with the tuple). Any failure is reported
- * via ereport().
+ * This routine may be used to update a tuple when concurrent updates of the
+ * target tuple are not expected (for example, because we have a lock on the
+ * relation associated with the tuple). Any failure is reported via ereport().
*/
void
-simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup,
+simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TM_FailureData tmfd;
LockTupleMode lockmode;
+ Buffer buffer;
+ Buffer vmbuffer = InvalidBuffer;
+ Page page;
+ BlockNumber block;
+ Bitmapset *hot_attrs,
+ *sum_attrs,
+ *pk_attrs,
+ *rid_attrs,
+ *mix_attrs,
+ *idx_attrs;
+ ItemId lp;
+ HeapTupleData oldtup;
+ bool rep_id_key_required = false;
+
+ Assert(ItemPointerIsValid(otid));
+
+ /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
+ Assert(HeapTupleHeaderGetNatts(tuple->t_data) <=
+ RelationGetNumberOfAttributes(relation));
+
+ /*
+ * Forbid this during a parallel operation, lest it allocate a combo CID.
+ * Other workers might need that combo CID for visibility checks, and we
+ * have no provision for broadcasting it to them.
+ */
+ if (IsInParallelMode())
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
+ errmsg("cannot update tuples during a parallel operation")));
+
+#ifdef USE_ASSERT_CHECKING
+ check_lock_if_inplace_updateable_rel(relation, otid, tuple);
+#endif
+
+ /*
+ * Fetch the list of attributes to be checked for various operations.
+ *
+ * For HOT considerations, this is wasted effort if we fail to update or
+ * have to put the new tuple on a different page. But we must compute the
+ * list before obtaining buffer lock --- in the worst case, if we are
+ * doing an update on one of the relevant system catalogs, we could
+ * deadlock if we try to fetch the list later. In any case, the relcache
+ * caches the data so this is usually pretty cheap.
+ *
+ * We also need columns used by the replica identity and columns that are
+ * considered the "key" of rows in the table.
+ *
+ * Note that we get copies of each bitmap, so we need not worry about
+ * relcache flush happening midway through.
+ */
+ hot_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_HOT_BLOCKING);
+ sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+ pk_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
+ rid_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
+ idx_attrs = bms_copy(hot_attrs);
+ idx_attrs = bms_add_members(idx_attrs, sum_attrs);
+ idx_attrs = bms_add_members(idx_attrs, pk_attrs);
+ idx_attrs = bms_add_members(idx_attrs, rid_attrs);
+
+ block = ItemPointerGetBlockNumber(otid);
+ INJECTION_POINT("heap_update-before-pin", NULL);
+ buffer = ReadBuffer(relation, block);
+ page = BufferGetPage(buffer);
+
+ /*
+ * Before locking the buffer, pin the visibility map page if it appears to
+ * be necessary. Since we haven't got the lock yet, someone else might be
+ * in the middle of changing this, so we'll need to recheck after we have
+ * the lock.
+ */
+ if (PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, &vmbuffer);
+
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+
+ lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
+
+ /*
+ * Usually, a buffer pin and/or snapshot blocks pruning of otid, ensuring
+ * we see LP_NORMAL here. When the otid origin is a syscache, we may have
+ * neither a pin nor a snapshot. Hence, we may see other LP_ states, each
+ * of which indicates concurrent pruning.
+ *
+ * Failing with TM_Updated would be most accurate. However, unlike other
+ * TM_Updated scenarios, we don't know the successor ctid in LP_UNUSED and
+ * LP_DEAD cases. While the distinction between TM_Updated and TM_Deleted
+ * does matter to SQL statements UPDATE and MERGE, those SQL statements
+ * hold a snapshot that ensures LP_NORMAL. Hence, the choice between
+ * TM_Updated and TM_Deleted affects only the wording of error messages.
+ * Settle on TM_Deleted, for two reasons. First, it avoids complicating
+ * the specification of when tmfd->ctid is valid. Second, it creates
+ * error log evidence that we took this branch.
+ *
+ * Since it's possible to see LP_UNUSED at otid, it's also possible to see
+ * LP_NORMAL for a tuple that replaced LP_UNUSED. If it's a tuple for an
+ * unrelated row, we'll fail with "duplicate key value violates unique".
+ * XXX if otid is the live, newer version of the newtup row, we'll discard
+ * changes originating in versions of this catalog row after the version
+ * the caller got from syscache. See syscache-update-pruned.spec.
+ */
+ if (!ItemIdIsNormal(lp))
+ {
+ Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
+
+ UnlockReleaseBuffer(buffer);
+ if (vmbuffer != InvalidBuffer)
+ ReleaseBuffer(vmbuffer);
+ *update_indexes = TU_None;
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(idx_attrs);
+ /* mix_attrs not yet initialized */
+
+ elog(ERROR, "tuple concurrently deleted");
+
+ return;
+ }
+
+ /*
+ * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
+ * then pass that on to heap_update.
+ */
+ oldtup.t_tableOid = RelationGetRelid(relation);
+ oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
+ oldtup.t_len = ItemIdGetLength(lp);
+ oldtup.t_self = *otid;
+
+ mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
+ &oldtup, tuple, &rep_id_key_required);
+
+ /*
+ * We'll need to WAL log the replica identity attributes if either they
+ * overlap with the modified indexed attributes or, as we've checked for
+ * just now in HeapDetermineColumnsInfo, they were unmodified external
+ * indexed attributes.
+ */
+ rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+
+ result = heap_update(relation, &oldtup, tuple, GetCurrentCommandId(true),
+ InvalidSnapshot, true /* wait for commit */ , &tmfd, &lockmode,
+ buffer, page, block, lp, hot_attrs, sum_attrs, pk_attrs,
+ rid_attrs, mix_attrs, &vmbuffer, rep_id_key_required,
+ update_indexes);
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(mix_attrs);
+ bms_free(idx_attrs);
- result = heap_update(relation, otid, tup,
- GetCurrentCommandId(true), InvalidSnapshot,
- true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
switch (result)
{
case TM_SelfModified:
@@ -9164,12 +9157,11 @@ log_heap_new_cid(Relation relation, HeapTuple tup)
* the same tuple that was passed in.
*/
static HeapTuple
-ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
- bool *copy)
+ExtractReplicaIdentity(Relation relation, HeapTuple tp, Bitmapset *rid_attrs,
+ bool key_required, bool *copy)
{
TupleDesc desc = RelationGetDescr(relation);
char replident = relation->rd_rel->relreplident;
- Bitmapset *idattrs;
HeapTuple key_tuple;
bool nulls[MaxHeapAttributeNumber];
Datum values[MaxHeapAttributeNumber];
@@ -9200,17 +9192,13 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
if (!key_required)
return NULL;
- /* find out the replica identity columns */
- idattrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
-
/*
* If there's no defined replica identity columns, treat as !key_required.
* (This case should not be reachable from heap_update, since that should
* calculate key_required accurately. But heap_delete just passes
* constant true for key_required, so we can hit this case in deletes.)
*/
- if (bms_is_empty(idattrs))
+ if (bms_is_empty(rid_attrs))
return NULL;
/*
@@ -9223,7 +9211,7 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
for (int i = 0; i < desc->natts; i++)
{
if (bms_is_member(i + 1 - FirstLowInvalidHeapAttributeNumber,
- idattrs))
+ rid_attrs))
Assert(!nulls[i]);
else
nulls[i] = true;
@@ -9232,8 +9220,6 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
key_tuple = heap_form_tuple(desc, values, nulls);
*copy = true;
- bms_free(idattrs);
-
/*
* If the tuple, which by here only contains indexed columns, still has
* toasted columns, force them to be inlined. This is somewhat unlikely
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bcbac844bb6..1cf9a18775d 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -44,6 +44,7 @@
#include "storage/procarray.h"
#include "storage/smgr.h"
#include "utils/builtins.h"
+#include "utils/injection_point.h"
#include "utils/rel.h"
static void reform_and_rewrite_tuple(HeapTuple tuple,
@@ -312,23 +313,133 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
return heap_delete(relation, tid, cid, crosscheck, wait, tmfd, changingPart);
}
-
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd,
LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
{
+ bool rep_id_key_required = false;
bool shouldFree = true;
HeapTuple tuple = ExecFetchSlotHeapTuple(slot, true, &shouldFree);
+ HeapTupleData oldtup;
+ Buffer buffer;
+ Buffer vmbuffer = InvalidBuffer;
+ Page page;
+ BlockNumber block;
+ ItemId lp;
+ Bitmapset *hot_attrs,
+ *sum_attrs,
+ *pk_attrs,
+ *rid_attrs,
+ *mix_attrs,
+ *idx_attrs;
TM_Result result;
+ Assert(ItemPointerIsValid(otid));
+
+ /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
+ Assert(HeapTupleHeaderGetNatts(tuple->t_data) <=
+ RelationGetNumberOfAttributes(relation));
+
+ /*
+ * Forbid this during a parallel operation, lest it allocate a combo CID.
+ * Other workers might need that combo CID for visibility checks, and we
+ * have no provision for broadcasting it to them.
+ */
+ if (IsInParallelMode())
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
+ errmsg("cannot update tuples during a parallel operation")));
+
+#ifdef USE_ASSERT_CHECKING
+ check_lock_if_inplace_updateable_rel(relation, otid, tuple);
+#endif
+
+ /*
+ * Fetch the list of attributes to be checked for various operations.
+ *
+ * For HOT considerations, this is wasted effort if we fail to update or
+ * have to put the new tuple on a different page. But we must compute the
+ * list before obtaining buffer lock --- in the worst case, if we are
+ * doing an update on one of the relevant system catalogs, we could
+ * deadlock if we try to fetch the list later. In any case, the relcache
+ * caches the data so this is usually pretty cheap.
+ *
+ * We also need columns used by the replica identity and columns that are
+ * considered the "key" of rows in the table.
+ *
+ * Note that we get copies of each bitmap, so we need not worry about
+ * relcache flush happening midway through.
+ */
+ hot_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_HOT_BLOCKING);
+ sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+ pk_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
+ rid_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
+ idx_attrs = bms_copy(hot_attrs);
+ idx_attrs = bms_add_members(idx_attrs, sum_attrs);
+ idx_attrs = bms_add_members(idx_attrs, pk_attrs);
+ idx_attrs = bms_add_members(idx_attrs, rid_attrs);
+
+ block = ItemPointerGetBlockNumber(otid);
+ INJECTION_POINT("heap_update-before-pin", NULL);
+ buffer = ReadBuffer(relation, block);
+ page = BufferGetPage(buffer);
+
+ /*
+ * Before locking the buffer, pin the visibility map page if it appears to
+ * be necessary. Since we haven't got the lock yet, someone else might be
+ * in the middle of changing this, so we'll need to recheck after we have
+ * the lock.
+ */
+ if (PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, &vmbuffer);
+
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+
+ lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
+
+ Assert(ItemIdIsNormal(lp));
+
+ /*
+ * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
+ * then pass that on to heap_update.
+ */
+ oldtup.t_tableOid = RelationGetRelid(relation);
+ oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
+ oldtup.t_len = ItemIdGetLength(lp);
+ oldtup.t_self = *otid;
+
+ mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
+ &oldtup, tuple, &rep_id_key_required);
+
+ /*
+ * We'll need to WAL log the replica identity attributes if either they
+ * overlap with the modified indexed attributes or, as we've checked for
+ * just now in HeapDetermineColumnsInfo, they were unmodified external
+ * indexed attributes.
+ */
+ rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
- result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
- tmfd, lockmode, update_indexes);
+ result = heap_update(relation, &oldtup, tuple, cid, crosscheck, wait, tmfd, lockmode,
+ buffer, page, block, lp, hot_attrs, sum_attrs, pk_attrs,
+ rid_attrs, mix_attrs, &vmbuffer, rep_id_key_required, update_indexes);
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(mix_attrs);
+ bms_free(idx_attrs);
+
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 632c4332a8c..2f9a2b069cd 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -364,11 +364,13 @@ extern TM_Result heap_delete(Relation relation, const ItemPointerData *tid,
TM_FailureData *tmfd, bool changingPart);
extern void heap_finish_speculative(Relation relation, const ItemPointerData *tid);
extern void heap_abort_speculative(Relation relation, const ItemPointerData *tid);
-extern TM_Result heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+extern TM_Result heap_update(Relation relation, HeapTupleData *oldtup,
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
+ Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
+ Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
+ Bitmapset *mix_attrs, Buffer *vmbuffer,
+ bool rep_id_key_required, TU_UpdateIndexes *update_indexes);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -430,6 +432,18 @@ extern void log_heap_prune_and_freeze(Relation relation, Buffer buffer,
OffsetNumber *dead, int ndead,
OffsetNumber *unused, int nunused);
+/* in heap/heapam.c */
+extern Bitmapset *HeapDetermineColumnsInfo(Relation relation,
+ Bitmapset *interesting_cols,
+ Bitmapset *external_cols,
+ HeapTuple oldtup, HeapTuple newtup,
+ bool *has_external);
+#ifdef USE_ASSERT_CHECKING
+extern void check_lock_if_inplace_updateable_rel(Relation relation,
+ const ItemPointerData *otid,
+ HeapTuple newtup);
+#endif
+
/* in heap/vacuumlazy.c */
extern void heap_vacuum_rel(Relation rel,
const VacuumParams params, BufferAccessStrategy bstrategy);
--
2.51.2
[application/octet-stream] v25-0003-Replace-index_unchanged_by_update-with-ri_Change.patch (8.4K, 3-v25-0003-Replace-index_unchanged_by_update-with-ri_Change.patch)
download | inline diff:
From 6b95e4418b2bfc20245e24b75ea7173b4d83552e Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Fri, 31 Oct 2025 14:55:25 -0400
Subject: [PATCH v25 3/3] Replace index_unchanged_by_update() with
ri_ChangedIndexedCols
In execIndexing on updates we'd like to pass a hint to the indexing code
when the indexed attributes are unchanged. This commit replaces the now
redundant code in index_unchanged_by_update() with the same information
found earlier in ExecWhichIndexesRequireUpdates() and stashed in
ri_ChangedIndexedCols.
---
src/backend/catalog/toasting.c | 2 -
src/backend/executor/execIndexing.c | 156 +---------------------------
src/backend/nodes/makefuncs.c | 2 -
src/include/nodes/execnodes.h | 4 -
4 files changed, 1 insertion(+), 163 deletions(-)
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 9d7cb4438d5..c665aa744b3 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -304,8 +304,6 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
indexInfo->ii_Unique = true;
indexInfo->ii_NullsNotDistinct = false;
indexInfo->ii_ReadyForInserts = true;
- indexInfo->ii_CheckedUnchanged = false;
- indexInfo->ii_IndexUnchanged = false;
indexInfo->ii_Concurrent = false;
indexInfo->ii_BrokenHotChain = false;
indexInfo->ii_ParallelWorkers = 0;
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 05ae7eb9f65..ff9d49d620d 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -143,11 +143,6 @@ static bool check_exclusion_or_unique_constraint(Relation heap, Relation index,
static bool index_recheck_constraint(Relation index, const Oid *constr_procs,
const Datum *existing_values, const bool *existing_isnull,
const Datum *new_values);
-static bool index_unchanged_by_update(ResultRelInfo *resultRelInfo,
- EState *estate, IndexInfo *indexInfo,
- Relation indexRelation);
-static bool index_expression_changed_walker(Node *node,
- Bitmapset *allUpdatedCols);
static void ExecWithoutOverlapsNotEmpty(Relation rel, NameData attname, Datum attval,
char typtype, Oid atttypid);
@@ -451,10 +446,7 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* index. If we're being called as part of an UPDATE statement,
* consider if the 'indexUnchanged' = true hint should be passed.
*/
- indexUnchanged = update && index_unchanged_by_update(resultRelInfo,
- estate,
- indexInfo,
- indexRelation);
+ indexUnchanged = update && bms_is_empty(resultRelInfo->ri_ChangedIndexedCols);
satisfiesConstraint =
index_insert(indexRelation, /* index relation */
@@ -1014,152 +1006,6 @@ index_recheck_constraint(Relation index, const Oid *constr_procs,
return true;
}
-/*
- * Check if ExecInsertIndexTuples() should pass indexUnchanged hint.
- *
- * When the executor performs an UPDATE that requires a new round of index
- * tuples, determine if we should pass 'indexUnchanged' = true hint for one
- * single index.
- */
-static bool
-index_unchanged_by_update(ResultRelInfo *resultRelInfo, EState *estate,
- IndexInfo *indexInfo, Relation indexRelation)
-{
- Bitmapset *updatedCols;
- Bitmapset *extraUpdatedCols;
- Bitmapset *allUpdatedCols;
- bool hasexpression = false;
- List *idxExprs;
-
- /*
- * Check cache first
- */
- if (indexInfo->ii_CheckedUnchanged)
- return indexInfo->ii_IndexUnchanged;
- indexInfo->ii_CheckedUnchanged = true;
-
- /*
- * Check for indexed attribute overlap with updated columns.
- *
- * Only do this for key columns. A change to a non-key column within an
- * INCLUDE index should not be counted here. Non-key column values are
- * opaque payload state to the index AM, a little like an extra table TID.
- *
- * Note that row-level BEFORE triggers won't affect our behavior, since
- * they don't affect the updatedCols bitmaps generally. It doesn't seem
- * worth the trouble of checking which attributes were changed directly.
- */
- updatedCols = ExecGetUpdatedCols(resultRelInfo, estate);
- extraUpdatedCols = ExecGetExtraUpdatedCols(resultRelInfo, estate);
- for (int attr = 0; attr < indexInfo->ii_NumIndexKeyAttrs; attr++)
- {
- int keycol = indexInfo->ii_IndexAttrNumbers[attr];
-
- if (keycol <= 0)
- {
- /*
- * Skip expressions for now, but remember to deal with them later
- * on
- */
- hasexpression = true;
- continue;
- }
-
- if (bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
- updatedCols) ||
- bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
- extraUpdatedCols))
- {
- /* Changed key column -- don't hint for this index */
- indexInfo->ii_IndexUnchanged = false;
- return false;
- }
- }
-
- /*
- * When we get this far and index has no expressions, return true so that
- * index_insert() call will go on to pass 'indexUnchanged' = true hint.
- *
- * The _absence_ of an indexed key attribute that overlaps with updated
- * attributes (in addition to the total absence of indexed expressions)
- * shows that the index as a whole is logically unchanged by UPDATE.
- */
- if (!hasexpression)
- {
- indexInfo->ii_IndexUnchanged = true;
- return true;
- }
-
- /*
- * Need to pass only one bms to expression_tree_walker helper function.
- * Avoid allocating memory in common case where there are no extra cols.
- */
- if (!extraUpdatedCols)
- allUpdatedCols = updatedCols;
- else
- allUpdatedCols = bms_union(updatedCols, extraUpdatedCols);
-
- /*
- * We have to work slightly harder in the event of indexed expressions,
- * but the principle is the same as before: try to find columns (Vars,
- * actually) that overlap with known-updated columns.
- *
- * If we find any matching Vars, don't pass hint for index. Otherwise
- * pass hint.
- */
- idxExprs = RelationGetIndexExpressions(indexRelation);
- hasexpression = index_expression_changed_walker((Node *) idxExprs,
- allUpdatedCols);
- list_free(idxExprs);
- if (extraUpdatedCols)
- bms_free(allUpdatedCols);
-
- if (hasexpression)
- {
- indexInfo->ii_IndexUnchanged = false;
- return false;
- }
-
- /*
- * Deliberately don't consider index predicates. We should even give the
- * hint when result rel's "updated tuple" has no corresponding index
- * tuple, which is possible with a partial index (provided the usual
- * conditions are met).
- */
- indexInfo->ii_IndexUnchanged = true;
- return true;
-}
-
-/*
- * Indexed expression helper for index_unchanged_by_update().
- *
- * Returns true when Var that appears within allUpdatedCols located.
- */
-static bool
-index_expression_changed_walker(Node *node, Bitmapset *allUpdatedCols)
-{
- if (node == NULL)
- return false;
-
- if (IsA(node, Var))
- {
- Var *var = (Var *) node;
-
- if (bms_is_member(var->varattno - FirstLowInvalidHeapAttributeNumber,
- allUpdatedCols))
- {
- /* Var was updated -- indicates that we should not hint */
- return true;
- }
-
- /* Still haven't found a reason to not pass the hint */
- return false;
- }
-
- return expression_tree_walker(node, index_expression_changed_walker,
- allUpdatedCols);
-}
-
/*
* ExecWithoutOverlapsNotEmpty - raise an error if the tuple has an empty
* range or multirange in the given attribute.
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index 1f1364f9df9..e9a53b95caf 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -845,8 +845,6 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
n->ii_Unique = unique;
n->ii_NullsNotDistinct = nulls_not_distinct;
n->ii_ReadyForInserts = isready;
- n->ii_CheckedUnchanged = false;
- n->ii_IndexUnchanged = false;
n->ii_Concurrent = concurrent;
n->ii_Summarizing = summarizing;
n->ii_WithoutOverlaps = withoutoverlaps;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index e10f4239de9..1259897282e 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -216,10 +216,6 @@ typedef struct IndexInfo
bool ii_NullsNotDistinct;
/* is it valid for inserts? */
bool ii_ReadyForInserts;
- /* IndexUnchanged status determined yet? */
- bool ii_CheckedUnchanged;
- /* aminsert hint, cached for retail inserts */
- bool ii_IndexUnchanged;
/* are we doing a concurrent index build? */
bool ii_Concurrent;
/* did we detect any broken HOT chains? */
--
2.51.2
[application/octet-stream] v25-0002-Track-changed-indexed-columns-in-the-executor-du.patch (113.5K, 4-v25-0002-Track-changed-indexed-columns-in-the-executor-du.patch)
download | inline diff:
From fff7f1524821c37db3b512a417031b141c56f631 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 26 Oct 2025 10:49:25 -0400
Subject: [PATCH v25 2/3] Track changed indexed columns in the executor during
UPDATEs
Refactor executor update logic to determine which indexed columns have
actually changed during an UPDATE operation rather than leaving this up
to HeapDetermineColumnsInfo() in heap_update().
ExecWhichIndexesRequireUpdates() replaces HeapDeterminesColumnsInfo()
when invoked from the table AM API via heapam_tuple_update(). The
test for equality remains datumIsEqual() as before.
This change necessitated some logic changes in execReplication() as it
performs updates now must provide the set of attributes that are both
changed and referenced by indexes. Luckilly, this is available within
calls to slot_modify_data() where LogicalRepTupleData is processed and
has a record of updated attributes. In this case rather than using
ExecWhichIndexesRequireUpdates() we can preseve what slot_modify_data()
identifies as the modified set and then intersect that with the set of
indexes on the relation and get the correct set of modified indexed
attributes required on heap_update().
This commit also extends the role index AMs play determining if they
require an update. A new optional index AM API, amcomparedatums(), is
added to allow index access methods to provide custom logic for
comparing datums. Hash and Gin indexes now implement this function. When
not implemented the executor will compare TupleTableSlot datum for
equality using datumIsEqual() as before.
Because heap_update() now requires the caller to provide the modified
indexed columns simple_heap_update() has become a tad more complex. It
is only called from CatalogTupleUpdate() which either updates heap
tuples via their Form_XXX or by calling heap_modify_tuple(). In both
cases the caller does know the modified set of attributes, but sadly
those attributes are lost before being provided to simple_heap_update().
Due to that the "simple" path has to (for now) retain the
HeapDetermineColumnsInfo() logic in order for catalog updates to
potentially take the HOT path.
---
src/backend/access/brin/brin.c | 1 +
src/backend/access/gin/ginutil.c | 90 ++-
src/backend/access/hash/hash.c | 44 ++
src/backend/access/heap/heapam.c | 20 +-
src/backend/access/heap/heapam_handler.c | 76 +-
src/backend/access/nbtree/nbtree.c | 1 +
src/backend/access/table/tableam.c | 5 +-
src/backend/bootstrap/bootstrap.c | 8 +
src/backend/catalog/index.c | 54 ++
src/backend/catalog/indexing.c | 16 +-
src/backend/catalog/toasting.c | 4 +
src/backend/executor/execIndexing.c | 41 +-
src/backend/executor/execMain.c | 1 +
src/backend/executor/execReplication.c | 7 +
src/backend/executor/nodeModifyTable.c | 283 +++++++-
src/backend/nodes/bitmapset.c | 4 +
src/backend/nodes/makefuncs.c | 4 +
src/backend/replication/logical/worker.c | 70 +-
src/backend/utils/cache/relcache.c | 15 +
src/include/access/amapi.h | 28 +
src/include/access/gin.h | 3 +
src/include/access/heapam.h | 6 +-
src/include/access/nbtree.h | 4 +
src/include/access/tableam.h | 8 +-
src/include/catalog/index.h | 1 +
src/include/executor/executor.h | 9 +
src/include/nodes/execnodes.h | 20 +
src/include/utils/rel.h | 1 +
src/include/utils/relcache.h | 1 +
.../expected/insert-conflict-specconflict.out | 20 +
.../regress/expected/heap_hot_updates.out | 650 ++++++++++++++++++
src/test/regress/parallel_schedule | 6 +
src/test/regress/sql/heap_hot_updates.sql | 513 ++++++++++++++
src/tools/pgindent/typedefs.list | 1 +
34 files changed, 1941 insertions(+), 74 deletions(-)
create mode 100644 src/test/regress/expected/heap_hot_updates.out
create mode 100644 src/test/regress/sql/heap_hot_updates.sql
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index cb3331921cb..36e639552e6 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -290,6 +290,7 @@ brinhandler(PG_FUNCTION_ARGS)
amroutine->amproperty = NULL;
amroutine->ambuildphasename = NULL;
amroutine->amvalidate = brinvalidate;
+ amroutine->amcomparedatums = NULL;
amroutine->amadjustmembers = NULL;
amroutine->ambeginscan = brinbeginscan;
amroutine->amrescan = brinrescan;
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index 78f7b7a2495..cc5410960a6 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -26,6 +26,7 @@
#include "storage/indexfsm.h"
#include "utils/builtins.h"
#include "utils/index_selfuncs.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/typcache.h"
@@ -78,6 +79,7 @@ ginhandler(PG_FUNCTION_ARGS)
amroutine->amproperty = NULL;
amroutine->ambuildphasename = ginbuildphasename;
amroutine->amvalidate = ginvalidate;
+ amroutine->amcomparedatums = gincomparedatums;
amroutine->amadjustmembers = ginadjustmembers;
amroutine->ambeginscan = ginbeginscan;
amroutine->amrescan = ginrescan;
@@ -477,13 +479,6 @@ cmpEntries(const void *a, const void *b, void *arg)
return res;
}
-
-/*
- * Extract the index key values from an indexable item
- *
- * The resulting key values are sorted, and any duplicates are removed.
- * This avoids generating redundant index entries.
- */
Datum *
ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
@@ -729,3 +724,84 @@ ginbuildphasename(int64 phasenum)
return NULL;
}
}
+
+/*
+ * gincomparedatums - Compare datums to determine if they produce identical keys
+ *
+ * This function extracts keys from both old_datum and new_datum using the
+ * opclass's extractValue function, then compares the extracted key arrays.
+ * Returns true if the key sets are identical (same keys, same counts).
+ *
+ * This enables HOT updates for GIN indexes when the indexed portions of a
+ * value haven't changed, even if the value itself has changed.
+ *
+ * Example: JSONB column with GIN index. If an update changes a non-indexed
+ * key in the JSONB document, the extracted keys are identical and we can
+ * do a HOT update.
+ */
+bool
+gincomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull)
+{
+ GinState ginstate;
+ Datum *old_keys;
+ Datum *new_keys;
+ GinNullCategory *old_categories;
+ GinNullCategory *new_categories;
+ int32 old_nkeys;
+ int32 new_nkeys;
+ MemoryContext tmpcontext;
+ MemoryContext oldcontext;
+ bool result = true;
+
+ /* Handle NULL cases */
+ if (old_isnull != new_isnull)
+ return false;
+ if (old_isnull)
+ return true;
+
+ /* Create temporary context for extraction work */
+ tmpcontext = AllocSetContextCreate(CurrentMemoryContext,
+ "GIN datum comparison",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(tmpcontext);
+
+ initGinState(&ginstate, index);
+
+ /* Extract keys from both datums using existing GIN infrastructure */
+ old_keys = ginExtractEntries(&ginstate, attnum, old_datum, old_isnull,
+ &old_nkeys, &old_categories);
+ new_keys = ginExtractEntries(&ginstate, attnum, new_datum, new_isnull,
+ &new_nkeys, &new_categories);
+
+ /* Different number of keys, definitely different */
+ if (old_nkeys != new_nkeys)
+ {
+ result = false;
+ goto cleanup;
+ }
+
+ /*
+ * Compare the sorted key arrays element-by-element. Since both arrays are
+ * already sorted by ginExtractEntries, we can do a simple O(n)
+ * comparison.
+ */
+ for (int i = 0; i < old_nkeys; i++)
+ {
+ if (ginCompareEntries(&ginstate, attnum,
+ old_keys[i], old_categories[i],
+ new_keys[i], new_categories[i]) != 0)
+ {
+ result = false;
+ break;
+ }
+ }
+
+cleanup:
+ /* Clean up */
+ MemoryContextSwitchTo(oldcontext);
+ MemoryContextDelete(tmpcontext);
+
+ return result;
+}
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 53061c819fb..91371dfdacd 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -50,6 +50,10 @@ static void hashbuildCallback(Relation index,
void *state);
+static bool hashcomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* Hash handler function: return IndexAmRoutine with access method parameters
* and callbacks.
@@ -98,6 +102,7 @@ hashhandler(PG_FUNCTION_ARGS)
amroutine->amproperty = NULL;
amroutine->ambuildphasename = NULL;
amroutine->amvalidate = hashvalidate;
+ amroutine->amcomparedatums = hashcomparedatums;
amroutine->amadjustmembers = hashadjustmembers;
amroutine->ambeginscan = hashbeginscan;
amroutine->amrescan = hashrescan;
@@ -944,3 +949,42 @@ hashtranslatecmptype(CompareType cmptype, Oid opfamily)
return HTEqualStrategyNumber;
return InvalidStrategy;
}
+
+/*
+ * hashcomparedatums - Compare datums to determine if they produce identical keys
+ *
+ * Returns true if the hash values are identical (index doesn't need update).
+ */
+bool
+hashcomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull)
+{
+ uint32 old_hashkey;
+ uint32 new_hashkey;
+
+ /* If both are NULL, they're equal */
+ if (old_isnull && new_isnull)
+ return true;
+
+ /* If NULL status differs, they're not equal */
+ if (old_isnull != new_isnull)
+ return false;
+
+ /*
+ * _hash_datum2hashkey() is used because we know this can't be a cross
+ * type comparison.
+ */
+ old_hashkey = _hash_datum2hashkey(index, old_datum);
+ new_hashkey = _hash_datum2hashkey(index, new_datum);
+
+ /*
+ * If hash keys are identical, the index entry would be the same. Return
+ * true to indicate no index update needed.
+ *
+ * Note: Hash collisions are rare but possible. If hash(x) == hash(y) but
+ * x != y, the hash index still treats them identically, so we correctly
+ * return true.
+ */
+ return (old_hashkey == new_hashkey);
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 30847db1fe3..3e88bdbbda8 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -3278,12 +3278,12 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* generated by another transaction).
*/
TM_Result
-heap_update(Relation relation, HeapTupleData *oldtup,
- HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
- Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
- Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
- Bitmapset *mix_attrs, Buffer *vmbuffer,
+heap_update(Relation relation, HeapTupleData *oldtup, HeapTuple newtup,
+ CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode,
+ Buffer buffer, Page page, BlockNumber block, ItemId lp,
+ Bitmapset *hot_attrs, Bitmapset *sum_attrs, Bitmapset *pk_attrs,
+ Bitmapset *rid_attrs, const Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -4352,8 +4352,9 @@ HeapDetermineColumnsInfo(Relation relation,
* This routine may be used to update a tuple when concurrent updates of the
* target tuple are not expected (for example, because we have a lock on the
* relation associated with the tuple). Any failure is reported via ereport().
+ * Returns the set of modified indexed attributes.
*/
-void
+Bitmapset *
simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
@@ -4482,7 +4483,7 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
elog(ERROR, "tuple concurrently deleted");
- return;
+ return NULL;
}
/*
@@ -4515,7 +4516,6 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
bms_free(sum_attrs);
bms_free(pk_attrs);
bms_free(rid_attrs);
- bms_free(mix_attrs);
bms_free(idx_attrs);
switch (result)
@@ -4541,6 +4541,8 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
elog(ERROR, "unrecognized heap_update status: %u", result);
break;
}
+
+ return mix_attrs;
}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 1cf9a18775d..7527809ec08 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -315,9 +315,12 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
- CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd,
- LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
+ CommandId cid, Snapshot snapshot,
+ Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd,
+ LockTupleMode *lockmode,
+ const Bitmapset *mix_attrs,
+ TU_UpdateIndexes *update_indexes)
{
bool rep_id_key_required = false;
bool shouldFree = true;
@@ -332,7 +335,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
*sum_attrs,
*pk_attrs,
*rid_attrs,
- *mix_attrs,
*idx_attrs;
TM_Result result;
@@ -405,25 +407,66 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
Assert(ItemIdIsNormal(lp));
- /*
- * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
- * then pass that on to heap_update.
- */
oldtup.t_tableOid = RelationGetRelid(relation);
oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
oldtup.t_len = ItemIdGetLength(lp);
oldtup.t_self = *otid;
- mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
- &oldtup, tuple, &rep_id_key_required);
-
/*
- * We'll need to WAL log the replica identity attributes if either they
- * overlap with the modified indexed attributes or, as we've checked for
- * just now in HeapDetermineColumnsInfo, they were unmodified external
- * indexed attributes.
+ * We'll need to include the replica identity key when either the identity
+ * key attributes overlap with the modified index attributes or when the
+ * replica identity attributes are stored externally. This is required
+ * because for such attributes the flattened value won't be WAL logged as
+ * part of the new tuple so we must determine if we need to extract and
+ * include them as part of the old_key_tuple (see ExtractReplicaIdentity).
*/
- rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+ rep_id_key_required = bms_overlap(mix_attrs, rid_attrs);
+ if (!rep_id_key_required)
+ {
+ Bitmapset *attrs;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ int attidx = -1;
+
+ /*
+ * We don't own idx_attrs so we'll copy it and remove the modified set
+ * to reduce the attributes we need to test in the while loop and
+ * avoid a two branches in the loop.
+ */
+ attrs = bms_difference(idx_attrs, mix_attrs);
+ attrs = bms_int_members(attrs, rid_attrs);
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /*
+ * attidx is zero-based, attrnum is the normal attribute number
+ */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value;
+ bool isnull;
+
+ /*
+ * System attributes are not added into interesting_attrs in
+ * relcache
+ */
+ Assert(attrnum > 0);
+
+ value = heap_getattr(&oldtup, attrnum, tupdesc, &isnull);
+
+ /* No need to check attributes that can't be stored externally */
+ if (isnull ||
+ TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
+ continue;
+
+ /* Check if the old tuple's attribute is stored externally */
+ if (VARATT_IS_EXTERNAL((struct varlena *) DatumGetPointer(value)))
+ {
+ rep_id_key_required = true;
+ break;
+ }
+ }
+
+ bms_free(attrs);
+ }
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
@@ -437,7 +480,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
bms_free(sum_attrs);
bms_free(pk_attrs);
bms_free(rid_attrs);
- bms_free(mix_attrs);
bms_free(idx_attrs);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index fdff960c130..e435f0d5db4 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -155,6 +155,7 @@ bthandler(PG_FUNCTION_ARGS)
amroutine->amproperty = btproperty;
amroutine->ambuildphasename = btbuildphasename;
amroutine->amvalidate = btvalidate;
+ amroutine->amcomparedatums = NULL;
amroutine->amadjustmembers = btadjustmembers;
amroutine->ambeginscan = btbeginscan;
amroutine->amrescan = btrescan;
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 1e099febdc8..15f0dd7aa28 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -367,6 +367,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -377,7 +378,9 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
GetCurrentCommandId(true),
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, &lockmode,
+ mix_attrs,
+ update_indexes);
switch (result)
{
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index fc8638c1b61..329c110d0bf 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -961,10 +961,18 @@ index_register(Oid heap,
newind->il_info->ii_Expressions =
copyObject(indexInfo->ii_Expressions);
newind->il_info->ii_ExpressionsState = NIL;
+ /* expression attrs will likely be null, but may as well copy it */
+ newind->il_info->ii_ExpressionsAttrs =
+ copyObject(indexInfo->ii_ExpressionsAttrs);
/* predicate will likely be null, but may as well copy it */
newind->il_info->ii_Predicate =
copyObject(indexInfo->ii_Predicate);
newind->il_info->ii_PredicateState = NULL;
+ /* predicate attrs will likely be null, but may as well copy it */
+ newind->il_info->ii_PredicateAttrs =
+ copyObject(indexInfo->ii_PredicateAttrs);
+ newind->il_info->ii_CheckedPredicate = false;
+ newind->il_info->ii_PredicateSatisfied = false;
/* no exclusion constraints at bootstrap time, so no need to copy */
Assert(indexInfo->ii_ExclusionOps == NULL);
Assert(indexInfo->ii_ExclusionProcs == NULL);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 5d9db167e59..e88db7e919b 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -27,6 +27,7 @@
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/relscan.h"
+#include "access/sysattr.h"
#include "access/tableam.h"
#include "access/toast_compression.h"
#include "access/transam.h"
@@ -58,6 +59,7 @@
#include "commands/trigger.h"
#include "executor/executor.h"
#include "miscadmin.h"
+#include "nodes/execnodes.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/optimizer.h"
@@ -2414,6 +2416,58 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* ----------------------------------------------------------------
*/
+/* ----------------
+ * BuildUpdateIndexInfo
+ *
+ * For expression indexes updates may not change the indexed value allowing
+ * for a HOT update. Add information to the IndexInfo to allow for checking
+ * if the indexed value has changed.
+ *
+ * Do this processing here rather than in BuildIndexInfo() to not incur the
+ * overhead in the common non-expression cases.
+ * ----------------
+ */
+void
+BuildUpdateIndexInfo(ResultRelInfo *resultRelInfo)
+{
+ for (int j = 0; j < resultRelInfo->ri_NumIndices; j++)
+ {
+ int i;
+ int indnatts;
+ Bitmapset *attrs = NULL;
+ IndexInfo *ii = resultRelInfo->ri_IndexRelationInfo[j];
+
+ indnatts = ii->ii_NumIndexAttrs;
+
+ /* Collect key attributes used by the index, key and including */
+ for (i = 0; i < indnatts; i++)
+ {
+ AttrNumber attnum = ii->ii_IndexAttrNumbers[i];
+
+ if (attnum != 0)
+ attrs = bms_add_member(attrs, attnum - FirstLowInvalidHeapAttributeNumber);
+ }
+
+ /* Collect attributes used in the expression */
+ if (ii->ii_Expressions)
+ pull_varattnos((Node *) ii->ii_Expressions,
+ resultRelInfo->ri_RangeTableIndex,
+ &ii->ii_ExpressionsAttrs);
+
+ /* Collect attributes used in the predicate */
+ if (ii->ii_Predicate)
+ pull_varattnos((Node *) ii->ii_Predicate,
+ resultRelInfo->ri_RangeTableIndex,
+ &ii->ii_PredicateAttrs);
+
+ /* Combine key, including, and expression attributes, but not predicate */
+ ii->ii_IndexedAttrs = bms_union(attrs, ii->ii_ExpressionsAttrs);
+
+ /* All indexes should index *something*! */
+ Assert(!bms_is_empty(ii->ii_IndexedAttrs));
+ }
+}
+
/* ----------------
* BuildIndexInfo
* Construct an IndexInfo record for an open index
diff --git a/src/backend/catalog/indexing.c b/src/backend/catalog/indexing.c
index 004c5121000..a361c215490 100644
--- a/src/backend/catalog/indexing.c
+++ b/src/backend/catalog/indexing.c
@@ -102,7 +102,7 @@ CatalogIndexInsert(CatalogIndexState indstate, HeapTuple heapTuple,
* Get information from the state structure. Fall out if nothing to do.
*/
numIndexes = indstate->ri_NumIndices;
- if (numIndexes == 0)
+ if (numIndexes == 0 || updateIndexes == TU_None)
return;
relationDescs = indstate->ri_IndexRelationDescs;
indexInfoArray = indstate->ri_IndexRelationInfo;
@@ -314,15 +314,18 @@ CatalogTupleUpdate(Relation heapRel, const ItemPointerData *otid, HeapTuple tup)
{
CatalogIndexState indstate;
TU_UpdateIndexes updateIndexes = TU_All;
+ Bitmapset *updatedAttrs;
CatalogTupleCheckConstraints(heapRel, tup);
indstate = CatalogOpenIndexes(heapRel);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
-
+ updatedAttrs = simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = updatedAttrs;
CatalogIndexInsert(indstate, tup, updateIndexes);
+
CatalogCloseIndexes(indstate);
+ bms_free(updatedAttrs);
}
/*
@@ -338,12 +341,15 @@ CatalogTupleUpdateWithInfo(Relation heapRel, const ItemPointerData *otid, HeapTu
CatalogIndexState indstate)
{
TU_UpdateIndexes updateIndexes = TU_All;
+ Bitmapset *updatedAttrs;
CatalogTupleCheckConstraints(heapRel, tup);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
-
+ updatedAttrs = simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = updatedAttrs;
CatalogIndexInsert(indstate, tup, updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = NULL;
+ bms_free(updatedAttrs);
}
/*
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 874a8fc89ad..9d7cb4438d5 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -292,8 +292,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
indexInfo->ii_IndexAttrNumbers[1] = 2;
indexInfo->ii_Expressions = NIL;
indexInfo->ii_ExpressionsState = NIL;
+ indexInfo->ii_ExpressionsAttrs = NULL;
indexInfo->ii_Predicate = NIL;
indexInfo->ii_PredicateState = NULL;
+ indexInfo->ii_PredicateAttrs = NULL;
+ indexInfo->ii_CheckedPredicate = false;
+ indexInfo->ii_PredicateSatisfied = false;
indexInfo->ii_ExclusionOps = NULL;
indexInfo->ii_ExclusionProcs = NULL;
indexInfo->ii_ExclusionStrats = NULL;
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index dd323c9b9fd..05ae7eb9f65 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -109,11 +109,15 @@
#include "access/genam.h"
#include "access/relscan.h"
#include "access/tableam.h"
+#include "access/sysattr.h"
#include "access/xact.h"
#include "catalog/index.h"
#include "executor/executor.h"
+#include "nodes/bitmapset.h"
+#include "nodes/execnodes.h"
#include "nodes/nodeFuncs.h"
#include "storage/lmgr.h"
+#include "utils/datum.h"
#include "utils/injection_point.h"
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
@@ -324,8 +328,8 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
Relation heapRelation;
IndexInfo **indexInfoArray;
ExprContext *econtext;
- Datum values[INDEX_MAX_KEYS];
- bool isnull[INDEX_MAX_KEYS];
+ Datum loc_values[INDEX_MAX_KEYS];
+ bool loc_isnull[INDEX_MAX_KEYS];
Assert(ItemPointerIsValid(tupleid));
@@ -349,13 +353,13 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
/* Arrange for econtext's scan tuple to be the tuple under test */
econtext->ecxt_scantuple = slot;
- /*
- * for each index, form and insert the index tuple
- */
+ /* Insert into each index that needs updating */
for (i = 0; i < numIndices; i++)
{
Relation indexRelation = relationDescs[i];
IndexInfo *indexInfo;
+ Datum *values;
+ bool *isnull;
bool applyNoDupErr;
IndexUniqueCheck checkUnique;
bool indexUnchanged;
@@ -372,7 +376,7 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
/*
* Skip processing of non-summarizing indexes if we only update
- * summarizing indexes
+ * summarizing indexes or if this index is unchanged.
*/
if (onlySummarizing && !indexInfo->ii_Summarizing)
continue;
@@ -393,8 +397,15 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
indexInfo->ii_PredicateState = predicate;
}
+ /* Check the index predicate if we haven't done so earlier on */
+ if (!indexInfo->ii_CheckedPredicate)
+ {
+ indexInfo->ii_PredicateSatisfied = ExecQual(predicate, econtext);
+ indexInfo->ii_CheckedPredicate = true;
+ }
+
/* Skip this index-update if the predicate isn't satisfied */
- if (!ExecQual(predicate, econtext))
+ if (!indexInfo->ii_PredicateSatisfied)
continue;
}
@@ -402,11 +413,10 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* FormIndexDatum fills in its values and isnull parameters with the
* appropriate values for the column(s) of the index.
*/
- FormIndexDatum(indexInfo,
- slot,
- estate,
- values,
- isnull);
+ FormIndexDatum(indexInfo, slot, estate, loc_values, loc_isnull);
+
+ values = loc_values;
+ isnull = loc_isnull;
/* Check whether to apply noDupErr to this index */
applyNoDupErr = noDupErr &&
@@ -613,7 +623,12 @@ ExecCheckIndexConstraints(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
checkedIndex = true;
/* Check for partial index */
- if (indexInfo->ii_Predicate != NIL)
+ if (indexInfo->ii_CheckedPredicate && !indexInfo->ii_PredicateSatisfied)
+ {
+ /* We've already checked and the predicate wasn't satisfied. */
+ continue;
+ }
+ else if (indexInfo->ii_Predicate != NIL)
{
ExprState *predicate;
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 27c9eec697b..6b7b6bc8019 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1282,6 +1282,7 @@ InitResultRelInfo(ResultRelInfo *resultRelInfo,
/* The following fields are set later if needed */
resultRelInfo->ri_RowIdAttNo = 0;
resultRelInfo->ri_extraUpdatedCols = NULL;
+ resultRelInfo->ri_ChangedIndexedCols = NULL;
resultRelInfo->ri_projectNew = NULL;
resultRelInfo->ri_newTupleSlot = NULL;
resultRelInfo->ri_oldTupleSlot = NULL;
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index def32774c90..2709e2db0f2 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -32,6 +32,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
#include "utils/typcache.h"
@@ -936,7 +937,13 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ /*
+ * We're not going to call ExecCheckIndexedAttrsForChanges here
+ * because we've already identified the changes earlier on thanks to
+ * slot_modify_data.
+ */
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
+ resultRelInfo->ri_ChangedIndexedCols,
&update_indexes);
conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index e44f1223886..a06cf34ca70 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -17,6 +17,7 @@
* ExecModifyTable - retrieve the next tuple from the node
* ExecEndModifyTable - shut down the ModifyTable node
* ExecReScanModifyTable - rescan the ModifyTable node
+ * ExecCheckIndexedAttrsForChanges - find set of updated indexed columns
*
* NOTES
* The ModifyTable node receives input from its outerPlan, which is
@@ -53,12 +54,18 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/attnum.h"
+#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/tupconvert.h"
+#include "access/tupdesc.h"
#include "access/xact.h"
+#include "catalog/index.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
#include "executor/nodeModifyTable.h"
+#include "executor/tuptable.h"
#include "foreign/fdwapi.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
@@ -68,8 +75,11 @@
#include "storage/lmgr.h"
#include "utils/builtins.h"
#include "utils/datum.h"
+#include "utils/float.h"
#include "utils/injection_point.h"
+#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
@@ -176,6 +186,220 @@ static TupleTableSlot *ExecMergeNotMatched(ModifyTableContext *context,
ResultRelInfo *resultRelInfo,
bool canSetTag);
+/*
+ * ExecCheckIndexedAttrsForChanges
+ *
+ * Determine which indexes need updating by finding the set of modified indexed
+ * attributes.
+ *
+ * For which implement the amcomparedatums() index AM API we'll need to form
+ * index datum and compare each attribute to see if anything actually changed.
+ *
+ * The goal is for the executor to know, ahead of calling into the table AM to
+ * process the update and before calling into the index AM for inserting new
+ * index tuples, which attributes in the new TupleTableSlot, if any, truely
+ * necessitate a new index tuple.
+ *
+ * Returns a Bitmapset of attributes that intersects with indexes which require
+ * a new index tuple.
+ */
+Bitmapset *
+ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts)
+{
+ Relation relation = relinfo->ri_RelationDesc;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ Bitmapset *mix_attrs = NULL; /* modified indexed attributes */
+
+ /* If no indexes, we're done */
+ if (relinfo->ri_NumIndices == 0)
+ return NULL;
+
+ /* Find the indexes that reference this attribute */
+ for (int i = 0; i < relinfo->ri_NumIndices; i++)
+ {
+ Relation index = relinfo->ri_IndexRelationDescs[i];
+ IndexAmRoutine *amroutine = index->rd_indam;
+ IndexInfo *indexInfo = relinfo->ri_IndexRelationInfo[i];
+ Bitmapset *m_attrs = NULL; /* (possibly) modified indexed attributes */
+ Bitmapset *u_attrs = NULL; /* unmodified indexed attributes */
+ bool has_am_compare = (amroutine->amcomparedatums != NULL);
+ bool supports_ios = (amroutine->amcanreturn != NULL);
+ bool is_partial = (indexInfo->ii_Predicate != NIL);
+ ExprContext *econtext = GetPerTupleExprContext(estate);
+ int num_datums = supports_ios ?
+ indexInfo->ii_NumIndexAttrs : indexInfo->ii_NumIndexKeyAttrs;
+
+ /* If we've reviewed all the attributes on this index, move on */
+ if (bms_is_subset(indexInfo->ii_IndexedAttrs, mix_attrs))
+ continue;
+
+ /* Add partial index attributes */
+ if (is_partial)
+ m_attrs = bms_add_members(m_attrs, indexInfo->ii_PredicateAttrs);
+
+ /* Compare the index datums for equality */
+ for (int j = 0; j < num_datums; j++)
+ {
+ AttrNumber rel_attrnum = indexInfo->ii_IndexAttrNumbers[j];
+ int rel_attridx = rel_attrnum - FirstLowInvalidHeapAttributeNumber;
+ int nth_expr = 0;
+ int16 typlen;
+ bool typbyval;
+ Datum old_value;
+ Datum new_value;
+ bool old_null;
+ bool new_null;
+ bool values_equal = false;
+
+ /* System attributes */
+ if (rel_attrnum < 0)
+ {
+ /* Extract system values from both slots for this attribute */
+ old_value = slot_getsysattr(old_tts, rel_attrnum, &old_null);
+ new_value = slot_getsysattr(new_tts, rel_attrnum, &new_null);
+
+ /* The only allowed system columns are OIDs, so do this */
+ values_equal = (DatumGetObjectId(old_value) == DatumGetObjectId(new_value));
+ goto equality_determined;
+ }
+
+ /*
+ * This is an expression attribute, but in an effort to avoid the
+ * expense of IndexFormDatum we're now faced with testing for
+ * equality so we'll have to exec the expressions and test for
+ * binary equality of the results.
+ */
+ else if (rel_attrnum == 0)
+ {
+ TupleTableSlot *save_scantuple = econtext->ecxt_scantuple;
+ Oid expr_type_oid;
+ Expr *expr = (Expr *) list_nth(indexInfo->ii_Expressions, nth_expr);
+ ExprState *state;
+
+ if (indexInfo->ii_ExpressionsState == NIL)
+ {
+ /* First time through, set up expression evaluation state */
+ indexInfo->ii_ExpressionsState =
+ ExecPrepareExprList(indexInfo->ii_Expressions, estate);
+ }
+
+ state = (ExprState *) list_nth(indexInfo->ii_ExpressionsState, nth_expr);
+
+ econtext->ecxt_scantuple = old_tts;
+ old_value = ExecEvalExprSwitchContext(state,
+ GetPerTupleExprContext(estate),
+ &old_null);
+
+ econtext->ecxt_scantuple = new_tts;
+ new_value = ExecEvalExprSwitchContext(state,
+ GetPerTupleExprContext(estate),
+ &new_null);
+
+ econtext->ecxt_scantuple = save_scantuple;
+
+ /*
+ * NOTE: test for NULL cases here to potentially avoid looking
+ * up the type information. It's a tad redundant, but worth
+ * it.
+ */
+
+ /* A change to/from NULL, so not equal */
+ if (old_null != new_null)
+ {
+ values_equal = false;
+ goto equality_determined;
+ }
+
+ /* Both NULL, no change record as unmodified */
+ if (old_null)
+ {
+ values_equal = true;
+ goto equality_determined;
+ }
+
+ /* Get type OID from the expression */
+ expr_type_oid = exprType((Node *) expr);
+
+ /* Get type information from the OID */
+ get_typlenbyval(expr_type_oid, &typlen, &typbyval);
+ }
+ /* Not a system or expression attribute */
+ else
+ {
+ CompactAttribute *att = TupleDescCompactAttr(tupdesc, rel_attrnum - 1);
+
+ /* Extract values from both slots for this attribute */
+ old_value = slot_getattr(old_tts, rel_attrnum, &old_null);
+ new_value = slot_getattr(new_tts, rel_attrnum, &new_null);
+
+ typlen = att->attlen;
+ typbyval = att->attbyval;
+ }
+
+ /* A change to/from NULL, so not equal */
+ if (old_null != new_null)
+ {
+ values_equal = false;
+ goto equality_determined;
+ }
+
+ /* Both NULL, no change record as unmodified */
+ if (old_null)
+ {
+ values_equal = true;
+ goto equality_determined;
+ }
+
+ if (has_am_compare)
+ {
+ /*
+ * NOTE: For AM comparison, pass the 1-based index attribute
+ * number. The AM's compare function expects the same
+ * numbering as used internally by the AM.
+ */
+ values_equal = amroutine->amcomparedatums(index, j + 1,
+ old_value, old_null,
+ new_value, new_null);
+ }
+ else
+ {
+ values_equal = datumIsEqual(old_value, new_value, typbyval, typlen);
+ }
+
+ equality_determined:;
+ if (!values_equal)
+ if (rel_attrnum == 0)
+ {
+ Expr *expr = (Expr *) list_nth(indexInfo->ii_Expressions, nth_expr);
+
+ pull_varattnos((Node *) expr, relinfo->ri_RangeTableIndex, &m_attrs);
+ }
+ else
+ m_attrs = bms_add_member(m_attrs, rel_attridx);
+ else
+ u_attrs = bms_add_member(u_attrs, rel_attridx);
+
+ if (rel_attrnum == 0)
+ nth_expr++;
+ }
+
+ /*
+ * Here we know all the attributes that might be modified and all
+ * those we know haven't been across all indexes. Take the difference
+ * and add it to the modified indexed attributes set.
+ */
+ m_attrs = bms_del_members(m_attrs, u_attrs);
+ mix_attrs = bms_add_members(mix_attrs, m_attrs);
+
+ bms_free(m_attrs);
+ bms_free(u_attrs);
+ }
+
+ return mix_attrs;
+}
/*
* Verify that the tuples to be produced by INSERT match the
@@ -2170,14 +2394,17 @@ ExecUpdatePrepareSlot(ResultRelInfo *resultRelInfo,
*/
static TM_Result
ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
- ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *slot,
- bool canSetTag, UpdateContext *updateCxt)
+ ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *oldSlot,
+ TupleTableSlot *slot, bool canSetTag, UpdateContext *updateCxt)
{
EState *estate = context->estate;
Relation resultRelationDesc = resultRelInfo->ri_RelationDesc;
bool partition_constraint_failed;
TM_Result result;
+ /* The set of modified indexed attributes that trigger new index entries */
+ Bitmapset *mix_attrs = NULL;
+
updateCxt->crossPartUpdate = false;
/*
@@ -2294,9 +2521,38 @@ lreplace:
ExecConstraints(resultRelInfo, slot, estate);
/*
- * replace the heap tuple
+ * Identify which, if any, indexed attributes were modified here so that
+ * we might reuse it in a few places.
+ */
+ bms_free(resultRelInfo->ri_ChangedIndexedCols);
+ resultRelInfo->ri_ChangedIndexedCols = NULL;
+
+ /*
+ * During updates we'll need a bit more information in IndexInfo but we've
+ * delayed adding it until here. We check to ensure that there are
+ * indexes, that something has changed that is indexed, and that the first
+ * index doesn't yet have ii_IndexedAttrs set as a way to ensure we only
+ * build this when needed and only once. We don't build this in
+ * ExecOpenIndicies() as it is unnecessary overhead when not performing an
+ * update.
+ */
+ if (resultRelInfo->ri_NumIndices > 0 &&
+ bms_is_empty(resultRelInfo->ri_IndexRelationInfo[0]->ii_IndexedAttrs))
+ BuildUpdateIndexInfo(resultRelInfo);
+
+ /*
+ * Next up we need to find out the set of indexed attributes that have
+ * changed in value and should trigger a new index tuple. We could start
+ * with the set of updated columns via ExecGetUpdatedCols(), but if we do
+ * we will overlook attributes directly modified by heap_modify_tuple()
+ * which are not known to ExecGetUpdatedCols().
+ */
+ mix_attrs = ExecCheckIndexedAttrsForChanges(resultRelInfo, estate, oldSlot, slot);
+
+ /*
+ * Call into the table AM to update the heap tuple.
*
- * Note: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
+ * NOTE: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
* the row to be updated is visible to that snapshot, and throw a
* can't-serialize error if not. This is a special-case behavior needed
* for referential integrity updates in transaction-snapshot mode
@@ -2308,8 +2564,12 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
+ mix_attrs,
&updateCxt->updateIndexes);
+ Assert(bms_is_empty(resultRelInfo->ri_ChangedIndexedCols));
+ resultRelInfo->ri_ChangedIndexedCols = mix_attrs;
+
return result;
}
@@ -2327,7 +2587,7 @@ ExecUpdateEpilogue(ModifyTableContext *context, UpdateContext *updateCxt,
ModifyTableState *mtstate = context->mtstate;
List *recheckIndexes = NIL;
- /* insert index entries for tuple if necessary */
+ /* Insert index entries for tuple if necessary */
if (resultRelInfo->ri_NumIndices > 0 && (updateCxt->updateIndexes != TU_None))
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
slot, context->estate,
@@ -2526,8 +2786,9 @@ ExecUpdate(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
*/
redo_act:
lockedtid = *tupleid;
- result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, slot,
- canSetTag, &updateCxt);
+
+ result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, oldSlot,
+ slot, canSetTag, &updateCxt);
/*
* If ExecUpdateAct reports that a cross-partition update was done,
@@ -3224,8 +3485,8 @@ lmerge_matched:
Assert(oldtuple == NULL);
result = ExecUpdateAct(context, resultRelInfo, tupleid,
- NULL, newslot, canSetTag,
- &updateCxt);
+ NULL, resultRelInfo->ri_oldTupleSlot,
+ newslot, canSetTag, &updateCxt);
/*
* As in ExecUpdate(), if ExecUpdateAct() reports that a
@@ -3250,6 +3511,7 @@ lmerge_matched:
tupleid, NULL, newslot);
mtstate->mt_merge_updated += 1;
}
+
break;
case CMD_DELETE:
@@ -4356,7 +4618,7 @@ ExecModifyTable(PlanState *pstate)
* For UPDATE/DELETE/MERGE, fetch the row identity info for the tuple
* to be updated/deleted/merged. For a heap relation, that's a TID;
* otherwise we may have a wholerow junk attr that carries the old
- * tuple in toto. Keep this in step with the part of
+ * tuple in total. Keep this in step with the part of
* ExecInitModifyTable that sets up ri_RowIdAttNo.
*/
if (operation == CMD_UPDATE || operation == CMD_DELETE ||
@@ -4532,6 +4794,7 @@ ExecModifyTable(PlanState *pstate)
/* Now apply the update. */
slot = ExecUpdate(&context, resultRelInfo, tupleid, oldtuple,
oldSlot, slot, node->canSetTag);
+
if (tuplock)
UnlockTuple(resultRelInfo->ri_RelationDesc, tupleid,
InplaceUpdateTupleLock);
diff --git a/src/backend/nodes/bitmapset.c b/src/backend/nodes/bitmapset.c
index 7b1e9d94103..c522971a37c 100644
--- a/src/backend/nodes/bitmapset.c
+++ b/src/backend/nodes/bitmapset.c
@@ -238,6 +238,10 @@ bms_make_singleton(int x)
void
bms_free(Bitmapset *a)
{
+#if USE_ASSERT_CHECKING
+ Assert(bms_is_valid_set(a));
+#endif
+
if (a)
pfree(a);
}
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index e2d9e9be41a..1f1364f9df9 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -857,10 +857,14 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
/* expressions */
n->ii_Expressions = expressions;
n->ii_ExpressionsState = NIL;
+ n->ii_ExpressionsAttrs = NULL;
/* predicates */
n->ii_Predicate = predicates;
n->ii_PredicateState = NULL;
+ n->ii_PredicateAttrs = NULL;
+ n->ii_CheckedPredicate = false;
+ n->ii_PredicateSatisfied = false;
/* exclusion constraints */
n->ii_ExclusionOps = NULL;
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 93970c6af29..b72bf2055f6 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -275,7 +275,6 @@
#include "replication/logicalrelation.h"
#include "replication/logicalworker.h"
#include "replication/origin.h"
-#include "replication/slot.h"
#include "replication/walreceiver.h"
#include "replication/worker_internal.h"
#include "rewrite/rewriteHandler.h"
@@ -285,12 +284,14 @@
#include "storage/procarray.h"
#include "tcop/tcopprot.h"
#include "utils/acl.h"
+#include "utils/datum.h"
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/pg_lsn.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/rls.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
@@ -1110,15 +1111,18 @@ slot_store_data(TupleTableSlot *slot, LogicalRepRelMapEntry *rel,
* "slot" is filled with a copy of the tuple in "srcslot", replacing
* columns provided in "tupleData" and leaving others as-is.
*
+ * Returns a bitmap of the modified columns.
+ *
* Caution: unreplaced pass-by-ref columns in "slot" will point into the
* storage for "srcslot". This is OK for current usage, but someday we may
* need to materialize "slot" at the end to make it independent of "srcslot".
*/
-static void
+static Bitmapset *
slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
LogicalRepRelMapEntry *rel,
LogicalRepTupleData *tupleData)
{
+ Bitmapset *modified = NULL;
int natts = slot->tts_tupleDescriptor->natts;
int i;
@@ -1195,6 +1199,27 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
slot->tts_isnull[i] = true;
}
+ /*
+ * Determine if the replicated value changed the local value by
+ * comparing slots. This is a subset of
+ * ExecCheckIndexedAttrsForChanges.
+ */
+ if (srcslot->tts_isnull[i] != slot->tts_isnull[i])
+ {
+ /* One is NULL, the other is not so the value changed */
+ modified = bms_add_member(modified, i + 1 - FirstLowInvalidHeapAttributeNumber);
+ }
+ else if (!srcslot->tts_isnull[i])
+ {
+ /* Both are not NULL, compare their values */
+
+ if (!datumIsEqual(srcslot->tts_values[i],
+ slot->tts_values[i],
+ att->attbyval,
+ att->attlen))
+ modified = bms_add_member(modified, i + 1 - FirstLowInvalidHeapAttributeNumber);
+ }
+
/* Reset attnum for error callback */
apply_error_callback_arg.remote_attnum = -1;
}
@@ -1202,6 +1227,8 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
/* And finally, declare that "slot" contains a valid virtual tuple */
ExecStoreVirtualTuple(slot);
+
+ return modified;
}
/*
@@ -2918,6 +2945,7 @@ apply_handle_update_internal(ApplyExecutionData *edata,
ConflictTupleInfo conflicttuple = {0};
bool found;
MemoryContext oldctx;
+ Bitmapset *indexed = NULL;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
ExecOpenIndices(relinfo, false);
@@ -2934,6 +2962,8 @@ apply_handle_update_internal(ApplyExecutionData *edata,
*/
if (found)
{
+ Bitmapset *modified = NULL;
+
/*
* Report the conflict if the tuple was modified by a different
* origin.
@@ -2957,15 +2987,29 @@ apply_handle_update_internal(ApplyExecutionData *edata,
/* Process and store remote tuple in the slot */
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot, localslot, relmapentry, newtup);
+ modified = slot_modify_data(remoteslot, localslot, relmapentry, newtup);
MemoryContextSwitchTo(oldctx);
+ /*
+ * Normally we'd call ExecCheckIndexedAttrForChanges but here we have
+ * the record of changed columns in the replication state, so let's
+ * use that instead.
+ */
+ indexed = RelationGetIndexAttrBitmap(relinfo->ri_RelationDesc,
+ INDEX_ATTR_BITMAP_INDEXED);
+
+ bms_free(relinfo->ri_ChangedIndexedCols);
+ relinfo->ri_ChangedIndexedCols = bms_int_members(modified, indexed);
+ bms_free(indexed);
+
EvalPlanQualSetSlot(&epqstate, remoteslot);
InitConflictIndexes(relinfo);
- /* Do the actual update. */
+ /* First check privileges */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
+
+ /* Then do the actual update. */
ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
remoteslot);
}
@@ -3455,6 +3499,8 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
bool found;
EPQState epqstate;
ConflictTupleInfo conflicttuple = {0};
+ Bitmapset *modified = NULL;
+ Bitmapset *indexed;
/* Get the matching local tuple from the partition. */
found = FindReplTupleInLocalRel(edata, partrel,
@@ -3523,8 +3569,8 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
* remoteslot_part.
*/
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot_part, localslot, part_entry,
- newtup);
+ modified = slot_modify_data(remoteslot_part, localslot, part_entry,
+ newtup);
MemoryContextSwitchTo(oldctx);
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
@@ -3549,6 +3595,18 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
EvalPlanQualSetSlot(&epqstate, remoteslot_part);
TargetPrivilegesCheck(partrelinfo->ri_RelationDesc,
ACL_UPDATE);
+
+ /*
+ * Normally we'd call ExecCheckIndexedAttrForChanges but
+ * here we have the record of changed columns in the
+ * replication state, so let's use that instead.
+ */
+ indexed = RelationGetIndexAttrBitmap(partrelinfo->ri_RelationDesc,
+ INDEX_ATTR_BITMAP_INDEXED);
+ bms_free(partrelinfo->ri_ChangedIndexedCols);
+ partrelinfo->ri_ChangedIndexedCols = bms_int_members(modified, indexed);
+ bms_free(indexed);
+
ExecSimpleRelationUpdate(partrelinfo, estate, &epqstate,
localslot, remoteslot_part);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 915d0bc9084..32825596be1 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -2482,6 +2482,7 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
bms_free(relation->rd_idattr);
bms_free(relation->rd_hotblockingattr);
bms_free(relation->rd_summarizedattr);
+ bms_free(relation->rd_indexedattr);
if (relation->rd_pubdesc)
pfree(relation->rd_pubdesc);
if (relation->rd_options)
@@ -5283,6 +5284,7 @@ RelationGetIndexPredicate(Relation relation)
* index (empty if FULL)
* INDEX_ATTR_BITMAP_HOT_BLOCKING Columns that block updates from being HOT
* INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
+ * INDEX_ATTR_BITMAP_INDEXED Columns referenced by indexes
*
* Attribute numbers are offset by FirstLowInvalidHeapAttributeNumber so that
* we can include system attributes (e.g., OID) in the bitmap representation.
@@ -5307,6 +5309,7 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
Bitmapset *idindexattrs; /* columns in the replica identity */
Bitmapset *hotblockingattrs; /* columns with HOT blocking indexes */
Bitmapset *summarizedattrs; /* columns with summarizing indexes */
+ Bitmapset *indexedattrs; /* columns referenced by indexes */
List *indexoidlist;
List *newindexoidlist;
Oid relpkindex;
@@ -5329,6 +5332,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
return bms_copy(relation->rd_hotblockingattr);
case INDEX_ATTR_BITMAP_SUMMARIZED:
return bms_copy(relation->rd_summarizedattr);
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return bms_copy(relation->rd_indexedattr);
default:
elog(ERROR, "unknown attrKind %u", attrKind);
}
@@ -5373,6 +5378,7 @@ restart:
idindexattrs = NULL;
hotblockingattrs = NULL;
summarizedattrs = NULL;
+ indexedattrs = NULL;
foreach(l, indexoidlist)
{
Oid indexOid = lfirst_oid(l);
@@ -5505,10 +5511,14 @@ restart:
bms_free(idindexattrs);
bms_free(hotblockingattrs);
bms_free(summarizedattrs);
+ bms_free(indexedattrs);
goto restart;
}
+ /* Combine all index attributes */
+ indexedattrs = bms_union(hotblockingattrs, summarizedattrs);
+
/* Don't leak the old values of these bitmaps, if any */
relation->rd_attrsvalid = false;
bms_free(relation->rd_keyattr);
@@ -5521,6 +5531,8 @@ restart:
relation->rd_hotblockingattr = NULL;
bms_free(relation->rd_summarizedattr);
relation->rd_summarizedattr = NULL;
+ bms_free(relation->rd_indexedattr);
+ relation->rd_indexedattr = NULL;
/*
* Now save copies of the bitmaps in the relcache entry. We intentionally
@@ -5535,6 +5547,7 @@ restart:
relation->rd_idattr = bms_copy(idindexattrs);
relation->rd_hotblockingattr = bms_copy(hotblockingattrs);
relation->rd_summarizedattr = bms_copy(summarizedattrs);
+ relation->rd_indexedattr = bms_copy(indexedattrs);
relation->rd_attrsvalid = true;
MemoryContextSwitchTo(oldcxt);
@@ -5551,6 +5564,8 @@ restart:
return hotblockingattrs;
case INDEX_ATTR_BITMAP_SUMMARIZED:
return summarizedattrs;
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return indexedattrs;
default:
elog(ERROR, "unknown attrKind %u", attrKind);
return NULL;
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index 63dd41c1f21..9bdf73eda59 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -211,6 +211,33 @@ typedef void (*ammarkpos_function) (IndexScanDesc scan);
/* restore marked scan position */
typedef void (*amrestrpos_function) (IndexScanDesc scan);
+/*
+ * amcomparedatums - Compare datums to determine if index update is needed
+ *
+ * This function compares old_datum and new_datum to determine if they would
+ * produce different index entries. For extraction-based indexes (GIN, RUM),
+ * this should:
+ * 1. Extract keys from old_datum using the opclass's extractValue function
+ * 2. Extract keys from new_datum using the opclass's extractValue function
+ * 3. Compare the two sets of keys using appropriate equality operators
+ * 4. Return true if the sets are equal (no index update needed)
+ *
+ * The comparison should account for:
+ * - Different numbers of extracted keys
+ * - NULL values
+ * - Type-specific equality (not just binary equality)
+ * - Opclass parameters (e.g., path in bson_rum_single_path_ops)
+ *
+ * For the DocumentDB example with path='a', this would extract values at
+ * path 'a' from both old and new BSON documents and compare them using
+ * BSON's equality operator.
+ */
+/* identify if updated datums would produce one or more index entries */
+typedef bool (*amcomparedatums_function) (Relation indexRelation,
+ int attno,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* Callback function signatures - for parallel index scans.
*/
@@ -313,6 +340,7 @@ typedef struct IndexAmRoutine
amendscan_function amendscan;
ammarkpos_function ammarkpos; /* can be NULL */
amrestrpos_function amrestrpos; /* can be NULL */
+ amcomparedatums_function amcomparedatums; /* can be NULL */
/* interface functions to support parallel index scans */
amestimateparallelscan_function amestimateparallelscan; /* can be NULL */
diff --git a/src/include/access/gin.h b/src/include/access/gin.h
index 13ea91922ef..2f265f4816c 100644
--- a/src/include/access/gin.h
+++ b/src/include/access/gin.h
@@ -100,6 +100,9 @@ extern PGDLLIMPORT int gin_pending_list_limit;
extern void ginGetStats(Relation index, GinStatsData *stats);
extern void ginUpdateStats(Relation index, const GinStatsData *stats,
bool is_build);
+extern bool gincomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
extern void _gin_parallel_build_main(dsm_segment *seg, shm_toc *toc);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 2f9a2b069cd..5783dbebff0 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -369,7 +369,7 @@ extern TM_Result heap_update(Relation relation, HeapTupleData *oldtup,
TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
- Bitmapset *mix_attrs, Buffer *vmbuffer,
+ const Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
@@ -404,8 +404,8 @@ extern bool heap_tuple_needs_eventual_freeze(HeapTupleHeader tuple);
extern void simple_heap_insert(Relation relation, HeapTuple tup);
extern void simple_heap_delete(Relation relation, const ItemPointerData *tid);
-extern void simple_heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple tup, TU_UpdateIndexes *update_indexes);
+extern Bitmapset *simple_heap_update(Relation relation, const ItemPointerData *otid,
+ HeapTuple tup, TU_UpdateIndexes *update_indexes);
extern TransactionId heap_index_delete_tuples(Relation rel,
TM_IndexDeleteOp *delstate);
diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h
index 16be5c7a9c1..42bd329eaad 100644
--- a/src/include/access/nbtree.h
+++ b/src/include/access/nbtree.h
@@ -1210,6 +1210,10 @@ extern int btgettreeheight(Relation rel);
extern CompareType bttranslatestrategy(StrategyNumber strategy, Oid opfamily);
extern StrategyNumber bttranslatecmptype(CompareType cmptype, Oid opfamily);
+extern bool btcomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* prototypes for internal functions in nbtree.c
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 2fa790b6bf5..d94dfc9b41d 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -549,6 +549,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
+ const Bitmapset *updated_cols,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1512,12 +1513,12 @@ static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ const Bitmapset *mix_cols, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
- wait, tmfd,
- lockmode, update_indexes);
+ wait, tmfd, lockmode,
+ mix_cols, update_indexes);
}
/*
@@ -2020,6 +2021,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index dda95e54903..8d364f8b30f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -132,6 +132,7 @@ extern bool CompareIndexInfo(const IndexInfo *info1, const IndexInfo *info2,
const AttrMap *attmap);
extern void BuildSpeculativeIndexInfo(Relation index, IndexInfo *ii);
+extern void BuildUpdateIndexInfo(ResultRelInfo *resultRelInfo);
extern void FormIndexDatum(IndexInfo *indexInfo,
TupleTableSlot *slot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index fa2b657fb2f..2b851d9964c 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -739,6 +739,11 @@ extern Bitmapset *ExecGetAllUpdatedCols(ResultRelInfo *relinfo, EState *estate);
*/
extern void ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative);
extern void ExecCloseIndices(ResultRelInfo *resultRelInfo);
+extern Bitmapset *ExecWhichIndexesRequireUpdates(ResultRelInfo *relinfo,
+ Bitmapset *mix_attrs,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
extern List *ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
TupleTableSlot *slot, EState *estate,
bool update,
@@ -800,5 +805,9 @@ extern ResultRelInfo *ExecLookupResultRelByOid(ModifyTableState *node,
Oid resultoid,
bool missing_ok,
bool update_cache);
+extern Bitmapset *ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 64ff6996431..e10f4239de9 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -174,15 +174,29 @@ typedef struct IndexInfo
*/
AttrNumber ii_IndexAttrNumbers[INDEX_MAX_KEYS];
+ /*
+ * All key, expression, sumarizing, and partition attributes referenced by
+ * this index
+ */
+ Bitmapset *ii_IndexedAttrs;
+
/* expr trees for expression entries, or NIL if none */
List *ii_Expressions; /* list of Expr */
/* exec state for expressions, or NIL if none */
List *ii_ExpressionsState; /* list of ExprState */
+ /* attributes exclusively referenced by expression indexes */
+ Bitmapset *ii_ExpressionsAttrs;
/* partial-index predicate, or NIL if none */
List *ii_Predicate; /* list of Expr */
/* exec state for expressions, or NIL if none */
ExprState *ii_PredicateState;
+ /* attributes referenced by the predicate */
+ Bitmapset *ii_PredicateAttrs;
+ /* partial index predicate determined yet? */
+ bool ii_CheckedPredicate;
+ /* amupdate hint used to avoid rechecking predicate */
+ bool ii_PredicateSatisfied;
/* Per-column exclusion operators, or NULL if none */
Oid *ii_ExclusionOps; /* array with one entry per column */
@@ -499,6 +513,12 @@ typedef struct ResultRelInfo
/* true if the above has been computed */
bool ri_extraUpdatedCols_valid;
+ /*
+ * For UPDATE a Bitmapset of the attributes that are both indexed and have
+ * changed in value.
+ */
+ Bitmapset *ri_ChangedIndexedCols;
+
/* Projection to generate new tuple in an INSERT/UPDATE */
ProjectionInfo *ri_projectNew;
/* Slot to hold that tuple */
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 80286076a11..b23a7306e69 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -164,6 +164,7 @@ typedef struct RelationData
Bitmapset *rd_idattr; /* included in replica identity index */
Bitmapset *rd_hotblockingattr; /* cols blocking HOT update */
Bitmapset *rd_summarizedattr; /* cols indexed by summarizing indexes */
+ Bitmapset *rd_indexedattr; /* all cols referenced by indexes */
PublicationDesc *rd_pubdesc; /* publication descriptor, or NULL */
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 3561c6bef0b..d3fbb8b093a 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -71,6 +71,7 @@ typedef enum IndexAttrBitmapKind
INDEX_ATTR_BITMAP_IDENTITY_KEY,
INDEX_ATTR_BITMAP_HOT_BLOCKING,
INDEX_ATTR_BITMAP_SUMMARIZED,
+ INDEX_ATTR_BITMAP_INDEXED,
} IndexAttrBitmapKind;
extern Bitmapset *RelationGetIndexAttrBitmap(Relation relation,
diff --git a/src/test/isolation/expected/insert-conflict-specconflict.out b/src/test/isolation/expected/insert-conflict-specconflict.out
index e34a821c403..54b3981918c 100644
--- a/src/test/isolation/expected/insert-conflict-specconflict.out
+++ b/src/test/isolation/expected/insert-conflict-specconflict.out
@@ -80,6 +80,10 @@ pg_advisory_unlock
t
(1 row)
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
s1: NOTICE: acquiring advisory lock on 2
s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
@@ -172,6 +176,10 @@ pg_advisory_unlock
t
(1 row)
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
s2: NOTICE: acquiring advisory lock on 2
s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
@@ -369,6 +377,10 @@ key|data
step s1_commit: COMMIT;
s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
s2: NOTICE: acquiring advisory lock on 2
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
step s2_upsert: <... completed>
step controller_show: SELECT * FROM upserttest;
key|data
@@ -530,6 +542,14 @@ isolation/insert-conflict-specconflict/s2|transactionid|ExclusiveLock|t
step s2_commit: COMMIT;
s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_4() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 4
+s1: NOTICE: blurt_and_lock_4() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 4
step s1_upsert: <... completed>
step s1_noop:
step controller_show: SELECT * FROM upserttest;
diff --git a/src/test/regress/expected/heap_hot_updates.out b/src/test/regress/expected/heap_hot_updates.out
new file mode 100644
index 00000000000..14276e3cbca
--- /dev/null
+++ b/src/test/regress/expected/heap_hot_updates.out
@@ -0,0 +1,650 @@
+-- ================================================================
+-- Test Suite for Heap-only (HOT) Updates
+-- ================================================================
+-- Setup: Create function to measure HOT updates
+CREATE OR REPLACE FUNCTION check_hot_updates(
+ expected INT,
+ p_table_name TEXT DEFAULT 't',
+ p_schema_name TEXT DEFAULT current_schema()
+)
+RETURNS TABLE (
+ table_name TEXT,
+ total_updates BIGINT,
+ hot_updates BIGINT,
+ hot_update_percentage NUMERIC,
+ matches_expected BOOLEAN
+)
+LANGUAGE plpgsql
+AS $$
+DECLARE
+ v_relid oid;
+ v_qualified_name TEXT;
+ v_hot_updates BIGINT;
+ v_updates BIGINT;
+ v_xact_hot_updates BIGINT;
+ v_xact_updates BIGINT;
+BEGIN
+ -- Force statistics update
+ PERFORM pg_stat_force_next_flush();
+
+ -- Get table OID
+ v_qualified_name := quote_ident(p_schema_name) || '.' || quote_ident(p_table_name);
+ v_relid := v_qualified_name::regclass;
+
+ IF v_relid IS NULL THEN
+ RAISE EXCEPTION 'Table %.% not found', p_schema_name, p_table_name;
+ END IF;
+
+ -- Get cumulative + transaction stats
+ v_hot_updates := COALESCE(pg_stat_get_tuples_hot_updated(v_relid), 0);
+ v_updates := COALESCE(pg_stat_get_tuples_updated(v_relid), 0);
+ v_xact_hot_updates := COALESCE(pg_stat_get_xact_tuples_hot_updated(v_relid), 0);
+ v_xact_updates := COALESCE(pg_stat_get_xact_tuples_updated(v_relid), 0);
+
+ v_hot_updates := v_hot_updates + v_xact_hot_updates;
+ v_updates := v_updates + v_xact_updates;
+
+ RETURN QUERY
+ SELECT
+ p_table_name::TEXT,
+ v_updates::BIGINT,
+ v_hot_updates::BIGINT,
+ CASE WHEN v_updates > 0
+ THEN ROUND((v_hot_updates::numeric / v_updates::numeric * 100)::numeric, 2)
+ ELSE 0
+ END,
+ (v_hot_updates = expected)::BOOLEAN;
+END;
+$$;
+CREATE COLLATION case_insensitive (
+ provider = libc,
+ locale = 'C'
+);
+-- ================================================================
+-- GIN Index on JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data);
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "database"]}');
+-- Change tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Change tags again - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Add field without changing existing keys - GIN keys changed (added "note"), NOT HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "note": "test"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Index with Unchanged Keys
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create GIN index on specific path
+CREATE INDEX t_gin_idx ON t USING gin((data->'tags'));
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "sql"], "status": "active"}');
+-- Change non-indexed field - GIN keys on 'tags' unchanged, should be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change indexed tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN with jsonb_path_ops
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data jsonb_path_ops);
+INSERT INTO t VALUES (1, '{"user": {"name": "alice"}, "tags": ["a", "b"]}');
+-- Change value at different path - keys changed, NOT HOT
+UPDATE t SET data = '{"user": {"name": "bob"}, "tags": ["a", "b"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Mixed Index Types (BRIN + Expression)
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_idx ON t USING brin(value);
+CREATE INDEX t_expr_idx ON t((data->'status'));
+INSERT INTO t VALUES (1, 100, '{"status": "active"}');
+-- Update only BRIN column - should be HOT
+UPDATE t SET value = 200 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update only expression column - should NOT be HOT
+UPDATE t SET data = '{"status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update both - should NOT be HOT
+UPDATE t SET value = 300, data = '{"status": "pending"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Array Index - Order Insensitive Extraction
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ data JSONB
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+-- GIN index on JSONB array (extracts all elements)
+CREATE INDEX t_items_gin ON t USING GIN ((data->'items'));
+INSERT INTO t VALUES (1, '{"items": [1, 2, 3], "status": "active"}');
+-- Update: Reorder array elements
+-- JSONB equality: NOT equal (different arrays)
+-- GIN extraction: Same elements extracted (might allow HOT if not careful)
+UPDATE t SET data = '{"items": [3, 2, 1], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update: Add/remove element
+UPDATE t SET data = '{"items": [1, 2, 3, 4], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- TEST: GIN with TOASTed TEXT (tsvector)
+-- ================================================================
+CREATE TABLE t(id INT, content TEXT, search_vec tsvector)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create trigger to maintain tsvector
+CREATE TRIGGER tsvectorupdate_toast
+ BEFORE INSERT OR UPDATE ON t
+ FOR EACH ROW EXECUTE FUNCTION
+ tsvector_update_trigger(search_vec, 'pg_catalog.english', content);
+CREATE INDEX t_gin ON t USING gin(search_vec);
+-- Insert with large content (will be TOASTed)
+INSERT INTO t (id, content) VALUES
+ (1, repeat('important keyword ', 1000) || repeat('filler text ', 10000));
+-- Verify initial state
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('important');
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+-- IMPORTANT: The BEFORE UPDATE trigger modifies search_vec, so by the time
+-- ExecWhichIndexesRequireUpdates() runs, search_vec has already changed.
+-- This means the comparison sees old tsvector vs. trigger-modified tsvector,
+-- not the natural progression. HOT won't happen because the trigger changed
+-- the indexed column.
+-- Update: Even though content keywords unchanged, trigger still fires
+UPDATE t
+SET content = repeat('important keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (trigger modifies search_vec, blocking HOT)
+-- This is actually correct behavior - the trigger updated an indexed column
+-- Update: Change indexed keywords
+UPDATE t
+SET content = repeat('critical keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (index keys changed)
+-- Verify query correctness
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('critical');
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+DROP TABLE t CASCADE;
+-- ================================================================
+-- TEST: GIN with Array of Large Strings
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin(tags);
+-- Insert with large array elements (might be TOASTed)
+INSERT INTO t (id, tags) VALUES
+ (1, ARRAY[repeat('tag1', 1000), repeat('tag2', 1000)]);
+-- Update: Change to different large values - NOT HOT
+UPDATE t
+SET tags = ARRAY[repeat('tag3', 1000), repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (keys actually changed)
+-- Update: Keep same tag values, just reorder - SHOULD BE HOT
+-- (GIN is order-insensitive: both [tag3,tag4] and [tag4,tag3]
+-- extract to the same sorted key set ['tag3','tag4'])
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000), repeat('tag3', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Expected: 1 HOT (GIN keys semantically identical)
+-- Update: Remove an element - NOT HOT (keys changed)
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Expected: Still 1 HOT (not this one)
+DROP TABLE t CASCADE;
+-- ================================================================
+-- BRIN Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ value INT,
+ description TEXT
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_partial_idx ON t USING brin(value) WHERE value > 100;
+INSERT INTO t VALUES (1, 50, 'below range');
+-- Test 1: Outside predicate
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Validate: Predicate query returns 0 rows
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+ cnt
+-----
+ 0
+(1 row)
+
+-- Test 2: Transition into predicate
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Validate: Predicate query returns 1 row with correct value
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+ cnt | max_val
+-----+---------
+ 1 | 150
+(1 row)
+
+-- Test 3: Inside predicate, value changes
+UPDATE t SET value = 160, description = 'updated again' WHERE id = 1;
+SELECT * FROM check_hot_updates(3);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 3 | 100.00 | t
+(1 row)
+
+-- Validate: Updated value (160) is returned
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+ cnt | max_val
+-----+---------
+ 1 | 160
+(1 row)
+
+-- Test 4: Transition out of predicate
+UPDATE t SET value = 50 WHERE id = 1;
+SELECT * FROM check_hot_updates(4);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 4 | 100.00 | t
+(1 row)
+
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+ cnt
+-----
+ 0
+(1 row)
+
+SELECT id, value, description FROM t;
+ id | value | description
+----+-------+---------------
+ 1 | 50 | updated again
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- HASH Index (Simple Column)
+-- ================================================================
+CREATE TABLE t(id INT, code VARCHAR(20), description TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_idx ON t USING hash(code);
+INSERT INTO t VALUES (1, 'CODE001', 'initial');
+-- Update non-indexed column - should be HOT
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update indexed column - HASH index requires update, NOT HOT
+UPDATE t SET code = 'CODE002' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update both - NOT HOT
+UPDATE t SET code = 'CODE003', description = 'changed' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Back to original code - NOT HOT (different hash bucket location)
+UPDATE t SET code = 'CODE001' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- HASH Index on Expression
+-- ================================================================
+CREATE TABLE t(id INT, email TEXT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_lower_email_idx ON t USING HASH(lower(email));
+INSERT INTO t VALUES (1, '[email protected]', '{"status": "new"}');
+-- Update non-indexed field - should be HOT
+UPDATE t SET data = '{"status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update email with case change only (same lowercase) - should be HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Update email to different lowercase - NOT HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 2 | 66.67 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- Multiple HASH Indexes
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, status VARCHAR, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+CREATE INDEX t_hash_status_idx ON t USING hash(status);
+INSERT INTO t VALUES (1, 'electronics', 'active', 100);
+-- Update non-indexed column - should be HOT
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update one indexed column - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update other indexed column - NOT HOT
+UPDATE t SET status = 'inactive' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Update both indexed columns - NOT HOT
+UPDATE t SET category = 'videos', status = 'pending' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- BRIN vs HASH Comparison
+-- ================================================================
+CREATE TABLE t_brin(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE TABLE t_hash(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_value_idx ON t_brin USING brin(value);
+CREATE INDEX t_hash_value_idx ON t_hash USING hash(value);
+INSERT INTO t_brin VALUES (1, 100, 'initial');
+INSERT INTO t_hash VALUES (1, 100, 'initial');
+-- Same update on both - different HOT behavior expected
+-- BRIN: might allow HOT (range summary unchanged)
+-- HASH: blocks HOT (hash bucket changed)
+UPDATE t_brin SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't_brin');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t_brin | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT (BRIN allows it for single row)
+UPDATE t_hash SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't_hash');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t_hash | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (HASH blocks it)
+DROP TABLE t_brin CASCADE;
+DROP TABLE t_hash CASCADE;
+-- ================================================================
+-- HASH Index with NULL Values
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'electronics', 'initial');
+-- Update indexed column to NULL - NOT HOT (hash value changed)
+UPDATE t SET category = NULL WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT
+-- Update indexed column from NULL to value - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Expected: 1 HOT
+DROP TABLE t CASCADE;
+-- ================================================================
+-- BRIN on JSONB Field
+-- ================================================================
+CREATE TABLE t(id INT, metrics JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- BRIN doesn't directly support JSONB, but we can test on expression
+CREATE INDEX t_brin_count_idx ON t USING brin(
+ CAST(metrics->>'count' AS INTEGER)
+);
+INSERT INTO t VALUES (1, '{"count": "100", "timestamp": "2024-01-01"}');
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET metrics = '{"count": "100", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT
+-- Update indexed field - BRIN allows HOT for single row
+UPDATE t SET metrics = '{"count": "150", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Expected: 2 HOT (BRIN permits single-row updates)
+DROP TABLE t CASCADE;
+-- ================================================================
+-- Mixed BRIN + HASH on Same Table
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, timestamp TIMESTAMP, price NUMERIC, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_timestamp_idx ON t USING brin(timestamp);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'books', '2024-01-01 10:00:00', 29.99, 'initial');
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT
+-- Update BRIN indexed column - allows HOT
+UPDATE t SET timestamp = '2024-01-02 10:00:00' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Expected: 2 HOT
+-- Update HASH indexed column - blocks HOT
+UPDATE t SET category = 'videos' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 2 | 66.67 | t
+(1 row)
+
+-- Expected: 2 HOT (HASH blocks it)
+-- Update price (non-indexed) - should be HOT
+UPDATE t SET price = 39.99 WHERE id = 1;
+SELECT * FROM check_hot_updates(3);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 3 | 75.00 | t
+(1 row)
+
+-- Expected: 3 HOT
+DROP TABLE t CASCADE;
+-- ================================================================
+-- Index both on a field in a JSONB document, and the document
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t((docs->'name'));
+CREATE INDEX t_docs_col_idx ON t(docs);
+INSERT INTO t VALUES (1, '{"name": "john", "data": "some data"}');
+-- Update impacts index on whole docment attribute, can't go HOT
+UPDATE t SET docs='{"name": "john", "data": "some other data"}' WHERE id=1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- Cleanup
+DROP FUNCTION check_hot_updates(int, text, text);
+DROP COLLATION case_insensitive;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index cc6d799bcea..f3db9270fe6 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -125,6 +125,12 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression compression_lz4 memoize stats predicate numa eager_aggregate
+
+# ----------
+# Another group of parallel tests, these focused on heap HOT updates
+# ----------
+test: heap_hot_updates
+
# event_trigger depends on create_am and cannot run concurrently with
# any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/sql/heap_hot_updates.sql b/src/test/regress/sql/heap_hot_updates.sql
new file mode 100644
index 00000000000..e047bcddf5c
--- /dev/null
+++ b/src/test/regress/sql/heap_hot_updates.sql
@@ -0,0 +1,513 @@
+-- ================================================================
+-- Test Suite for Heap-only (HOT) Updates
+-- ================================================================
+
+-- Setup: Create function to measure HOT updates
+CREATE OR REPLACE FUNCTION check_hot_updates(
+ expected INT,
+ p_table_name TEXT DEFAULT 't',
+ p_schema_name TEXT DEFAULT current_schema()
+)
+RETURNS TABLE (
+ table_name TEXT,
+ total_updates BIGINT,
+ hot_updates BIGINT,
+ hot_update_percentage NUMERIC,
+ matches_expected BOOLEAN
+)
+LANGUAGE plpgsql
+AS $$
+DECLARE
+ v_relid oid;
+ v_qualified_name TEXT;
+ v_hot_updates BIGINT;
+ v_updates BIGINT;
+ v_xact_hot_updates BIGINT;
+ v_xact_updates BIGINT;
+BEGIN
+ -- Force statistics update
+ PERFORM pg_stat_force_next_flush();
+
+ -- Get table OID
+ v_qualified_name := quote_ident(p_schema_name) || '.' || quote_ident(p_table_name);
+ v_relid := v_qualified_name::regclass;
+
+ IF v_relid IS NULL THEN
+ RAISE EXCEPTION 'Table %.% not found', p_schema_name, p_table_name;
+ END IF;
+
+ -- Get cumulative + transaction stats
+ v_hot_updates := COALESCE(pg_stat_get_tuples_hot_updated(v_relid), 0);
+ v_updates := COALESCE(pg_stat_get_tuples_updated(v_relid), 0);
+ v_xact_hot_updates := COALESCE(pg_stat_get_xact_tuples_hot_updated(v_relid), 0);
+ v_xact_updates := COALESCE(pg_stat_get_xact_tuples_updated(v_relid), 0);
+
+ v_hot_updates := v_hot_updates + v_xact_hot_updates;
+ v_updates := v_updates + v_xact_updates;
+
+ RETURN QUERY
+ SELECT
+ p_table_name::TEXT,
+ v_updates::BIGINT,
+ v_hot_updates::BIGINT,
+ CASE WHEN v_updates > 0
+ THEN ROUND((v_hot_updates::numeric / v_updates::numeric * 100)::numeric, 2)
+ ELSE 0
+ END,
+ (v_hot_updates = expected)::BOOLEAN;
+END;
+$$;
+
+CREATE COLLATION case_insensitive (
+ provider = libc,
+ locale = 'C'
+);
+
+
+-- ================================================================
+-- GIN Index on JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data);
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "database"]}');
+
+-- Change tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+
+-- Change tags again - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+
+-- Add field without changing existing keys - GIN keys changed (added "note"), NOT HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "note": "test"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+
+DROP TABLE t;
+
+
+-- ================================================================
+-- GIN Index with Unchanged Keys
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create GIN index on specific path
+CREATE INDEX t_gin_idx ON t USING gin((data->'tags'));
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "sql"], "status": "active"}');
+
+-- Change non-indexed field - GIN keys on 'tags' unchanged, should be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Change indexed tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t;
+
+
+-- ================================================================
+-- GIN with jsonb_path_ops
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data jsonb_path_ops);
+INSERT INTO t VALUES (1, '{"user": {"name": "alice"}, "tags": ["a", "b"]}');
+
+-- Change value at different path - keys changed, NOT HOT
+UPDATE t SET data = '{"user": {"name": "bob"}, "tags": ["a", "b"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+
+DROP TABLE t;
+
+
+-- ================================================================
+-- Mixed Index Types (BRIN + Expression)
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_idx ON t USING brin(value);
+CREATE INDEX t_expr_idx ON t((data->'status'));
+INSERT INTO t VALUES (1, 100, '{"status": "active"}');
+
+-- Update only BRIN column - should be HOT
+UPDATE t SET value = 200 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update only expression column - should NOT be HOT
+UPDATE t SET data = '{"status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update both - should NOT be HOT
+UPDATE t SET value = 300, data = '{"status": "pending"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t;
+
+
+-- ================================================================
+-- GIN Array Index - Order Insensitive Extraction
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ data JSONB
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+
+-- GIN index on JSONB array (extracts all elements)
+CREATE INDEX t_items_gin ON t USING GIN ((data->'items'));
+
+INSERT INTO t VALUES (1, '{"items": [1, 2, 3], "status": "active"}');
+
+-- Update: Reorder array elements
+-- JSONB equality: NOT equal (different arrays)
+-- GIN extraction: Same elements extracted (might allow HOT if not careful)
+UPDATE t SET data = '{"items": [3, 2, 1], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update: Add/remove element
+UPDATE t SET data = '{"items": [1, 2, 3, 4], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t;
+
+
+-- ================================================================
+-- TEST: GIN with TOASTed TEXT (tsvector)
+-- ================================================================
+CREATE TABLE t(id INT, content TEXT, search_vec tsvector)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+-- Create trigger to maintain tsvector
+CREATE TRIGGER tsvectorupdate_toast
+ BEFORE INSERT OR UPDATE ON t
+ FOR EACH ROW EXECUTE FUNCTION
+ tsvector_update_trigger(search_vec, 'pg_catalog.english', content);
+
+CREATE INDEX t_gin ON t USING gin(search_vec);
+
+-- Insert with large content (will be TOASTed)
+INSERT INTO t (id, content) VALUES
+ (1, repeat('important keyword ', 1000) || repeat('filler text ', 10000));
+
+-- Verify initial state
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('important');
+-- Expected: 1 row
+
+-- IMPORTANT: The BEFORE UPDATE trigger modifies search_vec, so by the time
+-- ExecWhichIndexesRequireUpdates() runs, search_vec has already changed.
+-- This means the comparison sees old tsvector vs. trigger-modified tsvector,
+-- not the natural progression. HOT won't happen because the trigger changed
+-- the indexed column.
+
+-- Update: Even though content keywords unchanged, trigger still fires
+UPDATE t
+SET content = repeat('important keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+-- Expected: 0 HOT (trigger modifies search_vec, blocking HOT)
+-- This is actually correct behavior - the trigger updated an indexed column
+
+-- Update: Change indexed keywords
+UPDATE t
+SET content = repeat('critical keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+-- Expected: 0 HOT (index keys changed)
+
+-- Verify query correctness
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('critical');
+-- Expected: 1 row
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- TEST: GIN with Array of Large Strings
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin(tags);
+
+-- Insert with large array elements (might be TOASTed)
+INSERT INTO t (id, tags) VALUES
+ (1, ARRAY[repeat('tag1', 1000), repeat('tag2', 1000)]);
+
+-- Update: Change to different large values - NOT HOT
+UPDATE t
+SET tags = ARRAY[repeat('tag3', 1000), repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+-- Expected: 0 HOT (keys actually changed)
+
+-- Update: Keep same tag values, just reorder - SHOULD BE HOT
+-- (GIN is order-insensitive: both [tag3,tag4] and [tag4,tag3]
+-- extract to the same sorted key set ['tag3','tag4'])
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000), repeat('tag3', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: 1 HOT (GIN keys semantically identical)
+
+-- Update: Remove an element - NOT HOT (keys changed)
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: Still 1 HOT (not this one)
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- BRIN Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ value INT,
+ description TEXT
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+
+CREATE INDEX t_brin_partial_idx ON t USING brin(value) WHERE value > 100;
+
+INSERT INTO t VALUES (1, 50, 'below range');
+
+-- Test 1: Outside predicate
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Validate: Predicate query returns 0 rows
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+
+-- Test 2: Transition into predicate
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+
+-- Validate: Predicate query returns 1 row with correct value
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+
+-- Test 3: Inside predicate, value changes
+UPDATE t SET value = 160, description = 'updated again' WHERE id = 1;
+SELECT * FROM check_hot_updates(3);
+
+-- Validate: Updated value (160) is returned
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+
+-- Test 4: Transition out of predicate
+UPDATE t SET value = 50 WHERE id = 1;
+SELECT * FROM check_hot_updates(4);
+
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+
+SELECT id, value, description FROM t;
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- HASH Index (Simple Column)
+-- ================================================================
+CREATE TABLE t(id INT, code VARCHAR(20), description TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_idx ON t USING hash(code);
+INSERT INTO t VALUES (1, 'CODE001', 'initial');
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update indexed column - HASH index requires update, NOT HOT
+UPDATE t SET code = 'CODE002' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update both - NOT HOT
+UPDATE t SET code = 'CODE003', description = 'changed' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Back to original code - NOT HOT (different hash bucket location)
+UPDATE t SET code = 'CODE001' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- HASH Index on Expression
+-- ================================================================
+CREATE TABLE t(id INT, email TEXT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_lower_email_idx ON t USING HASH(lower(email));
+INSERT INTO t VALUES (1, '[email protected]', '{"status": "new"}');
+
+-- Update non-indexed field - should be HOT
+UPDATE t SET data = '{"status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update email with case change only (same lowercase) - should be HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+
+-- Update email to different lowercase - NOT HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- Multiple HASH Indexes
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, status VARCHAR, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+CREATE INDEX t_hash_status_idx ON t USING hash(status);
+INSERT INTO t VALUES (1, 'electronics', 'active', 100);
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update one indexed column - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update other indexed column - NOT HOT
+UPDATE t SET status = 'inactive' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update both indexed columns - NOT HOT
+UPDATE t SET category = 'videos', status = 'pending' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- BRIN vs HASH Comparison
+-- ================================================================
+CREATE TABLE t_brin(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE TABLE t_hash(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+CREATE INDEX t_brin_value_idx ON t_brin USING brin(value);
+CREATE INDEX t_hash_value_idx ON t_hash USING hash(value);
+
+INSERT INTO t_brin VALUES (1, 100, 'initial');
+INSERT INTO t_hash VALUES (1, 100, 'initial');
+
+-- Same update on both - different HOT behavior expected
+-- BRIN: might allow HOT (range summary unchanged)
+-- HASH: blocks HOT (hash bucket changed)
+UPDATE t_brin SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't_brin');
+-- Expected: 1 HOT (BRIN allows it for single row)
+
+UPDATE t_hash SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't_hash');
+-- Expected: 0 HOT (HASH blocks it)
+
+DROP TABLE t_brin CASCADE;
+DROP TABLE t_hash CASCADE;
+
+
+-- ================================================================
+-- HASH Index with NULL Values
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'electronics', 'initial');
+
+-- Update indexed column to NULL - NOT HOT (hash value changed)
+UPDATE t SET category = NULL WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+-- Expected: 0 HOT
+
+-- Update indexed column from NULL to value - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+-- Expected: 0 HOT
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: 1 HOT
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- BRIN on JSONB Field
+-- ================================================================
+CREATE TABLE t(id INT, metrics JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- BRIN doesn't directly support JSONB, but we can test on expression
+CREATE INDEX t_brin_count_idx ON t USING brin(
+ CAST(metrics->>'count' AS INTEGER)
+);
+INSERT INTO t VALUES (1, '{"count": "100", "timestamp": "2024-01-01"}');
+
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET metrics = '{"count": "100", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: 1 HOT
+
+-- Update indexed field - BRIN allows HOT for single row
+UPDATE t SET metrics = '{"count": "150", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+-- Expected: 2 HOT (BRIN permits single-row updates)
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- Mixed BRIN + HASH on Same Table
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, timestamp TIMESTAMP, price NUMERIC, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_timestamp_idx ON t USING brin(timestamp);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'books', '2024-01-01 10:00:00', 29.99, 'initial');
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: 1 HOT
+
+-- Update BRIN indexed column - allows HOT
+UPDATE t SET timestamp = '2024-01-02 10:00:00' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+-- Expected: 2 HOT
+
+-- Update HASH indexed column - blocks HOT
+UPDATE t SET category = 'videos' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+-- Expected: 2 HOT (HASH blocks it)
+
+-- Update price (non-indexed) - should be HOT
+UPDATE t SET price = 39.99 WHERE id = 1;
+SELECT * FROM check_hot_updates(3);
+-- Expected: 3 HOT
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- Index both on a field in a JSONB document, and the document
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t((docs->'name'));
+CREATE INDEX t_docs_col_idx ON t(docs);
+INSERT INTO t VALUES (1, '{"name": "john", "data": "some data"}');
+
+-- Update impacts index on whole docment attribute, can't go HOT
+UPDATE t SET docs='{"name": "john", "data": "some other data"}' WHERE id=1;
+SELECT * FROM check_hot_updates(0);
+
+DROP TABLE t CASCADE;
+
+
+-- Cleanup
+DROP FUNCTION check_hot_updates(int, text, text);
+DROP COLLATION case_insensitive;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index cf3f6a7dafd..4cc7a9d4c7d 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -390,6 +390,7 @@ CachedFunctionCompileCallback
CachedFunctionDeleteCallback
CachedFunctionHashEntry
CachedFunctionHashKey
+CachedIndexDatum
CachedPlan
CachedPlanSource
CallContext
--
2.51.2
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-07 22:47 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-03-25 14:21 ` Re: Expanding HOT updates for expression and partial indexes Burd, Greg <[email protected]>
2025-07-02 18:10 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-07 21:36 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-10-09 19:27 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2025-11-16 18:53 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-11-19 18:00 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-11-21 15:25 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-11-22 21:30 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2025-11-24 18:59 ` Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-12-03 22:06 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
@ 2025-12-15 21:46 ` Greg Burd <[email protected]>
0 siblings, 0 replies; 37+ messages in thread
From: Greg Burd @ 2025-12-15 21:46 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Matthias van de Meent <[email protected]>
I've updated the patch set a tad and I've got some benchmark results
(and questions).
PATCHES
===========================================================
* 0001 - Prepare heapam_tuple_update() and simple_heap_update() for divergence
Unchanged.
* 0002 - Track changed indexed columns in the executor during UPDATEs
Bug/oversight minor fix related to partial index attributes.
Also, I mistakenly said that v25 removed the $subject (ability to allow
expression indexes to be HOT). That's not true, they can go HOT with
this patch provided that the result of the expression evaluated using
the before/after attribute values are equal using datumIsEqual(). When
that is the case, as can happen with updates to fields within JSONB
columns when indexes are on other fields, the update can be HOT should
the heap find room on the page to store the new tuple.
* 0003 - Replace index_unchanged_by_update() with ri_ChangedIndexedCols
Unchanged.
* 0004 - Identify if partial indexes are impacted by an update
This is the new piece, it existed in the v24 patch set and now it is
back. This checks the before/after partial index expression and when
both are outside the predicate then it is possible that heap can use the
HOT path whereas in the past this couldn't happen. In the past any
update to an attribute in an index, even if it was outside the
predicate, was (is) HOT blocking.
SUMMARY
===========================================================
I've just started to scratch the surface of performance testing for
this, attached is a very simple comparison of master/patch for a basic
update load that should always go HOT in either case. It shows about 1%
variance between the two (-O0), tests run on my laptop so that's
essentially no difference despite more overhead of the new function and
that it seems to be called more frequently due to (guessing here) more
opportunity for TM_Updated to be the return from heapam_tuple_update.
Your thoughts welcome here, or best/worst case ideas for tests to run.
Next up I plan to layer the controversial type-specific piece into this
patch set if nothing else just as a record of what's left over. Then
I'll try to better isolate good/bad performance implications of this
patch set.
Ideally, this patch set and the one (under development) for catalog
tuples could combine to completely restructure the heap update process
and open the door to more HOT updates and faster catalog updates. But,
I still have to demonstrate that. For JSONB heavy applications this
should be a net win, for the rest it should be a minor or zero
regression. For other custom implementations of indexes over
specialized types (as is the case for the new open sourced DocumentDB
work) this opens the door for HOT updates when possible. All of that is
the the hope, it's time to measure hope against reality. :)
This patch set does start to move the executor away from a heap-specific
view of the world where updates are all/none/summarizing. This
potentially eases the integration of WARM or PHOT-like solutions where
we only update those indexes that are materially impacted by an update.
It should be clear by now, that's my ultimate goal.
best.
-greg
pgbench (19devel)
progress: 1765546063.294 s, 65870.5 tps, lat 0.298 ms stddev 0.136, 0 failed
progress: 1765546068.294 s, 62181.9 tps, lat 0.316 ms stddev 0.137, 0 failed
progress: 1765546073.294 s, 61075.5 tps, lat 0.322 ms stddev 0.141, 0 failed
progress: 1765546078.294 s, 60555.0 tps, lat 0.325 ms stddev 0.140, 0 failed
progress: 1765546083.294 s, 59835.7 tps, lat 0.329 ms stddev 0.144, 0 failed
progress: 1765546088.294 s, 59338.7 tps, lat 0.331 ms stddev 0.148, 0 failed
progress: 1765546093.294 s, 58621.2 tps, lat 0.335 ms stddev 0.147, 0 failed
progress: 1765546098.294 s, 58402.5 tps, lat 0.337 ms stddev 0.154, 0 failed
progress: 1765546103.294 s, 58237.6 tps, lat 0.338 ms stddev 0.153, 0 failed
progress: 1765546108.294 s, 57781.4 tps, lat 0.340 ms stddev 0.145, 0 failed
progress: 1765546113.294 s, 57349.2 tps, lat 0.343 ms stddev 0.146, 0 failed
progress: 1765546118.294 s, 57205.5 tps, lat 0.344 ms stddev 0.144, 0 failed
progress: 1765546123.294 s, 56764.3 tps, lat 0.346 ms stddev 0.147, 0 failed
progress: 1765546128.294 s, 56694.2 tps, lat 0.347 ms stddev 0.155, 0 failed
progress: 1765546133.294 s, 56636.1 tps, lat 0.347 ms stddev 0.144, 0 failed
progress: 1765546138.294 s, 56498.2 tps, lat 0.348 ms stddev 0.146, 0 failed
progress: 1765546143.294 s, 56283.5 tps, lat 0.349 ms stddev 0.146, 0 failed
progress: 1765546148.294 s, 56153.6 tps, lat 0.350 ms stddev 0.147, 0 failed
progress: 1765546153.294 s, 56238.9 tps, lat 0.350 ms stddev 0.148, 0 failed
progress: 1765546158.294 s, 56077.8 tps, lat 0.351 ms stddev 0.148, 0 failed
progress: 1765546163.294 s, 56395.6 tps, lat 0.349 ms stddev 0.146, 0 failed
progress: 1765546168.294 s, 56297.3 tps, lat 0.349 ms stddev 0.146, 0 failed
progress: 1765546173.294 s, 56443.2 tps, lat 0.348 ms stddev 0.145, 0 failed
progress: 1765546178.294 s, 56345.4 tps, lat 0.349 ms stddev 0.145, 0 failed
progress: 1765546183.294 s, 56386.4 tps, lat 0.349 ms stddev 0.147, 0 failed
progress: 1765546188.294 s, 56389.0 tps, lat 0.349 ms stddev 0.150, 0 failed
progress: 1765546193.294 s, 56217.2 tps, lat 0.350 ms stddev 0.149, 0 failed
progress: 1765546198.294 s, 56250.8 tps, lat 0.350 ms stddev 0.152, 0 failed
progress: 1765546203.294 s, 56072.9 tps, lat 0.351 ms stddev 0.151, 0 failed
progress: 1765546208.294 s, 56065.7 tps, lat 0.351 ms stddev 0.146, 0 failed
progress: 1765546213.294 s, 56176.4 tps, lat 0.350 ms stddev 0.145, 0 failed
progress: 1765546218.294 s, 55823.8 tps, lat 0.352 ms stddev 0.146, 0 failed
progress: 1765546223.294 s, 56159.1 tps, lat 0.350 ms stddev 0.142, 0 failed
progress: 1765546228.294 s, 56028.1 tps, lat 0.351 ms stddev 0.142, 0 failed
progress: 1765546233.294 s, 56129.9 tps, lat 0.350 ms stddev 0.143, 0 failed
progress: 1765546238.294 s, 56168.9 tps, lat 0.350 ms stddev 0.141, 0 failed
progress: 1765546243.294 s, 56247.2 tps, lat 0.349 ms stddev 0.142, 0 failed
progress: 1765546248.294 s, 56092.8 tps, lat 0.350 ms stddev 0.146, 0 failed
progress: 1765546253.294 s, 56133.7 tps, lat 0.350 ms stddev 0.143, 0 failed
progress: 1765546258.294 s, 56178.2 tps, lat 0.350 ms stddev 0.144, 0 failed
progress: 1765546263.294 s, 56160.4 tps, lat 0.350 ms stddev 0.143, 0 failed
progress: 1765546268.294 s, 56067.4 tps, lat 0.351 ms stddev 0.142, 0 failed
progress: 1765546273.294 s, 56113.7 tps, lat 0.350 ms stddev 0.140, 0 failed
progress: 1765546278.294 s, 55889.0 tps, lat 0.352 ms stddev 0.142, 0 failed
progress: 1765546283.294 s, 56056.4 tps, lat 0.351 ms stddev 0.143, 0 failed
progress: 1765546288.294 s, 56131.6 tps, lat 0.350 ms stddev 0.142, 0 failed
progress: 1765546293.294 s, 56003.5 tps, lat 0.351 ms stddev 0.143, 0 failed
progress: 1765546298.294 s, 56040.5 tps, lat 0.351 ms stddev 0.139, 0 failed
progress: 1765546303.294 s, 55905.1 tps, lat 0.352 ms stddev 0.142, 0 failed
progress: 1765546308.294 s, 55932.0 tps, lat 0.352 ms stddev 0.142, 0 failed
progress: 1765546313.294 s, 55791.3 tps, lat 0.352 ms stddev 0.141, 0 failed
progress: 1765546318.294 s, 56021.1 tps, lat 0.351 ms stddev 0.141, 0 failed
progress: 1765546323.294 s, 55936.4 tps, lat 0.352 ms stddev 0.142, 0 failed
progress: 1765546328.294 s, 55782.7 tps, lat 0.352 ms stddev 0.142, 0 failed
progress: 1765546333.294 s, 55716.8 tps, lat 0.353 ms stddev 0.144, 0 failed
progress: 1765546338.294 s, 55571.8 tps, lat 0.354 ms stddev 0.145, 0 failed
progress: 1765546343.294 s, 55521.4 tps, lat 0.354 ms stddev 0.143, 0 failed
progress: 1765546348.294 s, 55767.0 tps, lat 0.353 ms stddev 0.140, 0 failed
progress: 1765546353.294 s, 55840.2 tps, lat 0.352 ms stddev 0.142, 0 failed
progress: 1765546358.294 s, 55840.3 tps, lat 0.352 ms stddev 0.144, 0 failed
progress: 1765546363.294 s, 55546.0 tps, lat 0.354 ms stddev 0.140, 0 failed
progress: 1765546368.294 s, 55789.7 tps, lat 0.352 ms stddev 0.144, 0 failed
progress: 1765546373.294 s, 55772.5 tps, lat 0.352 ms stddev 0.143, 0 failed
progress: 1765546378.294 s, 55674.0 tps, lat 0.353 ms stddev 0.143, 0 failed
progress: 1765546383.294 s, 55580.5 tps, lat 0.354 ms stddev 0.144, 0 failed
progress: 1765546388.294 s, 55417.2 tps, lat 0.355 ms stddev 0.142, 0 failed
progress: 1765546393.294 s, 55707.8 tps, lat 0.353 ms stddev 0.145, 0 failed
progress: 1765546398.294 s, 55542.2 tps, lat 0.354 ms stddev 0.146, 0 failed
progress: 1765546403.294 s, 55894.5 tps, lat 0.352 ms stddev 0.143, 0 failed
progress: 1765546408.294 s, 55926.0 tps, lat 0.352 ms stddev 0.140, 0 failed
progress: 1765546413.294 s, 55600.4 tps, lat 0.354 ms stddev 0.143, 0 failed
progress: 1765546418.299 s, 55548.0 tps, lat 0.354 ms stddev 0.141, 0 failed
transaction type: ../_bench/cf-5556/a/hot_test.sql
scaling factor: 1
query mode: simple
number of clients: 20
number of threads: 8
maximum number of tries: 1
duration: 360 s
number of transactions actually processed: 20389706
number of failed transactions: 0 (0.000%)
latency average = 0.347 ms
latency stddev = 0.145 ms
initial connection time = 9.011 ms
tps = 56638.715544 (without initial connection time)
table_name | total_updates | hot_updates | hot_percentage
------------+---------------+-------------+----------------
t | 20389706 | 20380083 | 99.95
(1 row)
schemaname | tablename | indexname | idx_scan | idx_tup_read | idx_tup_fetch
------------+-----------+-----------+----------+--------------+---------------
public | t | t_pkey | 20389706 | 20395813 | 20389706
(1 row)
relname | n_live_tup | n_dead_tup | dead_pct | last_vacuum | last_autovacuum
---------+------------+------------+----------+-------------+-------------------------------
t | 5000 | 0 | 0.00 | | 2025-12-12 08:34:28.502717-05
(1 row)
pgbench (19devel)
progress: 1765546808.169 s, 65602.3 tps, lat 0.299 ms stddev 0.142, 0 failed
progress: 1765546813.169 s, 62570.1 tps, lat 0.314 ms stddev 0.134, 0 failed
progress: 1765546818.169 s, 61340.7 tps, lat 0.321 ms stddev 0.141, 0 failed
progress: 1765546823.169 s, 60863.1 tps, lat 0.323 ms stddev 0.139, 0 failed
progress: 1765546828.169 s, 60407.0 tps, lat 0.326 ms stddev 0.144, 0 failed
progress: 1765546833.169 s, 58721.0 tps, lat 0.335 ms stddev 0.153, 0 failed
progress: 1765546838.169 s, 59566.4 tps, lat 0.330 ms stddev 0.149, 0 failed
progress: 1765546843.169 s, 58650.7 tps, lat 0.335 ms stddev 0.145, 0 failed
progress: 1765546848.169 s, 58056.5 tps, lat 0.339 ms stddev 0.152, 0 failed
progress: 1765546853.169 s, 57890.3 tps, lat 0.340 ms stddev 0.146, 0 failed
progress: 1765546858.169 s, 57613.9 tps, lat 0.341 ms stddev 0.146, 0 failed
progress: 1765546863.169 s, 57045.3 tps, lat 0.345 ms stddev 0.155, 0 failed
progress: 1765546868.169 s, 57455.6 tps, lat 0.342 ms stddev 0.149, 0 failed
progress: 1765546873.169 s, 57153.1 tps, lat 0.344 ms stddev 0.147, 0 failed
progress: 1765546878.169 s, 57054.2 tps, lat 0.345 ms stddev 0.146, 0 failed
progress: 1765546883.169 s, 56485.9 tps, lat 0.348 ms stddev 0.150, 0 failed
progress: 1765546888.169 s, 56779.8 tps, lat 0.346 ms stddev 0.146, 0 failed
progress: 1765546893.169 s, 56518.7 tps, lat 0.348 ms stddev 0.150, 0 failed
progress: 1765546898.169 s, 56803.5 tps, lat 0.346 ms stddev 0.148, 0 failed
progress: 1765546903.169 s, 56828.7 tps, lat 0.346 ms stddev 0.145, 0 failed
progress: 1765546908.169 s, 56692.3 tps, lat 0.347 ms stddev 0.149, 0 failed
progress: 1765546913.169 s, 56599.2 tps, lat 0.347 ms stddev 0.146, 0 failed
progress: 1765546918.169 s, 56619.7 tps, lat 0.347 ms stddev 0.147, 0 failed
progress: 1765546923.171 s, 56553.4 tps, lat 0.348 ms stddev 0.146, 0 failed
progress: 1765546928.169 s, 56541.0 tps, lat 0.348 ms stddev 0.147, 0 failed
progress: 1765546933.169 s, 56485.8 tps, lat 0.348 ms stddev 0.146, 0 failed
progress: 1765546938.169 s, 56377.9 tps, lat 0.349 ms stddev 0.156, 0 failed
progress: 1765546943.169 s, 55958.7 tps, lat 0.351 ms stddev 0.151, 0 failed
progress: 1765546948.169 s, 56291.2 tps, lat 0.349 ms stddev 0.148, 0 failed
progress: 1765546953.169 s, 56467.2 tps, lat 0.348 ms stddev 0.147, 0 failed
progress: 1765546958.169 s, 56267.9 tps, lat 0.349 ms stddev 0.151, 0 failed
progress: 1765546963.169 s, 56496.5 tps, lat 0.348 ms stddev 0.139, 0 failed
progress: 1765546968.169 s, 56280.4 tps, lat 0.349 ms stddev 0.146, 0 failed
progress: 1765546973.169 s, 56377.1 tps, lat 0.349 ms stddev 0.141, 0 failed
progress: 1765546978.169 s, 56081.8 tps, lat 0.351 ms stddev 0.146, 0 failed
progress: 1765546983.169 s, 56055.3 tps, lat 0.351 ms stddev 0.142, 0 failed
progress: 1765546988.169 s, 56430.8 tps, lat 0.348 ms stddev 0.141, 0 failed
progress: 1765546993.169 s, 56250.5 tps, lat 0.349 ms stddev 0.142, 0 failed
progress: 1765546998.169 s, 56193.0 tps, lat 0.350 ms stddev 0.144, 0 failed
progress: 1765547003.169 s, 56063.7 tps, lat 0.351 ms stddev 0.144, 0 failed
progress: 1765547008.169 s, 56181.7 tps, lat 0.350 ms stddev 0.143, 0 failed
progress: 1765547013.169 s, 56113.7 tps, lat 0.350 ms stddev 0.144, 0 failed
progress: 1765547018.169 s, 56215.2 tps, lat 0.350 ms stddev 0.142, 0 failed
progress: 1765547023.169 s, 56192.0 tps, lat 0.350 ms stddev 0.143, 0 failed
progress: 1765547028.169 s, 55851.8 tps, lat 0.352 ms stddev 0.144, 0 failed
progress: 1765547033.169 s, 56043.4 tps, lat 0.351 ms stddev 0.143, 0 failed
progress: 1765547038.169 s, 55929.4 tps, lat 0.351 ms stddev 0.142, 0 failed
progress: 1765547043.169 s, 55845.0 tps, lat 0.352 ms stddev 0.140, 0 failed
progress: 1765547048.169 s, 55332.0 tps, lat 0.355 ms stddev 0.148, 0 failed
progress: 1765547053.169 s, 55800.3 tps, lat 0.352 ms stddev 0.141, 0 failed
progress: 1765547058.169 s, 55691.2 tps, lat 0.353 ms stddev 0.146, 0 failed
progress: 1765547063.169 s, 55722.8 tps, lat 0.353 ms stddev 0.144, 0 failed
progress: 1765547068.169 s, 55907.4 tps, lat 0.352 ms stddev 0.142, 0 failed
progress: 1765547073.169 s, 55247.6 tps, lat 0.356 ms stddev 0.146, 0 failed
progress: 1765547078.169 s, 55753.3 tps, lat 0.352 ms stddev 0.143, 0 failed
progress: 1765547083.169 s, 55602.9 tps, lat 0.354 ms stddev 0.143, 0 failed
progress: 1765547088.169 s, 56078.6 tps, lat 0.351 ms stddev 0.146, 0 failed
progress: 1765547093.169 s, 55520.2 tps, lat 0.354 ms stddev 0.143, 0 failed
progress: 1765547098.169 s, 55919.5 tps, lat 0.352 ms stddev 0.141, 0 failed
progress: 1765547103.169 s, 55386.3 tps, lat 0.355 ms stddev 0.145, 0 failed
progress: 1765547108.169 s, 55562.4 tps, lat 0.354 ms stddev 0.144, 0 failed
progress: 1765547113.169 s, 55481.5 tps, lat 0.354 ms stddev 0.144, 0 failed
progress: 1765547118.169 s, 55409.7 tps, lat 0.355 ms stddev 0.145, 0 failed
progress: 1765547123.169 s, 55557.7 tps, lat 0.354 ms stddev 0.144, 0 failed
progress: 1765547128.169 s, 55173.7 tps, lat 0.356 ms stddev 0.146, 0 failed
progress: 1765547133.169 s, 55129.8 tps, lat 0.357 ms stddev 0.146, 0 failed
progress: 1765547138.169 s, 55724.8 tps, lat 0.353 ms stddev 0.143, 0 failed
progress: 1765547143.169 s, 54879.1 tps, lat 0.358 ms stddev 0.146, 0 failed
progress: 1765547148.169 s, 54758.7 tps, lat 0.359 ms stddev 0.149, 0 failed
progress: 1765547153.169 s, 55343.8 tps, lat 0.355 ms stddev 0.143, 0 failed
progress: 1765547158.169 s, 54764.9 tps, lat 0.359 ms stddev 0.146, 0 failed
transaction type: ../_bench/cf-5556/a/hot_test.sql
scaling factor: 1
query mode: simple
number of clients: 20
number of threads: 8
maximum number of tries: 1
duration: 360 s
number of transactions actually processed: 20401562
number of failed transactions: 0 (0.000%)
latency average = 0.347 ms
latency stddev = 0.146 ms
initial connection time = 12.057 ms
tps = 56672.534455 (without initial connection time)
table_name | total_updates | hot_updates | hot_percentage
------------+---------------+-------------+----------------
t | 20401562 | 20393424 | 99.96
(1 row)
schemaname | tablename | indexname | idx_scan | idx_tup_read | idx_tup_fetch
------------+-----------+-----------+----------+--------------+---------------
public | t | t_pkey | 20401562 | 20406533 | 20401562
(1 row)
relname | n_live_tup | n_dead_tup | dead_pct | last_vacuum | last_autovacuum
---------+------------+------------+----------+-------------+-------------------------------
t | 5000 | 0 | 0.00 | | 2025-12-12 08:46:09.781718-05
(1 row)
Attachments:
[application/octet-stream] v26-0001-Prepare-heapam_tuple_update-and-simple_heap_upda.patch (47.8K, 2-v26-0001-Prepare-heapam_tuple_update-and-simple_heap_upda.patch)
download | inline diff:
From f4756df0c566a10dcf978f2b76726e8bb3b30b0c Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 2 Nov 2025 11:36:20 -0500
Subject: [PATCH v26 1/4] Prepare heapam_tuple_update() and
simple_heap_update() for divergence
This commit lays the foundation for larger changes to come by taking the
first portion of heap_update() through the HeapDeterminColumnsInfo() and
replicating that logic in both heapam_tuple_update() and
simple_heap_upate(). This is done so that these two paths might diverge
in implementation later on. The simple_heap_update() path deals solely
with updates to catalog tuples which could record their modified
attributes rather than relearn them. The remaining calls from the
executor into the table AM update API could include the set of updated
attributes. This is foreshadowing... of course, as that's what the next
commit will start to do.
As part of this reorganization, the handling of replica identity key
attributes has been adjusted. Instead of fetching a second copy of
the bitmap during an update operation, the caller is now required to
provide it. This change applies to both heap_update() and
heap_delete().
---
src/backend/access/heap/heapam.c | 568 +++++++++++------------
src/backend/access/heap/heapam_handler.c | 117 ++++-
src/include/access/heapam.h | 24 +-
3 files changed, 410 insertions(+), 299 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 9636bb53ddd..d5a1b844b5a 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -39,18 +39,24 @@
#include "access/syncscan.h"
#include "access/valid.h"
#include "access/visibilitymap.h"
+#include "access/xact.h"
#include "access/xloginsert.h"
+#include "catalog/catalog.h"
#include "catalog/pg_database.h"
#include "catalog/pg_database_d.h"
#include "commands/vacuum.h"
+#include "nodes/bitmapset.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
+#include "storage/bufmgr.h"
+#include "storage/itemptr.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
#include "storage/procarray.h"
#include "utils/datum.h"
#include "utils/injection_point.h"
#include "utils/inval.h"
+#include "utils/relcache.h"
#include "utils/spccache.h"
#include "utils/syscache.h"
@@ -62,16 +68,8 @@ static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
HeapTuple newtup, HeapTuple old_key_tuple,
bool all_visible_cleared, bool new_all_visible_cleared);
#ifdef USE_ASSERT_CHECKING
-static void check_lock_if_inplace_updateable_rel(Relation relation,
- const ItemPointerData *otid,
- HeapTuple newtup);
static void check_inplace_rel_lock(HeapTuple oldtup);
#endif
-static Bitmapset *HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external);
static bool heap_acquire_tuplock(Relation relation, const ItemPointerData *tid,
LockTupleMode mode, LockWaitPolicy wait_policy,
bool *have_tuple_lock);
@@ -103,10 +101,10 @@ static bool ConditionalMultiXactIdWait(MultiXactId multi, MultiXactStatus status
static void index_delete_sort(TM_IndexDeleteOp *delstate);
static int bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate);
static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
-static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
+static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp,
+ Bitmapset *rid_attrs, bool key_required,
bool *copy);
-
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
* corresponding MultiXactStatuses (one to merely lock tuples, another one to
@@ -2814,6 +2812,7 @@ heap_delete(Relation relation, const ItemPointerData *tid,
Buffer buffer;
Buffer vmbuffer = InvalidBuffer;
TransactionId new_xmax;
+ Bitmapset *rid_attrs;
uint16 new_infomask,
new_infomask2;
bool have_tuple_lock = false;
@@ -2826,6 +2825,8 @@ heap_delete(Relation relation, const ItemPointerData *tid,
AssertHasSnapshotForToast(relation);
+ rid_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
/*
* Forbid this during a parallel operation, lest it allocate a combo CID.
* Other workers might need that combo CID for visibility checks, and we
@@ -3029,6 +3030,7 @@ l1:
UnlockTupleTuplock(relation, &(tp.t_self), LockTupleExclusive);
if (vmbuffer != InvalidBuffer)
ReleaseBuffer(vmbuffer);
+ bms_free(rid_attrs);
return result;
}
@@ -3050,7 +3052,10 @@ l1:
* Compute replica identity tuple before entering the critical section so
* we don't PANIC upon a memory allocation failure.
*/
- old_key_tuple = ExtractReplicaIdentity(relation, &tp, true, &old_key_copied);
+ old_key_tuple = ExtractReplicaIdentity(relation, &tp, rid_attrs,
+ true, &old_key_copied);
+ bms_free(rid_attrs);
+ rid_attrs = NULL;
/*
* If this is the first possibly-multixact-able operation in the current
@@ -3262,7 +3267,10 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* heap_update - replace a tuple
*
* See table_tuple_update() for an explanation of the parameters, except that
- * this routine directly takes a tuple rather than a slot.
+ * this routine directly takes a heap tuple rather than a slot.
+ *
+ * It's required that the caller has acquired the pin and lock on the buffer.
+ * That lock and pin will be managed here, not in the caller.
*
* In the failure cases, the routine fills *tmfd with the tuple's t_ctid,
* t_xmax (resolving a possible MultiXact, if necessary), and t_cmax (the last
@@ -3270,30 +3278,21 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* generated by another transaction).
*/
TM_Result
-heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+heap_update(Relation relation, HeapTupleData *oldtup,
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
+ Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
+ Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
+ Bitmapset *mix_attrs, Buffer *vmbuffer,
+ bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
- Bitmapset *hot_attrs;
- Bitmapset *sum_attrs;
- Bitmapset *key_attrs;
- Bitmapset *id_attrs;
- Bitmapset *interesting_attrs;
- Bitmapset *modified_attrs;
- ItemId lp;
- HeapTupleData oldtup;
HeapTuple heaptup;
HeapTuple old_key_tuple = NULL;
bool old_key_copied = false;
- Page page;
- BlockNumber block;
MultiXactStatus mxact_status;
- Buffer buffer,
- newbuf,
- vmbuffer = InvalidBuffer,
+ Buffer newbuf,
vmbuffer_new = InvalidBuffer;
bool need_toast;
Size newtupsize,
@@ -3307,7 +3306,6 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
bool all_visible_cleared_new = false;
bool checked_lockers;
bool locker_remains;
- bool id_has_external = false;
TransactionId xmax_new_tuple,
xmax_old_tuple;
uint16 infomask_old_tuple,
@@ -3315,144 +3313,13 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
infomask_new_tuple,
infomask2_new_tuple;
- Assert(ItemPointerIsValid(otid));
-
- /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
- Assert(HeapTupleHeaderGetNatts(newtup->t_data) <=
- RelationGetNumberOfAttributes(relation));
-
+ Assert(BufferIsLockedByMe(buffer));
+ Assert(ItemIdIsNormal(lp));
AssertHasSnapshotForToast(relation);
- /*
- * Forbid this during a parallel operation, lest it allocate a combo CID.
- * Other workers might need that combo CID for visibility checks, and we
- * have no provision for broadcasting it to them.
- */
- if (IsInParallelMode())
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
- errmsg("cannot update tuples during a parallel operation")));
-
-#ifdef USE_ASSERT_CHECKING
- check_lock_if_inplace_updateable_rel(relation, otid, newtup);
-#endif
-
- /*
- * Fetch the list of attributes to be checked for various operations.
- *
- * For HOT considerations, this is wasted effort if we fail to update or
- * have to put the new tuple on a different page. But we must compute the
- * list before obtaining buffer lock --- in the worst case, if we are
- * doing an update on one of the relevant system catalogs, we could
- * deadlock if we try to fetch the list later. In any case, the relcache
- * caches the data so this is usually pretty cheap.
- *
- * We also need columns used by the replica identity and columns that are
- * considered the "key" of rows in the table.
- *
- * Note that we get copies of each bitmap, so we need not worry about
- * relcache flush happening midway through.
- */
- hot_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_HOT_BLOCKING);
- sum_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_SUMMARIZED);
- key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
- id_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
- interesting_attrs = NULL;
- interesting_attrs = bms_add_members(interesting_attrs, hot_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, sum_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, key_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, id_attrs);
-
- block = ItemPointerGetBlockNumber(otid);
- INJECTION_POINT("heap_update-before-pin", NULL);
- buffer = ReadBuffer(relation, block);
- page = BufferGetPage(buffer);
-
- /*
- * Before locking the buffer, pin the visibility map page if it appears to
- * be necessary. Since we haven't got the lock yet, someone else might be
- * in the middle of changing this, so we'll need to recheck after we have
- * the lock.
- */
- if (PageIsAllVisible(page))
- visibilitymap_pin(relation, block, &vmbuffer);
-
- LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
-
- lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
-
- /*
- * Usually, a buffer pin and/or snapshot blocks pruning of otid, ensuring
- * we see LP_NORMAL here. When the otid origin is a syscache, we may have
- * neither a pin nor a snapshot. Hence, we may see other LP_ states, each
- * of which indicates concurrent pruning.
- *
- * Failing with TM_Updated would be most accurate. However, unlike other
- * TM_Updated scenarios, we don't know the successor ctid in LP_UNUSED and
- * LP_DEAD cases. While the distinction between TM_Updated and TM_Deleted
- * does matter to SQL statements UPDATE and MERGE, those SQL statements
- * hold a snapshot that ensures LP_NORMAL. Hence, the choice between
- * TM_Updated and TM_Deleted affects only the wording of error messages.
- * Settle on TM_Deleted, for two reasons. First, it avoids complicating
- * the specification of when tmfd->ctid is valid. Second, it creates
- * error log evidence that we took this branch.
- *
- * Since it's possible to see LP_UNUSED at otid, it's also possible to see
- * LP_NORMAL for a tuple that replaced LP_UNUSED. If it's a tuple for an
- * unrelated row, we'll fail with "duplicate key value violates unique".
- * XXX if otid is the live, newer version of the newtup row, we'll discard
- * changes originating in versions of this catalog row after the version
- * the caller got from syscache. See syscache-update-pruned.spec.
- */
- if (!ItemIdIsNormal(lp))
- {
- Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
-
- UnlockReleaseBuffer(buffer);
- Assert(!have_tuple_lock);
- if (vmbuffer != InvalidBuffer)
- ReleaseBuffer(vmbuffer);
- tmfd->ctid = *otid;
- tmfd->xmax = InvalidTransactionId;
- tmfd->cmax = InvalidCommandId;
- *update_indexes = TU_None;
-
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- /* modified_attrs not yet initialized */
- bms_free(interesting_attrs);
- return TM_Deleted;
- }
-
- /*
- * Fill in enough data in oldtup for HeapDetermineColumnsInfo to work
- * properly.
- */
- oldtup.t_tableOid = RelationGetRelid(relation);
- oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
- oldtup.t_len = ItemIdGetLength(lp);
- oldtup.t_self = *otid;
-
- /* the new tuple is ready, except for this: */
+ /* The new tuple is ready, except for this */
newtup->t_tableOid = RelationGetRelid(relation);
- /*
- * Determine columns modified by the update. Additionally, identify
- * whether any of the unmodified replica identity key attributes in the
- * old tuple is externally stored or not. This is required because for
- * such attributes the flattened value won't be WAL logged as part of the
- * new tuple so we must include it as part of the old_key_tuple. See
- * ExtractReplicaIdentity.
- */
- modified_attrs = HeapDetermineColumnsInfo(relation, interesting_attrs,
- id_attrs, &oldtup,
- newtup, &id_has_external);
-
/*
* If we're not updating any "key" column, we can grab a weaker lock type.
* This allows for more concurrency when we are running simultaneously
@@ -3464,7 +3331,7 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
* is updates that don't manipulate key columns, not those that
* serendipitously arrive at the same key values.
*/
- if (!bms_overlap(modified_attrs, key_attrs))
+ if (!bms_overlap(mix_attrs, pk_attrs))
{
*lockmode = LockTupleNoKeyExclusive;
mxact_status = MultiXactStatusNoKeyUpdate;
@@ -3488,17 +3355,10 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
key_intact = false;
}
- /*
- * Note: beyond this point, use oldtup not otid to refer to old tuple.
- * otid may very well point at newtup->t_self, which we will overwrite
- * with the new tuple's location, so there's great risk of confusion if we
- * use otid anymore.
- */
-
l2:
checked_lockers = false;
locker_remains = false;
- result = HeapTupleSatisfiesUpdate(&oldtup, cid, buffer);
+ result = HeapTupleSatisfiesUpdate(oldtup, cid, buffer);
/* see below about the "no wait" case */
Assert(result != TM_BeingModified || wait);
@@ -3530,8 +3390,8 @@ l2:
*/
/* must copy state data before unlocking buffer */
- xwait = HeapTupleHeaderGetRawXmax(oldtup.t_data);
- infomask = oldtup.t_data->t_infomask;
+ xwait = HeapTupleHeaderGetRawXmax(oldtup->t_data);
+ infomask = oldtup->t_data->t_infomask;
/*
* Now we have to do something about the existing locker. If it's a
@@ -3571,13 +3431,12 @@ l2:
* requesting a lock and already have one; avoids deadlock).
*/
if (!current_is_member)
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &oldtup->t_self, *lockmode,
LockWaitBlock, &have_tuple_lock);
/* wait for multixact */
MultiXactIdWait((MultiXactId) xwait, mxact_status, infomask,
- relation, &oldtup.t_self, XLTW_Update,
- &remain);
+ relation, &oldtup->t_self, XLTW_Update, &remain);
checked_lockers = true;
locker_remains = remain != 0;
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
@@ -3587,9 +3446,9 @@ l2:
* could update this tuple before we get to this point. Check
* for xmax change, and start over if so.
*/
- if (xmax_infomask_changed(oldtup.t_data->t_infomask,
+ if (xmax_infomask_changed(oldtup->t_data->t_infomask,
infomask) ||
- !TransactionIdEquals(HeapTupleHeaderGetRawXmax(oldtup.t_data),
+ !TransactionIdEquals(HeapTupleHeaderGetRawXmax(oldtup->t_data),
xwait))
goto l2;
}
@@ -3614,8 +3473,8 @@ l2:
* before this one, which are important to keep in case this
* subxact aborts.
*/
- if (!HEAP_XMAX_IS_LOCKED_ONLY(oldtup.t_data->t_infomask))
- update_xact = HeapTupleGetUpdateXid(oldtup.t_data);
+ if (!HEAP_XMAX_IS_LOCKED_ONLY(oldtup->t_data->t_infomask))
+ update_xact = HeapTupleGetUpdateXid(oldtup->t_data);
else
update_xact = InvalidTransactionId;
@@ -3656,9 +3515,9 @@ l2:
* lock.
*/
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &oldtup->t_self, *lockmode,
LockWaitBlock, &have_tuple_lock);
- XactLockTableWait(xwait, relation, &oldtup.t_self,
+ XactLockTableWait(xwait, relation, &oldtup->t_self,
XLTW_Update);
checked_lockers = true;
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
@@ -3668,20 +3527,20 @@ l2:
* other xact could update this tuple before we get to this point.
* Check for xmax change, and start over if so.
*/
- if (xmax_infomask_changed(oldtup.t_data->t_infomask, infomask) ||
+ if (xmax_infomask_changed(oldtup->t_data->t_infomask, infomask) ||
!TransactionIdEquals(xwait,
- HeapTupleHeaderGetRawXmax(oldtup.t_data)))
+ HeapTupleHeaderGetRawXmax(oldtup->t_data)))
goto l2;
/* Otherwise check if it committed or aborted */
- UpdateXmaxHintBits(oldtup.t_data, buffer, xwait);
- if (oldtup.t_data->t_infomask & HEAP_XMAX_INVALID)
+ UpdateXmaxHintBits(oldtup->t_data, buffer, xwait);
+ if (oldtup->t_data->t_infomask & HEAP_XMAX_INVALID)
can_continue = true;
}
if (can_continue)
result = TM_Ok;
- else if (!ItemPointerEquals(&oldtup.t_self, &oldtup.t_data->t_ctid))
+ else if (!ItemPointerEquals(&oldtup->t_self, &oldtup->t_data->t_ctid))
result = TM_Updated;
else
result = TM_Deleted;
@@ -3694,39 +3553,33 @@ l2:
result == TM_Updated ||
result == TM_Deleted ||
result == TM_BeingModified);
- Assert(!(oldtup.t_data->t_infomask & HEAP_XMAX_INVALID));
+ Assert(!(oldtup->t_data->t_infomask & HEAP_XMAX_INVALID));
Assert(result != TM_Updated ||
- !ItemPointerEquals(&oldtup.t_self, &oldtup.t_data->t_ctid));
+ !ItemPointerEquals(&oldtup->t_self, &oldtup->t_data->t_ctid));
}
if (crosscheck != InvalidSnapshot && result == TM_Ok)
{
/* Perform additional check for transaction-snapshot mode RI updates */
- if (!HeapTupleSatisfiesVisibility(&oldtup, crosscheck, buffer))
+ if (!HeapTupleSatisfiesVisibility(oldtup, crosscheck, buffer))
result = TM_Updated;
}
if (result != TM_Ok)
{
- tmfd->ctid = oldtup.t_data->t_ctid;
- tmfd->xmax = HeapTupleHeaderGetUpdateXid(oldtup.t_data);
+ tmfd->ctid = oldtup->t_data->t_ctid;
+ tmfd->xmax = HeapTupleHeaderGetUpdateXid(oldtup->t_data);
if (result == TM_SelfModified)
- tmfd->cmax = HeapTupleHeaderGetCmax(oldtup.t_data);
+ tmfd->cmax = HeapTupleHeaderGetCmax(oldtup->t_data);
else
tmfd->cmax = InvalidCommandId;
UnlockReleaseBuffer(buffer);
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
- if (vmbuffer != InvalidBuffer)
- ReleaseBuffer(vmbuffer);
+ UnlockTupleTuplock(relation, &oldtup->t_self, *lockmode);
+ if (*vmbuffer != InvalidBuffer)
+ ReleaseBuffer(*vmbuffer);
*update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
return result;
}
@@ -3739,10 +3592,10 @@ l2:
* tuple has been locked or updated under us, but hopefully it won't
* happen very often.
*/
- if (vmbuffer == InvalidBuffer && PageIsAllVisible(page))
+ if (*vmbuffer == InvalidBuffer && PageIsAllVisible(page))
{
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- visibilitymap_pin(relation, block, &vmbuffer);
+ visibilitymap_pin(relation, block, vmbuffer);
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
goto l2;
}
@@ -3753,9 +3606,9 @@ l2:
* If the tuple we're updating is locked, we need to preserve the locking
* info in the old tuple's Xmax. Prepare a new Xmax value for this.
*/
- compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
- oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2,
+ compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup->t_data),
+ oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2,
xid, *lockmode, true,
&xmax_old_tuple, &infomask_old_tuple,
&infomask2_old_tuple);
@@ -3767,12 +3620,12 @@ l2:
* tuple. (In rare cases that might also be InvalidTransactionId and yet
* not have the HEAP_XMAX_INVALID bit set; that's fine.)
*/
- if ((oldtup.t_data->t_infomask & HEAP_XMAX_INVALID) ||
- HEAP_LOCKED_UPGRADED(oldtup.t_data->t_infomask) ||
+ if ((oldtup->t_data->t_infomask & HEAP_XMAX_INVALID) ||
+ HEAP_LOCKED_UPGRADED(oldtup->t_data->t_infomask) ||
(checked_lockers && !locker_remains))
xmax_new_tuple = InvalidTransactionId;
else
- xmax_new_tuple = HeapTupleHeaderGetRawXmax(oldtup.t_data);
+ xmax_new_tuple = HeapTupleHeaderGetRawXmax(oldtup->t_data);
if (!TransactionIdIsValid(xmax_new_tuple))
{
@@ -3787,7 +3640,7 @@ l2:
* Note that since we're doing an update, the only possibility is that
* the lockers had FOR KEY SHARE lock.
*/
- if (oldtup.t_data->t_infomask & HEAP_XMAX_IS_MULTI)
+ if (oldtup->t_data->t_infomask & HEAP_XMAX_IS_MULTI)
{
GetMultiXactIdHintBits(xmax_new_tuple, &infomask_new_tuple,
&infomask2_new_tuple);
@@ -3815,7 +3668,7 @@ l2:
* Replace cid with a combo CID if necessary. Note that we already put
* the plain cid into the new tuple.
*/
- HeapTupleHeaderAdjustCmax(oldtup.t_data, &cid, &iscombo);
+ HeapTupleHeaderAdjustCmax(oldtup->t_data, &cid, &iscombo);
/*
* If the toaster needs to be activated, OR if the new tuple will not fit
@@ -3832,12 +3685,12 @@ l2:
relation->rd_rel->relkind != RELKIND_MATVIEW)
{
/* toast table entries should never be recursively toasted */
- Assert(!HeapTupleHasExternal(&oldtup));
+ Assert(!HeapTupleHasExternal(oldtup));
Assert(!HeapTupleHasExternal(newtup));
need_toast = false;
}
else
- need_toast = (HeapTupleHasExternal(&oldtup) ||
+ need_toast = (HeapTupleHasExternal(oldtup) ||
HeapTupleHasExternal(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
@@ -3870,9 +3723,9 @@ l2:
* updating, because the potentially created multixact would otherwise
* be wrong.
*/
- compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
- oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2,
+ compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup->t_data),
+ oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2,
xid, *lockmode, false,
&xmax_lock_old_tuple, &infomask_lock_old_tuple,
&infomask2_lock_old_tuple);
@@ -3882,18 +3735,18 @@ l2:
START_CRIT_SECTION();
/* Clear obsolete visibility flags ... */
- oldtup.t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
- oldtup.t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
- HeapTupleClearHotUpdated(&oldtup);
+ oldtup->t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
+ oldtup->t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
+ HeapTupleClearHotUpdated(oldtup);
/* ... and store info about transaction updating this tuple */
Assert(TransactionIdIsValid(xmax_lock_old_tuple));
- HeapTupleHeaderSetXmax(oldtup.t_data, xmax_lock_old_tuple);
- oldtup.t_data->t_infomask |= infomask_lock_old_tuple;
- oldtup.t_data->t_infomask2 |= infomask2_lock_old_tuple;
- HeapTupleHeaderSetCmax(oldtup.t_data, cid, iscombo);
+ HeapTupleHeaderSetXmax(oldtup->t_data, xmax_lock_old_tuple);
+ oldtup->t_data->t_infomask |= infomask_lock_old_tuple;
+ oldtup->t_data->t_infomask2 |= infomask2_lock_old_tuple;
+ HeapTupleHeaderSetCmax(oldtup->t_data, cid, iscombo);
/* temporarily make it look not-updated, but locked */
- oldtup.t_data->t_ctid = oldtup.t_self;
+ oldtup->t_data->t_ctid = oldtup->t_self;
/*
* Clear all-frozen bit on visibility map if needed. We could
@@ -3902,7 +3755,7 @@ l2:
* worthwhile.
*/
if (PageIsAllVisible(page) &&
- visibilitymap_clear(relation, block, vmbuffer,
+ visibilitymap_clear(relation, block, *vmbuffer,
VISIBILITYMAP_ALL_FROZEN))
cleared_all_frozen = true;
@@ -3916,10 +3769,10 @@ l2:
XLogBeginInsert();
XLogRegisterBuffer(0, buffer, REGBUF_STANDARD);
- xlrec.offnum = ItemPointerGetOffsetNumber(&oldtup.t_self);
+ xlrec.offnum = ItemPointerGetOffsetNumber(&oldtup->t_self);
xlrec.xmax = xmax_lock_old_tuple;
- xlrec.infobits_set = compute_infobits(oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2);
+ xlrec.infobits_set = compute_infobits(oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2);
xlrec.flags =
cleared_all_frozen ? XLH_LOCK_ALL_FROZEN_CLEARED : 0;
XLogRegisterData(&xlrec, SizeOfHeapLock);
@@ -3941,7 +3794,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = heap_toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = heap_toast_insert_or_update(relation, newtup, oldtup, 0);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
@@ -3977,20 +3830,20 @@ l2:
/* It doesn't fit, must use RelationGetBufferForTuple. */
newbuf = RelationGetBufferForTuple(relation, heaptup->t_len,
buffer, 0, NULL,
- &vmbuffer_new, &vmbuffer,
+ &vmbuffer_new, vmbuffer,
0);
/* We're all done. */
break;
}
/* Acquire VM page pin if needed and we don't have it. */
- if (vmbuffer == InvalidBuffer && PageIsAllVisible(page))
- visibilitymap_pin(relation, block, &vmbuffer);
+ if (*vmbuffer == InvalidBuffer && PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, vmbuffer);
/* Re-acquire the lock on the old tuple's page. */
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
/* Re-check using the up-to-date free space */
pagefree = PageGetHeapFreeSpace(page);
if (newtupsize > pagefree ||
- (vmbuffer == InvalidBuffer && PageIsAllVisible(page)))
+ (*vmbuffer == InvalidBuffer && PageIsAllVisible(page)))
{
/*
* Rats, it doesn't fit anymore, or somebody just now set the
@@ -4028,7 +3881,7 @@ l2:
* will include checking the relation level, there is no benefit to a
* separate check for the new tuple.
*/
- CheckForSerializableConflictIn(relation, &oldtup.t_self,
+ CheckForSerializableConflictIn(relation, &oldtup->t_self,
BufferGetBlockNumber(buffer));
/*
@@ -4036,7 +3889,6 @@ l2:
* has enough space for the new tuple. If they are the same buffer, only
* one pin is held.
*/
-
if (newbuf == buffer)
{
/*
@@ -4044,7 +3896,7 @@ l2:
* to do a HOT update. Check if any of the index columns have been
* changed.
*/
- if (!bms_overlap(modified_attrs, hot_attrs))
+ if (!bms_overlap(mix_attrs, hot_attrs))
{
use_hot_update = true;
@@ -4055,7 +3907,7 @@ l2:
* indexes if the columns were updated, or we may fail to detect
* e.g. value bound changes in BRIN minmax indexes.
*/
- if (bms_overlap(modified_attrs, sum_attrs))
+ if (bms_overlap(mix_attrs, sum_attrs))
summarized_update = true;
}
}
@@ -4072,10 +3924,8 @@ l2:
* logged. Pass old key required as true only if the replica identity key
* columns are modified or it has external data.
*/
- old_key_tuple = ExtractReplicaIdentity(relation, &oldtup,
- bms_overlap(modified_attrs, id_attrs) ||
- id_has_external,
- &old_key_copied);
+ old_key_tuple = ExtractReplicaIdentity(relation, oldtup, rid_attrs,
+ rep_id_key_required, &old_key_copied);
/* NO EREPORT(ERROR) from here till changes are logged */
START_CRIT_SECTION();
@@ -4097,7 +3947,7 @@ l2:
if (use_hot_update)
{
/* Mark the old tuple as HOT-updated */
- HeapTupleSetHotUpdated(&oldtup);
+ HeapTupleSetHotUpdated(oldtup);
/* And mark the new tuple as heap-only */
HeapTupleSetHeapOnly(heaptup);
/* Mark the caller's copy too, in case different from heaptup */
@@ -4106,7 +3956,7 @@ l2:
else
{
/* Make sure tuples are correctly marked as not-HOT */
- HeapTupleClearHotUpdated(&oldtup);
+ HeapTupleClearHotUpdated(oldtup);
HeapTupleClearHeapOnly(heaptup);
HeapTupleClearHeapOnly(newtup);
}
@@ -4115,17 +3965,17 @@ l2:
/* Clear obsolete visibility flags, possibly set by ourselves above... */
- oldtup.t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
- oldtup.t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
+ oldtup->t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
+ oldtup->t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
/* ... and store info about transaction updating this tuple */
Assert(TransactionIdIsValid(xmax_old_tuple));
- HeapTupleHeaderSetXmax(oldtup.t_data, xmax_old_tuple);
- oldtup.t_data->t_infomask |= infomask_old_tuple;
- oldtup.t_data->t_infomask2 |= infomask2_old_tuple;
- HeapTupleHeaderSetCmax(oldtup.t_data, cid, iscombo);
+ HeapTupleHeaderSetXmax(oldtup->t_data, xmax_old_tuple);
+ oldtup->t_data->t_infomask |= infomask_old_tuple;
+ oldtup->t_data->t_infomask2 |= infomask2_old_tuple;
+ HeapTupleHeaderSetCmax(oldtup->t_data, cid, iscombo);
/* record address of new tuple in t_ctid of old one */
- oldtup.t_data->t_ctid = heaptup->t_self;
+ oldtup->t_data->t_ctid = heaptup->t_self;
/* clear PD_ALL_VISIBLE flags, reset all visibilitymap bits */
if (PageIsAllVisible(BufferGetPage(buffer)))
@@ -4133,7 +3983,7 @@ l2:
all_visible_cleared = true;
PageClearAllVisible(BufferGetPage(buffer));
visibilitymap_clear(relation, BufferGetBlockNumber(buffer),
- vmbuffer, VISIBILITYMAP_VALID_BITS);
+ *vmbuffer, VISIBILITYMAP_VALID_BITS);
}
if (newbuf != buffer && PageIsAllVisible(BufferGetPage(newbuf)))
{
@@ -4158,12 +4008,12 @@ l2:
*/
if (RelationIsAccessibleInLogicalDecoding(relation))
{
- log_heap_new_cid(relation, &oldtup);
+ log_heap_new_cid(relation, oldtup);
log_heap_new_cid(relation, heaptup);
}
recptr = log_heap_update(relation, buffer,
- newbuf, &oldtup, heaptup,
+ newbuf, oldtup, heaptup,
old_key_tuple,
all_visible_cleared,
all_visible_cleared_new);
@@ -4188,7 +4038,7 @@ l2:
* both tuple versions in one call to inval.c so we can avoid redundant
* sinval messages.)
*/
- CacheInvalidateHeapTuple(relation, &oldtup, heaptup);
+ CacheInvalidateHeapTuple(relation, oldtup, heaptup);
/* Now we can release the buffer(s) */
if (newbuf != buffer)
@@ -4196,14 +4046,14 @@ l2:
ReleaseBuffer(buffer);
if (BufferIsValid(vmbuffer_new))
ReleaseBuffer(vmbuffer_new);
- if (BufferIsValid(vmbuffer))
- ReleaseBuffer(vmbuffer);
+ if (BufferIsValid(*vmbuffer))
+ ReleaseBuffer(*vmbuffer);
/*
* Release the lmgr tuple lock, if we had it.
*/
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &oldtup->t_self, *lockmode);
pgstat_count_heap_update(relation, use_hot_update, newbuf != buffer);
@@ -4236,13 +4086,6 @@ l2:
if (old_key_tuple != NULL && old_key_copied)
heap_freetuple(old_key_tuple);
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
-
return TM_Ok;
}
@@ -4251,7 +4094,7 @@ l2:
* Confirm adequate lock held during heap_update(), per rules from
* README.tuplock section "Locking to write inplace-updated tables".
*/
-static void
+void
check_lock_if_inplace_updateable_rel(Relation relation,
const ItemPointerData *otid,
HeapTuple newtup)
@@ -4423,7 +4266,7 @@ heap_attr_equals(TupleDesc tupdesc, int attrnum, Datum value1, Datum value2,
* listed as interesting) of the old tuple is a member of external_cols and is
* stored externally.
*/
-static Bitmapset *
+Bitmapset *
HeapDetermineColumnsInfo(Relation relation,
Bitmapset *interesting_cols,
Bitmapset *external_cols,
@@ -4506,25 +4349,175 @@ HeapDetermineColumnsInfo(Relation relation,
}
/*
- * simple_heap_update - replace a tuple
- *
- * This routine may be used to update a tuple when concurrent updates of
- * the target tuple are not expected (for example, because we have a lock
- * on the relation associated with the tuple). Any failure is reported
- * via ereport().
+ * This routine may be used to update a tuple when concurrent updates of the
+ * target tuple are not expected (for example, because we have a lock on the
+ * relation associated with the tuple). Any failure is reported via ereport().
*/
void
-simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup,
+simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TM_FailureData tmfd;
LockTupleMode lockmode;
+ Buffer buffer;
+ Buffer vmbuffer = InvalidBuffer;
+ Page page;
+ BlockNumber block;
+ Bitmapset *hot_attrs,
+ *sum_attrs,
+ *pk_attrs,
+ *rid_attrs,
+ *mix_attrs,
+ *idx_attrs;
+ ItemId lp;
+ HeapTupleData oldtup;
+ bool rep_id_key_required = false;
+
+ Assert(ItemPointerIsValid(otid));
+
+ /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
+ Assert(HeapTupleHeaderGetNatts(tuple->t_data) <=
+ RelationGetNumberOfAttributes(relation));
+
+ /*
+ * Forbid this during a parallel operation, lest it allocate a combo CID.
+ * Other workers might need that combo CID for visibility checks, and we
+ * have no provision for broadcasting it to them.
+ */
+ if (IsInParallelMode())
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
+ errmsg("cannot update tuples during a parallel operation")));
+
+#ifdef USE_ASSERT_CHECKING
+ check_lock_if_inplace_updateable_rel(relation, otid, tuple);
+#endif
+
+ /*
+ * Fetch the list of attributes to be checked for various operations.
+ *
+ * For HOT considerations, this is wasted effort if we fail to update or
+ * have to put the new tuple on a different page. But we must compute the
+ * list before obtaining buffer lock --- in the worst case, if we are
+ * doing an update on one of the relevant system catalogs, we could
+ * deadlock if we try to fetch the list later. In any case, the relcache
+ * caches the data so this is usually pretty cheap.
+ *
+ * We also need columns used by the replica identity and columns that are
+ * considered the "key" of rows in the table.
+ *
+ * Note that we get copies of each bitmap, so we need not worry about
+ * relcache flush happening midway through.
+ */
+ hot_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_HOT_BLOCKING);
+ sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+ pk_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
+ rid_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
+ idx_attrs = bms_copy(hot_attrs);
+ idx_attrs = bms_add_members(idx_attrs, sum_attrs);
+ idx_attrs = bms_add_members(idx_attrs, pk_attrs);
+ idx_attrs = bms_add_members(idx_attrs, rid_attrs);
+
+ block = ItemPointerGetBlockNumber(otid);
+ INJECTION_POINT("heap_update-before-pin", NULL);
+ buffer = ReadBuffer(relation, block);
+ page = BufferGetPage(buffer);
+
+ /*
+ * Before locking the buffer, pin the visibility map page if it appears to
+ * be necessary. Since we haven't got the lock yet, someone else might be
+ * in the middle of changing this, so we'll need to recheck after we have
+ * the lock.
+ */
+ if (PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, &vmbuffer);
+
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+
+ lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
+
+ /*
+ * Usually, a buffer pin and/or snapshot blocks pruning of otid, ensuring
+ * we see LP_NORMAL here. When the otid origin is a syscache, we may have
+ * neither a pin nor a snapshot. Hence, we may see other LP_ states, each
+ * of which indicates concurrent pruning.
+ *
+ * Failing with TM_Updated would be most accurate. However, unlike other
+ * TM_Updated scenarios, we don't know the successor ctid in LP_UNUSED and
+ * LP_DEAD cases. While the distinction between TM_Updated and TM_Deleted
+ * does matter to SQL statements UPDATE and MERGE, those SQL statements
+ * hold a snapshot that ensures LP_NORMAL. Hence, the choice between
+ * TM_Updated and TM_Deleted affects only the wording of error messages.
+ * Settle on TM_Deleted, for two reasons. First, it avoids complicating
+ * the specification of when tmfd->ctid is valid. Second, it creates
+ * error log evidence that we took this branch.
+ *
+ * Since it's possible to see LP_UNUSED at otid, it's also possible to see
+ * LP_NORMAL for a tuple that replaced LP_UNUSED. If it's a tuple for an
+ * unrelated row, we'll fail with "duplicate key value violates unique".
+ * XXX if otid is the live, newer version of the newtup row, we'll discard
+ * changes originating in versions of this catalog row after the version
+ * the caller got from syscache. See syscache-update-pruned.spec.
+ */
+ if (!ItemIdIsNormal(lp))
+ {
+ Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
+
+ UnlockReleaseBuffer(buffer);
+ if (vmbuffer != InvalidBuffer)
+ ReleaseBuffer(vmbuffer);
+ *update_indexes = TU_None;
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(idx_attrs);
+ /* mix_attrs not yet initialized */
+
+ elog(ERROR, "tuple concurrently deleted");
+
+ return;
+ }
+
+ /*
+ * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
+ * then pass that on to heap_update.
+ */
+ oldtup.t_tableOid = RelationGetRelid(relation);
+ oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
+ oldtup.t_len = ItemIdGetLength(lp);
+ oldtup.t_self = *otid;
+
+ mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
+ &oldtup, tuple, &rep_id_key_required);
+
+ /*
+ * We'll need to WAL log the replica identity attributes if either they
+ * overlap with the modified indexed attributes or, as we've checked for
+ * just now in HeapDetermineColumnsInfo, they were unmodified external
+ * indexed attributes.
+ */
+ rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+
+ result = heap_update(relation, &oldtup, tuple, GetCurrentCommandId(true),
+ InvalidSnapshot, true /* wait for commit */ , &tmfd, &lockmode,
+ buffer, page, block, lp, hot_attrs, sum_attrs, pk_attrs,
+ rid_attrs, mix_attrs, &vmbuffer, rep_id_key_required,
+ update_indexes);
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(mix_attrs);
+ bms_free(idx_attrs);
- result = heap_update(relation, otid, tup,
- GetCurrentCommandId(true), InvalidSnapshot,
- true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
switch (result)
{
case TM_SelfModified:
@@ -9164,12 +9157,11 @@ log_heap_new_cid(Relation relation, HeapTuple tup)
* the same tuple that was passed in.
*/
static HeapTuple
-ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
- bool *copy)
+ExtractReplicaIdentity(Relation relation, HeapTuple tp, Bitmapset *rid_attrs,
+ bool key_required, bool *copy)
{
TupleDesc desc = RelationGetDescr(relation);
char replident = relation->rd_rel->relreplident;
- Bitmapset *idattrs;
HeapTuple key_tuple;
bool nulls[MaxHeapAttributeNumber];
Datum values[MaxHeapAttributeNumber];
@@ -9200,17 +9192,13 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
if (!key_required)
return NULL;
- /* find out the replica identity columns */
- idattrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
-
/*
* If there's no defined replica identity columns, treat as !key_required.
* (This case should not be reachable from heap_update, since that should
* calculate key_required accurately. But heap_delete just passes
* constant true for key_required, so we can hit this case in deletes.)
*/
- if (bms_is_empty(idattrs))
+ if (bms_is_empty(rid_attrs))
return NULL;
/*
@@ -9223,7 +9211,7 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
for (int i = 0; i < desc->natts; i++)
{
if (bms_is_member(i + 1 - FirstLowInvalidHeapAttributeNumber,
- idattrs))
+ rid_attrs))
Assert(!nulls[i]);
else
nulls[i] = true;
@@ -9232,8 +9220,6 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
key_tuple = heap_form_tuple(desc, values, nulls);
*copy = true;
- bms_free(idattrs);
-
/*
* If the tuple, which by here only contains indexed columns, still has
* toasted columns, force them to be inlined. This is somewhat unlikely
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index dd4fe6bf62f..8708af01f8c 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -44,6 +44,7 @@
#include "storage/procarray.h"
#include "storage/smgr.h"
#include "utils/builtins.h"
+#include "utils/injection_point.h"
#include "utils/rel.h"
static void reform_and_rewrite_tuple(HeapTuple tuple,
@@ -312,23 +313,133 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
return heap_delete(relation, tid, cid, crosscheck, wait, tmfd, changingPart);
}
-
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd,
LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
{
+ bool rep_id_key_required = false;
bool shouldFree = true;
HeapTuple tuple = ExecFetchSlotHeapTuple(slot, true, &shouldFree);
+ HeapTupleData oldtup;
+ Buffer buffer;
+ Buffer vmbuffer = InvalidBuffer;
+ Page page;
+ BlockNumber block;
+ ItemId lp;
+ Bitmapset *hot_attrs,
+ *sum_attrs,
+ *pk_attrs,
+ *rid_attrs,
+ *mix_attrs,
+ *idx_attrs;
TM_Result result;
+ Assert(ItemPointerIsValid(otid));
+
+ /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
+ Assert(HeapTupleHeaderGetNatts(tuple->t_data) <=
+ RelationGetNumberOfAttributes(relation));
+
+ /*
+ * Forbid this during a parallel operation, lest it allocate a combo CID.
+ * Other workers might need that combo CID for visibility checks, and we
+ * have no provision for broadcasting it to them.
+ */
+ if (IsInParallelMode())
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
+ errmsg("cannot update tuples during a parallel operation")));
+
+#ifdef USE_ASSERT_CHECKING
+ check_lock_if_inplace_updateable_rel(relation, otid, tuple);
+#endif
+
+ /*
+ * Fetch the list of attributes to be checked for various operations.
+ *
+ * For HOT considerations, this is wasted effort if we fail to update or
+ * have to put the new tuple on a different page. But we must compute the
+ * list before obtaining buffer lock --- in the worst case, if we are
+ * doing an update on one of the relevant system catalogs, we could
+ * deadlock if we try to fetch the list later. In any case, the relcache
+ * caches the data so this is usually pretty cheap.
+ *
+ * We also need columns used by the replica identity and columns that are
+ * considered the "key" of rows in the table.
+ *
+ * Note that we get copies of each bitmap, so we need not worry about
+ * relcache flush happening midway through.
+ */
+ hot_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_HOT_BLOCKING);
+ sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+ pk_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
+ rid_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
+ idx_attrs = bms_copy(hot_attrs);
+ idx_attrs = bms_add_members(idx_attrs, sum_attrs);
+ idx_attrs = bms_add_members(idx_attrs, pk_attrs);
+ idx_attrs = bms_add_members(idx_attrs, rid_attrs);
+
+ block = ItemPointerGetBlockNumber(otid);
+ INJECTION_POINT("heap_update-before-pin", NULL);
+ buffer = ReadBuffer(relation, block);
+ page = BufferGetPage(buffer);
+
+ /*
+ * Before locking the buffer, pin the visibility map page if it appears to
+ * be necessary. Since we haven't got the lock yet, someone else might be
+ * in the middle of changing this, so we'll need to recheck after we have
+ * the lock.
+ */
+ if (PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, &vmbuffer);
+
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+
+ lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
+
+ Assert(ItemIdIsNormal(lp));
+
+ /*
+ * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
+ * then pass that on to heap_update.
+ */
+ oldtup.t_tableOid = RelationGetRelid(relation);
+ oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
+ oldtup.t_len = ItemIdGetLength(lp);
+ oldtup.t_self = *otid;
+
+ mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
+ &oldtup, tuple, &rep_id_key_required);
+
+ /*
+ * We'll need to WAL log the replica identity attributes if either they
+ * overlap with the modified indexed attributes or, as we've checked for
+ * just now in HeapDetermineColumnsInfo, they were unmodified external
+ * indexed attributes.
+ */
+ rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
- result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
- tmfd, lockmode, update_indexes);
+ result = heap_update(relation, &oldtup, tuple, cid, crosscheck, wait, tmfd, lockmode,
+ buffer, page, block, lp, hot_attrs, sum_attrs, pk_attrs,
+ rid_attrs, mix_attrs, &vmbuffer, rep_id_key_required, update_indexes);
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(mix_attrs);
+ bms_free(idx_attrs);
+
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index f7e4ae3843c..9398216d5d9 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -364,11 +364,13 @@ extern TM_Result heap_delete(Relation relation, const ItemPointerData *tid,
TM_FailureData *tmfd, bool changingPart);
extern void heap_finish_speculative(Relation relation, const ItemPointerData *tid);
extern void heap_abort_speculative(Relation relation, const ItemPointerData *tid);
-extern TM_Result heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+extern TM_Result heap_update(Relation relation, HeapTupleData *oldtup,
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
+ Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
+ Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
+ Bitmapset *mix_attrs, Buffer *vmbuffer,
+ bool rep_id_key_required, TU_UpdateIndexes *update_indexes);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -430,6 +432,18 @@ extern void log_heap_prune_and_freeze(Relation relation, Buffer buffer,
OffsetNumber *dead, int ndead,
OffsetNumber *unused, int nunused);
+/* in heap/heapam.c */
+extern Bitmapset *HeapDetermineColumnsInfo(Relation relation,
+ Bitmapset *interesting_cols,
+ Bitmapset *external_cols,
+ HeapTuple oldtup, HeapTuple newtup,
+ bool *has_external);
+#ifdef USE_ASSERT_CHECKING
+extern void check_lock_if_inplace_updateable_rel(Relation relation,
+ const ItemPointerData *otid,
+ HeapTuple newtup);
+#endif
+
/* in heap/vacuumlazy.c */
extern void heap_vacuum_rel(Relation rel,
const VacuumParams params, BufferAccessStrategy bstrategy);
--
2.51.2
[application/octet-stream] v26-0004-Identify-if-partial-indexes-are-impacted-by-an-u.patch (3.8K, 3-v26-0004-Identify-if-partial-indexes-are-impacted-by-an-u.patch)
download | inline diff:
From 516f799c70e3d6bdf123afc9ab7650bb0edc4de6 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Fri, 5 Dec 2025 13:42:13 -0500
Subject: [PATCH v26 4/4] Identify if partial indexes are impacted by an
update.
The executor now determines which, if any, attributes that are indexed
are both modified and force new index tuples to be inserted ahead of
calling into the table AM update function. Prior to this commit the
test for partial indexes happened after table update, this changes that
to before so that in cases where the before and after tuples both lie
outside the predicate the attributes for the predicate are not included
in the "modified indexed attributes" bitmapset.
---
src/backend/executor/nodeModifyTable.c | 53 ++++++++++++++++++++++++--
1 file changed, 49 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 52a74479502..c55d84f86f8 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -226,9 +226,11 @@ ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
Bitmapset *m_attrs = NULL; /* (possibly) modified indexed attrs */
Bitmapset *p_attrs = NULL; /* (possibly) modified predicate attrs */
Bitmapset *u_attrs = NULL; /* unmodified indexed attrs */
+ Bitmapset *pre_attrs = indexInfo->ii_PredicateAttrs;
bool has_am_compare = (amroutine->amcomparedatums != NULL);
bool supports_ios = (amroutine->amcanreturn != NULL);
bool is_partial = (indexInfo->ii_Predicate != NIL);
+ TupleTableSlot *save_scantuple;
ExprContext *econtext = GetPerTupleExprContext(estate);
int num_datums = supports_ios ?
indexInfo->ii_NumIndexAttrs : indexInfo->ii_NumIndexKeyAttrs;
@@ -237,9 +239,51 @@ ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
if (bms_is_subset(indexInfo->ii_IndexedAttrs, mix_attrs))
continue;
- /* Add partial index attributes */
- if (is_partial)
- p_attrs = bms_add_members(p_attrs, indexInfo->ii_PredicateAttrs);
+ /* Checking partial at this point isn't viable when we're serializable */
+ if (is_partial && IsolationIsSerializable())
+ {
+ p_attrs = bms_add_members(p_attrs, pre_attrs);
+ }
+ /* Check partial index predicate */
+ else if (is_partial)
+ {
+ ExprState *pstate;
+ bool old_qualifies,
+ new_qualifies;
+
+
+ if (!indexInfo->ii_CheckedPredicate)
+ pstate = ExecPrepareQual(indexInfo->ii_Predicate, estate);
+ else
+ pstate = indexInfo->ii_PredicateState;
+
+ save_scantuple = econtext->ecxt_scantuple;
+
+ econtext->ecxt_scantuple = old_tts;
+ old_qualifies = ExecQual(pstate, econtext);
+
+ econtext->ecxt_scantuple = new_tts;
+ new_qualifies = ExecQual(pstate, econtext);
+
+ econtext->ecxt_scantuple = save_scantuple;
+
+ indexInfo->ii_CheckedPredicate = true;
+ indexInfo->ii_PredicateState = pstate;
+ indexInfo->ii_PredicateSatisfied = new_qualifies;
+
+ /* Both outside predicate, index doesn't need update */
+ if (!old_qualifies && !new_qualifies)
+ continue;
+
+ /* A transition means we need to update the index */
+ if (old_qualifies != new_qualifies)
+ p_attrs = bms_copy(pre_attrs);
+
+ /*
+ * When both are within the predicate we must update this index,
+ * but only if one of the index key attributes changed.
+ */
+ }
/* Compare the index datums for equality */
for (int j = 0; j < num_datums; j++)
@@ -275,11 +319,12 @@ ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
*/
else if (rel_attrnum == 0)
{
- TupleTableSlot *save_scantuple = econtext->ecxt_scantuple;
Oid expr_type_oid;
Expr *expr = (Expr *) list_nth(indexInfo->ii_Expressions, nth_expr);
ExprState *state;
+ save_scantuple = econtext->ecxt_scantuple;
+
if (indexInfo->ii_ExpressionsState == NIL)
{
/* First time through, set up expression evaluation state */
--
2.51.2
[application/octet-stream] v26-0003-Replace-index_unchanged_by_update-with-ri_Change.patch (8.4K, 4-v26-0003-Replace-index_unchanged_by_update-with-ri_Change.patch)
download | inline diff:
From 99659e8c9398790ea3d1bd34d46013498bff2f52 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Fri, 31 Oct 2025 14:55:25 -0400
Subject: [PATCH v26 3/4] Replace index_unchanged_by_update() with
ri_ChangedIndexedCols
In execIndexing on updates we'd like to pass a hint to the indexing code
when the indexed attributes are unchanged. This commit replaces the now
redundant code in index_unchanged_by_update() with the same information
found earlier in ExecWhichIndexesRequireUpdates() and stashed in
ri_ChangedIndexedCols.
---
src/backend/catalog/toasting.c | 2 -
src/backend/executor/execIndexing.c | 156 +---------------------------
src/backend/nodes/makefuncs.c | 2 -
src/include/nodes/execnodes.h | 4 -
4 files changed, 1 insertion(+), 163 deletions(-)
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 9d7cb4438d5..c665aa744b3 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -304,8 +304,6 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
indexInfo->ii_Unique = true;
indexInfo->ii_NullsNotDistinct = false;
indexInfo->ii_ReadyForInserts = true;
- indexInfo->ii_CheckedUnchanged = false;
- indexInfo->ii_IndexUnchanged = false;
indexInfo->ii_Concurrent = false;
indexInfo->ii_BrokenHotChain = false;
indexInfo->ii_ParallelWorkers = 0;
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 1b17d8d135f..a634ebd49aa 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -143,11 +143,6 @@ static bool check_exclusion_or_unique_constraint(Relation heap, Relation index,
static bool index_recheck_constraint(Relation index, const Oid *constr_procs,
const Datum *existing_values, const bool *existing_isnull,
const Datum *new_values);
-static bool index_unchanged_by_update(ResultRelInfo *resultRelInfo,
- EState *estate, IndexInfo *indexInfo,
- Relation indexRelation);
-static bool index_expression_changed_walker(Node *node,
- Bitmapset *allUpdatedCols);
static void ExecWithoutOverlapsNotEmpty(Relation rel, NameData attname, Datum attval,
char typtype, Oid atttypid);
@@ -451,10 +446,7 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* index. If we're being called as part of an UPDATE statement,
* consider if the 'indexUnchanged' = true hint should be passed.
*/
- indexUnchanged = update && index_unchanged_by_update(resultRelInfo,
- estate,
- indexInfo,
- indexRelation);
+ indexUnchanged = update && bms_is_empty(resultRelInfo->ri_ChangedIndexedCols);
satisfiesConstraint =
index_insert(indexRelation, /* index relation */
@@ -1014,152 +1006,6 @@ index_recheck_constraint(Relation index, const Oid *constr_procs,
return true;
}
-/*
- * Check if ExecInsertIndexTuples() should pass indexUnchanged hint.
- *
- * When the executor performs an UPDATE that requires a new round of index
- * tuples, determine if we should pass 'indexUnchanged' = true hint for one
- * single index.
- */
-static bool
-index_unchanged_by_update(ResultRelInfo *resultRelInfo, EState *estate,
- IndexInfo *indexInfo, Relation indexRelation)
-{
- Bitmapset *updatedCols;
- Bitmapset *extraUpdatedCols;
- Bitmapset *allUpdatedCols;
- bool hasexpression = false;
- List *idxExprs;
-
- /*
- * Check cache first
- */
- if (indexInfo->ii_CheckedUnchanged)
- return indexInfo->ii_IndexUnchanged;
- indexInfo->ii_CheckedUnchanged = true;
-
- /*
- * Check for indexed attribute overlap with updated columns.
- *
- * Only do this for key columns. A change to a non-key column within an
- * INCLUDE index should not be counted here. Non-key column values are
- * opaque payload state to the index AM, a little like an extra table TID.
- *
- * Note that row-level BEFORE triggers won't affect our behavior, since
- * they don't affect the updatedCols bitmaps generally. It doesn't seem
- * worth the trouble of checking which attributes were changed directly.
- */
- updatedCols = ExecGetUpdatedCols(resultRelInfo, estate);
- extraUpdatedCols = ExecGetExtraUpdatedCols(resultRelInfo, estate);
- for (int attr = 0; attr < indexInfo->ii_NumIndexKeyAttrs; attr++)
- {
- int keycol = indexInfo->ii_IndexAttrNumbers[attr];
-
- if (keycol <= 0)
- {
- /*
- * Skip expressions for now, but remember to deal with them later
- * on
- */
- hasexpression = true;
- continue;
- }
-
- if (bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
- updatedCols) ||
- bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
- extraUpdatedCols))
- {
- /* Changed key column -- don't hint for this index */
- indexInfo->ii_IndexUnchanged = false;
- return false;
- }
- }
-
- /*
- * When we get this far and index has no expressions, return true so that
- * index_insert() call will go on to pass 'indexUnchanged' = true hint.
- *
- * The _absence_ of an indexed key attribute that overlaps with updated
- * attributes (in addition to the total absence of indexed expressions)
- * shows that the index as a whole is logically unchanged by UPDATE.
- */
- if (!hasexpression)
- {
- indexInfo->ii_IndexUnchanged = true;
- return true;
- }
-
- /*
- * Need to pass only one bms to expression_tree_walker helper function.
- * Avoid allocating memory in common case where there are no extra cols.
- */
- if (!extraUpdatedCols)
- allUpdatedCols = updatedCols;
- else
- allUpdatedCols = bms_union(updatedCols, extraUpdatedCols);
-
- /*
- * We have to work slightly harder in the event of indexed expressions,
- * but the principle is the same as before: try to find columns (Vars,
- * actually) that overlap with known-updated columns.
- *
- * If we find any matching Vars, don't pass hint for index. Otherwise
- * pass hint.
- */
- idxExprs = RelationGetIndexExpressions(indexRelation);
- hasexpression = index_expression_changed_walker((Node *) idxExprs,
- allUpdatedCols);
- list_free(idxExprs);
- if (extraUpdatedCols)
- bms_free(allUpdatedCols);
-
- if (hasexpression)
- {
- indexInfo->ii_IndexUnchanged = false;
- return false;
- }
-
- /*
- * Deliberately don't consider index predicates. We should even give the
- * hint when result rel's "updated tuple" has no corresponding index
- * tuple, which is possible with a partial index (provided the usual
- * conditions are met).
- */
- indexInfo->ii_IndexUnchanged = true;
- return true;
-}
-
-/*
- * Indexed expression helper for index_unchanged_by_update().
- *
- * Returns true when Var that appears within allUpdatedCols located.
- */
-static bool
-index_expression_changed_walker(Node *node, Bitmapset *allUpdatedCols)
-{
- if (node == NULL)
- return false;
-
- if (IsA(node, Var))
- {
- Var *var = (Var *) node;
-
- if (bms_is_member(var->varattno - FirstLowInvalidHeapAttributeNumber,
- allUpdatedCols))
- {
- /* Var was updated -- indicates that we should not hint */
- return true;
- }
-
- /* Still haven't found a reason to not pass the hint */
- return false;
- }
-
- return expression_tree_walker(node, index_expression_changed_walker,
- allUpdatedCols);
-}
-
/*
* ExecWithoutOverlapsNotEmpty - raise an error if the tuple has an empty
* range or multirange in the given attribute.
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index 1f1364f9df9..e9a53b95caf 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -845,8 +845,6 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
n->ii_Unique = unique;
n->ii_NullsNotDistinct = nulls_not_distinct;
n->ii_ReadyForInserts = isready;
- n->ii_CheckedUnchanged = false;
- n->ii_IndexUnchanged = false;
n->ii_Concurrent = concurrent;
n->ii_Summarizing = summarizing;
n->ii_WithoutOverlaps = withoutoverlaps;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index dfc93c2cc98..901162bf09d 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -216,10 +216,6 @@ typedef struct IndexInfo
bool ii_NullsNotDistinct;
/* is it valid for inserts? */
bool ii_ReadyForInserts;
- /* IndexUnchanged status determined yet? */
- bool ii_CheckedUnchanged;
- /* aminsert hint, cached for retail inserts */
- bool ii_IndexUnchanged;
/* are we doing a concurrent index build? */
bool ii_Concurrent;
/* did we detect any broken HOT chains? */
--
2.51.2
[application/octet-stream] v26-0002-Track-changed-indexed-columns-in-the-executor-du.patch (113.7K, 5-v26-0002-Track-changed-indexed-columns-in-the-executor-du.patch)
download | inline diff:
From 4e50176da1b03e57b37738d9beb03653ed03a1ad Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 26 Oct 2025 10:49:25 -0400
Subject: [PATCH v26 2/4] Track changed indexed columns in the executor during
UPDATEs
Refactor executor update logic to determine which indexed columns have
actually changed during an UPDATE operation rather than leaving this up
to HeapDetermineColumnsInfo() in heap_update().
ExecWhichIndexesRequireUpdates() replaces HeapDeterminesColumnsInfo()
when invoked from the table AM API via heapam_tuple_update(). The
test for equality remains datumIsEqual() as before.
This change necessitated some logic changes in execReplication() as it
performs updates now must provide the set of attributes that are both
changed and referenced by indexes. Luckilly, this is available within
calls to slot_modify_data() where LogicalRepTupleData is processed and
has a record of updated attributes. In this case rather than using
ExecWhichIndexesRequireUpdates() we can preseve what slot_modify_data()
identifies as the modified set and then intersect that with the set of
indexes on the relation and get the correct set of modified indexed
attributes required on heap_update().
This commit also extends the role index AMs play determining if they
require an update. A new optional index AM API, amcomparedatums(), is
added to allow index access methods to provide custom logic for
comparing datums. Hash and Gin indexes now implement this function. When
not implemented the executor will compare TupleTableSlot datum for
equality using datumIsEqual() as before.
Because heap_update() now requires the caller to provide the modified
indexed columns simple_heap_update() has become a tad more complex. It
is only called from CatalogTupleUpdate() which either updates heap
tuples via their Form_XXX or by calling heap_modify_tuple(). In both
cases the caller does know the modified set of attributes, but sadly
those attributes are lost before being provided to simple_heap_update().
Due to that the "simple" path has to (for now) retain the
HeapDetermineColumnsInfo() logic in order for catalog updates to
potentially take the HOT path.
---
src/backend/access/brin/brin.c | 1 +
src/backend/access/gin/ginutil.c | 90 ++-
src/backend/access/hash/hash.c | 44 ++
src/backend/access/heap/heapam.c | 20 +-
src/backend/access/heap/heapam_handler.c | 76 +-
src/backend/access/nbtree/nbtree.c | 1 +
src/backend/access/table/tableam.c | 5 +-
src/backend/bootstrap/bootstrap.c | 8 +
src/backend/catalog/index.c | 57 ++
src/backend/catalog/indexing.c | 16 +-
src/backend/catalog/toasting.c | 4 +
src/backend/executor/execIndexing.c | 41 +-
src/backend/executor/execMain.c | 1 +
src/backend/executor/execReplication.c | 7 +
src/backend/executor/nodeModifyTable.c | 287 +++++++-
src/backend/nodes/bitmapset.c | 4 +
src/backend/nodes/makefuncs.c | 4 +
src/backend/replication/logical/worker.c | 70 +-
src/backend/utils/cache/relcache.c | 15 +
src/include/access/amapi.h | 28 +
src/include/access/gin.h | 3 +
src/include/access/heapam.h | 6 +-
src/include/access/nbtree.h | 4 +
src/include/access/tableam.h | 8 +-
src/include/catalog/index.h | 1 +
src/include/executor/executor.h | 9 +
src/include/nodes/execnodes.h | 20 +
src/include/utils/rel.h | 1 +
src/include/utils/relcache.h | 1 +
.../expected/insert-conflict-specconflict.out | 20 +
.../regress/expected/heap_hot_updates.out | 650 ++++++++++++++++++
src/test/regress/parallel_schedule | 6 +
src/test/regress/sql/heap_hot_updates.sql | 513 ++++++++++++++
src/tools/pgindent/typedefs.list | 1 +
34 files changed, 1948 insertions(+), 74 deletions(-)
create mode 100644 src/test/regress/expected/heap_hot_updates.out
create mode 100644 src/test/regress/sql/heap_hot_updates.sql
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 26cb75058d1..e0388271614 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -290,6 +290,7 @@ brinhandler(PG_FUNCTION_ARGS)
amroutine->amproperty = NULL;
amroutine->ambuildphasename = NULL;
amroutine->amvalidate = brinvalidate;
+ amroutine->amcomparedatums = NULL;
amroutine->amadjustmembers = NULL;
amroutine->ambeginscan = brinbeginscan;
amroutine->amrescan = brinrescan;
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index 605f80aad39..fd1d365c828 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -26,6 +26,7 @@
#include "storage/indexfsm.h"
#include "utils/builtins.h"
#include "utils/index_selfuncs.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/typcache.h"
@@ -78,6 +79,7 @@ ginhandler(PG_FUNCTION_ARGS)
amroutine->amproperty = NULL;
amroutine->ambuildphasename = ginbuildphasename;
amroutine->amvalidate = ginvalidate;
+ amroutine->amcomparedatums = gincomparedatums;
amroutine->amadjustmembers = ginadjustmembers;
amroutine->ambeginscan = ginbeginscan;
amroutine->amrescan = ginrescan;
@@ -477,13 +479,6 @@ cmpEntries(const void *a, const void *b, void *arg)
return res;
}
-
-/*
- * Extract the index key values from an indexable item
- *
- * The resulting key values are sorted, and any duplicates are removed.
- * This avoids generating redundant index entries.
- */
Datum *
ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
@@ -729,3 +724,84 @@ ginbuildphasename(int64 phasenum)
return NULL;
}
}
+
+/*
+ * gincomparedatums - Compare datums to determine if they produce identical keys
+ *
+ * This function extracts keys from both old_datum and new_datum using the
+ * opclass's extractValue function, then compares the extracted key arrays.
+ * Returns true if the key sets are identical (same keys, same counts).
+ *
+ * This enables HOT updates for GIN indexes when the indexed portions of a
+ * value haven't changed, even if the value itself has changed.
+ *
+ * Example: JSONB column with GIN index. If an update changes a non-indexed
+ * key in the JSONB document, the extracted keys are identical and we can
+ * do a HOT update.
+ */
+bool
+gincomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull)
+{
+ GinState ginstate;
+ Datum *old_keys;
+ Datum *new_keys;
+ GinNullCategory *old_categories;
+ GinNullCategory *new_categories;
+ int32 old_nkeys;
+ int32 new_nkeys;
+ MemoryContext tmpcontext;
+ MemoryContext oldcontext;
+ bool result = true;
+
+ /* Handle NULL cases */
+ if (old_isnull != new_isnull)
+ return false;
+ if (old_isnull)
+ return true;
+
+ /* Create temporary context for extraction work */
+ tmpcontext = AllocSetContextCreate(CurrentMemoryContext,
+ "GIN datum comparison",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(tmpcontext);
+
+ initGinState(&ginstate, index);
+
+ /* Extract keys from both datums using existing GIN infrastructure */
+ old_keys = ginExtractEntries(&ginstate, attnum, old_datum, old_isnull,
+ &old_nkeys, &old_categories);
+ new_keys = ginExtractEntries(&ginstate, attnum, new_datum, new_isnull,
+ &new_nkeys, &new_categories);
+
+ /* Different number of keys, definitely different */
+ if (old_nkeys != new_nkeys)
+ {
+ result = false;
+ goto cleanup;
+ }
+
+ /*
+ * Compare the sorted key arrays element-by-element. Since both arrays are
+ * already sorted by ginExtractEntries, we can do a simple O(n)
+ * comparison.
+ */
+ for (int i = 0; i < old_nkeys; i++)
+ {
+ if (ginCompareEntries(&ginstate, attnum,
+ old_keys[i], old_categories[i],
+ new_keys[i], new_categories[i]) != 0)
+ {
+ result = false;
+ break;
+ }
+ }
+
+cleanup:
+ /* Clean up */
+ MemoryContextSwitchTo(oldcontext);
+ MemoryContextDelete(tmpcontext);
+
+ return result;
+}
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index e388252afdc..ada646c6671 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -50,6 +50,10 @@ static void hashbuildCallback(Relation index,
void *state);
+static bool hashcomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* Hash handler function: return IndexAmRoutine with access method parameters
* and callbacks.
@@ -98,6 +102,7 @@ hashhandler(PG_FUNCTION_ARGS)
amroutine->amproperty = NULL;
amroutine->ambuildphasename = NULL;
amroutine->amvalidate = hashvalidate;
+ amroutine->amcomparedatums = hashcomparedatums;
amroutine->amadjustmembers = hashadjustmembers;
amroutine->ambeginscan = hashbeginscan;
amroutine->amrescan = hashrescan;
@@ -943,3 +948,42 @@ hashtranslatecmptype(CompareType cmptype, Oid opfamily)
return HTEqualStrategyNumber;
return InvalidStrategy;
}
+
+/*
+ * hashcomparedatums - Compare datums to determine if they produce identical keys
+ *
+ * Returns true if the hash values are identical (index doesn't need update).
+ */
+bool
+hashcomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull)
+{
+ uint32 old_hashkey;
+ uint32 new_hashkey;
+
+ /* If both are NULL, they're equal */
+ if (old_isnull && new_isnull)
+ return true;
+
+ /* If NULL status differs, they're not equal */
+ if (old_isnull != new_isnull)
+ return false;
+
+ /*
+ * _hash_datum2hashkey() is used because we know this can't be a cross
+ * type comparison.
+ */
+ old_hashkey = _hash_datum2hashkey(index, old_datum);
+ new_hashkey = _hash_datum2hashkey(index, new_datum);
+
+ /*
+ * If hash keys are identical, the index entry would be the same. Return
+ * true to indicate no index update needed.
+ *
+ * Note: Hash collisions are rare but possible. If hash(x) == hash(y) but
+ * x != y, the hash index still treats them identically, so we correctly
+ * return true.
+ */
+ return (old_hashkey == new_hashkey);
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index d5a1b844b5a..887aeee4bcd 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -3278,12 +3278,12 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* generated by another transaction).
*/
TM_Result
-heap_update(Relation relation, HeapTupleData *oldtup,
- HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
- Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
- Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
- Bitmapset *mix_attrs, Buffer *vmbuffer,
+heap_update(Relation relation, HeapTupleData *oldtup, HeapTuple newtup,
+ CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode,
+ Buffer buffer, Page page, BlockNumber block, ItemId lp,
+ Bitmapset *hot_attrs, Bitmapset *sum_attrs, Bitmapset *pk_attrs,
+ Bitmapset *rid_attrs, const Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -4352,8 +4352,9 @@ HeapDetermineColumnsInfo(Relation relation,
* This routine may be used to update a tuple when concurrent updates of the
* target tuple are not expected (for example, because we have a lock on the
* relation associated with the tuple). Any failure is reported via ereport().
+ * Returns the set of modified indexed attributes.
*/
-void
+Bitmapset *
simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
@@ -4482,7 +4483,7 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
elog(ERROR, "tuple concurrently deleted");
- return;
+ return NULL;
}
/*
@@ -4515,7 +4516,6 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
bms_free(sum_attrs);
bms_free(pk_attrs);
bms_free(rid_attrs);
- bms_free(mix_attrs);
bms_free(idx_attrs);
switch (result)
@@ -4541,6 +4541,8 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
elog(ERROR, "unrecognized heap_update status: %u", result);
break;
}
+
+ return mix_attrs;
}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 8708af01f8c..6b12f49f2b5 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -315,9 +315,12 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
- CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd,
- LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
+ CommandId cid, Snapshot snapshot,
+ Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd,
+ LockTupleMode *lockmode,
+ const Bitmapset *mix_attrs,
+ TU_UpdateIndexes *update_indexes)
{
bool rep_id_key_required = false;
bool shouldFree = true;
@@ -332,7 +335,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
*sum_attrs,
*pk_attrs,
*rid_attrs,
- *mix_attrs,
*idx_attrs;
TM_Result result;
@@ -405,25 +407,66 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
Assert(ItemIdIsNormal(lp));
- /*
- * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
- * then pass that on to heap_update.
- */
oldtup.t_tableOid = RelationGetRelid(relation);
oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
oldtup.t_len = ItemIdGetLength(lp);
oldtup.t_self = *otid;
- mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
- &oldtup, tuple, &rep_id_key_required);
-
/*
- * We'll need to WAL log the replica identity attributes if either they
- * overlap with the modified indexed attributes or, as we've checked for
- * just now in HeapDetermineColumnsInfo, they were unmodified external
- * indexed attributes.
+ * We'll need to include the replica identity key when either the identity
+ * key attributes overlap with the modified index attributes or when the
+ * replica identity attributes are stored externally. This is required
+ * because for such attributes the flattened value won't be WAL logged as
+ * part of the new tuple so we must determine if we need to extract and
+ * include them as part of the old_key_tuple (see ExtractReplicaIdentity).
*/
- rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+ rep_id_key_required = bms_overlap(mix_attrs, rid_attrs);
+ if (!rep_id_key_required)
+ {
+ Bitmapset *attrs;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ int attidx = -1;
+
+ /*
+ * We don't own idx_attrs so we'll copy it and remove the modified set
+ * to reduce the attributes we need to test in the while loop and
+ * avoid a two branches in the loop.
+ */
+ attrs = bms_difference(idx_attrs, mix_attrs);
+ attrs = bms_int_members(attrs, rid_attrs);
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /*
+ * attidx is zero-based, attrnum is the normal attribute number
+ */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value;
+ bool isnull;
+
+ /*
+ * System attributes are not added into interesting_attrs in
+ * relcache
+ */
+ Assert(attrnum > 0);
+
+ value = heap_getattr(&oldtup, attrnum, tupdesc, &isnull);
+
+ /* No need to check attributes that can't be stored externally */
+ if (isnull ||
+ TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
+ continue;
+
+ /* Check if the old tuple's attribute is stored externally */
+ if (VARATT_IS_EXTERNAL((struct varlena *) DatumGetPointer(value)))
+ {
+ rep_id_key_required = true;
+ break;
+ }
+ }
+
+ bms_free(attrs);
+ }
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
@@ -437,7 +480,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
bms_free(sum_attrs);
bms_free(pk_attrs);
bms_free(rid_attrs);
- bms_free(mix_attrs);
bms_free(idx_attrs);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index 6197b725fb1..e411be020b4 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -156,6 +156,7 @@ bthandler(PG_FUNCTION_ARGS)
amroutine->amproperty = btproperty;
amroutine->ambuildphasename = btbuildphasename;
amroutine->amvalidate = btvalidate;
+ amroutine->amcomparedatums = NULL;
amroutine->amadjustmembers = btadjustmembers;
amroutine->ambeginscan = btbeginscan;
amroutine->amrescan = btrescan;
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 73ebc01a08f..4f8fed45c05 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -367,6 +367,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -377,7 +378,9 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
GetCurrentCommandId(true),
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, &lockmode,
+ mix_attrs,
+ update_indexes);
switch (result)
{
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 4986b1ea7ed..43442a61f39 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -961,10 +961,18 @@ index_register(Oid heap,
newind->il_info->ii_Expressions =
copyObject(indexInfo->ii_Expressions);
newind->il_info->ii_ExpressionsState = NIL;
+ /* expression attrs will likely be null, but may as well copy it */
+ newind->il_info->ii_ExpressionsAttrs =
+ copyObject(indexInfo->ii_ExpressionsAttrs);
/* predicate will likely be null, but may as well copy it */
newind->il_info->ii_Predicate =
copyObject(indexInfo->ii_Predicate);
newind->il_info->ii_PredicateState = NULL;
+ /* predicate attrs will likely be null, but may as well copy it */
+ newind->il_info->ii_PredicateAttrs =
+ copyObject(indexInfo->ii_PredicateAttrs);
+ newind->il_info->ii_CheckedPredicate = false;
+ newind->il_info->ii_PredicateSatisfied = false;
/* no exclusion constraints at bootstrap time, so no need to copy */
Assert(indexInfo->ii_ExclusionOps == NULL);
Assert(indexInfo->ii_ExclusionProcs == NULL);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 8dea58ad96b..ed57dde1fc8 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -27,6 +27,7 @@
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/relscan.h"
+#include "access/sysattr.h"
#include "access/tableam.h"
#include "access/toast_compression.h"
#include "access/transam.h"
@@ -58,6 +59,7 @@
#include "commands/trigger.h"
#include "executor/executor.h"
#include "miscadmin.h"
+#include "nodes/execnodes.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/optimizer.h"
@@ -2414,6 +2416,61 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* ----------------------------------------------------------------
*/
+/* ----------------
+ * BuildUpdateIndexInfo
+ *
+ * For expression indexes updates may not change the indexed value allowing
+ * for a HOT update. Add information to the IndexInfo to allow for checking
+ * if the indexed value has changed.
+ *
+ * Do this processing here rather than in BuildIndexInfo() to not incur the
+ * overhead in the common non-expression cases.
+ * ----------------
+ */
+void
+BuildUpdateIndexInfo(ResultRelInfo *resultRelInfo)
+{
+ for (int j = 0; j < resultRelInfo->ri_NumIndices; j++)
+ {
+ int i;
+ int indnatts;
+ Bitmapset *attrs = NULL;
+ IndexInfo *ii = resultRelInfo->ri_IndexRelationInfo[j];
+
+ indnatts = ii->ii_NumIndexAttrs;
+
+ /* Collect key attributes used by the index, key and including */
+ for (i = 0; i < indnatts; i++)
+ {
+ AttrNumber attnum = ii->ii_IndexAttrNumbers[i];
+
+ if (attnum != 0)
+ attrs = bms_add_member(attrs, attnum - FirstLowInvalidHeapAttributeNumber);
+ }
+
+ /* Collect attributes used in the expression */
+ if (ii->ii_Expressions)
+ pull_varattnos((Node *) ii->ii_Expressions,
+ resultRelInfo->ri_RangeTableIndex,
+ &ii->ii_ExpressionsAttrs);
+
+ /* Collect attributes used in the predicate */
+ if (ii->ii_Predicate)
+ pull_varattnos((Node *) ii->ii_Predicate,
+ resultRelInfo->ri_RangeTableIndex,
+ &ii->ii_PredicateAttrs);
+
+ /*
+ * Combine key, including, and expression, but not partial index
+ * predicate attributes.
+ */
+ ii->ii_IndexedAttrs = bms_union(attrs, ii->ii_ExpressionsAttrs);
+
+ /* All indexes should index *something*! */
+ Assert(!bms_is_empty(ii->ii_IndexedAttrs));
+ }
+}
+
/* ----------------
* BuildIndexInfo
* Construct an IndexInfo record for an open index
diff --git a/src/backend/catalog/indexing.c b/src/backend/catalog/indexing.c
index 004c5121000..a361c215490 100644
--- a/src/backend/catalog/indexing.c
+++ b/src/backend/catalog/indexing.c
@@ -102,7 +102,7 @@ CatalogIndexInsert(CatalogIndexState indstate, HeapTuple heapTuple,
* Get information from the state structure. Fall out if nothing to do.
*/
numIndexes = indstate->ri_NumIndices;
- if (numIndexes == 0)
+ if (numIndexes == 0 || updateIndexes == TU_None)
return;
relationDescs = indstate->ri_IndexRelationDescs;
indexInfoArray = indstate->ri_IndexRelationInfo;
@@ -314,15 +314,18 @@ CatalogTupleUpdate(Relation heapRel, const ItemPointerData *otid, HeapTuple tup)
{
CatalogIndexState indstate;
TU_UpdateIndexes updateIndexes = TU_All;
+ Bitmapset *updatedAttrs;
CatalogTupleCheckConstraints(heapRel, tup);
indstate = CatalogOpenIndexes(heapRel);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
-
+ updatedAttrs = simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = updatedAttrs;
CatalogIndexInsert(indstate, tup, updateIndexes);
+
CatalogCloseIndexes(indstate);
+ bms_free(updatedAttrs);
}
/*
@@ -338,12 +341,15 @@ CatalogTupleUpdateWithInfo(Relation heapRel, const ItemPointerData *otid, HeapTu
CatalogIndexState indstate)
{
TU_UpdateIndexes updateIndexes = TU_All;
+ Bitmapset *updatedAttrs;
CatalogTupleCheckConstraints(heapRel, tup);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
-
+ updatedAttrs = simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = updatedAttrs;
CatalogIndexInsert(indstate, tup, updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = NULL;
+ bms_free(updatedAttrs);
}
/*
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 874a8fc89ad..9d7cb4438d5 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -292,8 +292,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
indexInfo->ii_IndexAttrNumbers[1] = 2;
indexInfo->ii_Expressions = NIL;
indexInfo->ii_ExpressionsState = NIL;
+ indexInfo->ii_ExpressionsAttrs = NULL;
indexInfo->ii_Predicate = NIL;
indexInfo->ii_PredicateState = NULL;
+ indexInfo->ii_PredicateAttrs = NULL;
+ indexInfo->ii_CheckedPredicate = false;
+ indexInfo->ii_PredicateSatisfied = false;
indexInfo->ii_ExclusionOps = NULL;
indexInfo->ii_ExclusionProcs = NULL;
indexInfo->ii_ExclusionStrats = NULL;
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 0b3a31f1703..1b17d8d135f 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -109,11 +109,15 @@
#include "access/genam.h"
#include "access/relscan.h"
#include "access/tableam.h"
+#include "access/sysattr.h"
#include "access/xact.h"
#include "catalog/index.h"
#include "executor/executor.h"
+#include "nodes/bitmapset.h"
+#include "nodes/execnodes.h"
#include "nodes/nodeFuncs.h"
#include "storage/lmgr.h"
+#include "utils/datum.h"
#include "utils/injection_point.h"
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
@@ -324,8 +328,8 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
Relation heapRelation;
IndexInfo **indexInfoArray;
ExprContext *econtext;
- Datum values[INDEX_MAX_KEYS];
- bool isnull[INDEX_MAX_KEYS];
+ Datum loc_values[INDEX_MAX_KEYS];
+ bool loc_isnull[INDEX_MAX_KEYS];
Assert(ItemPointerIsValid(tupleid));
@@ -349,13 +353,13 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
/* Arrange for econtext's scan tuple to be the tuple under test */
econtext->ecxt_scantuple = slot;
- /*
- * for each index, form and insert the index tuple
- */
+ /* Insert into each index that needs updating */
for (i = 0; i < numIndices; i++)
{
Relation indexRelation = relationDescs[i];
IndexInfo *indexInfo;
+ Datum *values;
+ bool *isnull;
bool applyNoDupErr;
IndexUniqueCheck checkUnique;
bool indexUnchanged;
@@ -372,7 +376,7 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
/*
* Skip processing of non-summarizing indexes if we only update
- * summarizing indexes
+ * summarizing indexes or if this index is unchanged.
*/
if (onlySummarizing && !indexInfo->ii_Summarizing)
continue;
@@ -393,8 +397,15 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
indexInfo->ii_PredicateState = predicate;
}
+ /* Check the index predicate if we haven't done so earlier on */
+ if (!indexInfo->ii_CheckedPredicate)
+ {
+ indexInfo->ii_PredicateSatisfied = ExecQual(predicate, econtext);
+ indexInfo->ii_CheckedPredicate = true;
+ }
+
/* Skip this index-update if the predicate isn't satisfied */
- if (!ExecQual(predicate, econtext))
+ if (!indexInfo->ii_PredicateSatisfied)
continue;
}
@@ -402,11 +413,10 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* FormIndexDatum fills in its values and isnull parameters with the
* appropriate values for the column(s) of the index.
*/
- FormIndexDatum(indexInfo,
- slot,
- estate,
- values,
- isnull);
+ FormIndexDatum(indexInfo, slot, estate, loc_values, loc_isnull);
+
+ values = loc_values;
+ isnull = loc_isnull;
/* Check whether to apply noDupErr to this index */
applyNoDupErr = noDupErr &&
@@ -613,7 +623,12 @@ ExecCheckIndexConstraints(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
checkedIndex = true;
/* Check for partial index */
- if (indexInfo->ii_Predicate != NIL)
+ if (indexInfo->ii_CheckedPredicate && !indexInfo->ii_PredicateSatisfied)
+ {
+ /* We've already checked and the predicate wasn't satisfied. */
+ continue;
+ }
+ else if (indexInfo->ii_Predicate != NIL)
{
ExprState *predicate;
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 797d8b1ca1c..3d140556a1a 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1282,6 +1282,7 @@ InitResultRelInfo(ResultRelInfo *resultRelInfo,
/* The following fields are set later if needed */
resultRelInfo->ri_RowIdAttNo = 0;
resultRelInfo->ri_extraUpdatedCols = NULL;
+ resultRelInfo->ri_ChangedIndexedCols = NULL;
resultRelInfo->ri_projectNew = NULL;
resultRelInfo->ri_newTupleSlot = NULL;
resultRelInfo->ri_oldTupleSlot = NULL;
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index 860f79f9cc1..d5860fb2c3f 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -32,6 +32,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
#include "utils/typcache.h"
@@ -936,7 +937,13 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ /*
+ * We're not going to call ExecCheckIndexedAttrsForChanges here
+ * because we've already identified the changes earlier on thanks to
+ * slot_modify_data.
+ */
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
+ resultRelInfo->ri_ChangedIndexedCols,
&update_indexes);
conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 874b71e6608..52a74479502 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -17,6 +17,7 @@
* ExecModifyTable - retrieve the next tuple from the node
* ExecEndModifyTable - shut down the ModifyTable node
* ExecReScanModifyTable - rescan the ModifyTable node
+ * ExecCheckIndexedAttrsForChanges - find set of updated indexed columns
*
* NOTES
* The ModifyTable node receives input from its outerPlan, which is
@@ -53,12 +54,18 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/attnum.h"
+#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/tupconvert.h"
+#include "access/tupdesc.h"
#include "access/xact.h"
+#include "catalog/index.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
#include "executor/nodeModifyTable.h"
+#include "executor/tuptable.h"
#include "foreign/fdwapi.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
@@ -68,8 +75,11 @@
#include "storage/lmgr.h"
#include "utils/builtins.h"
#include "utils/datum.h"
+#include "utils/float.h"
#include "utils/injection_point.h"
+#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
@@ -176,6 +186,224 @@ static TupleTableSlot *ExecMergeNotMatched(ModifyTableContext *context,
ResultRelInfo *resultRelInfo,
bool canSetTag);
+/*
+ * ExecCheckIndexedAttrsForChanges
+ *
+ * Determine which indexes need updating by finding the set of modified indexed
+ * attributes.
+ *
+ * For which implement the amcomparedatums() index AM API we'll need to form
+ * index datum and compare each attribute to see if anything actually changed.
+ *
+ * The goal is for the executor to know, ahead of calling into the table AM to
+ * process the update and before calling into the index AM for inserting new
+ * index tuples, which attributes in the new TupleTableSlot, if any, truely
+ * necessitate a new index tuple.
+ *
+ * Returns a Bitmapset of attributes that intersects with indexes which require
+ * a new index tuple.
+ */
+Bitmapset *
+ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts)
+{
+ Relation relation = relinfo->ri_RelationDesc;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ Bitmapset *mix_attrs = NULL; /* modified indexed attributes */
+
+ /* If no indexes, we're done */
+ if (relinfo->ri_NumIndices == 0)
+ return NULL;
+
+ /* Find the indexes that reference this attribute */
+ for (int i = 0; i < relinfo->ri_NumIndices; i++)
+ {
+ Relation index = relinfo->ri_IndexRelationDescs[i];
+ IndexAmRoutine *amroutine = index->rd_indam;
+ IndexInfo *indexInfo = relinfo->ri_IndexRelationInfo[i];
+ Bitmapset *m_attrs = NULL; /* (possibly) modified indexed attrs */
+ Bitmapset *p_attrs = NULL; /* (possibly) modified predicate attrs */
+ Bitmapset *u_attrs = NULL; /* unmodified indexed attrs */
+ bool has_am_compare = (amroutine->amcomparedatums != NULL);
+ bool supports_ios = (amroutine->amcanreturn != NULL);
+ bool is_partial = (indexInfo->ii_Predicate != NIL);
+ ExprContext *econtext = GetPerTupleExprContext(estate);
+ int num_datums = supports_ios ?
+ indexInfo->ii_NumIndexAttrs : indexInfo->ii_NumIndexKeyAttrs;
+
+ /* If we've reviewed all the attributes on this index, move on */
+ if (bms_is_subset(indexInfo->ii_IndexedAttrs, mix_attrs))
+ continue;
+
+ /* Add partial index attributes */
+ if (is_partial)
+ p_attrs = bms_add_members(p_attrs, indexInfo->ii_PredicateAttrs);
+
+ /* Compare the index datums for equality */
+ for (int j = 0; j < num_datums; j++)
+ {
+ AttrNumber rel_attrnum = indexInfo->ii_IndexAttrNumbers[j];
+ int rel_attridx = rel_attrnum - FirstLowInvalidHeapAttributeNumber;
+ int nth_expr = 0;
+ int16 typlen;
+ bool typbyval;
+ Datum old_value;
+ Datum new_value;
+ bool old_null;
+ bool new_null;
+ bool values_equal = false;
+
+ /* System attributes */
+ if (rel_attrnum < 0)
+ {
+ /* Extract system values from both slots for this attribute */
+ old_value = slot_getsysattr(old_tts, rel_attrnum, &old_null);
+ new_value = slot_getsysattr(new_tts, rel_attrnum, &new_null);
+
+ /* The only allowed system columns are OIDs, so do this */
+ values_equal = (DatumGetObjectId(old_value) == DatumGetObjectId(new_value));
+ goto equality_determined;
+ }
+
+ /*
+ * This is an expression attribute, but in an effort to avoid the
+ * expense of IndexFormDatum we're now faced with testing for
+ * equality so we'll have to exec the expressions and test for
+ * binary equality of the results.
+ */
+ else if (rel_attrnum == 0)
+ {
+ TupleTableSlot *save_scantuple = econtext->ecxt_scantuple;
+ Oid expr_type_oid;
+ Expr *expr = (Expr *) list_nth(indexInfo->ii_Expressions, nth_expr);
+ ExprState *state;
+
+ if (indexInfo->ii_ExpressionsState == NIL)
+ {
+ /* First time through, set up expression evaluation state */
+ indexInfo->ii_ExpressionsState =
+ ExecPrepareExprList(indexInfo->ii_Expressions, estate);
+ }
+
+ state = (ExprState *) list_nth(indexInfo->ii_ExpressionsState, nth_expr);
+
+ econtext->ecxt_scantuple = old_tts;
+ old_value = ExecEvalExprSwitchContext(state,
+ GetPerTupleExprContext(estate),
+ &old_null);
+
+ econtext->ecxt_scantuple = new_tts;
+ new_value = ExecEvalExprSwitchContext(state,
+ GetPerTupleExprContext(estate),
+ &new_null);
+
+ econtext->ecxt_scantuple = save_scantuple;
+
+ /*
+ * NOTE: test for NULL cases here to potentially avoid looking
+ * up the type information. It's a tad redundant, but worth
+ * it.
+ */
+
+ /* A change to/from NULL, so not equal */
+ if (old_null != new_null)
+ {
+ values_equal = false;
+ goto equality_determined;
+ }
+
+ /* Both NULL, no change record as unmodified */
+ if (old_null)
+ {
+ values_equal = true;
+ goto equality_determined;
+ }
+
+ /* Get type OID from the expression */
+ expr_type_oid = exprType((Node *) expr);
+
+ /* Get type information from the OID */
+ get_typlenbyval(expr_type_oid, &typlen, &typbyval);
+ }
+ /* Not a system or expression attribute */
+ else
+ {
+ CompactAttribute *att = TupleDescCompactAttr(tupdesc, rel_attrnum - 1);
+
+ /* Extract values from both slots for this attribute */
+ old_value = slot_getattr(old_tts, rel_attrnum, &old_null);
+ new_value = slot_getattr(new_tts, rel_attrnum, &new_null);
+
+ typlen = att->attlen;
+ typbyval = att->attbyval;
+ }
+
+ /* A change to/from NULL, so not equal */
+ if (old_null != new_null)
+ {
+ values_equal = false;
+ goto equality_determined;
+ }
+
+ /* Both NULL, no change record as unmodified */
+ if (old_null)
+ {
+ values_equal = true;
+ goto equality_determined;
+ }
+
+ if (has_am_compare)
+ {
+ /*
+ * NOTE: For AM comparison, pass the 1-based index attribute
+ * number. The AM's compare function expects the same
+ * numbering as used internally by the AM.
+ */
+ values_equal = amroutine->amcomparedatums(index, j + 1,
+ old_value, old_null,
+ new_value, new_null);
+ }
+ else
+ {
+ values_equal = datumIsEqual(old_value, new_value, typbyval, typlen);
+ }
+
+ equality_determined:;
+ if (!values_equal)
+ if (rel_attrnum == 0)
+ {
+ Expr *expr = (Expr *) list_nth(indexInfo->ii_Expressions, nth_expr);
+
+ pull_varattnos((Node *) expr, relinfo->ri_RangeTableIndex, &m_attrs);
+ }
+ else
+ m_attrs = bms_add_member(m_attrs, rel_attridx);
+ else
+ u_attrs = bms_add_member(u_attrs, rel_attridx);
+
+ if (rel_attrnum == 0)
+ nth_expr++;
+ }
+
+ /*
+ * Here we know all the attributes that might be modified and all
+ * those we know haven't been across all indexes. Take the difference
+ * and add it to the modified indexed attributes set.
+ */
+ m_attrs = bms_del_members(m_attrs, u_attrs);
+ p_attrs = bms_del_members(p_attrs, u_attrs);
+ mix_attrs = bms_add_members(mix_attrs, m_attrs);
+ mix_attrs = bms_add_members(mix_attrs, p_attrs);
+
+ bms_free(m_attrs);
+ bms_free(u_attrs);
+ bms_free(p_attrs);
+ }
+
+ return mix_attrs;
+}
/*
* Verify that the tuples to be produced by INSERT match the
@@ -2168,14 +2396,17 @@ ExecUpdatePrepareSlot(ResultRelInfo *resultRelInfo,
*/
static TM_Result
ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
- ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *slot,
- bool canSetTag, UpdateContext *updateCxt)
+ ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *oldSlot,
+ TupleTableSlot *slot, bool canSetTag, UpdateContext *updateCxt)
{
EState *estate = context->estate;
Relation resultRelationDesc = resultRelInfo->ri_RelationDesc;
bool partition_constraint_failed;
TM_Result result;
+ /* The set of modified indexed attributes that trigger new index entries */
+ Bitmapset *mix_attrs = NULL;
+
updateCxt->crossPartUpdate = false;
/*
@@ -2292,9 +2523,38 @@ lreplace:
ExecConstraints(resultRelInfo, slot, estate);
/*
- * replace the heap tuple
+ * Identify which, if any, indexed attributes were modified here so that
+ * we might reuse it in a few places.
+ */
+ bms_free(resultRelInfo->ri_ChangedIndexedCols);
+ resultRelInfo->ri_ChangedIndexedCols = NULL;
+
+ /*
+ * During updates we'll need a bit more information in IndexInfo but we've
+ * delayed adding it until here. We check to ensure that there are
+ * indexes, that something has changed that is indexed, and that the first
+ * index doesn't yet have ii_IndexedAttrs set as a way to ensure we only
+ * build this when needed and only once. We don't build this in
+ * ExecOpenIndicies() as it is unnecessary overhead when not performing an
+ * update.
+ */
+ if (resultRelInfo->ri_NumIndices > 0 &&
+ bms_is_empty(resultRelInfo->ri_IndexRelationInfo[0]->ii_IndexedAttrs))
+ BuildUpdateIndexInfo(resultRelInfo);
+
+ /*
+ * Next up we need to find out the set of indexed attributes that have
+ * changed in value and should trigger a new index tuple. We could start
+ * with the set of updated columns via ExecGetUpdatedCols(), but if we do
+ * we will overlook attributes directly modified by heap_modify_tuple()
+ * which are not known to ExecGetUpdatedCols().
+ */
+ mix_attrs = ExecCheckIndexedAttrsForChanges(resultRelInfo, estate, oldSlot, slot);
+
+ /*
+ * Call into the table AM to update the heap tuple.
*
- * Note: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
+ * NOTE: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
* the row to be updated is visible to that snapshot, and throw a
* can't-serialize error if not. This is a special-case behavior needed
* for referential integrity updates in transaction-snapshot mode
@@ -2306,8 +2566,12 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
+ mix_attrs,
&updateCxt->updateIndexes);
+ Assert(bms_is_empty(resultRelInfo->ri_ChangedIndexedCols));
+ resultRelInfo->ri_ChangedIndexedCols = mix_attrs;
+
return result;
}
@@ -2325,7 +2589,7 @@ ExecUpdateEpilogue(ModifyTableContext *context, UpdateContext *updateCxt,
ModifyTableState *mtstate = context->mtstate;
List *recheckIndexes = NIL;
- /* insert index entries for tuple if necessary */
+ /* Insert index entries for tuple if necessary */
if (resultRelInfo->ri_NumIndices > 0 && (updateCxt->updateIndexes != TU_None))
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
slot, context->estate,
@@ -2524,8 +2788,9 @@ ExecUpdate(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
*/
redo_act:
lockedtid = *tupleid;
- result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, slot,
- canSetTag, &updateCxt);
+
+ result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, oldSlot,
+ slot, canSetTag, &updateCxt);
/*
* If ExecUpdateAct reports that a cross-partition update was done,
@@ -3222,8 +3487,8 @@ lmerge_matched:
Assert(oldtuple == NULL);
result = ExecUpdateAct(context, resultRelInfo, tupleid,
- NULL, newslot, canSetTag,
- &updateCxt);
+ NULL, resultRelInfo->ri_oldTupleSlot,
+ newslot, canSetTag, &updateCxt);
/*
* As in ExecUpdate(), if ExecUpdateAct() reports that a
@@ -3248,6 +3513,7 @@ lmerge_matched:
tupleid, NULL, newslot);
mtstate->mt_merge_updated += 1;
}
+
break;
case CMD_DELETE:
@@ -4354,7 +4620,7 @@ ExecModifyTable(PlanState *pstate)
* For UPDATE/DELETE/MERGE, fetch the row identity info for the tuple
* to be updated/deleted/merged. For a heap relation, that's a TID;
* otherwise we may have a wholerow junk attr that carries the old
- * tuple in toto. Keep this in step with the part of
+ * tuple in total. Keep this in step with the part of
* ExecInitModifyTable that sets up ri_RowIdAttNo.
*/
if (operation == CMD_UPDATE || operation == CMD_DELETE ||
@@ -4530,6 +4796,7 @@ ExecModifyTable(PlanState *pstate)
/* Now apply the update. */
slot = ExecUpdate(&context, resultRelInfo, tupleid, oldtuple,
oldSlot, slot, node->canSetTag);
+
if (tuplock)
UnlockTuple(resultRelInfo->ri_RelationDesc, tupleid,
InplaceUpdateTupleLock);
diff --git a/src/backend/nodes/bitmapset.c b/src/backend/nodes/bitmapset.c
index 7b1e9d94103..c522971a37c 100644
--- a/src/backend/nodes/bitmapset.c
+++ b/src/backend/nodes/bitmapset.c
@@ -238,6 +238,10 @@ bms_make_singleton(int x)
void
bms_free(Bitmapset *a)
{
+#if USE_ASSERT_CHECKING
+ Assert(bms_is_valid_set(a));
+#endif
+
if (a)
pfree(a);
}
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index e2d9e9be41a..1f1364f9df9 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -857,10 +857,14 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
/* expressions */
n->ii_Expressions = expressions;
n->ii_ExpressionsState = NIL;
+ n->ii_ExpressionsAttrs = NULL;
/* predicates */
n->ii_Predicate = predicates;
n->ii_PredicateState = NULL;
+ n->ii_PredicateAttrs = NULL;
+ n->ii_CheckedPredicate = false;
+ n->ii_PredicateSatisfied = false;
/* exclusion constraints */
n->ii_ExclusionOps = NULL;
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index fc64476a9ef..3fca4cd292b 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -275,7 +275,6 @@
#include "replication/logicalrelation.h"
#include "replication/logicalworker.h"
#include "replication/origin.h"
-#include "replication/slot.h"
#include "replication/walreceiver.h"
#include "replication/worker_internal.h"
#include "rewrite/rewriteHandler.h"
@@ -285,12 +284,14 @@
#include "storage/procarray.h"
#include "tcop/tcopprot.h"
#include "utils/acl.h"
+#include "utils/datum.h"
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/pg_lsn.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/rls.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
@@ -1110,15 +1111,18 @@ slot_store_data(TupleTableSlot *slot, LogicalRepRelMapEntry *rel,
* "slot" is filled with a copy of the tuple in "srcslot", replacing
* columns provided in "tupleData" and leaving others as-is.
*
+ * Returns a bitmap of the modified columns.
+ *
* Caution: unreplaced pass-by-ref columns in "slot" will point into the
* storage for "srcslot". This is OK for current usage, but someday we may
* need to materialize "slot" at the end to make it independent of "srcslot".
*/
-static void
+static Bitmapset *
slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
LogicalRepRelMapEntry *rel,
LogicalRepTupleData *tupleData)
{
+ Bitmapset *modified = NULL;
int natts = slot->tts_tupleDescriptor->natts;
int i;
@@ -1195,6 +1199,27 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
slot->tts_isnull[i] = true;
}
+ /*
+ * Determine if the replicated value changed the local value by
+ * comparing slots. This is a subset of
+ * ExecCheckIndexedAttrsForChanges.
+ */
+ if (srcslot->tts_isnull[i] != slot->tts_isnull[i])
+ {
+ /* One is NULL, the other is not so the value changed */
+ modified = bms_add_member(modified, i + 1 - FirstLowInvalidHeapAttributeNumber);
+ }
+ else if (!srcslot->tts_isnull[i])
+ {
+ /* Both are not NULL, compare their values */
+
+ if (!datumIsEqual(srcslot->tts_values[i],
+ slot->tts_values[i],
+ att->attbyval,
+ att->attlen))
+ modified = bms_add_member(modified, i + 1 - FirstLowInvalidHeapAttributeNumber);
+ }
+
/* Reset attnum for error callback */
apply_error_callback_arg.remote_attnum = -1;
}
@@ -1202,6 +1227,8 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
/* And finally, declare that "slot" contains a valid virtual tuple */
ExecStoreVirtualTuple(slot);
+
+ return modified;
}
/*
@@ -2918,6 +2945,7 @@ apply_handle_update_internal(ApplyExecutionData *edata,
ConflictTupleInfo conflicttuple = {0};
bool found;
MemoryContext oldctx;
+ Bitmapset *indexed = NULL;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
ExecOpenIndices(relinfo, false);
@@ -2934,6 +2962,8 @@ apply_handle_update_internal(ApplyExecutionData *edata,
*/
if (found)
{
+ Bitmapset *modified = NULL;
+
/*
* Report the conflict if the tuple was modified by a different
* origin.
@@ -2957,15 +2987,29 @@ apply_handle_update_internal(ApplyExecutionData *edata,
/* Process and store remote tuple in the slot */
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot, localslot, relmapentry, newtup);
+ modified = slot_modify_data(remoteslot, localslot, relmapentry, newtup);
MemoryContextSwitchTo(oldctx);
+ /*
+ * Normally we'd call ExecCheckIndexedAttrForChanges but here we have
+ * the record of changed columns in the replication state, so let's
+ * use that instead.
+ */
+ indexed = RelationGetIndexAttrBitmap(relinfo->ri_RelationDesc,
+ INDEX_ATTR_BITMAP_INDEXED);
+
+ bms_free(relinfo->ri_ChangedIndexedCols);
+ relinfo->ri_ChangedIndexedCols = bms_int_members(modified, indexed);
+ bms_free(indexed);
+
EvalPlanQualSetSlot(&epqstate, remoteslot);
InitConflictIndexes(relinfo);
- /* Do the actual update. */
+ /* First check privileges */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
+
+ /* Then do the actual update. */
ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
remoteslot);
}
@@ -3455,6 +3499,8 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
bool found;
EPQState epqstate;
ConflictTupleInfo conflicttuple = {0};
+ Bitmapset *modified = NULL;
+ Bitmapset *indexed;
/* Get the matching local tuple from the partition. */
found = FindReplTupleInLocalRel(edata, partrel,
@@ -3523,8 +3569,8 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
* remoteslot_part.
*/
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot_part, localslot, part_entry,
- newtup);
+ modified = slot_modify_data(remoteslot_part, localslot, part_entry,
+ newtup);
MemoryContextSwitchTo(oldctx);
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
@@ -3549,6 +3595,18 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
EvalPlanQualSetSlot(&epqstate, remoteslot_part);
TargetPrivilegesCheck(partrelinfo->ri_RelationDesc,
ACL_UPDATE);
+
+ /*
+ * Normally we'd call ExecCheckIndexedAttrForChanges but
+ * here we have the record of changed columns in the
+ * replication state, so let's use that instead.
+ */
+ indexed = RelationGetIndexAttrBitmap(partrelinfo->ri_RelationDesc,
+ INDEX_ATTR_BITMAP_INDEXED);
+ bms_free(partrelinfo->ri_ChangedIndexedCols);
+ partrelinfo->ri_ChangedIndexedCols = bms_int_members(modified, indexed);
+ bms_free(indexed);
+
ExecSimpleRelationUpdate(partrelinfo, estate, &epqstate,
localslot, remoteslot_part);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 2d0cb7bcfd4..415ad1019b2 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -2482,6 +2482,7 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
bms_free(relation->rd_idattr);
bms_free(relation->rd_hotblockingattr);
bms_free(relation->rd_summarizedattr);
+ bms_free(relation->rd_indexedattr);
if (relation->rd_pubdesc)
pfree(relation->rd_pubdesc);
if (relation->rd_options)
@@ -5283,6 +5284,7 @@ RelationGetIndexPredicate(Relation relation)
* index (empty if FULL)
* INDEX_ATTR_BITMAP_HOT_BLOCKING Columns that block updates from being HOT
* INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
+ * INDEX_ATTR_BITMAP_INDEXED Columns referenced by indexes
*
* Attribute numbers are offset by FirstLowInvalidHeapAttributeNumber so that
* we can include system attributes (e.g., OID) in the bitmap representation.
@@ -5307,6 +5309,7 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
Bitmapset *idindexattrs; /* columns in the replica identity */
Bitmapset *hotblockingattrs; /* columns with HOT blocking indexes */
Bitmapset *summarizedattrs; /* columns with summarizing indexes */
+ Bitmapset *indexedattrs; /* columns referenced by indexes */
List *indexoidlist;
List *newindexoidlist;
Oid relpkindex;
@@ -5329,6 +5332,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
return bms_copy(relation->rd_hotblockingattr);
case INDEX_ATTR_BITMAP_SUMMARIZED:
return bms_copy(relation->rd_summarizedattr);
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return bms_copy(relation->rd_indexedattr);
default:
elog(ERROR, "unknown attrKind %u", attrKind);
}
@@ -5373,6 +5378,7 @@ restart:
idindexattrs = NULL;
hotblockingattrs = NULL;
summarizedattrs = NULL;
+ indexedattrs = NULL;
foreach(l, indexoidlist)
{
Oid indexOid = lfirst_oid(l);
@@ -5505,10 +5511,14 @@ restart:
bms_free(idindexattrs);
bms_free(hotblockingattrs);
bms_free(summarizedattrs);
+ bms_free(indexedattrs);
goto restart;
}
+ /* Combine all index attributes */
+ indexedattrs = bms_union(hotblockingattrs, summarizedattrs);
+
/* Don't leak the old values of these bitmaps, if any */
relation->rd_attrsvalid = false;
bms_free(relation->rd_keyattr);
@@ -5521,6 +5531,8 @@ restart:
relation->rd_hotblockingattr = NULL;
bms_free(relation->rd_summarizedattr);
relation->rd_summarizedattr = NULL;
+ bms_free(relation->rd_indexedattr);
+ relation->rd_indexedattr = NULL;
/*
* Now save copies of the bitmaps in the relcache entry. We intentionally
@@ -5535,6 +5547,7 @@ restart:
relation->rd_idattr = bms_copy(idindexattrs);
relation->rd_hotblockingattr = bms_copy(hotblockingattrs);
relation->rd_summarizedattr = bms_copy(summarizedattrs);
+ relation->rd_indexedattr = bms_copy(indexedattrs);
relation->rd_attrsvalid = true;
MemoryContextSwitchTo(oldcxt);
@@ -5551,6 +5564,8 @@ restart:
return hotblockingattrs;
case INDEX_ATTR_BITMAP_SUMMARIZED:
return summarizedattrs;
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return indexedattrs;
default:
elog(ERROR, "unknown attrKind %u", attrKind);
return NULL;
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index 63dd41c1f21..9bdf73eda59 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -211,6 +211,33 @@ typedef void (*ammarkpos_function) (IndexScanDesc scan);
/* restore marked scan position */
typedef void (*amrestrpos_function) (IndexScanDesc scan);
+/*
+ * amcomparedatums - Compare datums to determine if index update is needed
+ *
+ * This function compares old_datum and new_datum to determine if they would
+ * produce different index entries. For extraction-based indexes (GIN, RUM),
+ * this should:
+ * 1. Extract keys from old_datum using the opclass's extractValue function
+ * 2. Extract keys from new_datum using the opclass's extractValue function
+ * 3. Compare the two sets of keys using appropriate equality operators
+ * 4. Return true if the sets are equal (no index update needed)
+ *
+ * The comparison should account for:
+ * - Different numbers of extracted keys
+ * - NULL values
+ * - Type-specific equality (not just binary equality)
+ * - Opclass parameters (e.g., path in bson_rum_single_path_ops)
+ *
+ * For the DocumentDB example with path='a', this would extract values at
+ * path 'a' from both old and new BSON documents and compare them using
+ * BSON's equality operator.
+ */
+/* identify if updated datums would produce one or more index entries */
+typedef bool (*amcomparedatums_function) (Relation indexRelation,
+ int attno,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* Callback function signatures - for parallel index scans.
*/
@@ -313,6 +340,7 @@ typedef struct IndexAmRoutine
amendscan_function amendscan;
ammarkpos_function ammarkpos; /* can be NULL */
amrestrpos_function amrestrpos; /* can be NULL */
+ amcomparedatums_function amcomparedatums; /* can be NULL */
/* interface functions to support parallel index scans */
amestimateparallelscan_function amestimateparallelscan; /* can be NULL */
diff --git a/src/include/access/gin.h b/src/include/access/gin.h
index 13ea91922ef..2f265f4816c 100644
--- a/src/include/access/gin.h
+++ b/src/include/access/gin.h
@@ -100,6 +100,9 @@ extern PGDLLIMPORT int gin_pending_list_limit;
extern void ginGetStats(Relation index, GinStatsData *stats);
extern void ginUpdateStats(Relation index, const GinStatsData *stats,
bool is_build);
+extern bool gincomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
extern void _gin_parallel_build_main(dsm_segment *seg, shm_toc *toc);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 9398216d5d9..072749ef417 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -369,7 +369,7 @@ extern TM_Result heap_update(Relation relation, HeapTupleData *oldtup,
TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
- Bitmapset *mix_attrs, Buffer *vmbuffer,
+ const Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
@@ -404,8 +404,8 @@ extern bool heap_tuple_needs_eventual_freeze(HeapTupleHeader tuple);
extern void simple_heap_insert(Relation relation, HeapTuple tup);
extern void simple_heap_delete(Relation relation, const ItemPointerData *tid);
-extern void simple_heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple tup, TU_UpdateIndexes *update_indexes);
+extern Bitmapset *simple_heap_update(Relation relation, const ItemPointerData *otid,
+ HeapTuple tup, TU_UpdateIndexes *update_indexes);
extern TransactionId heap_index_delete_tuples(Relation rel,
TM_IndexDeleteOp *delstate);
diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h
index 7a3efd209bc..2a28a791df0 100644
--- a/src/include/access/nbtree.h
+++ b/src/include/access/nbtree.h
@@ -1179,6 +1179,10 @@ extern int btgettreeheight(Relation rel);
extern CompareType bttranslatestrategy(StrategyNumber strategy, Oid opfamily);
extern StrategyNumber bttranslatecmptype(CompareType cmptype, Oid opfamily);
+extern bool btcomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* prototypes for internal functions in nbtree.c
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 2fa790b6bf5..d94dfc9b41d 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -549,6 +549,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
+ const Bitmapset *updated_cols,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1512,12 +1513,12 @@ static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ const Bitmapset *mix_cols, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
- wait, tmfd,
- lockmode, update_indexes);
+ wait, tmfd, lockmode,
+ mix_cols, update_indexes);
}
/*
@@ -2020,6 +2021,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index dda95e54903..8d364f8b30f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -132,6 +132,7 @@ extern bool CompareIndexInfo(const IndexInfo *info1, const IndexInfo *info2,
const AttrMap *attmap);
extern void BuildSpeculativeIndexInfo(Relation index, IndexInfo *ii);
+extern void BuildUpdateIndexInfo(ResultRelInfo *resultRelInfo);
extern void FormIndexDatum(IndexInfo *indexInfo,
TupleTableSlot *slot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 7cd6a49309f..d0461a0baec 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -739,6 +739,11 @@ extern Bitmapset *ExecGetAllUpdatedCols(ResultRelInfo *relinfo, EState *estate);
*/
extern void ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative);
extern void ExecCloseIndices(ResultRelInfo *resultRelInfo);
+extern Bitmapset *ExecWhichIndexesRequireUpdates(ResultRelInfo *relinfo,
+ Bitmapset *mix_attrs,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
extern List *ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
TupleTableSlot *slot, EState *estate,
bool update,
@@ -800,5 +805,9 @@ extern ResultRelInfo *ExecLookupResultRelByOid(ModifyTableState *node,
Oid resultoid,
bool missing_ok,
bool update_cache);
+extern Bitmapset *ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 3968429f991..dfc93c2cc98 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -174,15 +174,29 @@ typedef struct IndexInfo
*/
AttrNumber ii_IndexAttrNumbers[INDEX_MAX_KEYS];
+ /*
+ * All key, expression, sumarizing, and partition attributes referenced by
+ * this index
+ */
+ Bitmapset *ii_IndexedAttrs;
+
/* expr trees for expression entries, or NIL if none */
List *ii_Expressions; /* list of Expr */
/* exec state for expressions, or NIL if none */
List *ii_ExpressionsState; /* list of ExprState */
+ /* attributes exclusively referenced by expression indexes */
+ Bitmapset *ii_ExpressionsAttrs;
/* partial-index predicate, or NIL if none */
List *ii_Predicate; /* list of Expr */
/* exec state for expressions, or NIL if none */
ExprState *ii_PredicateState;
+ /* attributes referenced by the predicate */
+ Bitmapset *ii_PredicateAttrs;
+ /* partial index predicate determined yet? */
+ bool ii_CheckedPredicate;
+ /* amupdate hint used to avoid rechecking predicate */
+ bool ii_PredicateSatisfied;
/* Per-column exclusion operators, or NULL if none */
Oid *ii_ExclusionOps; /* array with one entry per column */
@@ -499,6 +513,12 @@ typedef struct ResultRelInfo
/* true if the above has been computed */
bool ri_extraUpdatedCols_valid;
+ /*
+ * For UPDATE a Bitmapset of the attributes that are both indexed and have
+ * changed in value.
+ */
+ Bitmapset *ri_ChangedIndexedCols;
+
/* Projection to generate new tuple in an INSERT/UPDATE */
ProjectionInfo *ri_projectNew;
/* Slot to hold that tuple */
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 80286076a11..b23a7306e69 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -164,6 +164,7 @@ typedef struct RelationData
Bitmapset *rd_idattr; /* included in replica identity index */
Bitmapset *rd_hotblockingattr; /* cols blocking HOT update */
Bitmapset *rd_summarizedattr; /* cols indexed by summarizing indexes */
+ Bitmapset *rd_indexedattr; /* all cols referenced by indexes */
PublicationDesc *rd_pubdesc; /* publication descriptor, or NULL */
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 3561c6bef0b..d3fbb8b093a 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -71,6 +71,7 @@ typedef enum IndexAttrBitmapKind
INDEX_ATTR_BITMAP_IDENTITY_KEY,
INDEX_ATTR_BITMAP_HOT_BLOCKING,
INDEX_ATTR_BITMAP_SUMMARIZED,
+ INDEX_ATTR_BITMAP_INDEXED,
} IndexAttrBitmapKind;
extern Bitmapset *RelationGetIndexAttrBitmap(Relation relation,
diff --git a/src/test/isolation/expected/insert-conflict-specconflict.out b/src/test/isolation/expected/insert-conflict-specconflict.out
index e34a821c403..54b3981918c 100644
--- a/src/test/isolation/expected/insert-conflict-specconflict.out
+++ b/src/test/isolation/expected/insert-conflict-specconflict.out
@@ -80,6 +80,10 @@ pg_advisory_unlock
t
(1 row)
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
s1: NOTICE: acquiring advisory lock on 2
s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
@@ -172,6 +176,10 @@ pg_advisory_unlock
t
(1 row)
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
s2: NOTICE: acquiring advisory lock on 2
s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
@@ -369,6 +377,10 @@ key|data
step s1_commit: COMMIT;
s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
s2: NOTICE: acquiring advisory lock on 2
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
+s2: NOTICE: blurt_and_lock_123() called for k1 in session 2
+s2: NOTICE: acquiring advisory lock on 2
step s2_upsert: <... completed>
step controller_show: SELECT * FROM upserttest;
key|data
@@ -530,6 +542,14 @@ isolation/insert-conflict-specconflict/s2|transactionid|ExclusiveLock|t
step s2_commit: COMMIT;
s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_123() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 2
+s1: NOTICE: blurt_and_lock_4() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 4
+s1: NOTICE: blurt_and_lock_4() called for k1 in session 1
+s1: NOTICE: acquiring advisory lock on 4
step s1_upsert: <... completed>
step s1_noop:
step controller_show: SELECT * FROM upserttest;
diff --git a/src/test/regress/expected/heap_hot_updates.out b/src/test/regress/expected/heap_hot_updates.out
new file mode 100644
index 00000000000..14276e3cbca
--- /dev/null
+++ b/src/test/regress/expected/heap_hot_updates.out
@@ -0,0 +1,650 @@
+-- ================================================================
+-- Test Suite for Heap-only (HOT) Updates
+-- ================================================================
+-- Setup: Create function to measure HOT updates
+CREATE OR REPLACE FUNCTION check_hot_updates(
+ expected INT,
+ p_table_name TEXT DEFAULT 't',
+ p_schema_name TEXT DEFAULT current_schema()
+)
+RETURNS TABLE (
+ table_name TEXT,
+ total_updates BIGINT,
+ hot_updates BIGINT,
+ hot_update_percentage NUMERIC,
+ matches_expected BOOLEAN
+)
+LANGUAGE plpgsql
+AS $$
+DECLARE
+ v_relid oid;
+ v_qualified_name TEXT;
+ v_hot_updates BIGINT;
+ v_updates BIGINT;
+ v_xact_hot_updates BIGINT;
+ v_xact_updates BIGINT;
+BEGIN
+ -- Force statistics update
+ PERFORM pg_stat_force_next_flush();
+
+ -- Get table OID
+ v_qualified_name := quote_ident(p_schema_name) || '.' || quote_ident(p_table_name);
+ v_relid := v_qualified_name::regclass;
+
+ IF v_relid IS NULL THEN
+ RAISE EXCEPTION 'Table %.% not found', p_schema_name, p_table_name;
+ END IF;
+
+ -- Get cumulative + transaction stats
+ v_hot_updates := COALESCE(pg_stat_get_tuples_hot_updated(v_relid), 0);
+ v_updates := COALESCE(pg_stat_get_tuples_updated(v_relid), 0);
+ v_xact_hot_updates := COALESCE(pg_stat_get_xact_tuples_hot_updated(v_relid), 0);
+ v_xact_updates := COALESCE(pg_stat_get_xact_tuples_updated(v_relid), 0);
+
+ v_hot_updates := v_hot_updates + v_xact_hot_updates;
+ v_updates := v_updates + v_xact_updates;
+
+ RETURN QUERY
+ SELECT
+ p_table_name::TEXT,
+ v_updates::BIGINT,
+ v_hot_updates::BIGINT,
+ CASE WHEN v_updates > 0
+ THEN ROUND((v_hot_updates::numeric / v_updates::numeric * 100)::numeric, 2)
+ ELSE 0
+ END,
+ (v_hot_updates = expected)::BOOLEAN;
+END;
+$$;
+CREATE COLLATION case_insensitive (
+ provider = libc,
+ locale = 'C'
+);
+-- ================================================================
+-- GIN Index on JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data);
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "database"]}');
+-- Change tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Change tags again - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Add field without changing existing keys - GIN keys changed (added "note"), NOT HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "note": "test"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Index with Unchanged Keys
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create GIN index on specific path
+CREATE INDEX t_gin_idx ON t USING gin((data->'tags'));
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "sql"], "status": "active"}');
+-- Change non-indexed field - GIN keys on 'tags' unchanged, should be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change indexed tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN with jsonb_path_ops
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data jsonb_path_ops);
+INSERT INTO t VALUES (1, '{"user": {"name": "alice"}, "tags": ["a", "b"]}');
+-- Change value at different path - keys changed, NOT HOT
+UPDATE t SET data = '{"user": {"name": "bob"}, "tags": ["a", "b"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Mixed Index Types (BRIN + Expression)
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_idx ON t USING brin(value);
+CREATE INDEX t_expr_idx ON t((data->'status'));
+INSERT INTO t VALUES (1, 100, '{"status": "active"}');
+-- Update only BRIN column - should be HOT
+UPDATE t SET value = 200 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update only expression column - should NOT be HOT
+UPDATE t SET data = '{"status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update both - should NOT be HOT
+UPDATE t SET value = 300, data = '{"status": "pending"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Array Index - Order Insensitive Extraction
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ data JSONB
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+-- GIN index on JSONB array (extracts all elements)
+CREATE INDEX t_items_gin ON t USING GIN ((data->'items'));
+INSERT INTO t VALUES (1, '{"items": [1, 2, 3], "status": "active"}');
+-- Update: Reorder array elements
+-- JSONB equality: NOT equal (different arrays)
+-- GIN extraction: Same elements extracted (might allow HOT if not careful)
+UPDATE t SET data = '{"items": [3, 2, 1], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update: Add/remove element
+UPDATE t SET data = '{"items": [1, 2, 3, 4], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- TEST: GIN with TOASTed TEXT (tsvector)
+-- ================================================================
+CREATE TABLE t(id INT, content TEXT, search_vec tsvector)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create trigger to maintain tsvector
+CREATE TRIGGER tsvectorupdate_toast
+ BEFORE INSERT OR UPDATE ON t
+ FOR EACH ROW EXECUTE FUNCTION
+ tsvector_update_trigger(search_vec, 'pg_catalog.english', content);
+CREATE INDEX t_gin ON t USING gin(search_vec);
+-- Insert with large content (will be TOASTed)
+INSERT INTO t (id, content) VALUES
+ (1, repeat('important keyword ', 1000) || repeat('filler text ', 10000));
+-- Verify initial state
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('important');
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+-- IMPORTANT: The BEFORE UPDATE trigger modifies search_vec, so by the time
+-- ExecWhichIndexesRequireUpdates() runs, search_vec has already changed.
+-- This means the comparison sees old tsvector vs. trigger-modified tsvector,
+-- not the natural progression. HOT won't happen because the trigger changed
+-- the indexed column.
+-- Update: Even though content keywords unchanged, trigger still fires
+UPDATE t
+SET content = repeat('important keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (trigger modifies search_vec, blocking HOT)
+-- This is actually correct behavior - the trigger updated an indexed column
+-- Update: Change indexed keywords
+UPDATE t
+SET content = repeat('critical keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (index keys changed)
+-- Verify query correctness
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('critical');
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+DROP TABLE t CASCADE;
+-- ================================================================
+-- TEST: GIN with Array of Large Strings
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin(tags);
+-- Insert with large array elements (might be TOASTed)
+INSERT INTO t (id, tags) VALUES
+ (1, ARRAY[repeat('tag1', 1000), repeat('tag2', 1000)]);
+-- Update: Change to different large values - NOT HOT
+UPDATE t
+SET tags = ARRAY[repeat('tag3', 1000), repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (keys actually changed)
+-- Update: Keep same tag values, just reorder - SHOULD BE HOT
+-- (GIN is order-insensitive: both [tag3,tag4] and [tag4,tag3]
+-- extract to the same sorted key set ['tag3','tag4'])
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000), repeat('tag3', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Expected: 1 HOT (GIN keys semantically identical)
+-- Update: Remove an element - NOT HOT (keys changed)
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Expected: Still 1 HOT (not this one)
+DROP TABLE t CASCADE;
+-- ================================================================
+-- BRIN Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ value INT,
+ description TEXT
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_partial_idx ON t USING brin(value) WHERE value > 100;
+INSERT INTO t VALUES (1, 50, 'below range');
+-- Test 1: Outside predicate
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Validate: Predicate query returns 0 rows
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+ cnt
+-----
+ 0
+(1 row)
+
+-- Test 2: Transition into predicate
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Validate: Predicate query returns 1 row with correct value
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+ cnt | max_val
+-----+---------
+ 1 | 150
+(1 row)
+
+-- Test 3: Inside predicate, value changes
+UPDATE t SET value = 160, description = 'updated again' WHERE id = 1;
+SELECT * FROM check_hot_updates(3);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 3 | 100.00 | t
+(1 row)
+
+-- Validate: Updated value (160) is returned
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+ cnt | max_val
+-----+---------
+ 1 | 160
+(1 row)
+
+-- Test 4: Transition out of predicate
+UPDATE t SET value = 50 WHERE id = 1;
+SELECT * FROM check_hot_updates(4);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 4 | 100.00 | t
+(1 row)
+
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+ cnt
+-----
+ 0
+(1 row)
+
+SELECT id, value, description FROM t;
+ id | value | description
+----+-------+---------------
+ 1 | 50 | updated again
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- HASH Index (Simple Column)
+-- ================================================================
+CREATE TABLE t(id INT, code VARCHAR(20), description TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_idx ON t USING hash(code);
+INSERT INTO t VALUES (1, 'CODE001', 'initial');
+-- Update non-indexed column - should be HOT
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update indexed column - HASH index requires update, NOT HOT
+UPDATE t SET code = 'CODE002' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update both - NOT HOT
+UPDATE t SET code = 'CODE003', description = 'changed' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Back to original code - NOT HOT (different hash bucket location)
+UPDATE t SET code = 'CODE001' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- HASH Index on Expression
+-- ================================================================
+CREATE TABLE t(id INT, email TEXT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_lower_email_idx ON t USING HASH(lower(email));
+INSERT INTO t VALUES (1, '[email protected]', '{"status": "new"}');
+-- Update non-indexed field - should be HOT
+UPDATE t SET data = '{"status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update email with case change only (same lowercase) - should be HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Update email to different lowercase - NOT HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 2 | 66.67 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- Multiple HASH Indexes
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, status VARCHAR, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+CREATE INDEX t_hash_status_idx ON t USING hash(status);
+INSERT INTO t VALUES (1, 'electronics', 'active', 100);
+-- Update non-indexed column - should be HOT
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update one indexed column - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update other indexed column - NOT HOT
+UPDATE t SET status = 'inactive' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Update both indexed columns - NOT HOT
+UPDATE t SET category = 'videos', status = 'pending' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- ================================================================
+-- BRIN vs HASH Comparison
+-- ================================================================
+CREATE TABLE t_brin(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE TABLE t_hash(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_value_idx ON t_brin USING brin(value);
+CREATE INDEX t_hash_value_idx ON t_hash USING hash(value);
+INSERT INTO t_brin VALUES (1, 100, 'initial');
+INSERT INTO t_hash VALUES (1, 100, 'initial');
+-- Same update on both - different HOT behavior expected
+-- BRIN: might allow HOT (range summary unchanged)
+-- HASH: blocks HOT (hash bucket changed)
+UPDATE t_brin SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't_brin');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t_brin | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT (BRIN allows it for single row)
+UPDATE t_hash SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't_hash');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t_hash | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (HASH blocks it)
+DROP TABLE t_brin CASCADE;
+DROP TABLE t_hash CASCADE;
+-- ================================================================
+-- HASH Index with NULL Values
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'electronics', 'initial');
+-- Update indexed column to NULL - NOT HOT (hash value changed)
+UPDATE t SET category = NULL WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT
+-- Update indexed column from NULL to value - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Expected: 1 HOT
+DROP TABLE t CASCADE;
+-- ================================================================
+-- BRIN on JSONB Field
+-- ================================================================
+CREATE TABLE t(id INT, metrics JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- BRIN doesn't directly support JSONB, but we can test on expression
+CREATE INDEX t_brin_count_idx ON t USING brin(
+ CAST(metrics->>'count' AS INTEGER)
+);
+INSERT INTO t VALUES (1, '{"count": "100", "timestamp": "2024-01-01"}');
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET metrics = '{"count": "100", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT
+-- Update indexed field - BRIN allows HOT for single row
+UPDATE t SET metrics = '{"count": "150", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Expected: 2 HOT (BRIN permits single-row updates)
+DROP TABLE t CASCADE;
+-- ================================================================
+-- Mixed BRIN + HASH on Same Table
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, timestamp TIMESTAMP, price NUMERIC, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_timestamp_idx ON t USING brin(timestamp);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'books', '2024-01-01 10:00:00', 29.99, 'initial');
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT
+-- Update BRIN indexed column - allows HOT
+UPDATE t SET timestamp = '2024-01-02 10:00:00' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Expected: 2 HOT
+-- Update HASH indexed column - blocks HOT
+UPDATE t SET category = 'videos' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 2 | 66.67 | t
+(1 row)
+
+-- Expected: 2 HOT (HASH blocks it)
+-- Update price (non-indexed) - should be HOT
+UPDATE t SET price = 39.99 WHERE id = 1;
+SELECT * FROM check_hot_updates(3);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 3 | 75.00 | t
+(1 row)
+
+-- Expected: 3 HOT
+DROP TABLE t CASCADE;
+-- ================================================================
+-- Index both on a field in a JSONB document, and the document
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t((docs->'name'));
+CREATE INDEX t_docs_col_idx ON t(docs);
+INSERT INTO t VALUES (1, '{"name": "john", "data": "some data"}');
+-- Update impacts index on whole docment attribute, can't go HOT
+UPDATE t SET docs='{"name": "john", "data": "some other data"}' WHERE id=1;
+SELECT * FROM check_hot_updates(0);
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t CASCADE;
+-- Cleanup
+DROP FUNCTION check_hot_updates(int, text, text);
+DROP COLLATION case_insensitive;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 905f9bca959..3f0eab3131b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -125,6 +125,12 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
test: partition_merge partition_split partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression compression_lz4 memoize stats predicate numa eager_aggregate
+
+# ----------
+# Another group of parallel tests, these focused on heap HOT updates
+# ----------
+test: heap_hot_updates
+
# event_trigger depends on create_am and cannot run concurrently with
# any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/sql/heap_hot_updates.sql b/src/test/regress/sql/heap_hot_updates.sql
new file mode 100644
index 00000000000..e047bcddf5c
--- /dev/null
+++ b/src/test/regress/sql/heap_hot_updates.sql
@@ -0,0 +1,513 @@
+-- ================================================================
+-- Test Suite for Heap-only (HOT) Updates
+-- ================================================================
+
+-- Setup: Create function to measure HOT updates
+CREATE OR REPLACE FUNCTION check_hot_updates(
+ expected INT,
+ p_table_name TEXT DEFAULT 't',
+ p_schema_name TEXT DEFAULT current_schema()
+)
+RETURNS TABLE (
+ table_name TEXT,
+ total_updates BIGINT,
+ hot_updates BIGINT,
+ hot_update_percentage NUMERIC,
+ matches_expected BOOLEAN
+)
+LANGUAGE plpgsql
+AS $$
+DECLARE
+ v_relid oid;
+ v_qualified_name TEXT;
+ v_hot_updates BIGINT;
+ v_updates BIGINT;
+ v_xact_hot_updates BIGINT;
+ v_xact_updates BIGINT;
+BEGIN
+ -- Force statistics update
+ PERFORM pg_stat_force_next_flush();
+
+ -- Get table OID
+ v_qualified_name := quote_ident(p_schema_name) || '.' || quote_ident(p_table_name);
+ v_relid := v_qualified_name::regclass;
+
+ IF v_relid IS NULL THEN
+ RAISE EXCEPTION 'Table %.% not found', p_schema_name, p_table_name;
+ END IF;
+
+ -- Get cumulative + transaction stats
+ v_hot_updates := COALESCE(pg_stat_get_tuples_hot_updated(v_relid), 0);
+ v_updates := COALESCE(pg_stat_get_tuples_updated(v_relid), 0);
+ v_xact_hot_updates := COALESCE(pg_stat_get_xact_tuples_hot_updated(v_relid), 0);
+ v_xact_updates := COALESCE(pg_stat_get_xact_tuples_updated(v_relid), 0);
+
+ v_hot_updates := v_hot_updates + v_xact_hot_updates;
+ v_updates := v_updates + v_xact_updates;
+
+ RETURN QUERY
+ SELECT
+ p_table_name::TEXT,
+ v_updates::BIGINT,
+ v_hot_updates::BIGINT,
+ CASE WHEN v_updates > 0
+ THEN ROUND((v_hot_updates::numeric / v_updates::numeric * 100)::numeric, 2)
+ ELSE 0
+ END,
+ (v_hot_updates = expected)::BOOLEAN;
+END;
+$$;
+
+CREATE COLLATION case_insensitive (
+ provider = libc,
+ locale = 'C'
+);
+
+
+-- ================================================================
+-- GIN Index on JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data);
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "database"]}');
+
+-- Change tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+
+-- Change tags again - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+
+-- Add field without changing existing keys - GIN keys changed (added "note"), NOT HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "note": "test"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+
+DROP TABLE t;
+
+
+-- ================================================================
+-- GIN Index with Unchanged Keys
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create GIN index on specific path
+CREATE INDEX t_gin_idx ON t USING gin((data->'tags'));
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "sql"], "status": "active"}');
+
+-- Change non-indexed field - GIN keys on 'tags' unchanged, should be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Change indexed tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t;
+
+
+-- ================================================================
+-- GIN with jsonb_path_ops
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data jsonb_path_ops);
+INSERT INTO t VALUES (1, '{"user": {"name": "alice"}, "tags": ["a", "b"]}');
+
+-- Change value at different path - keys changed, NOT HOT
+UPDATE t SET data = '{"user": {"name": "bob"}, "tags": ["a", "b"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+
+DROP TABLE t;
+
+
+-- ================================================================
+-- Mixed Index Types (BRIN + Expression)
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_idx ON t USING brin(value);
+CREATE INDEX t_expr_idx ON t((data->'status'));
+INSERT INTO t VALUES (1, 100, '{"status": "active"}');
+
+-- Update only BRIN column - should be HOT
+UPDATE t SET value = 200 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update only expression column - should NOT be HOT
+UPDATE t SET data = '{"status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update both - should NOT be HOT
+UPDATE t SET value = 300, data = '{"status": "pending"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t;
+
+
+-- ================================================================
+-- GIN Array Index - Order Insensitive Extraction
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ data JSONB
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+
+-- GIN index on JSONB array (extracts all elements)
+CREATE INDEX t_items_gin ON t USING GIN ((data->'items'));
+
+INSERT INTO t VALUES (1, '{"items": [1, 2, 3], "status": "active"}');
+
+-- Update: Reorder array elements
+-- JSONB equality: NOT equal (different arrays)
+-- GIN extraction: Same elements extracted (might allow HOT if not careful)
+UPDATE t SET data = '{"items": [3, 2, 1], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update: Add/remove element
+UPDATE t SET data = '{"items": [1, 2, 3, 4], "status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t;
+
+
+-- ================================================================
+-- TEST: GIN with TOASTed TEXT (tsvector)
+-- ================================================================
+CREATE TABLE t(id INT, content TEXT, search_vec tsvector)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+-- Create trigger to maintain tsvector
+CREATE TRIGGER tsvectorupdate_toast
+ BEFORE INSERT OR UPDATE ON t
+ FOR EACH ROW EXECUTE FUNCTION
+ tsvector_update_trigger(search_vec, 'pg_catalog.english', content);
+
+CREATE INDEX t_gin ON t USING gin(search_vec);
+
+-- Insert with large content (will be TOASTed)
+INSERT INTO t (id, content) VALUES
+ (1, repeat('important keyword ', 1000) || repeat('filler text ', 10000));
+
+-- Verify initial state
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('important');
+-- Expected: 1 row
+
+-- IMPORTANT: The BEFORE UPDATE trigger modifies search_vec, so by the time
+-- ExecWhichIndexesRequireUpdates() runs, search_vec has already changed.
+-- This means the comparison sees old tsvector vs. trigger-modified tsvector,
+-- not the natural progression. HOT won't happen because the trigger changed
+-- the indexed column.
+
+-- Update: Even though content keywords unchanged, trigger still fires
+UPDATE t
+SET content = repeat('important keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+-- Expected: 0 HOT (trigger modifies search_vec, blocking HOT)
+-- This is actually correct behavior - the trigger updated an indexed column
+
+-- Update: Change indexed keywords
+UPDATE t
+SET content = repeat('critical keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+-- Expected: 0 HOT (index keys changed)
+
+-- Verify query correctness
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('critical');
+-- Expected: 1 row
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- TEST: GIN with Array of Large Strings
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin(tags);
+
+-- Insert with large array elements (might be TOASTed)
+INSERT INTO t (id, tags) VALUES
+ (1, ARRAY[repeat('tag1', 1000), repeat('tag2', 1000)]);
+
+-- Update: Change to different large values - NOT HOT
+UPDATE t
+SET tags = ARRAY[repeat('tag3', 1000), repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+-- Expected: 0 HOT (keys actually changed)
+
+-- Update: Keep same tag values, just reorder - SHOULD BE HOT
+-- (GIN is order-insensitive: both [tag3,tag4] and [tag4,tag3]
+-- extract to the same sorted key set ['tag3','tag4'])
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000), repeat('tag3', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: 1 HOT (GIN keys semantically identical)
+
+-- Update: Remove an element - NOT HOT (keys changed)
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: Still 1 HOT (not this one)
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- BRIN Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(
+ id INT PRIMARY KEY,
+ value INT,
+ description TEXT
+) WITH (autovacuum_enabled = off, fillfactor = 70);
+
+CREATE INDEX t_brin_partial_idx ON t USING brin(value) WHERE value > 100;
+
+INSERT INTO t VALUES (1, 50, 'below range');
+
+-- Test 1: Outside predicate
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Validate: Predicate query returns 0 rows
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+
+-- Test 2: Transition into predicate
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+
+-- Validate: Predicate query returns 1 row with correct value
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+
+-- Test 3: Inside predicate, value changes
+UPDATE t SET value = 160, description = 'updated again' WHERE id = 1;
+SELECT * FROM check_hot_updates(3);
+
+-- Validate: Updated value (160) is returned
+SELECT COUNT(*) as cnt, MAX(value) as max_val FROM t WHERE value > 100;
+
+-- Test 4: Transition out of predicate
+UPDATE t SET value = 50 WHERE id = 1;
+SELECT * FROM check_hot_updates(4);
+
+SELECT COUNT(*) as cnt FROM t WHERE value > 100;
+
+SELECT id, value, description FROM t;
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- HASH Index (Simple Column)
+-- ================================================================
+CREATE TABLE t(id INT, code VARCHAR(20), description TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_idx ON t USING hash(code);
+INSERT INTO t VALUES (1, 'CODE001', 'initial');
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET description = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update indexed column - HASH index requires update, NOT HOT
+UPDATE t SET code = 'CODE002' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update both - NOT HOT
+UPDATE t SET code = 'CODE003', description = 'changed' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Back to original code - NOT HOT (different hash bucket location)
+UPDATE t SET code = 'CODE001' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- HASH Index on Expression
+-- ================================================================
+CREATE TABLE t(id INT, email TEXT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_lower_email_idx ON t USING HASH(lower(email));
+INSERT INTO t VALUES (1, '[email protected]', '{"status": "new"}');
+
+-- Update non-indexed field - should be HOT
+UPDATE t SET data = '{"status": "active"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update email with case change only (same lowercase) - should be HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+
+-- Update email to different lowercase - NOT HOT
+UPDATE t SET email = '[email protected]' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- Multiple HASH Indexes
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, status VARCHAR, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+CREATE INDEX t_hash_status_idx ON t USING hash(status);
+INSERT INTO t VALUES (1, 'electronics', 'active', 100);
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update one indexed column - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update other indexed column - NOT HOT
+UPDATE t SET status = 'inactive' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+-- Update both indexed columns - NOT HOT
+UPDATE t SET category = 'videos', status = 'pending' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- BRIN vs HASH Comparison
+-- ================================================================
+CREATE TABLE t_brin(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE TABLE t_hash(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+CREATE INDEX t_brin_value_idx ON t_brin USING brin(value);
+CREATE INDEX t_hash_value_idx ON t_hash USING hash(value);
+
+INSERT INTO t_brin VALUES (1, 100, 'initial');
+INSERT INTO t_hash VALUES (1, 100, 'initial');
+
+-- Same update on both - different HOT behavior expected
+-- BRIN: might allow HOT (range summary unchanged)
+-- HASH: blocks HOT (hash bucket changed)
+UPDATE t_brin SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't_brin');
+-- Expected: 1 HOT (BRIN allows it for single row)
+
+UPDATE t_hash SET value = 150 WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't_hash');
+-- Expected: 0 HOT (HASH blocks it)
+
+DROP TABLE t_brin CASCADE;
+DROP TABLE t_hash CASCADE;
+
+
+-- ================================================================
+-- HASH Index with NULL Values
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'electronics', 'initial');
+
+-- Update indexed column to NULL - NOT HOT (hash value changed)
+UPDATE t SET category = NULL WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+-- Expected: 0 HOT
+
+-- Update indexed column from NULL to value - NOT HOT
+UPDATE t SET category = 'books' WHERE id = 1;
+SELECT * FROM check_hot_updates(0);
+-- Expected: 0 HOT
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: 1 HOT
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- BRIN on JSONB Field
+-- ================================================================
+CREATE TABLE t(id INT, metrics JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- BRIN doesn't directly support JSONB, but we can test on expression
+CREATE INDEX t_brin_count_idx ON t USING brin(
+ CAST(metrics->>'count' AS INTEGER)
+);
+INSERT INTO t VALUES (1, '{"count": "100", "timestamp": "2024-01-01"}');
+
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET metrics = '{"count": "100", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: 1 HOT
+
+-- Update indexed field - BRIN allows HOT for single row
+UPDATE t SET metrics = '{"count": "150", "timestamp": "2024-01-02"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+-- Expected: 2 HOT (BRIN permits single-row updates)
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- Mixed BRIN + HASH on Same Table
+-- ================================================================
+CREATE TABLE t(id INT, category VARCHAR, timestamp TIMESTAMP, price NUMERIC, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_timestamp_idx ON t USING brin(timestamp);
+CREATE INDEX t_hash_category_idx ON t USING hash(category);
+INSERT INTO t VALUES (1, 'books', '2024-01-01 10:00:00', 29.99, 'initial');
+
+-- Update non-indexed column - should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1);
+-- Expected: 1 HOT
+
+-- Update BRIN indexed column - allows HOT
+UPDATE t SET timestamp = '2024-01-02 10:00:00' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+-- Expected: 2 HOT
+
+-- Update HASH indexed column - blocks HOT
+UPDATE t SET category = 'videos' WHERE id = 1;
+SELECT * FROM check_hot_updates(2);
+-- Expected: 2 HOT (HASH blocks it)
+
+-- Update price (non-indexed) - should be HOT
+UPDATE t SET price = 39.99 WHERE id = 1;
+SELECT * FROM check_hot_updates(3);
+-- Expected: 3 HOT
+
+DROP TABLE t CASCADE;
+
+
+-- ================================================================
+-- Index both on a field in a JSONB document, and the document
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB) WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_idx ON t((docs->'name'));
+CREATE INDEX t_docs_col_idx ON t(docs);
+INSERT INTO t VALUES (1, '{"name": "john", "data": "some data"}');
+
+-- Update impacts index on whole docment attribute, can't go HOT
+UPDATE t SET docs='{"name": "john", "data": "some other data"}' WHERE id=1;
+SELECT * FROM check_hot_updates(0);
+
+DROP TABLE t CASCADE;
+
+
+-- Cleanup
+DROP FUNCTION check_hot_updates(int, text, text);
+DROP COLLATION case_insensitive;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 3451538565e..2c457af2257 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -398,6 +398,7 @@ CachedFunctionCompileCallback
CachedFunctionDeleteCallback
CachedFunctionHashEntry
CachedFunctionHashKey
+CachedIndexDatum
CachedPlan
CachedPlanSource
CallContext
--
2.51.2
[application/octet-stream] hot_test.sql (7.8K, 6-hot_test.sql)
download
[application/octet-stream] setup.sql (22.3K, 7-setup.sql)
download
[application/octet-stream] cf-5556-flame-a.svg (933.3K, 8-cf-5556-flame-a.svg)
download
[application/octet-stream] master-flame-a.svg (928.2K, 9-master-flame-a.svg)
download
[text/plain] cf-5556-a.txt (6.5K, 10-cf-5556-a.txt)
download | inline:
pgbench (19devel)
progress: 1765546063.294 s, 65870.5 tps, lat 0.298 ms stddev 0.136, 0 failed
progress: 1765546068.294 s, 62181.9 tps, lat 0.316 ms stddev 0.137, 0 failed
progress: 1765546073.294 s, 61075.5 tps, lat 0.322 ms stddev 0.141, 0 failed
progress: 1765546078.294 s, 60555.0 tps, lat 0.325 ms stddev 0.140, 0 failed
progress: 1765546083.294 s, 59835.7 tps, lat 0.329 ms stddev 0.144, 0 failed
progress: 1765546088.294 s, 59338.7 tps, lat 0.331 ms stddev 0.148, 0 failed
progress: 1765546093.294 s, 58621.2 tps, lat 0.335 ms stddev 0.147, 0 failed
progress: 1765546098.294 s, 58402.5 tps, lat 0.337 ms stddev 0.154, 0 failed
progress: 1765546103.294 s, 58237.6 tps, lat 0.338 ms stddev 0.153, 0 failed
progress: 1765546108.294 s, 57781.4 tps, lat 0.340 ms stddev 0.145, 0 failed
progress: 1765546113.294 s, 57349.2 tps, lat 0.343 ms stddev 0.146, 0 failed
progress: 1765546118.294 s, 57205.5 tps, lat 0.344 ms stddev 0.144, 0 failed
progress: 1765546123.294 s, 56764.3 tps, lat 0.346 ms stddev 0.147, 0 failed
progress: 1765546128.294 s, 56694.2 tps, lat 0.347 ms stddev 0.155, 0 failed
progress: 1765546133.294 s, 56636.1 tps, lat 0.347 ms stddev 0.144, 0 failed
progress: 1765546138.294 s, 56498.2 tps, lat 0.348 ms stddev 0.146, 0 failed
progress: 1765546143.294 s, 56283.5 tps, lat 0.349 ms stddev 0.146, 0 failed
progress: 1765546148.294 s, 56153.6 tps, lat 0.350 ms stddev 0.147, 0 failed
progress: 1765546153.294 s, 56238.9 tps, lat 0.350 ms stddev 0.148, 0 failed
progress: 1765546158.294 s, 56077.8 tps, lat 0.351 ms stddev 0.148, 0 failed
progress: 1765546163.294 s, 56395.6 tps, lat 0.349 ms stddev 0.146, 0 failed
progress: 1765546168.294 s, 56297.3 tps, lat 0.349 ms stddev 0.146, 0 failed
progress: 1765546173.294 s, 56443.2 tps, lat 0.348 ms stddev 0.145, 0 failed
progress: 1765546178.294 s, 56345.4 tps, lat 0.349 ms stddev 0.145, 0 failed
progress: 1765546183.294 s, 56386.4 tps, lat 0.349 ms stddev 0.147, 0 failed
progress: 1765546188.294 s, 56389.0 tps, lat 0.349 ms stddev 0.150, 0 failed
progress: 1765546193.294 s, 56217.2 tps, lat 0.350 ms stddev 0.149, 0 failed
progress: 1765546198.294 s, 56250.8 tps, lat 0.350 ms stddev 0.152, 0 failed
progress: 1765546203.294 s, 56072.9 tps, lat 0.351 ms stddev 0.151, 0 failed
progress: 1765546208.294 s, 56065.7 tps, lat 0.351 ms stddev 0.146, 0 failed
progress: 1765546213.294 s, 56176.4 tps, lat 0.350 ms stddev 0.145, 0 failed
progress: 1765546218.294 s, 55823.8 tps, lat 0.352 ms stddev 0.146, 0 failed
progress: 1765546223.294 s, 56159.1 tps, lat 0.350 ms stddev 0.142, 0 failed
progress: 1765546228.294 s, 56028.1 tps, lat 0.351 ms stddev 0.142, 0 failed
progress: 1765546233.294 s, 56129.9 tps, lat 0.350 ms stddev 0.143, 0 failed
progress: 1765546238.294 s, 56168.9 tps, lat 0.350 ms stddev 0.141, 0 failed
progress: 1765546243.294 s, 56247.2 tps, lat 0.349 ms stddev 0.142, 0 failed
progress: 1765546248.294 s, 56092.8 tps, lat 0.350 ms stddev 0.146, 0 failed
progress: 1765546253.294 s, 56133.7 tps, lat 0.350 ms stddev 0.143, 0 failed
progress: 1765546258.294 s, 56178.2 tps, lat 0.350 ms stddev 0.144, 0 failed
progress: 1765546263.294 s, 56160.4 tps, lat 0.350 ms stddev 0.143, 0 failed
progress: 1765546268.294 s, 56067.4 tps, lat 0.351 ms stddev 0.142, 0 failed
progress: 1765546273.294 s, 56113.7 tps, lat 0.350 ms stddev 0.140, 0 failed
progress: 1765546278.294 s, 55889.0 tps, lat 0.352 ms stddev 0.142, 0 failed
progress: 1765546283.294 s, 56056.4 tps, lat 0.351 ms stddev 0.143, 0 failed
progress: 1765546288.294 s, 56131.6 tps, lat 0.350 ms stddev 0.142, 0 failed
progress: 1765546293.294 s, 56003.5 tps, lat 0.351 ms stddev 0.143, 0 failed
progress: 1765546298.294 s, 56040.5 tps, lat 0.351 ms stddev 0.139, 0 failed
progress: 1765546303.294 s, 55905.1 tps, lat 0.352 ms stddev 0.142, 0 failed
progress: 1765546308.294 s, 55932.0 tps, lat 0.352 ms stddev 0.142, 0 failed
progress: 1765546313.294 s, 55791.3 tps, lat 0.352 ms stddev 0.141, 0 failed
progress: 1765546318.294 s, 56021.1 tps, lat 0.351 ms stddev 0.141, 0 failed
progress: 1765546323.294 s, 55936.4 tps, lat 0.352 ms stddev 0.142, 0 failed
progress: 1765546328.294 s, 55782.7 tps, lat 0.352 ms stddev 0.142, 0 failed
progress: 1765546333.294 s, 55716.8 tps, lat 0.353 ms stddev 0.144, 0 failed
progress: 1765546338.294 s, 55571.8 tps, lat 0.354 ms stddev 0.145, 0 failed
progress: 1765546343.294 s, 55521.4 tps, lat 0.354 ms stddev 0.143, 0 failed
progress: 1765546348.294 s, 55767.0 tps, lat 0.353 ms stddev 0.140, 0 failed
progress: 1765546353.294 s, 55840.2 tps, lat 0.352 ms stddev 0.142, 0 failed
progress: 1765546358.294 s, 55840.3 tps, lat 0.352 ms stddev 0.144, 0 failed
progress: 1765546363.294 s, 55546.0 tps, lat 0.354 ms stddev 0.140, 0 failed
progress: 1765546368.294 s, 55789.7 tps, lat 0.352 ms stddev 0.144, 0 failed
progress: 1765546373.294 s, 55772.5 tps, lat 0.352 ms stddev 0.143, 0 failed
progress: 1765546378.294 s, 55674.0 tps, lat 0.353 ms stddev 0.143, 0 failed
progress: 1765546383.294 s, 55580.5 tps, lat 0.354 ms stddev 0.144, 0 failed
progress: 1765546388.294 s, 55417.2 tps, lat 0.355 ms stddev 0.142, 0 failed
progress: 1765546393.294 s, 55707.8 tps, lat 0.353 ms stddev 0.145, 0 failed
progress: 1765546398.294 s, 55542.2 tps, lat 0.354 ms stddev 0.146, 0 failed
progress: 1765546403.294 s, 55894.5 tps, lat 0.352 ms stddev 0.143, 0 failed
progress: 1765546408.294 s, 55926.0 tps, lat 0.352 ms stddev 0.140, 0 failed
progress: 1765546413.294 s, 55600.4 tps, lat 0.354 ms stddev 0.143, 0 failed
progress: 1765546418.299 s, 55548.0 tps, lat 0.354 ms stddev 0.141, 0 failed
transaction type: ../_bench/cf-5556/a/hot_test.sql
scaling factor: 1
query mode: simple
number of clients: 20
number of threads: 8
maximum number of tries: 1
duration: 360 s
number of transactions actually processed: 20389706
number of failed transactions: 0 (0.000%)
latency average = 0.347 ms
latency stddev = 0.145 ms
initial connection time = 9.011 ms
tps = 56638.715544 (without initial connection time)
table_name | total_updates | hot_updates | hot_percentage
------------+---------------+-------------+----------------
t | 20389706 | 20380083 | 99.95
(1 row)
schemaname | tablename | indexname | idx_scan | idx_tup_read | idx_tup_fetch
------------+-----------+-----------+----------+--------------+---------------
public | t | t_pkey | 20389706 | 20395813 | 20389706
(1 row)
relname | n_live_tup | n_dead_tup | dead_pct | last_vacuum | last_autovacuum
---------+------------+------------+----------+-------------+-------------------------------
t | 5000 | 0 | 0.00 | | 2025-12-12 08:34:28.502717-05
(1 row)
[text/plain] master-a.txt (6.5K, 11-master-a.txt)
download | inline:
pgbench (19devel)
progress: 1765546808.169 s, 65602.3 tps, lat 0.299 ms stddev 0.142, 0 failed
progress: 1765546813.169 s, 62570.1 tps, lat 0.314 ms stddev 0.134, 0 failed
progress: 1765546818.169 s, 61340.7 tps, lat 0.321 ms stddev 0.141, 0 failed
progress: 1765546823.169 s, 60863.1 tps, lat 0.323 ms stddev 0.139, 0 failed
progress: 1765546828.169 s, 60407.0 tps, lat 0.326 ms stddev 0.144, 0 failed
progress: 1765546833.169 s, 58721.0 tps, lat 0.335 ms stddev 0.153, 0 failed
progress: 1765546838.169 s, 59566.4 tps, lat 0.330 ms stddev 0.149, 0 failed
progress: 1765546843.169 s, 58650.7 tps, lat 0.335 ms stddev 0.145, 0 failed
progress: 1765546848.169 s, 58056.5 tps, lat 0.339 ms stddev 0.152, 0 failed
progress: 1765546853.169 s, 57890.3 tps, lat 0.340 ms stddev 0.146, 0 failed
progress: 1765546858.169 s, 57613.9 tps, lat 0.341 ms stddev 0.146, 0 failed
progress: 1765546863.169 s, 57045.3 tps, lat 0.345 ms stddev 0.155, 0 failed
progress: 1765546868.169 s, 57455.6 tps, lat 0.342 ms stddev 0.149, 0 failed
progress: 1765546873.169 s, 57153.1 tps, lat 0.344 ms stddev 0.147, 0 failed
progress: 1765546878.169 s, 57054.2 tps, lat 0.345 ms stddev 0.146, 0 failed
progress: 1765546883.169 s, 56485.9 tps, lat 0.348 ms stddev 0.150, 0 failed
progress: 1765546888.169 s, 56779.8 tps, lat 0.346 ms stddev 0.146, 0 failed
progress: 1765546893.169 s, 56518.7 tps, lat 0.348 ms stddev 0.150, 0 failed
progress: 1765546898.169 s, 56803.5 tps, lat 0.346 ms stddev 0.148, 0 failed
progress: 1765546903.169 s, 56828.7 tps, lat 0.346 ms stddev 0.145, 0 failed
progress: 1765546908.169 s, 56692.3 tps, lat 0.347 ms stddev 0.149, 0 failed
progress: 1765546913.169 s, 56599.2 tps, lat 0.347 ms stddev 0.146, 0 failed
progress: 1765546918.169 s, 56619.7 tps, lat 0.347 ms stddev 0.147, 0 failed
progress: 1765546923.171 s, 56553.4 tps, lat 0.348 ms stddev 0.146, 0 failed
progress: 1765546928.169 s, 56541.0 tps, lat 0.348 ms stddev 0.147, 0 failed
progress: 1765546933.169 s, 56485.8 tps, lat 0.348 ms stddev 0.146, 0 failed
progress: 1765546938.169 s, 56377.9 tps, lat 0.349 ms stddev 0.156, 0 failed
progress: 1765546943.169 s, 55958.7 tps, lat 0.351 ms stddev 0.151, 0 failed
progress: 1765546948.169 s, 56291.2 tps, lat 0.349 ms stddev 0.148, 0 failed
progress: 1765546953.169 s, 56467.2 tps, lat 0.348 ms stddev 0.147, 0 failed
progress: 1765546958.169 s, 56267.9 tps, lat 0.349 ms stddev 0.151, 0 failed
progress: 1765546963.169 s, 56496.5 tps, lat 0.348 ms stddev 0.139, 0 failed
progress: 1765546968.169 s, 56280.4 tps, lat 0.349 ms stddev 0.146, 0 failed
progress: 1765546973.169 s, 56377.1 tps, lat 0.349 ms stddev 0.141, 0 failed
progress: 1765546978.169 s, 56081.8 tps, lat 0.351 ms stddev 0.146, 0 failed
progress: 1765546983.169 s, 56055.3 tps, lat 0.351 ms stddev 0.142, 0 failed
progress: 1765546988.169 s, 56430.8 tps, lat 0.348 ms stddev 0.141, 0 failed
progress: 1765546993.169 s, 56250.5 tps, lat 0.349 ms stddev 0.142, 0 failed
progress: 1765546998.169 s, 56193.0 tps, lat 0.350 ms stddev 0.144, 0 failed
progress: 1765547003.169 s, 56063.7 tps, lat 0.351 ms stddev 0.144, 0 failed
progress: 1765547008.169 s, 56181.7 tps, lat 0.350 ms stddev 0.143, 0 failed
progress: 1765547013.169 s, 56113.7 tps, lat 0.350 ms stddev 0.144, 0 failed
progress: 1765547018.169 s, 56215.2 tps, lat 0.350 ms stddev 0.142, 0 failed
progress: 1765547023.169 s, 56192.0 tps, lat 0.350 ms stddev 0.143, 0 failed
progress: 1765547028.169 s, 55851.8 tps, lat 0.352 ms stddev 0.144, 0 failed
progress: 1765547033.169 s, 56043.4 tps, lat 0.351 ms stddev 0.143, 0 failed
progress: 1765547038.169 s, 55929.4 tps, lat 0.351 ms stddev 0.142, 0 failed
progress: 1765547043.169 s, 55845.0 tps, lat 0.352 ms stddev 0.140, 0 failed
progress: 1765547048.169 s, 55332.0 tps, lat 0.355 ms stddev 0.148, 0 failed
progress: 1765547053.169 s, 55800.3 tps, lat 0.352 ms stddev 0.141, 0 failed
progress: 1765547058.169 s, 55691.2 tps, lat 0.353 ms stddev 0.146, 0 failed
progress: 1765547063.169 s, 55722.8 tps, lat 0.353 ms stddev 0.144, 0 failed
progress: 1765547068.169 s, 55907.4 tps, lat 0.352 ms stddev 0.142, 0 failed
progress: 1765547073.169 s, 55247.6 tps, lat 0.356 ms stddev 0.146, 0 failed
progress: 1765547078.169 s, 55753.3 tps, lat 0.352 ms stddev 0.143, 0 failed
progress: 1765547083.169 s, 55602.9 tps, lat 0.354 ms stddev 0.143, 0 failed
progress: 1765547088.169 s, 56078.6 tps, lat 0.351 ms stddev 0.146, 0 failed
progress: 1765547093.169 s, 55520.2 tps, lat 0.354 ms stddev 0.143, 0 failed
progress: 1765547098.169 s, 55919.5 tps, lat 0.352 ms stddev 0.141, 0 failed
progress: 1765547103.169 s, 55386.3 tps, lat 0.355 ms stddev 0.145, 0 failed
progress: 1765547108.169 s, 55562.4 tps, lat 0.354 ms stddev 0.144, 0 failed
progress: 1765547113.169 s, 55481.5 tps, lat 0.354 ms stddev 0.144, 0 failed
progress: 1765547118.169 s, 55409.7 tps, lat 0.355 ms stddev 0.145, 0 failed
progress: 1765547123.169 s, 55557.7 tps, lat 0.354 ms stddev 0.144, 0 failed
progress: 1765547128.169 s, 55173.7 tps, lat 0.356 ms stddev 0.146, 0 failed
progress: 1765547133.169 s, 55129.8 tps, lat 0.357 ms stddev 0.146, 0 failed
progress: 1765547138.169 s, 55724.8 tps, lat 0.353 ms stddev 0.143, 0 failed
progress: 1765547143.169 s, 54879.1 tps, lat 0.358 ms stddev 0.146, 0 failed
progress: 1765547148.169 s, 54758.7 tps, lat 0.359 ms stddev 0.149, 0 failed
progress: 1765547153.169 s, 55343.8 tps, lat 0.355 ms stddev 0.143, 0 failed
progress: 1765547158.169 s, 54764.9 tps, lat 0.359 ms stddev 0.146, 0 failed
transaction type: ../_bench/cf-5556/a/hot_test.sql
scaling factor: 1
query mode: simple
number of clients: 20
number of threads: 8
maximum number of tries: 1
duration: 360 s
number of transactions actually processed: 20401562
number of failed transactions: 0 (0.000%)
latency average = 0.347 ms
latency stddev = 0.146 ms
initial connection time = 12.057 ms
tps = 56672.534455 (without initial connection time)
table_name | total_updates | hot_updates | hot_percentage
------------+---------------+-------------+----------------
t | 20401562 | 20393424 | 99.96
(1 row)
schemaname | tablename | indexname | idx_scan | idx_tup_read | idx_tup_fetch
------------+-----------+-----------+----------+--------------+---------------
public | t | t_pkey | 20401562 | 20406533 | 20401562
(1 row)
relname | n_live_tup | n_dead_tup | dead_pct | last_vacuum | last_autovacuum
---------+------------+------------+----------+-------------+-------------------------------
t | 5000 | 0 | 0.00 | | 2025-12-12 08:46:09.781718-05
(1 row)
[application/octet-stream] run_bench_perf.sh (11.8K, 12-run_bench_perf.sh)
download
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
@ 2026-03-02 19:08 Greg Burd <[email protected]>
2026-03-11 15:51 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Greg Burd @ 2026-03-02 19:08 UTC (permalink / raw)
To: Jeff Davis <[email protected]>; +Cc: pgsql-hackers
Hello Jeff, hackers,
In v33 I've updated a test in triggers.sql to address differences across platforms identified by the cf-bot and rebased the work.
I thought it might be prudent to add tests that validate all the corner cases of HOT that I could come up with, maybe too many (you tell me). In addition, because code that impacts HOT is also involved in what is WAL logged for the purposes of logical replication, I've added tests that try to explore the corners of that too. The goal of these first few patches is to NOT change the behavior of these things, but to only move the logic into the executor and out of heap then it makes sense to validate that explicitly.
At some point when I get back to $subjet I'll want to document how things changed. The best way to do that is by changing tests along with code. So, that is "0001" in this v33 patch set.
I've also run longer performance tests which show minimal performance differences between master and patched.
Workload 60s 300s 600s
jsonb_write_batch +14.8% -7.0% +0.1%
jsonb_write_single +0.3% +0.2% -0.0%
license_write_single +0.4% +0.2% -0.1%
gin_write_single -0.3% -0.2% -0.4%
pgbench_simple-update +3.2% +6.2% +0.9%
pgbench_tpcb-like -1.1% +0.8% -2.0%
Changing tests isn't something I take lightly, I dug into this quite a bit. I ran an analysis of ALL regression tests comparing master vs patched after instrumenting the code (see below) so I could record HOT and replica identity decisions and record where the tuple landed on the page.
Patched code produced:
simple_heap_update: 17,028 calls (72.5% - catalog updates, direct heap ops)
heapam_tuple_update: 6,462 calls (27.5% - executor path via table AM)
Total entry points: 23,490
This matched master's log line output for the same tests.
Replica identity decisions were identical, 342 unique patterns with 0 differences.
HOT eligibility was also identical, 398 unique patterns matched, again 0 differences.
The physical placement of tuples on pages was 99.991% identical, only 2 of 23,473 updates had different buffer placement.
Across test runs there were a few differences noted for pg_sequence, target, and wslot. Both master and patched agreed on hot_allowed=1 (logic identical), but in some cases use_hot_update differed (buffer placement, newbuf =?= buffer). To me this reads as non-deterministic behavior, not a bug introduced in this patch.
At this point I'd say that v33 patch is functionally correct and performance neutral. This set of changes isn't exactly exciting on the surface, but I feel that it opens the door to other changes that will be more interesting/valuable down the line.
Thank you for your time and interest.
best.
-greg
COMPARISON TESTING NOTES:
---------------------------------------------------------------------------------------
src/backend/access/heap/heapam.c
3514: elog(LOG, "PATCHED heap_update (replica check): rel=%s otid=(%u,%u) rep_id_key_required=%d",
3515- RelationGetRelationName(relation),
3516- ItemPointerGetBlockNumber(otid),
3517- ItemPointerGetOffsetNumber(otid),
3518- rep_id_key_required);
3519-
--
4106: elog(LOG, "PATCHED heap_update (final HOT): rel=%s otid=(%u,%u) hot_allowed=%d newbuf==buffer=%d use_hot_update=%d",
4107- RelationGetRelationName(relation),
4108- ItemPointerGetBlockNumber(otid),
4109- ItemPointerGetOffsetNumber(otid),
4110- hot_allowed, (newbuf == buffer), use_hot_update);
4111-
--
4693: elog(LOG, "PATCHED simple_heap_update: rel=%s otid=(%u,%u) hot_allowed=%d summarized_only=%d lockmode=%d",
4694- RelationGetRelationName(relation),
4695- ItemPointerGetBlockNumber(otid),
4696- ItemPointerGetOffsetNumber(otid),
4697- hot_allowed, summarized_only, lockmode);
4698-
src/backend/access/heap/heapam_handler.c
333: elog(LOG, "PATCHED heapam_tuple_update: rel=%s otid=(%u,%u) hot_allowed=%d summarized_only=%d lockmode=%d",
334- RelationGetRelationName(relation),
335- ItemPointerGetBlockNumber(otid),
336- ItemPointerGetOffsetNumber(otid),
337- hot_allowed, summarized_only, *lockmode);
Attachments:
[text/x-patch] v33-0001-Add-comprehensive-tests-for-HOT-updates-and-repl.patch (102.7K, 2-v33-0001-Add-comprehensive-tests-for-HOT-updates-and-repl.patch)
download | inline diff:
From 95cda56b5e9d53232c9b5f95abe423e878b1fe78 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Fri, 27 Feb 2026 12:19:06 -0500
Subject: [PATCH v33 1/2] Add comprehensive tests for HOT updates and replica
identity
Adds regression and isolation tests covering:
- HOT update decisions across various index types (B-tree, BRIN,
partial, expression, multi-column, unique constraints)
- Replica identity key extraction for logical replication
(DEFAULT, FULL, USING INDEX, NOTHING modes)
- Concurrent HOT update scenarios (locking, blocking, index scans,
HOT chains, FOR UPDATE/KEY SHARE interactions)
Regression tests:
- hot_updates.sql: 10 scenarios testing HOT eligibility
- replica_identity_logging.sql: 11 scenarios verifying replica
identity keys logged to WAL via test_decoding
Isolation tests:
- hot_updates_concurrent.spec: concurrent updates on same/different rows
- hot_updates_index_scan.spec: interactions with index scans and row locks
- hot_updates_chain.spec: HOT chain building and snapshot isolation
---
.../isolation/expected/hot_updates_chain.out | 144 ++++
.../expected/hot_updates_concurrent.out | 143 ++++
.../expected/hot_updates_index_scan.out | 126 +++
src/test/isolation/isolation_schedule | 3 +
.../isolation/specs/hot_updates_chain.spec | 110 +++
.../specs/hot_updates_concurrent.spec | 107 +++
.../specs/hot_updates_index_scan.spec | 91 +++
src/test/regress/expected/hot_updates.out | 725 ++++++++++++++++++
.../expected/replica_identity_logging.out | 396 ++++++++++
src/test/regress/parallel_schedule | 7 +
src/test/regress/sql/hot_updates.sql | 553 +++++++++++++
.../regress/sql/replica_identity_logging.sql | 349 +++++++++
12 files changed, 2754 insertions(+)
create mode 100644 src/test/isolation/expected/hot_updates_chain.out
create mode 100644 src/test/isolation/expected/hot_updates_concurrent.out
create mode 100644 src/test/isolation/expected/hot_updates_index_scan.out
create mode 100644 src/test/isolation/specs/hot_updates_chain.spec
create mode 100644 src/test/isolation/specs/hot_updates_concurrent.spec
create mode 100644 src/test/isolation/specs/hot_updates_index_scan.spec
create mode 100644 src/test/regress/expected/hot_updates.out
create mode 100644 src/test/regress/expected/replica_identity_logging.out
create mode 100644 src/test/regress/sql/hot_updates.sql
create mode 100644 src/test/regress/sql/replica_identity_logging.sql
diff --git a/src/test/isolation/expected/hot_updates_chain.out b/src/test/isolation/expected/hot_updates_chain.out
new file mode 100644
index 00000000000..503252009ea
--- /dev/null
+++ b/src/test/isolation/expected/hot_updates_chain.out
@@ -0,0 +1,144 @@
+Parsed test spec with 5 sessions
+
+starting permutation: s1_begin s1_hot_update1 s1_hot_update2 s1_hot_update3 s1_commit s1_select s1_verify_hot
+step s1_begin: BEGIN;
+step s1_hot_update1: UPDATE hot_test SET non_indexed_col = 'update1' WHERE id = 1;
+step s1_hot_update2: UPDATE hot_test SET non_indexed_col = 'update2' WHERE id = 1;
+step s1_hot_update3: UPDATE hot_test SET non_indexed_col = 'update3' WHERE id = 1;
+step s1_commit: COMMIT;
+step s1_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 100|update3
+(1 row)
+
+step s1_verify_hot:
+ -- Check for HOT chain: LP_REDIRECT or tuple with t_ctid pointing to same page
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2 -- LP_REDIRECT indicates HOT chain
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0 -- same page
+ AND t_ctid != ('(0,' || lp || ')')::tid); -- different offset
+
+has_hot_chain
+-------------
+t
+(1 row)
+
+
+starting permutation: s2_begin s2_select_before s1_begin s1_hot_update1 s1_hot_update2 s1_commit s2_select_after s2_commit
+step s2_begin: BEGIN ISOLATION LEVEL REPEATABLE READ;
+step s2_select_before: SELECT non_indexed_col FROM hot_test WHERE id = 1;
+non_indexed_col
+---------------
+initial
+(1 row)
+
+step s1_begin: BEGIN;
+step s1_hot_update1: UPDATE hot_test SET non_indexed_col = 'update1' WHERE id = 1;
+step s1_hot_update2: UPDATE hot_test SET non_indexed_col = 'update2' WHERE id = 1;
+step s1_commit: COMMIT;
+step s2_select_after: SELECT non_indexed_col FROM hot_test WHERE id = 1;
+non_indexed_col
+---------------
+initial
+(1 row)
+
+step s2_commit: COMMIT;
+
+starting permutation: s1_begin s1_hot_update1 s1_hot_update2 s1_commit s3_begin s3_non_hot_update s3_commit s1_select
+step s1_begin: BEGIN;
+step s1_hot_update1: UPDATE hot_test SET non_indexed_col = 'update1' WHERE id = 1;
+step s1_hot_update2: UPDATE hot_test SET non_indexed_col = 'update2' WHERE id = 1;
+step s1_commit: COMMIT;
+step s3_begin: BEGIN;
+step s3_non_hot_update: UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+step s3_commit: COMMIT;
+step s1_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 150|update2
+(1 row)
+
+
+starting permutation: s1_begin s1_hot_update1 s1_commit s3_begin s3_non_hot_update s3_commit s4_begin s4_hot_after_non_hot s4_commit s4_select s4_verify_hot
+step s1_begin: BEGIN;
+step s1_hot_update1: UPDATE hot_test SET non_indexed_col = 'update1' WHERE id = 1;
+step s1_commit: COMMIT;
+step s3_begin: BEGIN;
+step s3_non_hot_update: UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+step s3_commit: COMMIT;
+step s4_begin: BEGIN;
+step s4_hot_after_non_hot: UPDATE hot_test SET non_indexed_col = 'after_non_hot' WHERE id = 1;
+step s4_commit: COMMIT;
+step s4_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 150|after_non_hot
+(1 row)
+
+step s4_verify_hot:
+ -- Check for new HOT chain after non-HOT update broke the previous chain
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0
+ AND t_ctid != ('(0,' || lp || ')')::tid);
+
+has_hot_chain
+-------------
+t
+(1 row)
+
+
+starting permutation: s1_begin s1_hot_update1 s1_hot_update2 s5_begin s5_hot_update_row2_1 s5_hot_update_row2_2 s1_commit s5_commit s1_select s5_select s1_verify_hot s5_verify_hot
+step s1_begin: BEGIN;
+step s1_hot_update1: UPDATE hot_test SET non_indexed_col = 'update1' WHERE id = 1;
+step s1_hot_update2: UPDATE hot_test SET non_indexed_col = 'update2' WHERE id = 1;
+step s5_begin: BEGIN;
+step s5_hot_update_row2_1: UPDATE hot_test SET non_indexed_col = 'row2_update1' WHERE id = 2;
+step s5_hot_update_row2_2: UPDATE hot_test SET non_indexed_col = 'row2_update2' WHERE id = 2;
+step s1_commit: COMMIT;
+step s5_commit: COMMIT;
+step s1_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 100|update2
+(1 row)
+
+step s5_select: SELECT * FROM hot_test WHERE id = 2;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 2| 200|row2_update2
+(1 row)
+
+step s1_verify_hot:
+ -- Check for HOT chain: LP_REDIRECT or tuple with t_ctid pointing to same page
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2 -- LP_REDIRECT indicates HOT chain
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0 -- same page
+ AND t_ctid != ('(0,' || lp || ')')::tid); -- different offset
+
+has_hot_chain
+-------------
+t
+(1 row)
+
+step s5_verify_hot:
+ -- Check for HOT chain on page 0
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0
+ AND t_ctid != ('(0,' || lp || ')')::tid);
+
+has_hot_chain
+-------------
+t
+(1 row)
+
diff --git a/src/test/isolation/expected/hot_updates_concurrent.out b/src/test/isolation/expected/hot_updates_concurrent.out
new file mode 100644
index 00000000000..b1a8b0cb7b2
--- /dev/null
+++ b/src/test/isolation/expected/hot_updates_concurrent.out
@@ -0,0 +1,143 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s1_begin s1_hot_update s2_begin s2_hot_update s1_commit s2_commit s1_select s2_select s2_verify_hot
+step s1_begin: BEGIN;
+step s1_hot_update: UPDATE hot_test SET non_indexed_col = 'updated_s1' WHERE id = 1;
+step s2_begin: BEGIN;
+step s2_hot_update: UPDATE hot_test SET non_indexed_col = 'updated_s2' WHERE id = 1; <waiting ...>
+step s1_commit: COMMIT;
+step s2_hot_update: <... completed>
+step s2_commit: COMMIT;
+step s1_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 100|updated_s2
+(1 row)
+
+step s2_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 100|updated_s2
+(1 row)
+
+step s2_verify_hot:
+ -- Check for HOT chain: look for LP_REDIRECT (lp_flags=2) or tuple with t_ctid pointing to same page
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2 -- LP_REDIRECT indicates HOT chain
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0 -- same page
+ AND t_ctid != ('(0,' || lp || ')')::tid); -- different offset
+
+has_hot_chain
+-------------
+t
+(1 row)
+
+
+starting permutation: s1_begin s1_hot_update s3_begin s3_non_hot_update s1_commit s3_commit s3_select s3_verify_index
+step s1_begin: BEGIN;
+step s1_hot_update: UPDATE hot_test SET non_indexed_col = 'updated_s1' WHERE id = 1;
+step s3_begin: BEGIN;
+step s3_non_hot_update: UPDATE hot_test SET indexed_col = 150 WHERE id = 1; <waiting ...>
+step s1_commit: COMMIT;
+step s3_non_hot_update: <... completed>
+step s3_commit: COMMIT;
+step s3_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 150|updated_s1
+(1 row)
+
+step s3_verify_index:
+ -- Verify index was updated (proves non-HOT)
+ SELECT COUNT(*) = 1 AS index_updated FROM hot_test WHERE indexed_col = 150;
+ SELECT COUNT(*) = 0 AS old_value_gone FROM hot_test WHERE indexed_col = 100;
+
+index_updated
+-------------
+t
+(1 row)
+
+old_value_gone
+--------------
+t
+(1 row)
+
+
+starting permutation: s3_begin s3_non_hot_update s1_begin s1_hot_update s3_commit s1_commit s1_select s1_verify_hot
+step s3_begin: BEGIN;
+step s3_non_hot_update: UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+step s1_begin: BEGIN;
+step s1_hot_update: UPDATE hot_test SET non_indexed_col = 'updated_s1' WHERE id = 1; <waiting ...>
+step s3_commit: COMMIT;
+step s1_hot_update: <... completed>
+step s1_commit: COMMIT;
+step s1_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 150|updated_s1
+(1 row)
+
+step s1_verify_hot:
+ -- Check for HOT chain: look for LP_REDIRECT (lp_flags=2) or tuple with t_ctid pointing to same page
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2 -- LP_REDIRECT indicates HOT chain
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0 -- same page
+ AND t_ctid != ('(0,' || lp || ')')::tid); -- different offset
+
+has_hot_chain
+-------------
+t
+(1 row)
+
+
+starting permutation: s1_begin s1_hot_update s4_begin s4_hot_update_row2 s1_commit s4_commit s1_select s4_select s1_verify_hot s4_verify_hot
+step s1_begin: BEGIN;
+step s1_hot_update: UPDATE hot_test SET non_indexed_col = 'updated_s1' WHERE id = 1;
+step s4_begin: BEGIN;
+step s4_hot_update_row2: UPDATE hot_test SET non_indexed_col = 'updated_s4' WHERE id = 2;
+step s1_commit: COMMIT;
+step s4_commit: COMMIT;
+step s1_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 100|updated_s1
+(1 row)
+
+step s4_select: SELECT * FROM hot_test WHERE id = 2;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 2| 200|updated_s4
+(1 row)
+
+step s1_verify_hot:
+ -- Check for HOT chain: look for LP_REDIRECT (lp_flags=2) or tuple with t_ctid pointing to same page
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2 -- LP_REDIRECT indicates HOT chain
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0 -- same page
+ AND t_ctid != ('(0,' || lp || ')')::tid); -- different offset
+
+has_hot_chain
+-------------
+t
+(1 row)
+
+step s4_verify_hot:
+ -- Check for HOT chain on page 0
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0
+ AND t_ctid != ('(0,' || lp || ')')::tid);
+
+has_hot_chain
+-------------
+t
+(1 row)
+
diff --git a/src/test/isolation/expected/hot_updates_index_scan.out b/src/test/isolation/expected/hot_updates_index_scan.out
new file mode 100644
index 00000000000..d72322b2146
--- /dev/null
+++ b/src/test/isolation/expected/hot_updates_index_scan.out
@@ -0,0 +1,126 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s1_begin s1_hot_update s2_begin s2_index_scan s1_commit s2_commit
+step s1_begin: BEGIN;
+step s1_hot_update: UPDATE hot_test SET non_indexed_col = 'hot_updated' WHERE id = 50;
+step s2_begin: BEGIN;
+step s2_index_scan: SELECT * FROM hot_test WHERE indexed_col = 500;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+50| 500|initial50
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_commit: COMMIT;
+
+starting permutation: s1_begin s1_non_hot_update s1_commit s2_begin s2_index_scan_new s2_commit s2_verify_index
+step s1_begin: BEGIN;
+step s1_non_hot_update: UPDATE hot_test SET indexed_col = 555 WHERE id = 50;
+step s1_commit: COMMIT;
+step s2_begin: BEGIN;
+step s2_index_scan_new: SELECT * FROM hot_test WHERE indexed_col = 555;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+50| 555|initial50
+(1 row)
+
+step s2_commit: COMMIT;
+step s2_verify_index:
+ -- After non-HOT update, verify index reflects the change
+ SELECT COUNT(*) = 1 AS found_new_value FROM hot_test WHERE indexed_col = 555;
+ SELECT COUNT(*) = 0 AS old_value_gone FROM hot_test WHERE indexed_col = 500;
+
+found_new_value
+---------------
+t
+(1 row)
+
+old_value_gone
+--------------
+t
+(1 row)
+
+
+starting permutation: s3_begin s3_select_for_update s1_begin s1_hot_update s3_commit s1_commit s1_verify_hot
+step s3_begin: BEGIN;
+step s3_select_for_update: SELECT * FROM hot_test WHERE id = 50 FOR UPDATE;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+50| 500|initial50
+(1 row)
+
+step s1_begin: BEGIN;
+step s1_hot_update: UPDATE hot_test SET non_indexed_col = 'hot_updated' WHERE id = 50; <waiting ...>
+step s3_commit: COMMIT;
+step s1_hot_update: <... completed>
+step s1_commit: COMMIT;
+step s1_verify_hot:
+ -- Verify HOT chain exists for row with id=50
+ SELECT EXISTS (
+ SELECT 1 FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0
+ AND t_ctid != ('(0,' || lp || ')')::tid)
+ ) AS has_hot_chain;
+
+has_hot_chain
+-------------
+t
+(1 row)
+
+
+starting permutation: s1_begin s1_hot_update s3_begin s3_select_for_update s1_commit s3_commit
+step s1_begin: BEGIN;
+step s1_hot_update: UPDATE hot_test SET non_indexed_col = 'hot_updated' WHERE id = 50;
+step s3_begin: BEGIN;
+step s3_select_for_update: SELECT * FROM hot_test WHERE id = 50 FOR UPDATE; <waiting ...>
+step s1_commit: COMMIT;
+step s3_select_for_update: <... completed>
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+50| 500|hot_updated
+(1 row)
+
+step s3_commit: COMMIT;
+
+starting permutation: s4_begin s4_select_for_key_share s1_begin s1_hot_update s4_commit s1_commit s1_verify_hot
+step s4_begin: BEGIN;
+step s4_select_for_key_share: SELECT * FROM hot_test WHERE id = 50 FOR KEY SHARE;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+50| 500|initial50
+(1 row)
+
+step s1_begin: BEGIN;
+step s1_hot_update: UPDATE hot_test SET non_indexed_col = 'hot_updated' WHERE id = 50;
+step s4_commit: COMMIT;
+step s1_commit: COMMIT;
+step s1_verify_hot:
+ -- Verify HOT chain exists for row with id=50
+ SELECT EXISTS (
+ SELECT 1 FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0
+ AND t_ctid != ('(0,' || lp || ')')::tid)
+ ) AS has_hot_chain;
+
+has_hot_chain
+-------------
+t
+(1 row)
+
+
+starting permutation: s4_begin s4_select_for_key_share s1_begin s1_non_hot_update s4_commit s1_commit
+step s4_begin: BEGIN;
+step s4_select_for_key_share: SELECT * FROM hot_test WHERE id = 50 FOR KEY SHARE;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+50| 500|initial50
+(1 row)
+
+step s1_begin: BEGIN;
+step s1_non_hot_update: UPDATE hot_test SET indexed_col = 555 WHERE id = 50;
+step s4_commit: COMMIT;
+step s1_commit: COMMIT;
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 4e466580cd4..46525b0a62a 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -19,6 +19,9 @@ test: multiple-row-versions
test: index-only-scan
test: index-only-bitmapscan
test: predicate-lock-hot-tuple
+test: hot_updates_concurrent
+test: hot_updates_index_scan
+test: hot_updates_chain
test: update-conflict-out
test: deadlock-simple
test: deadlock-hard
diff --git a/src/test/isolation/specs/hot_updates_chain.spec b/src/test/isolation/specs/hot_updates_chain.spec
new file mode 100644
index 00000000000..85cd2176133
--- /dev/null
+++ b/src/test/isolation/specs/hot_updates_chain.spec
@@ -0,0 +1,110 @@
+# Test HOT update chains and their interaction with VACUUM and page pruning
+#
+# This test verifies that HOT update chains are correctly maintained when
+# multiple HOT updates occur on the same row, and that VACUUM correctly
+# handles HOT chains.
+
+setup
+{
+ CREATE EXTENSION IF NOT EXISTS pageinspect;
+
+ CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+ );
+
+ CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
+
+ INSERT INTO hot_test VALUES (1, 100, 'initial');
+ INSERT INTO hot_test VALUES (2, 200, 'initial');
+}
+
+teardown
+{
+ DROP TABLE hot_test;
+ DROP EXTENSION pageinspect;
+}
+
+# Session 1: Create HOT chain with multiple updates
+session s1
+step s1_begin { BEGIN; }
+step s1_hot_update1 { UPDATE hot_test SET non_indexed_col = 'update1' WHERE id = 1; }
+step s1_hot_update2 { UPDATE hot_test SET non_indexed_col = 'update2' WHERE id = 1; }
+step s1_hot_update3 { UPDATE hot_test SET non_indexed_col = 'update3' WHERE id = 1; }
+step s1_commit { COMMIT; }
+step s1_select { SELECT * FROM hot_test WHERE id = 1; }
+step s1_verify_hot {
+ -- Check for HOT chain: LP_REDIRECT or tuple with t_ctid pointing to same page
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2 -- LP_REDIRECT indicates HOT chain
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0 -- same page
+ AND t_ctid != ('(0,' || lp || ')')::tid); -- different offset
+}
+
+# Session 2: Read while HOT chain is being built
+session s2
+step s2_begin { BEGIN ISOLATION LEVEL REPEATABLE READ; }
+step s2_select_before { SELECT non_indexed_col FROM hot_test WHERE id = 1; }
+step s2_select_after { SELECT non_indexed_col FROM hot_test WHERE id = 1; }
+step s2_commit { COMMIT; }
+
+# Session 3: Break HOT chain with non-HOT update
+session s3
+step s3_begin { BEGIN; }
+step s3_non_hot_update { UPDATE hot_test SET indexed_col = 150 WHERE id = 1; }
+step s3_commit { COMMIT; }
+
+# Session 4: Try to build HOT chain after non-HOT update
+session s4
+step s4_begin { BEGIN; }
+step s4_hot_after_non_hot { UPDATE hot_test SET non_indexed_col = 'after_non_hot' WHERE id = 1; }
+step s4_commit { COMMIT; }
+step s4_select { SELECT * FROM hot_test WHERE id = 1; }
+step s4_verify_hot {
+ -- Check for new HOT chain after non-HOT update broke the previous chain
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0
+ AND t_ctid != ('(0,' || lp || ')')::tid);
+}
+
+# Session 5: Multiple sessions building separate HOT chains on different rows
+session s5
+step s5_begin { BEGIN; }
+step s5_hot_update_row2_1 { UPDATE hot_test SET non_indexed_col = 'row2_update1' WHERE id = 2; }
+step s5_hot_update_row2_2 { UPDATE hot_test SET non_indexed_col = 'row2_update2' WHERE id = 2; }
+step s5_commit { COMMIT; }
+step s5_select { SELECT * FROM hot_test WHERE id = 2; }
+step s5_verify_hot {
+ -- Check for HOT chain on page 0
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0
+ AND t_ctid != ('(0,' || lp || ')')::tid);
+}
+
+# Build HOT chain within single transaction
+# All updates should form a HOT chain
+permutation s1_begin s1_hot_update1 s1_hot_update2 s1_hot_update3 s1_commit s1_select s1_verify_hot
+
+# REPEATABLE READ should see consistent snapshot across HOT chain updates
+# Session 2 starts before updates, should see 'initial' throughout
+permutation s2_begin s2_select_before s1_begin s1_hot_update1 s1_hot_update2 s1_commit s2_select_after s2_commit
+
+# HOT chain followed by non-HOT update
+# Non-HOT update breaks the HOT chain
+permutation s1_begin s1_hot_update1 s1_hot_update2 s1_commit s3_begin s3_non_hot_update s3_commit s1_select
+
+# HOT update after non-HOT update can start new HOT chain
+# After breaking chain with indexed column update, new HOT updates can start fresh chain
+permutation s1_begin s1_hot_update1 s1_commit s3_begin s3_non_hot_update s3_commit s4_begin s4_hot_after_non_hot s4_commit s4_select s4_verify_hot
+
+# Multiple sessions building separate HOT chains on different rows
+permutation s1_begin s1_hot_update1 s1_hot_update2 s5_begin s5_hot_update_row2_1 s5_hot_update_row2_2 s1_commit s5_commit s1_select s5_select s1_verify_hot s5_verify_hot
diff --git a/src/test/isolation/specs/hot_updates_concurrent.spec b/src/test/isolation/specs/hot_updates_concurrent.spec
new file mode 100644
index 00000000000..eac78d62ac5
--- /dev/null
+++ b/src/test/isolation/specs/hot_updates_concurrent.spec
@@ -0,0 +1,107 @@
+# Test concurrent HOT updates and validate HOT chains
+#
+# This test verifies that HOT updates work correctly when multiple sessions
+# are updating the same table concurrently, and validates that HOT chains
+# are actually created using heap_page_items().
+
+setup
+{
+ CREATE EXTENSION IF NOT EXISTS pageinspect;
+
+ CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+ );
+
+ CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
+
+ INSERT INTO hot_test VALUES (1, 100, 'initial1');
+ INSERT INTO hot_test VALUES (2, 200, 'initial2');
+ INSERT INTO hot_test VALUES (3, 300, 'initial3');
+}
+
+teardown
+{
+ DROP TABLE hot_test;
+ DROP EXTENSION pageinspect;
+}
+
+# Session 1: HOT update (modify non-indexed column)
+session s1
+step s1_begin { BEGIN; }
+step s1_hot_update { UPDATE hot_test SET non_indexed_col = 'updated_s1' WHERE id = 1; }
+step s1_commit { COMMIT; }
+step s1_select { SELECT * FROM hot_test WHERE id = 1; }
+step s1_verify_hot {
+ -- Check for HOT chain: look for LP_REDIRECT (lp_flags=2) or tuple with t_ctid pointing to same page
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2 -- LP_REDIRECT indicates HOT chain
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0 -- same page
+ AND t_ctid != ('(0,' || lp || ')')::tid); -- different offset
+}
+
+# Session 2: HOT update (modify non-indexed column on same row)
+session s2
+step s2_begin { BEGIN; }
+step s2_hot_update { UPDATE hot_test SET non_indexed_col = 'updated_s2' WHERE id = 1; }
+step s2_commit { COMMIT; }
+step s2_select { SELECT * FROM hot_test WHERE id = 1; }
+step s2_verify_hot {
+ -- Check for HOT chain: look for LP_REDIRECT (lp_flags=2) or tuple with t_ctid pointing to same page
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2 -- LP_REDIRECT indicates HOT chain
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0 -- same page
+ AND t_ctid != ('(0,' || lp || ')')::tid); -- different offset
+}
+
+# Session 3: Non-HOT update (modify indexed column)
+session s3
+step s3_begin { BEGIN; }
+step s3_non_hot_update { UPDATE hot_test SET indexed_col = 150 WHERE id = 1; }
+step s3_commit { COMMIT; }
+step s3_select { SELECT * FROM hot_test WHERE id = 1; }
+step s3_verify_index {
+ -- Verify index was updated (proves non-HOT)
+ SELECT COUNT(*) = 1 AS index_updated FROM hot_test WHERE indexed_col = 150;
+ SELECT COUNT(*) = 0 AS old_value_gone FROM hot_test WHERE indexed_col = 100;
+}
+
+# Session 4: Concurrent HOT updates on different rows
+session s4
+step s4_begin { BEGIN; }
+step s4_hot_update_row2 { UPDATE hot_test SET non_indexed_col = 'updated_s4' WHERE id = 2; }
+step s4_commit { COMMIT; }
+step s4_select { SELECT * FROM hot_test WHERE id = 2; }
+step s4_verify_hot {
+ -- Check for HOT chain on page 0
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0
+ AND t_ctid != ('(0,' || lp || ')')::tid);
+}
+
+# Two sessions both doing HOT updates on same row
+# Second session should block until first commits
+# Both should create HOT chains
+permutation s1_begin s1_hot_update s2_begin s2_hot_update s1_commit s2_commit s1_select s2_select s2_verify_hot
+
+# HOT update followed by non-HOT update
+# Non-HOT update should wait for HOT update to commit
+# First update is HOT, second is non-HOT (index updated)
+permutation s1_begin s1_hot_update s3_begin s3_non_hot_update s1_commit s3_commit s3_select s3_verify_index
+
+# Non-HOT update followed by HOT update
+# HOT update should wait for non-HOT update to commit
+# First update is non-HOT (index), second is HOT
+permutation s3_begin s3_non_hot_update s1_begin s1_hot_update s3_commit s1_commit s1_select s1_verify_hot
+
+# Concurrent HOT updates on different rows (should not block)
+# Both sessions should be able to create HOT chains independently
+permutation s1_begin s1_hot_update s4_begin s4_hot_update_row2 s1_commit s4_commit s1_select s4_select s1_verify_hot s4_verify_hot
diff --git a/src/test/isolation/specs/hot_updates_index_scan.spec b/src/test/isolation/specs/hot_updates_index_scan.spec
new file mode 100644
index 00000000000..39db07cc80f
--- /dev/null
+++ b/src/test/isolation/specs/hot_updates_index_scan.spec
@@ -0,0 +1,91 @@
+# Test HOT updates interaction with index scans and SELECT FOR UPDATE
+#
+# This test verifies that HOT updates are correctly handled when concurrent
+# sessions are performing index scans, using SELECT FOR UPDATE, and validates
+# HOT chains using heap_page_items().
+
+setup
+{
+ CREATE EXTENSION IF NOT EXISTS pageinspect;
+
+ CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+ );
+
+ CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
+
+ INSERT INTO hot_test SELECT i, i * 10, 'initial' || i FROM generate_series(1, 100) i;
+}
+
+teardown
+{
+ DROP TABLE hot_test;
+ DROP EXTENSION pageinspect;
+}
+
+# Session 1: Perform HOT update
+session s1
+step s1_begin { BEGIN; }
+step s1_hot_update { UPDATE hot_test SET non_indexed_col = 'hot_updated' WHERE id = 50; }
+step s1_non_hot_update { UPDATE hot_test SET indexed_col = 555 WHERE id = 50; }
+step s1_commit { COMMIT; }
+step s1_verify_hot {
+ -- Verify HOT chain exists for row with id=50
+ SELECT EXISTS (
+ SELECT 1 FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0
+ AND t_ctid != ('(0,' || lp || ')')::tid)
+ ) AS has_hot_chain;
+}
+
+# Session 2: Index scan while HOT update in progress
+session s2
+step s2_begin { BEGIN; }
+step s2_index_scan { SELECT * FROM hot_test WHERE indexed_col = 500; }
+step s2_index_scan_new { SELECT * FROM hot_test WHERE indexed_col = 555; }
+step s2_commit { COMMIT; }
+step s2_verify_index {
+ -- After non-HOT update, verify index reflects the change
+ SELECT COUNT(*) = 1 AS found_new_value FROM hot_test WHERE indexed_col = 555;
+ SELECT COUNT(*) = 0 AS old_value_gone FROM hot_test WHERE indexed_col = 500;
+}
+
+# Session 3: SELECT FOR UPDATE
+session s3
+step s3_begin { BEGIN; }
+step s3_select_for_update { SELECT * FROM hot_test WHERE id = 50 FOR UPDATE; }
+step s3_commit { COMMIT; }
+
+# Session 4: SELECT FOR KEY SHARE (should not block HOT update of non-key column)
+session s4
+step s4_begin { BEGIN; }
+step s4_select_for_key_share { SELECT * FROM hot_test WHERE id = 50 FOR KEY SHARE; }
+step s4_commit { COMMIT; }
+
+# Index scan should see consistent snapshot during HOT update
+# Index scan starts before HOT update commits
+permutation s1_begin s1_hot_update s2_begin s2_index_scan s1_commit s2_commit
+
+# Index scan after non-HOT update should see new index entry
+# Index scan starts after non-HOT update commits
+permutation s1_begin s1_non_hot_update s1_commit s2_begin s2_index_scan_new s2_commit s2_verify_index
+
+# SELECT FOR UPDATE blocks HOT update
+# FOR UPDATE should block the UPDATE until SELECT commits
+permutation s3_begin s3_select_for_update s1_begin s1_hot_update s3_commit s1_commit s1_verify_hot
+
+# HOT update blocks SELECT FOR UPDATE
+# SELECT FOR UPDATE should wait for HOT update to commit
+permutation s1_begin s1_hot_update s3_begin s3_select_for_update s1_commit s3_commit
+
+# SELECT FOR KEY SHARE should not block HOT update (non-key column)
+# HOT update of non-indexed column should not conflict with FOR KEY SHARE
+permutation s4_begin s4_select_for_key_share s1_begin s1_hot_update s4_commit s1_commit s1_verify_hot
+
+# Non-HOT update (key column) should block after FOR KEY SHARE
+# Non-HOT update of indexed column should wait for FOR KEY SHARE
+permutation s4_begin s4_select_for_key_share s1_begin s1_non_hot_update s4_commit s1_commit
diff --git a/src/test/regress/expected/hot_updates.out b/src/test/regress/expected/hot_updates.out
new file mode 100644
index 00000000000..04fb86755db
--- /dev/null
+++ b/src/test/regress/expected/hot_updates.out
@@ -0,0 +1,725 @@
+--
+-- HOT_UPDATES
+-- Test Heap-Only Tuple (HOT) update decisions
+--
+-- This test systematically verifies that HOT updates are used when appropriate
+-- and avoided when necessary (e.g., when indexed columns are modified).
+--
+-- We use multiple validation methods:
+-- 1. Index verification (index still works = proves no index update for HOT)
+-- 2. Statistics functions (pg_stat_get_tuples_hot_updated)
+-- 3. pageinspect extension for HOT chain examination
+--
+-- Load required extensions
+CREATE EXTENSION IF NOT EXISTS pageinspect;
+-- Clean up from prior runs
+DROP TABLE IF EXISTS hot_test CASCADE;
+NOTICE: table "hot_test" does not exist, skipping
+-- Function to get HOT update count
+CREATE OR REPLACE FUNCTION get_hot_count(rel_name text)
+RETURNS TABLE (
+ updates BIGINT,
+ hot BIGINT
+) AS $$
+DECLARE
+ rel_oid oid;
+BEGIN
+ rel_oid := rel_name::regclass::oid;
+ PERFORM pg_stat_force_next_flush();
+
+ updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
+ hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
+
+ RETURN NEXT;
+END;
+$$ LANGUAGE plpgsql;
+-- Check if a tuple is part of a HOT chain (has a predecessor on same page)
+CREATE OR REPLACE FUNCTION has_hot_chain(rel_name text, target_ctid tid)
+RETURNS boolean AS $$
+DECLARE
+ block_num int;
+ page_item record;
+BEGIN
+ block_num := (target_ctid::text::point)[0]::int;
+
+ -- Look for a different tuple on the same page that points to our target tuple
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid IS NOT NULL
+ AND t_ctid = target_ctid
+ AND ('(' || block_num::text || ',' || lp::text || ')')::tid != target_ctid
+ LOOP
+ RETURN true;
+ END LOOP;
+
+ RETURN false;
+END;
+$$ LANGUAGE plpgsql;
+-- Print the HOT chain starting from a given tuple
+CREATE OR REPLACE FUNCTION print_hot_chain(rel_name text, start_ctid tid)
+RETURNS TABLE(chain_position int, ctid tid, lp_flags text, t_ctid tid, chain_end boolean) AS
+$$
+#variable_conflict use_column
+DECLARE
+ block_num int;
+ line_ptr int;
+ current_ctid tid := start_ctid;
+ next_ctid tid;
+ position int := 0;
+ max_iterations int := 100;
+ page_item record;
+ found_predecessor boolean := false;
+ flags_name text;
+BEGIN
+ block_num := (start_ctid::text::point)[0]::int;
+
+ -- Find the predecessor (old tuple pointing to our start_ctid)
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid = start_ctid
+ LOOP
+ current_ctid := ('(' || block_num::text || ',' || page_item.lp::text || ')')::tid;
+ found_predecessor := true;
+ EXIT;
+ END LOOP;
+
+ -- If no predecessor found, start with the given ctid
+ IF NOT found_predecessor THEN
+ current_ctid := start_ctid;
+ END IF;
+
+ -- Follow the chain forward
+ WHILE position < max_iterations LOOP
+ line_ptr := (current_ctid::text::point)[1]::int;
+
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp = line_ptr
+ LOOP
+ -- Map lp_flags to names
+ flags_name := CASE page_item.lp_flags
+ WHEN 0 THEN 'unused (0)'
+ WHEN 1 THEN 'normal (1)'
+ WHEN 2 THEN 'redirect (2)'
+ WHEN 3 THEN 'dead (3)'
+ ELSE 'unknown (' || page_item.lp_flags::text || ')'
+ END;
+
+ RETURN QUERY SELECT
+ position,
+ current_ctid,
+ flags_name,
+ page_item.t_ctid,
+ (page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid)::boolean
+ ;
+
+ IF page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid THEN
+ RETURN;
+ END IF;
+
+ next_ctid := page_item.t_ctid;
+
+ IF (next_ctid::text::point)[0]::int != block_num THEN
+ RETURN;
+ END IF;
+
+ current_ctid := next_ctid;
+ position := position + 1;
+ END LOOP;
+
+ IF position = 0 THEN
+ RETURN;
+ END IF;
+ END LOOP;
+END;
+$$ LANGUAGE plpgsql;
+-- Trigger page pruning via table scan
+CREATE OR REPLACE FUNCTION heap_prune_page(rel_name text, target_ctid tid)
+RETURNS void AS $$
+DECLARE
+ block_num int;
+BEGIN
+ -- Extract block number from ctid
+ block_num := (target_ctid::text::point)[0]::int;
+
+ -- Scan only the specific page to trigger pruning on that page
+ EXECUTE 'SELECT COUNT(*) FROM ' || quote_ident(rel_name) ||
+ ' WHERE ctid >= (' || block_num || ',0) AND ctid < (' || (block_num + 1) || ',0)';
+END;
+$$ LANGUAGE plpgsql;
+-- Basic HOT update (update non-indexed column)
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+) USING heap;
+CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
+INSERT INTO hot_test VALUES (1, 100, 'initial');
+INSERT INTO hot_test VALUES (2, 200, 'initial');
+INSERT INTO hot_test VALUES (3, 300, 'initial');
+-- Get baseline and initial ctid
+WITH initial_state AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ 'Initial State' AS phase,
+ initial_state.ctid,
+ (get_hot_count('hot_test')).updates,
+ (get_hot_count('hot_test')).hot
+FROM initial_state;
+ phase | ctid | updates | hot
+---------------+-------+---------+-----
+ Initial State | (0,1) | 0 | 0
+(1 row)
+
+-- Should be HOT updates (only non-indexed column modified)
+UPDATE hot_test SET non_indexed_col = 'updated1' WHERE id = 1;
+UPDATE hot_test SET non_indexed_col = 'updated2' WHERE id = 2;
+UPDATE hot_test SET non_indexed_col = 'updated3' WHERE id = 3;
+-- Verify HOT updates occurred
+SELECT
+ 'After Updates' AS phase,
+ (get_hot_count('hot_test')).updates,
+ (get_hot_count('hot_test')).hot;
+ phase | updates | hot
+---------------+---------+-----
+ After Updates | 3 | 3
+(1 row)
+
+-- Dump the HOT chain before pruning
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ 'Before VACUUM' AS phase,
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+ phase | has_chain | chain_position | ctid | lp_flags | t_ctid
+---------------+-----------+----------------+-------+------------+--------
+ Before VACUUM | t | 0 | (0,1) | normal (1) | (0,4)
+ Before VACUUM | t | 1 | (0,4) | normal (1) | (0,4)
+(2 rows)
+
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+-- Verify indexes still work
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 100;
+ QUERY PLAN
+---------------------------------------------------
+ Index Scan using hot_test_indexed_idx on hot_test
+ Index Cond: (indexed_col = 100)
+(2 rows)
+
+SELECT id, indexed_col FROM hot_test WHERE indexed_col = 100;
+ id | indexed_col
+----+-------------
+ 1 | 100
+(1 row)
+
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 200;
+ QUERY PLAN
+---------------------------------------------------
+ Index Scan using hot_test_indexed_idx on hot_test
+ Index Cond: (indexed_col = 200)
+(2 rows)
+
+SELECT id, indexed_col FROM hot_test WHERE indexed_col = 200;
+ id | indexed_col
+----+-------------
+ 2 | 200
+(1 row)
+
+-- Vacuum the relation, expect the HOT chain to collapse
+VACUUM hot_test;
+-- Show that there is no chain after vacuum
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ 'After VACUUM' AS phase,
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+ phase | has_chain | chain_position | ctid | lp_flags | t_ctid
+--------------+-----------+----------------+-------+------------+--------
+ After VACUUM | f | 0 | (0,4) | normal (1) | (0,4)
+(1 row)
+
+-- Non-HOT update (update indexed column)
+UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (4,3)
+(1 row)
+
+-- Verify index was updated (new value findable)
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+ QUERY PLAN
+---------------------------------------------------
+ Index Scan using hot_test_indexed_idx on hot_test
+ Index Cond: (indexed_col = 150)
+(2 rows)
+
+SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+ id | indexed_col
+----+-------------
+ 1 | 150
+(1 row)
+
+-- Verify old value no longer in index
+EXPLAIN (COSTS OFF) SELECT id FROM hot_test WHERE indexed_col = 100;
+ QUERY PLAN
+---------------------------------------------------
+ Index Scan using hot_test_indexed_idx on hot_test
+ Index Cond: (indexed_col = 100)
+(2 rows)
+
+SELECT id FROM hot_test WHERE indexed_col = 100;
+ id
+----
+(0 rows)
+
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+-- All-or-none property: updating one indexed column requires ALL index updates
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ non_indexed text
+) USING heap;
+CREATE INDEX hot_test_a_idx ON hot_test(col_a);
+CREATE INDEX hot_test_b_idx ON hot_test(col_b);
+CREATE INDEX hot_test_c_idx ON hot_test(col_c);
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'initial');
+-- Update only col_a - should NOT be HOT because an indexed column changed
+-- This means ALL indexes must be updated (all-or-none property)
+UPDATE hot_test SET col_a = 15 WHERE id = 1;
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (1,0)
+(1 row)
+
+-- Verify all three indexes still work correctly
+SELECT id, col_a FROM hot_test WHERE col_a = 15; -- updated index
+ id | col_a
+----+-------
+ 1 | 15
+(1 row)
+
+SELECT id, col_b FROM hot_test WHERE col_b = 20; -- unchanged index
+ id | col_b
+----+-------
+ 1 | 20
+(1 row)
+
+SELECT id, col_c FROM hot_test WHERE col_c = 30; -- unchanged index
+ id | col_c
+----+-------
+ 1 | 30
+(1 row)
+
+-- Now update only non-indexed column - should be HOT
+UPDATE hot_test SET non_indexed = 'updated';
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (2,1)
+(1 row)
+
+-- Verify all indexes still work
+SELECT id FROM hot_test WHERE col_a = 15 AND col_b = 20 AND col_c = 30;
+ id
+----
+ 1
+(1 row)
+
+-- Partial index: both old and new outside predicate (conservative = non-HOT)
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ status text,
+ data text
+) USING heap;
+-- Partial index only covers status = 'active'
+CREATE INDEX hot_test_active_idx ON hot_test(status) WHERE status = 'active';
+INSERT INTO hot_test VALUES (1, 'active', 'data1');
+INSERT INTO hot_test VALUES (2, 'inactive', 'data2');
+INSERT INTO hot_test VALUES (3, 'deleted', 'data3');
+-- Update non-indexed column on 'active' row (in predicate, status unchanged)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated1' WHERE id = 1;
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (1,1)
+(1 row)
+
+-- Update non-indexed column on 'inactive' row (outside predicate)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated2' WHERE id = 2;
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (2,2)
+(1 row)
+
+-- Update status from 'inactive' to 'deleted' (both outside predicate)
+-- PostgreSQL is conservative: heap insert happens before predicate check
+-- So this is NON-HOT even though both values are outside predicate
+UPDATE hot_test SET status = 'deleted' WHERE id = 2;
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (3,2)
+(1 row)
+
+-- Verify index still works for 'active' rows
+SELECT id, status FROM hot_test WHERE status = 'active';
+ id | status
+----+--------
+ 1 | active
+(1 row)
+
+-- Expression index with JSONB
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ metadata jsonb
+) USING heap;
+-- Index on JSONB expression
+CREATE INDEX hot_test_user_id_idx ON hot_test((metadata->>'user_id'));
+CREATE INDEX hot_test_status_idx ON hot_test((metadata->>'status'));
+INSERT INTO hot_test VALUES (1, '{"user_id": "123", "status": "active"}'::jsonb);
+-- Update JSONB field used in expression index to the same value,
+-- this will be HOT because the entire JSONB field is observed to
+-- be unchanged.
+UPDATE hot_test SET metadata = jsonb_set(metadata, '{user_id}', '"123"')
+WHERE id = 1;
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (1,1)
+(1 row)
+
+-- Update JSONB field that is no used in any index to some new value, this
+-- will prevent a HOT update despite not changing what is used when forming
+-- the index key, this is counter intuitive and causes index bloat as well
+-- as slows down updates on JSONB data as any change will trigger all
+-- indexes to be updated.
+UPDATE hot_test SET metadata = jsonb_set(metadata, '{food}', '"apple"')
+WHERE id = 1;
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (2,1)
+(1 row)
+
+-- Use a few different methods for mutating JSONB data, but don't modify
+-- indexed portions of the document. None of these will be HOT.
+UPDATE hot_test SET metadata = jsonb_set(
+ jsonb_set(metadata, '{food}', '"pear"'),
+ '{timestamp}',
+ to_jsonb(now())
+)
+WHERE id = 1;
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (3,1)
+(1 row)
+
+UPDATE hot_test
+SET metadata = metadata || '{"user_id": "123", "timestamp": "2024-01-01"}'::jsonb
+WHERE id = 1;
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (4,1)
+(1 row)
+
+UPDATE hot_test SET metadata =
+ jsonb_set(
+ jsonb_set(metadata, '{user_id}', '"123"'),
+ '{fruit}',
+ '"plumb"'
+ );
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (5,1)
+(1 row)
+
+UPDATE hot_test SET metadata = metadata || jsonb_build_object(
+ 'user_id', '123',
+ 'timestamp', now(),
+ 'fruit', 'honeydew'
+);
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (6,1)
+(1 row)
+
+-- Only BRIN (summarizing) indexes on non-PK columns
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ ts timestamp,
+ value int,
+ brin_col int
+) USING heap;
+CREATE INDEX hot_test_ts_brin ON hot_test USING brin(ts);
+CREATE INDEX hot_test_brin_col_brin ON hot_test USING brin(brin_col);
+INSERT INTO hot_test VALUES (1, '2024-01-01', 100, 1000);
+-- Update both BRIN columns - should still be HOT (only summarizing indexes)
+UPDATE hot_test SET ts = '2024-01-02', brin_col = 2000 WHERE id = 1;
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (1,1)
+(1 row)
+
+-- Verify BRIN indexes work
+SELECT id FROM hot_test WHERE ts >= '2024-01-02';
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hot_test WHERE brin_col >= 2000;
+ id
+----
+ 1
+(1 row)
+
+-- Update non-indexed column - should also be HOT
+UPDATE hot_test SET value = 200 WHERE id = 1;
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (2,2)
+(1 row)
+
+-- TOAST and HOT: TOASTed columns can participate in HOT
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ large_text text,
+ small_text text
+) USING heap;
+CREATE INDEX hot_test_idx ON hot_test(indexed_col);
+-- Insert row with TOASTed column (> 2KB)
+INSERT INTO hot_test VALUES (1, 100, repeat('x', 3000), 'small');
+-- Update non-indexed, non-TOASTed column - should be HOT
+UPDATE hot_test SET small_text = 'updated';
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (1,1)
+(1 row)
+
+-- Update TOASTed column - should be HOT if indexed column unchanged
+UPDATE hot_test SET large_text = repeat('y', 3000);
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (2,2)
+(1 row)
+
+-- Verify index still works
+SELECT id FROM hot_test WHERE indexed_col = 100;
+ id
+----
+ 1
+(1 row)
+
+-- Update indexed column - should NOT be HOT
+UPDATE hot_test SET indexed_col = 200;
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (3,2)
+(1 row)
+
+-- Verify index was updated
+SELECT id FROM hot_test WHERE indexed_col = 200;
+ id
+----
+ 1
+(1 row)
+
+-- Unique constraint (unique index) behaves like regular index
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ unique_col int UNIQUE,
+ data text
+) USING heap;
+INSERT INTO hot_test VALUES (1, 100, 'data1');
+INSERT INTO hot_test VALUES (2, 200, 'data2');
+-- Update data (non-indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (2,2)
+(1 row)
+
+-- Verify unique constraint still enforced
+SELECT id, unique_col, data FROM hot_test ORDER BY id;
+ id | unique_col | data
+----+------------+---------
+ 1 | 100 | updated
+ 2 | 200 | updated
+(2 rows)
+
+-- This should fail (unique violation)
+UPDATE hot_test SET unique_col = 100 WHERE id = 2;
+ERROR: duplicate key value violates unique constraint "hot_test_unique_col_key"
+DETAIL: Key (unique_col)=(100) already exists.
+-- Multi-column index: any column change = non-HOT
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ data text
+) USING heap;
+CREATE INDEX hot_test_ab_idx ON hot_test(col_a, col_b);
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'data');
+-- Update col_a (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_a = 15;
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (1,0)
+(1 row)
+
+-- Reset
+UPDATE hot_test SET col_a = 10;
+-- Update col_b (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_b = 25;
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (3,0)
+(1 row)
+
+-- Reset
+UPDATE hot_test SET col_b = 20;
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (4,0)
+(1 row)
+
+-- Update col_c (not indexed) - should be HOT
+UPDATE hot_test SET col_c = 35;
+-- Update data (not indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT get_hot_count('hot_test');
+ get_hot_count
+---------------
+ (6,2)
+(1 row)
+
+-- Verify multi-column index works
+SELECT id FROM hot_test WHERE col_a = 10 AND col_b = 20;
+ id
+----
+ 1
+(1 row)
+
+-- Partitioned tables: HOT works within partitions
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+NOTICE: table "hot_test_partitioned" does not exist, skipping
+CREATE TABLE hot_test_partitioned (
+ id int,
+ partition_key int,
+ indexed_col int,
+ data text,
+ PRIMARY KEY (id, partition_key)
+) PARTITION BY RANGE (partition_key) USING heap;
+CREATE TABLE hot_test_part1 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (1) TO (100) USING heap;
+CREATE TABLE hot_test_part2 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (100) TO (200) USING heap;
+CREATE INDEX hot_test_part_idx ON hot_test_partitioned(indexed_col);
+INSERT INTO hot_test_partitioned VALUES (1, 50, 100, 'initial1');
+INSERT INTO hot_test_partitioned VALUES (2, 150, 200, 'initial2');
+-- Update in partition 1 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'updated1' WHERE id = 1;
+-- Update in partition 2 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'updated2' WHERE id = 2;
+SELECT get_hot_count('hot_test_part1');
+ get_hot_count
+---------------
+ (1,1)
+(1 row)
+
+SELECT get_hot_count('hot_test_part2');
+ get_hot_count
+---------------
+ (1,1)
+(1 row)
+
+-- Verify indexes work on partitions
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 100;
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 200;
+ id
+----
+ 2
+(1 row)
+
+-- Update indexed column in partition - should NOT be HOT
+UPDATE hot_test_partitioned SET indexed_col = 150 WHERE id = 1;
+SELECT get_hot_count('hot_test_part1');
+ get_hot_count
+---------------
+ (2,1)
+(1 row)
+
+-- Verify index was updated
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 150;
+ id
+----
+ 1
+(1 row)
+
+-- Cleanup
+DROP TABLE IF EXISTS hot_test;
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+DROP FUNCTION IF EXISTS has_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS print_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS heap_prune(text);
+NOTICE: function heap_prune(text) does not exist, skipping
+DROP FUNCTION IF EXISTS get_hot_count(text);
+DROP EXTENSION pageinspect;
diff --git a/src/test/regress/expected/replica_identity_logging.out b/src/test/regress/expected/replica_identity_logging.out
new file mode 100644
index 00000000000..2096510b924
--- /dev/null
+++ b/src/test/regress/expected/replica_identity_logging.out
@@ -0,0 +1,396 @@
+--
+-- REPLICA_IDENTITY_LOGGING
+-- Test that replica identity keys are correctly extracted and logged for logical replication
+--
+-- This test verifies that the correct old key columns are included in WAL records
+-- for logical replication, based on the table's replica identity setting.
+--
+-- Clean up from prior runs
+DROP TABLE IF EXISTS repid_test CASCADE;
+NOTICE: table "repid_test" does not exist, skipping
+-- Drop replication slot if it exists from prior run
+SELECT pg_drop_replication_slot('repid_test_slot') FROM pg_replication_slots WHERE slot_name = 'repid_test_slot';
+ pg_drop_replication_slot
+--------------------------
+(0 rows)
+
+-- Enable logical decoding to verify what gets logged
+SELECT 'init' FROM pg_create_logical_replication_slot('repid_test_slot', 'test_decoding');
+ ?column?
+----------
+ init
+(1 row)
+
+-- REPLICA IDENTITY DEFAULT (primary key columns only)
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ data text
+);
+CREATE INDEX repid_test_idx ON repid_test(indexed_col);
+INSERT INTO repid_test VALUES (1, 100, 'initial');
+INSERT INTO repid_test VALUES (2, 200, 'initial');
+-- Advance slot to skip inserts (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+ data
+----------------------------------------------------------------------------------------------
+ table public.repid_test: INSERT: id[integer]:1 indexed_col[integer]:100 data[text]:'initial'
+ table public.repid_test: INSERT: id[integer]:2 indexed_col[integer]:200 data[text]:'initial'
+(2 rows)
+
+-- Update non-key column - should log only id in old key
+UPDATE repid_test SET data = 'updated' WHERE id = 1;
+-- Update indexed non-key column - should still log only id in old key
+UPDATE repid_test SET indexed_col = 150 WHERE id = 2;
+-- Check logical decoding output - should see old key with only id
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+ data
+----------------------------------------------------------------------------------------------
+ table public.repid_test: UPDATE: id[integer]:1 indexed_col[integer]:100 data[text]:'updated'
+ table public.repid_test: UPDATE: id[integer]:2 indexed_col[integer]:150 data[text]:'initial'
+(2 rows)
+
+-- REPLICA IDENTITY FULL (all columns in old key)
+DROP TABLE repid_test;
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ data text
+);
+ALTER TABLE repid_test REPLICA IDENTITY FULL;
+CREATE INDEX repid_test_idx ON repid_test(indexed_col);
+INSERT INTO repid_test VALUES (1, 100, 'initial');
+-- Advance slot past insert (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+ data
+----------------------------------------------------------------------------------------------
+ table public.repid_test: INSERT: id[integer]:1 indexed_col[integer]:100 data[text]:'initial'
+(1 row)
+
+-- Update any column - should log ALL columns in old key
+UPDATE repid_test SET data = 'updated' WHERE id = 1;
+-- Check logical decoding output - should see old key with all columns
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+ data
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ table public.repid_test: UPDATE: old-key: id[integer]:1 indexed_col[integer]:100 data[text]:'initial' new-tuple: id[integer]:1 indexed_col[integer]:100 data[text]:'updated'
+(1 row)
+
+-- REPLICA IDENTITY USING INDEX (index columns only)
+DROP TABLE repid_test;
+CREATE TABLE repid_test (
+ id int,
+ unique_col int UNIQUE NOT NULL,
+ data text
+);
+ALTER TABLE repid_test REPLICA IDENTITY USING INDEX repid_test_unique_col_key;
+INSERT INTO repid_test VALUES (1, 100, 'initial');
+INSERT INTO repid_test VALUES (2, 200, 'initial');
+-- Advance slot past inserts (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+ data
+---------------------------------------------------------------------------------------------
+ table public.repid_test: INSERT: id[integer]:1 unique_col[integer]:100 data[text]:'initial'
+ table public.repid_test: INSERT: id[integer]:2 unique_col[integer]:200 data[text]:'initial'
+(2 rows)
+
+-- Update non-indexed column - should log only unique_col in old key
+UPDATE repid_test SET data = 'updated' WHERE unique_col = 100;
+-- Update id (not in replica identity index) - should still log only unique_col
+UPDATE repid_test SET id = 10 WHERE unique_col = 200;
+-- Check logical decoding output - should see old key with only unique_col
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+ data
+----------------------------------------------------------------------------------------------
+ table public.repid_test: UPDATE: id[integer]:1 unique_col[integer]:100 data[text]:'updated'
+ table public.repid_test: UPDATE: id[integer]:10 unique_col[integer]:200 data[text]:'initial'
+(2 rows)
+
+-- REPLICA IDENTITY NOTHING (no old key logged)
+DROP TABLE repid_test;
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ data text
+);
+ALTER TABLE repid_test REPLICA IDENTITY NOTHING;
+INSERT INTO repid_test VALUES (1, 'initial');
+-- Advance slot past insert (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+ data
+---------------------------------------------------------------------
+ table public.repid_test: INSERT: id[integer]:1 data[text]:'initial'
+(1 row)
+
+-- Update - should log no old key
+UPDATE repid_test SET data = 'updated' WHERE id = 1;
+-- Check logical decoding output - should see update with no old key
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+ data
+---------------------------------------------------------------------
+ table public.repid_test: UPDATE: id[integer]:1 data[text]:'updated'
+(1 row)
+
+-- Multi-column index replica identity
+DROP TABLE repid_test;
+CREATE TABLE repid_test (
+ id int,
+ col_a int NOT NULL,
+ col_b int NOT NULL,
+ col_c int,
+ data text,
+ UNIQUE (col_a, col_b)
+);
+ALTER TABLE repid_test REPLICA IDENTITY USING INDEX repid_test_col_a_col_b_key;
+INSERT INTO repid_test VALUES (1, 10, 20, 30, 'initial');
+-- Advance slot past insert (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+ data
+---------------------------------------------------------------------------------------------------------------------------
+ table public.repid_test: INSERT: id[integer]:1 col_a[integer]:10 col_b[integer]:20 col_c[integer]:30 data[text]:'initial'
+(1 row)
+
+-- Update non-indexed columns - should log col_a and col_b in old key
+UPDATE repid_test SET data = 'updated', col_c = 35 WHERE col_a = 10 AND col_b = 20;
+-- Check logical decoding output - should see old key with col_a and col_b
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+ data
+---------------------------------------------------------------------------------------------------------------------------
+ table public.repid_test: UPDATE: id[integer]:1 col_a[integer]:10 col_b[integer]:20 col_c[integer]:35 data[text]:'updated'
+(1 row)
+
+-- TOAST/external columns in replica identity
+DROP TABLE repid_test;
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ large_text text,
+ data text
+);
+-- REPLICA IDENTITY FULL includes toasted columns
+ALTER TABLE repid_test REPLICA IDENTITY FULL;
+-- Insert a large value (large enough to show the concept without excessive output)
+INSERT INTO repid_test VALUES (1, repeat('x', 100), 'initial');
+-- Advance slot past insert (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+ data
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ table public.repid_test: INSERT: id[integer]:1 large_text[text]:'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' data[text]:'initial'
+(1 row)
+
+-- Update small column - should still log large_text column in old key
+UPDATE repid_test SET data = 'updated' WHERE id = 1;
+-- Check logical decoding output - verify both old and new values are logged
+-- Just check that UPDATE happened and includes both large_text and data columns
+SELECT COUNT(*) as update_count FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%' AND data LIKE '%large_text%' AND data LIKE '%old-key%';
+ update_count
+--------------
+ 1
+(1 row)
+
+-- Test TOAST columns with REPLICA IDENTITY USING INDEX
+DROP TABLE repid_test;
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ indexed_large_text text NOT NULL,
+ data text
+);
+-- Create unique index on the large text column
+CREATE UNIQUE INDEX repid_test_large_idx ON repid_test(indexed_large_text);
+-- Set replica identity to use the index (not FULL)
+ALTER TABLE repid_test REPLICA IDENTITY USING INDEX repid_test_large_idx;
+-- Insert a large value (large enough to be TOASTed)
+INSERT INTO repid_test VALUES (1, repeat('x', 100000), 'initial');
+-- Advance slot past inserts
+SELECT COUNT(*) FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+ count
+-------
+ 1
+(1 row)
+
+-- Update non-indexed column - should still log indexed_large_text in old key
+-- despite being unmodified because it is TOASTed and in the replica key
+UPDATE repid_test SET data = 'updated' WHERE id = 1;
+-- Verify TOASTed indexed column part of the relica identity is logged in old key
+SELECT COUNT(*) AS toasted_index_logged FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%' AND data LIKE '%indexed_large_text%';
+ toasted_index_logged
+----------------------
+ 1
+(1 row)
+
+-- Dropped columns and replica identity
+DROP TABLE repid_test;
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ dropped_col int,
+ kept_col int,
+ data text
+);
+ALTER TABLE repid_test REPLICA IDENTITY FULL;
+INSERT INTO repid_test VALUES (1, 999, 100, 'initial');
+-- Drop a column
+ALTER TABLE repid_test DROP COLUMN dropped_col;
+-- Advance slot past insert and DDL (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+ data
+--------------------------------------------------------------------------------------------------------------------
+ table public.repid_test: INSERT: id[integer]:1 dropped_col[integer]:999 kept_col[integer]:100 data[text]:'initial'
+(1 row)
+
+-- Update - old key should handle dropped column
+UPDATE repid_test SET data = 'updated' WHERE id = 1;
+-- Check logical decoding output
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+ data
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ table public.repid_test: UPDATE: old-key: id[integer]:1 kept_col[integer]:100 data[text]:'initial' new-tuple: id[integer]:1 kept_col[integer]:100 data[text]:'updated'
+(1 row)
+
+-- DEFAULT replica identity with composite primary key
+DROP TABLE repid_test;
+CREATE TABLE repid_test (
+ id_a int,
+ id_b int,
+ indexed_col int,
+ data text,
+ PRIMARY KEY (id_a, id_b)
+);
+CREATE INDEX repid_test_idx ON repid_test(indexed_col);
+INSERT INTO repid_test VALUES (1, 10, 100, 'initial');
+-- Advance slot past insert (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+ data
+-----------------------------------------------------------------------------------------------------------------
+ table public.repid_test: INSERT: id_a[integer]:1 id_b[integer]:10 indexed_col[integer]:100 data[text]:'initial'
+(1 row)
+
+-- Update non-key columns - should log both id_a and id_b in old key
+UPDATE repid_test SET data = 'updated', indexed_col = 150 WHERE id_a = 1 AND id_b = 10;
+-- Check logical decoding output - should see old key with both primary key columns
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+ data
+-----------------------------------------------------------------------------------------------------------------
+ table public.repid_test: UPDATE: id_a[integer]:1 id_b[integer]:10 indexed_col[integer]:150 data[text]:'updated'
+(1 row)
+
+-- Expression index and replica identity
+DROP TABLE repid_test;
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ email text NOT NULL,
+ data text
+);
+-- Create unique expression index
+CREATE UNIQUE INDEX repid_test_lower_email_idx ON repid_test(lower(email));
+-- Cannot use expression index for replica identity (should fail)
+-- PostgreSQL requires the index to be on simple column references
+-- This should produce an error
+DO $$
+BEGIN
+ ALTER TABLE repid_test REPLICA IDENTITY USING INDEX repid_test_lower_email_idx;
+ RAISE EXCEPTION 'Should have failed - expression indexes cannot be used for replica identity';
+EXCEPTION
+ WHEN feature_not_supported THEN
+ RAISE NOTICE 'Correctly rejected expression index for replica identity';
+END$$;
+NOTICE: Correctly rejected expression index for replica identity
+-- Use FULL instead
+ALTER TABLE repid_test REPLICA IDENTITY FULL;
+INSERT INTO repid_test VALUES (1, '[email protected]', 'initial');
+-- Advance slot past insert (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+ data
+----------------------------------------------------------------------------------------------------
+ table public.repid_test: INSERT: id[integer]:1 email[text]:'[email protected]' data[text]:'initial'
+(1 row)
+
+-- Update - should log all columns in old key
+UPDATE repid_test SET data = 'updated' WHERE id = 1;
+-- Check logical decoding output
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+ data
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ table public.repid_test: UPDATE: old-key: id[integer]:1 email[text]:'[email protected]' data[text]:'initial' new-tuple: id[integer]:1 email[text]:'[email protected]' data[text]:'updated'
+(1 row)
+
+-- NULL values in replica identity columns
+DROP TABLE repid_test;
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ nullable_col int,
+ data text
+);
+ALTER TABLE repid_test REPLICA IDENTITY FULL;
+INSERT INTO repid_test VALUES (1, NULL, 'initial');
+-- Advance slot past insert (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+ data
+------------------------------------------------------------------------------------------------
+ table public.repid_test: INSERT: id[integer]:1 nullable_col[integer]:null data[text]:'initial'
+(1 row)
+
+-- Update - old key should include NULL value
+UPDATE repid_test SET data = 'updated' WHERE id = 1;
+-- Check logical decoding output - should see old key with NULL
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+ data
+-------------------------------------------------------------------------------------------------------------------------------------------------------
+ table public.repid_test: UPDATE: old-key: id[integer]:1 data[text]:'initial' new-tuple: id[integer]:1 nullable_col[integer]:null data[text]:'updated'
+(1 row)
+
+-- Generated columns and replica identity
+DROP TABLE repid_test;
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ base_col int,
+ generated_col int GENERATED ALWAYS AS (base_col * 2) STORED,
+ data text
+);
+ALTER TABLE repid_test REPLICA IDENTITY FULL;
+INSERT INTO repid_test (id, base_col, data) VALUES (1, 50, 'initial');
+-- Advance slot past insert (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+ data
+---------------------------------------------------------------------------------------------------------------------
+ table public.repid_test: INSERT: id[integer]:1 base_col[integer]:50 generated_col[integer]:100 data[text]:'initial'
+(1 row)
+
+-- Update base_col - generated_col will change automatically
+UPDATE repid_test SET base_col = 60 WHERE id = 1;
+-- Check logical decoding output - should include old generated_col value
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+ data
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ table public.repid_test: UPDATE: old-key: id[integer]:1 base_col[integer]:50 generated_col[integer]:100 data[text]:'initial' new-tuple: id[integer]:1 base_col[integer]:60 generated_col[integer]:120 data[text]:'initial'
+(1 row)
+
+-- Cleanup
+SELECT pg_drop_replication_slot('repid_test_slot');
+ pg_drop_replication_slot
+--------------------------
+
+(1 row)
+
+DROP TABLE repid_test;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 549e9b2d7be..01ed43eba18 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -137,6 +137,13 @@ test: event_trigger_login
# this test also uses event triggers, so likewise run it by itself
test: fast_default
+# ----------
+# HOT updates and replica identity logging tests
+# Run these sequentially to avoid logical replication slot interference
+# ----------
+test: hot_updates
+test: replica_identity_logging
+
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
diff --git a/src/test/regress/sql/hot_updates.sql b/src/test/regress/sql/hot_updates.sql
new file mode 100644
index 00000000000..7030f4fc6db
--- /dev/null
+++ b/src/test/regress/sql/hot_updates.sql
@@ -0,0 +1,553 @@
+--
+-- HOT_UPDATES
+-- Test Heap-Only Tuple (HOT) update decisions
+--
+-- This test systematically verifies that HOT updates are used when appropriate
+-- and avoided when necessary (e.g., when indexed columns are modified).
+--
+-- We use multiple validation methods:
+-- 1. Index verification (index still works = proves no index update for HOT)
+-- 2. Statistics functions (pg_stat_get_tuples_hot_updated)
+-- 3. pageinspect extension for HOT chain examination
+--
+
+-- Load required extensions
+CREATE EXTENSION IF NOT EXISTS pageinspect;
+
+-- Clean up from prior runs
+DROP TABLE IF EXISTS hot_test CASCADE;
+
+-- Function to get HOT update count
+CREATE OR REPLACE FUNCTION get_hot_count(rel_name text)
+RETURNS TABLE (
+ updates BIGINT,
+ hot BIGINT
+) AS $$
+DECLARE
+ rel_oid oid;
+BEGIN
+ rel_oid := rel_name::regclass::oid;
+ PERFORM pg_stat_force_next_flush();
+
+ updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
+ hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
+
+ RETURN NEXT;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Check if a tuple is part of a HOT chain (has a predecessor on same page)
+CREATE OR REPLACE FUNCTION has_hot_chain(rel_name text, target_ctid tid)
+RETURNS boolean AS $$
+DECLARE
+ block_num int;
+ page_item record;
+BEGIN
+ block_num := (target_ctid::text::point)[0]::int;
+
+ -- Look for a different tuple on the same page that points to our target tuple
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid IS NOT NULL
+ AND t_ctid = target_ctid
+ AND ('(' || block_num::text || ',' || lp::text || ')')::tid != target_ctid
+ LOOP
+ RETURN true;
+ END LOOP;
+
+ RETURN false;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Print the HOT chain starting from a given tuple
+CREATE OR REPLACE FUNCTION print_hot_chain(rel_name text, start_ctid tid)
+RETURNS TABLE(chain_position int, ctid tid, lp_flags text, t_ctid tid, chain_end boolean) AS
+$$
+#variable_conflict use_column
+DECLARE
+ block_num int;
+ line_ptr int;
+ current_ctid tid := start_ctid;
+ next_ctid tid;
+ position int := 0;
+ max_iterations int := 100;
+ page_item record;
+ found_predecessor boolean := false;
+ flags_name text;
+BEGIN
+ block_num := (start_ctid::text::point)[0]::int;
+
+ -- Find the predecessor (old tuple pointing to our start_ctid)
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid = start_ctid
+ LOOP
+ current_ctid := ('(' || block_num::text || ',' || page_item.lp::text || ')')::tid;
+ found_predecessor := true;
+ EXIT;
+ END LOOP;
+
+ -- If no predecessor found, start with the given ctid
+ IF NOT found_predecessor THEN
+ current_ctid := start_ctid;
+ END IF;
+
+ -- Follow the chain forward
+ WHILE position < max_iterations LOOP
+ line_ptr := (current_ctid::text::point)[1]::int;
+
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp = line_ptr
+ LOOP
+ -- Map lp_flags to names
+ flags_name := CASE page_item.lp_flags
+ WHEN 0 THEN 'unused (0)'
+ WHEN 1 THEN 'normal (1)'
+ WHEN 2 THEN 'redirect (2)'
+ WHEN 3 THEN 'dead (3)'
+ ELSE 'unknown (' || page_item.lp_flags::text || ')'
+ END;
+
+ RETURN QUERY SELECT
+ position,
+ current_ctid,
+ flags_name,
+ page_item.t_ctid,
+ (page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid)::boolean
+ ;
+
+ IF page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid THEN
+ RETURN;
+ END IF;
+
+ next_ctid := page_item.t_ctid;
+
+ IF (next_ctid::text::point)[0]::int != block_num THEN
+ RETURN;
+ END IF;
+
+ current_ctid := next_ctid;
+ position := position + 1;
+ END LOOP;
+
+ IF position = 0 THEN
+ RETURN;
+ END IF;
+ END LOOP;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Trigger page pruning via table scan
+CREATE OR REPLACE FUNCTION heap_prune_page(rel_name text, target_ctid tid)
+RETURNS void AS $$
+DECLARE
+ block_num int;
+BEGIN
+ -- Extract block number from ctid
+ block_num := (target_ctid::text::point)[0]::int;
+
+ -- Scan only the specific page to trigger pruning on that page
+ EXECUTE 'SELECT COUNT(*) FROM ' || quote_ident(rel_name) ||
+ ' WHERE ctid >= (' || block_num || ',0) AND ctid < (' || (block_num + 1) || ',0)';
+END;
+$$ LANGUAGE plpgsql;
+
+-- Basic HOT update (update non-indexed column)
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+) USING heap;
+
+CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
+
+INSERT INTO hot_test VALUES (1, 100, 'initial');
+INSERT INTO hot_test VALUES (2, 200, 'initial');
+INSERT INTO hot_test VALUES (3, 300, 'initial');
+
+-- Get baseline and initial ctid
+WITH initial_state AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ 'Initial State' AS phase,
+ initial_state.ctid,
+ (get_hot_count('hot_test')).updates,
+ (get_hot_count('hot_test')).hot
+FROM initial_state;
+
+-- Should be HOT updates (only non-indexed column modified)
+UPDATE hot_test SET non_indexed_col = 'updated1' WHERE id = 1;
+UPDATE hot_test SET non_indexed_col = 'updated2' WHERE id = 2;
+UPDATE hot_test SET non_indexed_col = 'updated3' WHERE id = 3;
+
+-- Verify HOT updates occurred
+SELECT
+ 'After Updates' AS phase,
+ (get_hot_count('hot_test')).updates,
+ (get_hot_count('hot_test')).hot;
+
+-- Dump the HOT chain before pruning
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ 'Before VACUUM' AS phase,
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+
+-- Verify indexes still work
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 100;
+SELECT id, indexed_col FROM hot_test WHERE indexed_col = 100;
+
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 200;
+SELECT id, indexed_col FROM hot_test WHERE indexed_col = 200;
+
+-- Vacuum the relation, expect the HOT chain to collapse
+VACUUM hot_test;
+
+-- Show that there is no chain after vacuum
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ 'After VACUUM' AS phase,
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+
+-- Non-HOT update (update indexed column)
+UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+SELECT get_hot_count('hot_test');
+
+-- Verify index was updated (new value findable)
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+
+-- Verify old value no longer in index
+EXPLAIN (COSTS OFF) SELECT id FROM hot_test WHERE indexed_col = 100;
+SELECT id FROM hot_test WHERE indexed_col = 100;
+
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+
+-- All-or-none property: updating one indexed column requires ALL index updates
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ non_indexed text
+) USING heap;
+
+CREATE INDEX hot_test_a_idx ON hot_test(col_a);
+CREATE INDEX hot_test_b_idx ON hot_test(col_b);
+CREATE INDEX hot_test_c_idx ON hot_test(col_c);
+
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'initial');
+
+-- Update only col_a - should NOT be HOT because an indexed column changed
+-- This means ALL indexes must be updated (all-or-none property)
+UPDATE hot_test SET col_a = 15 WHERE id = 1;
+SELECT get_hot_count('hot_test');
+
+-- Verify all three indexes still work correctly
+SELECT id, col_a FROM hot_test WHERE col_a = 15; -- updated index
+SELECT id, col_b FROM hot_test WHERE col_b = 20; -- unchanged index
+SELECT id, col_c FROM hot_test WHERE col_c = 30; -- unchanged index
+
+-- Now update only non-indexed column - should be HOT
+UPDATE hot_test SET non_indexed = 'updated';
+SELECT get_hot_count('hot_test');
+
+-- Verify all indexes still work
+SELECT id FROM hot_test WHERE col_a = 15 AND col_b = 20 AND col_c = 30;
+
+-- Partial index: both old and new outside predicate (conservative = non-HOT)
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ status text,
+ data text
+) USING heap;
+
+-- Partial index only covers status = 'active'
+CREATE INDEX hot_test_active_idx ON hot_test(status) WHERE status = 'active';
+
+INSERT INTO hot_test VALUES (1, 'active', 'data1');
+INSERT INTO hot_test VALUES (2, 'inactive', 'data2');
+INSERT INTO hot_test VALUES (3, 'deleted', 'data3');
+
+-- Update non-indexed column on 'active' row (in predicate, status unchanged)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated1' WHERE id = 1;
+SELECT get_hot_count('hot_test');
+
+-- Update non-indexed column on 'inactive' row (outside predicate)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated2' WHERE id = 2;
+SELECT get_hot_count('hot_test');
+
+-- Update status from 'inactive' to 'deleted' (both outside predicate)
+-- PostgreSQL is conservative: heap insert happens before predicate check
+-- So this is NON-HOT even though both values are outside predicate
+UPDATE hot_test SET status = 'deleted' WHERE id = 2;
+SELECT get_hot_count('hot_test');
+
+-- Verify index still works for 'active' rows
+SELECT id, status FROM hot_test WHERE status = 'active';
+
+-- Expression index with JSONB
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ metadata jsonb
+) USING heap;
+
+-- Index on JSONB expression
+CREATE INDEX hot_test_user_id_idx ON hot_test((metadata->>'user_id'));
+CREATE INDEX hot_test_status_idx ON hot_test((metadata->>'status'));
+
+INSERT INTO hot_test VALUES (1, '{"user_id": "123", "status": "active"}'::jsonb);
+
+-- Update JSONB field used in expression index to the same value,
+-- this will be HOT because the entire JSONB field is observed to
+-- be unchanged.
+UPDATE hot_test SET metadata = jsonb_set(metadata, '{user_id}', '"123"')
+WHERE id = 1;
+SELECT get_hot_count('hot_test');
+
+-- Update JSONB field that is no used in any index to some new value, this
+-- will prevent a HOT update despite not changing what is used when forming
+-- the index key, this is counter intuitive and causes index bloat as well
+-- as slows down updates on JSONB data as any change will trigger all
+-- indexes to be updated.
+UPDATE hot_test SET metadata = jsonb_set(metadata, '{food}', '"apple"')
+WHERE id = 1;
+SELECT get_hot_count('hot_test');
+
+-- Use a few different methods for mutating JSONB data, but don't modify
+-- indexed portions of the document. None of these will be HOT.
+UPDATE hot_test SET metadata = jsonb_set(
+ jsonb_set(metadata, '{food}', '"pear"'),
+ '{timestamp}',
+ to_jsonb(now())
+)
+WHERE id = 1;
+SELECT get_hot_count('hot_test');
+
+UPDATE hot_test
+SET metadata = metadata || '{"user_id": "123", "timestamp": "2024-01-01"}'::jsonb
+WHERE id = 1;
+SELECT get_hot_count('hot_test');
+
+UPDATE hot_test SET metadata =
+ jsonb_set(
+ jsonb_set(metadata, '{user_id}', '"123"'),
+ '{fruit}',
+ '"plumb"'
+ );
+SELECT get_hot_count('hot_test');
+
+UPDATE hot_test SET metadata = metadata || jsonb_build_object(
+ 'user_id', '123',
+ 'timestamp', now(),
+ 'fruit', 'honeydew'
+);
+SELECT get_hot_count('hot_test');
+
+-- Only BRIN (summarizing) indexes on non-PK columns
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ ts timestamp,
+ value int,
+ brin_col int
+) USING heap;
+
+CREATE INDEX hot_test_ts_brin ON hot_test USING brin(ts);
+CREATE INDEX hot_test_brin_col_brin ON hot_test USING brin(brin_col);
+
+INSERT INTO hot_test VALUES (1, '2024-01-01', 100, 1000);
+
+-- Update both BRIN columns - should still be HOT (only summarizing indexes)
+UPDATE hot_test SET ts = '2024-01-02', brin_col = 2000 WHERE id = 1;
+SELECT get_hot_count('hot_test');
+
+-- Verify BRIN indexes work
+SELECT id FROM hot_test WHERE ts >= '2024-01-02';
+SELECT id FROM hot_test WHERE brin_col >= 2000;
+
+-- Update non-indexed column - should also be HOT
+UPDATE hot_test SET value = 200 WHERE id = 1;
+SELECT get_hot_count('hot_test');
+
+-- TOAST and HOT: TOASTed columns can participate in HOT
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ large_text text,
+ small_text text
+) USING heap;
+
+CREATE INDEX hot_test_idx ON hot_test(indexed_col);
+
+-- Insert row with TOASTed column (> 2KB)
+INSERT INTO hot_test VALUES (1, 100, repeat('x', 3000), 'small');
+
+-- Update non-indexed, non-TOASTed column - should be HOT
+UPDATE hot_test SET small_text = 'updated';
+SELECT get_hot_count('hot_test');
+
+-- Update TOASTed column - should be HOT if indexed column unchanged
+UPDATE hot_test SET large_text = repeat('y', 3000);
+SELECT get_hot_count('hot_test');
+
+-- Verify index still works
+SELECT id FROM hot_test WHERE indexed_col = 100;
+
+-- Update indexed column - should NOT be HOT
+UPDATE hot_test SET indexed_col = 200;
+SELECT get_hot_count('hot_test');
+
+-- Verify index was updated
+SELECT id FROM hot_test WHERE indexed_col = 200;
+
+-- Unique constraint (unique index) behaves like regular index
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ unique_col int UNIQUE,
+ data text
+) USING heap;
+
+INSERT INTO hot_test VALUES (1, 100, 'data1');
+INSERT INTO hot_test VALUES (2, 200, 'data2');
+
+-- Update data (non-indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT get_hot_count('hot_test');
+
+-- Verify unique constraint still enforced
+SELECT id, unique_col, data FROM hot_test ORDER BY id;
+
+-- This should fail (unique violation)
+UPDATE hot_test SET unique_col = 100 WHERE id = 2;
+
+-- Multi-column index: any column change = non-HOT
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ data text
+) USING heap;
+
+CREATE INDEX hot_test_ab_idx ON hot_test(col_a, col_b);
+
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'data');
+
+-- Update col_a (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_a = 15;
+SELECT get_hot_count('hot_test');
+
+-- Reset
+UPDATE hot_test SET col_a = 10;
+
+-- Update col_b (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_b = 25;
+SELECT get_hot_count('hot_test');
+
+-- Reset
+UPDATE hot_test SET col_b = 20;
+SELECT get_hot_count('hot_test');
+
+-- Update col_c (not indexed) - should be HOT
+UPDATE hot_test SET col_c = 35;
+
+-- Update data (not indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT get_hot_count('hot_test');
+
+-- Verify multi-column index works
+SELECT id FROM hot_test WHERE col_a = 10 AND col_b = 20;
+
+-- Partitioned tables: HOT works within partitions
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+
+CREATE TABLE hot_test_partitioned (
+ id int,
+ partition_key int,
+ indexed_col int,
+ data text,
+ PRIMARY KEY (id, partition_key)
+) PARTITION BY RANGE (partition_key) USING heap;
+
+CREATE TABLE hot_test_part1 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (1) TO (100) USING heap;
+CREATE TABLE hot_test_part2 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (100) TO (200) USING heap;
+
+CREATE INDEX hot_test_part_idx ON hot_test_partitioned(indexed_col);
+
+INSERT INTO hot_test_partitioned VALUES (1, 50, 100, 'initial1');
+INSERT INTO hot_test_partitioned VALUES (2, 150, 200, 'initial2');
+
+-- Update in partition 1 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'updated1' WHERE id = 1;
+
+-- Update in partition 2 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'updated2' WHERE id = 2;
+
+SELECT get_hot_count('hot_test_part1');
+SELECT get_hot_count('hot_test_part2');
+
+-- Verify indexes work on partitions
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 100;
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 200;
+
+-- Update indexed column in partition - should NOT be HOT
+UPDATE hot_test_partitioned SET indexed_col = 150 WHERE id = 1;
+SELECT get_hot_count('hot_test_part1');
+
+-- Verify index was updated
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 150;
+
+-- Cleanup
+DROP TABLE IF EXISTS hot_test;
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+DROP FUNCTION IF EXISTS has_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS print_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS heap_prune(text);
+DROP FUNCTION IF EXISTS get_hot_count(text);
+DROP EXTENSION pageinspect;
diff --git a/src/test/regress/sql/replica_identity_logging.sql b/src/test/regress/sql/replica_identity_logging.sql
new file mode 100644
index 00000000000..4c45e76e15d
--- /dev/null
+++ b/src/test/regress/sql/replica_identity_logging.sql
@@ -0,0 +1,349 @@
+--
+-- REPLICA_IDENTITY_LOGGING
+-- Test that replica identity keys are correctly extracted and logged for logical replication
+--
+-- This test verifies that the correct old key columns are included in WAL records
+-- for logical replication, based on the table's replica identity setting.
+--
+
+-- Clean up from prior runs
+DROP TABLE IF EXISTS repid_test CASCADE;
+
+-- Drop replication slot if it exists from prior run
+SELECT pg_drop_replication_slot('repid_test_slot') FROM pg_replication_slots WHERE slot_name = 'repid_test_slot';
+
+-- Enable logical decoding to verify what gets logged
+SELECT 'init' FROM pg_create_logical_replication_slot('repid_test_slot', 'test_decoding');
+
+-- REPLICA IDENTITY DEFAULT (primary key columns only)
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ data text
+);
+
+CREATE INDEX repid_test_idx ON repid_test(indexed_col);
+
+INSERT INTO repid_test VALUES (1, 100, 'initial');
+INSERT INTO repid_test VALUES (2, 200, 'initial');
+
+-- Advance slot to skip inserts (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+
+-- Update non-key column - should log only id in old key
+UPDATE repid_test SET data = 'updated' WHERE id = 1;
+
+-- Update indexed non-key column - should still log only id in old key
+UPDATE repid_test SET indexed_col = 150 WHERE id = 2;
+
+-- Check logical decoding output - should see old key with only id
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+
+-- REPLICA IDENTITY FULL (all columns in old key)
+DROP TABLE repid_test;
+
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ data text
+);
+
+ALTER TABLE repid_test REPLICA IDENTITY FULL;
+
+CREATE INDEX repid_test_idx ON repid_test(indexed_col);
+
+INSERT INTO repid_test VALUES (1, 100, 'initial');
+
+-- Advance slot past insert (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+
+-- Update any column - should log ALL columns in old key
+UPDATE repid_test SET data = 'updated' WHERE id = 1;
+
+-- Check logical decoding output - should see old key with all columns
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+
+-- REPLICA IDENTITY USING INDEX (index columns only)
+DROP TABLE repid_test;
+
+CREATE TABLE repid_test (
+ id int,
+ unique_col int UNIQUE NOT NULL,
+ data text
+);
+
+ALTER TABLE repid_test REPLICA IDENTITY USING INDEX repid_test_unique_col_key;
+
+INSERT INTO repid_test VALUES (1, 100, 'initial');
+INSERT INTO repid_test VALUES (2, 200, 'initial');
+
+-- Advance slot past inserts (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+
+-- Update non-indexed column - should log only unique_col in old key
+UPDATE repid_test SET data = 'updated' WHERE unique_col = 100;
+
+-- Update id (not in replica identity index) - should still log only unique_col
+UPDATE repid_test SET id = 10 WHERE unique_col = 200;
+
+-- Check logical decoding output - should see old key with only unique_col
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+
+-- REPLICA IDENTITY NOTHING (no old key logged)
+DROP TABLE repid_test;
+
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ data text
+);
+
+ALTER TABLE repid_test REPLICA IDENTITY NOTHING;
+
+INSERT INTO repid_test VALUES (1, 'initial');
+
+-- Advance slot past insert (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+
+-- Update - should log no old key
+UPDATE repid_test SET data = 'updated' WHERE id = 1;
+
+-- Check logical decoding output - should see update with no old key
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+
+-- Multi-column index replica identity
+DROP TABLE repid_test;
+
+CREATE TABLE repid_test (
+ id int,
+ col_a int NOT NULL,
+ col_b int NOT NULL,
+ col_c int,
+ data text,
+ UNIQUE (col_a, col_b)
+);
+
+ALTER TABLE repid_test REPLICA IDENTITY USING INDEX repid_test_col_a_col_b_key;
+
+INSERT INTO repid_test VALUES (1, 10, 20, 30, 'initial');
+
+-- Advance slot past insert (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+
+-- Update non-indexed columns - should log col_a and col_b in old key
+UPDATE repid_test SET data = 'updated', col_c = 35 WHERE col_a = 10 AND col_b = 20;
+
+-- Check logical decoding output - should see old key with col_a and col_b
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+
+-- TOAST/external columns in replica identity
+DROP TABLE repid_test;
+
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ large_text text,
+ data text
+);
+
+-- REPLICA IDENTITY FULL includes toasted columns
+ALTER TABLE repid_test REPLICA IDENTITY FULL;
+
+-- Insert a large value (large enough to show the concept without excessive output)
+INSERT INTO repid_test VALUES (1, repeat('x', 100), 'initial');
+
+-- Advance slot past insert (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+
+-- Update small column - should still log large_text column in old key
+UPDATE repid_test SET data = 'updated' WHERE id = 1;
+
+-- Check logical decoding output - verify both old and new values are logged
+-- Just check that UPDATE happened and includes both large_text and data columns
+SELECT COUNT(*) as update_count FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%' AND data LIKE '%large_text%' AND data LIKE '%old-key%';
+
+-- Test TOAST columns with REPLICA IDENTITY USING INDEX
+DROP TABLE repid_test;
+
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ indexed_large_text text NOT NULL,
+ data text
+);
+
+-- Create unique index on the large text column
+CREATE UNIQUE INDEX repid_test_large_idx ON repid_test(indexed_large_text);
+
+-- Set replica identity to use the index (not FULL)
+ALTER TABLE repid_test REPLICA IDENTITY USING INDEX repid_test_large_idx;
+
+-- Insert a large value (large enough to be TOASTed)
+INSERT INTO repid_test VALUES (1, repeat('x', 100000), 'initial');
+
+-- Advance slot past inserts
+SELECT COUNT(*) FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+
+-- Update non-indexed column - should still log indexed_large_text in old key
+-- despite being unmodified because it is TOASTed and in the replica key
+UPDATE repid_test SET data = 'updated' WHERE id = 1;
+
+-- Verify TOASTed indexed column part of the relica identity is logged in old key
+SELECT COUNT(*) AS toasted_index_logged FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%' AND data LIKE '%indexed_large_text%';
+-- Dropped columns and replica identity
+DROP TABLE repid_test;
+
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ dropped_col int,
+ kept_col int,
+ data text
+);
+
+ALTER TABLE repid_test REPLICA IDENTITY FULL;
+
+INSERT INTO repid_test VALUES (1, 999, 100, 'initial');
+
+-- Drop a column
+ALTER TABLE repid_test DROP COLUMN dropped_col;
+
+-- Advance slot past insert and DDL (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+
+-- Update - old key should handle dropped column
+UPDATE repid_test SET data = 'updated' WHERE id = 1;
+
+-- Check logical decoding output
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+
+-- DEFAULT replica identity with composite primary key
+DROP TABLE repid_test;
+
+CREATE TABLE repid_test (
+ id_a int,
+ id_b int,
+ indexed_col int,
+ data text,
+ PRIMARY KEY (id_a, id_b)
+);
+
+CREATE INDEX repid_test_idx ON repid_test(indexed_col);
+
+INSERT INTO repid_test VALUES (1, 10, 100, 'initial');
+
+-- Advance slot past insert (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+
+-- Update non-key columns - should log both id_a and id_b in old key
+UPDATE repid_test SET data = 'updated', indexed_col = 150 WHERE id_a = 1 AND id_b = 10;
+
+-- Check logical decoding output - should see old key with both primary key columns
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+
+-- Expression index and replica identity
+DROP TABLE repid_test;
+
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ email text NOT NULL,
+ data text
+);
+
+-- Create unique expression index
+CREATE UNIQUE INDEX repid_test_lower_email_idx ON repid_test(lower(email));
+
+-- Cannot use expression index for replica identity (should fail)
+-- PostgreSQL requires the index to be on simple column references
+-- This should produce an error
+DO $$
+BEGIN
+ ALTER TABLE repid_test REPLICA IDENTITY USING INDEX repid_test_lower_email_idx;
+ RAISE EXCEPTION 'Should have failed - expression indexes cannot be used for replica identity';
+EXCEPTION
+ WHEN feature_not_supported THEN
+ RAISE NOTICE 'Correctly rejected expression index for replica identity';
+END$$;
+
+-- Use FULL instead
+ALTER TABLE repid_test REPLICA IDENTITY FULL;
+
+INSERT INTO repid_test VALUES (1, '[email protected]', 'initial');
+
+-- Advance slot past insert (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+
+-- Update - should log all columns in old key
+UPDATE repid_test SET data = 'updated' WHERE id = 1;
+
+-- Check logical decoding output
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+
+-- NULL values in replica identity columns
+DROP TABLE repid_test;
+
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ nullable_col int,
+ data text
+);
+
+ALTER TABLE repid_test REPLICA IDENTITY FULL;
+
+INSERT INTO repid_test VALUES (1, NULL, 'initial');
+
+-- Advance slot past insert (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+
+-- Update - old key should include NULL value
+UPDATE repid_test SET data = 'updated' WHERE id = 1;
+
+-- Check logical decoding output - should see old key with NULL
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+
+-- Generated columns and replica identity
+DROP TABLE repid_test;
+
+CREATE TABLE repid_test (
+ id int PRIMARY KEY,
+ base_col int,
+ generated_col int GENERATED ALWAYS AS (base_col * 2) STORED,
+ data text
+);
+
+ALTER TABLE repid_test REPLICA IDENTITY FULL;
+
+INSERT INTO repid_test (id, base_col, data) VALUES (1, 50, 'initial');
+
+-- Advance slot past insert (filter out transaction boundaries for stability)
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data NOT LIKE 'BEGIN %' AND data NOT LIKE 'COMMIT %';
+
+-- Update base_col - generated_col will change automatically
+UPDATE repid_test SET base_col = 60 WHERE id = 1;
+
+-- Check logical decoding output - should include old generated_col value
+SELECT data FROM pg_logical_slot_get_changes('repid_test_slot', NULL, NULL)
+WHERE data LIKE '%UPDATE%';
+
+-- Cleanup
+SELECT pg_drop_replication_slot('repid_test_slot');
+DROP TABLE repid_test;
--
2.51.2
[text/x-patch] v33-0002-Idenfity-modified-indexed-attributes-in-the-exec.patch (59.5K, 3-v33-0002-Idenfity-modified-indexed-attributes-in-the-exec.patch)
download | inline diff:
From 0ef7260343415f98201108778d0793af5ebcf1b3 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 2 Nov 2025 11:36:20 -0500
Subject: [PATCH v33 2/2] Idenfity modified indexed attributes in the executor
on UPDATE
Refactor executor update logic to determine which indexed columns have
actually changed during an UPDATE operation rather than leaving this up
to HeapDetermineColumnsInfo() in heap_update(). Finding this set of
attributes is not heap-specific, but more general to all table AMs and
having this information in the executor could inform other decisions
about when index inserts are required and when they are not regardless
of the table AM's MVCC implementation strategy.
The heap-only tuple decision (HOT) in heap functions as it always has,
but the determination of the "modified indexed attributes" (mix_attrs,
was known as modified_attrs) now happens outside the buffer lock and can
inform other decisions unrelated to heap.
ExecUpdateModIdxAttrs() replaces HeapDeterminesColumnsInfo() and is
called before table_tuple_update() crucially without the need for an
exclusive buffer lock on the page that holds the tuple being updated.
This reduces the time the lock is held later within
heapam_tuple_update() and heap_update().
ExecUpdateModIdxAttrs() in turn uses ExecCompareSlotAttrs() to identify
which attributes have changed and then intersects that with the set of
indexed attributes to identify the modified indexed set, the mix_attrs.
Besides identifying the set of modified indexed attributes
HeapDetermineColumnsInfo() was also responsible for part of the logic
involed in the decision to include the replica identity key or not.
This moved into heap_update() and out of HeapDetermineColumnsInfo()
which has been renamed to HeapUpdateModIdxAttrs() as it is still
required within simple_heap_update() to be able to identify mix_attrs
given only an old TID and a new HeapTuple.
Updates stemming from logical replication also use the new
ExecUpdateModIdxAttrs() in ExecSimpleRelationUpdate().
This patch also introduces a few helper functions: HeapUpdateHotAllowable(),
HeapUpdateDetermineLockmode(). These are used in both heap_update() and
simple_heap_update().
The heap_update() function is called now with lockmode pre-determined
and a booleaning indicating if the update allows HOT updates or not.
If during heap_update() the new tuple will fit on the same page and that
boolean is true, the update is HOT. None of the logic related to when
HOT is allowed has changed.
Triggers are free to use heap_modify_tuple() and update attributes not
found in the UPDATE statement or triggers that fire due to an UPDATE.
When that happens the executor has no knowledge of those changes. This
forced HeapDetermineColumnsInfo() to scan all indexed attributes on a
relation rather than only the intersection of indexed and those
identified by ExecGetAllUpdatedCols(). This occurs in at least one test
that uses the tsvector_update_trigger() function (tsearch.sql).
ExecBRUpdateTriggers() has been changed to identify changes to indexed
columns not found by ExecGetAllUpdateCols() and add those attributes to
ri_extraUpdatedCols.
Three tests were adjusted to avoid instability due to tuple ordering
during heap page scans. This avoids nondeterministic results.
---
src/backend/access/heap/heapam.c | 481 +++++++++++-------
src/backend/access/heap/heapam_handler.c | 32 +-
src/backend/access/table/tableam.c | 5 +-
src/backend/commands/trigger.c | 20 +-
src/backend/executor/execReplication.c | 7 +-
src/backend/executor/execTuples.c | 78 +++
src/backend/executor/nodeModifyTable.c | 93 +++-
src/backend/utils/cache/relcache.c | 44 +-
src/include/access/heapam.h | 13 +-
src/include/access/tableam.h | 8 +-
src/include/executor/executor.h | 9 +
src/include/utils/rel.h | 2 +-
src/include/utils/relcache.h | 2 +-
.../regress/expected/generated_virtual.out | 2 +-
src/test/regress/expected/triggers.out | 16 +-
src/test/regress/expected/updatable_views.out | 4 +-
src/test/regress/sql/generated_virtual.sql | 2 +-
src/test/regress/sql/triggers.sql | 4 +-
src/test/regress/sql/updatable_views.sql | 2 +-
19 files changed, 591 insertions(+), 233 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index a231563f0df..18961d714a3 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -37,14 +37,19 @@
#include "access/multixact.h"
#include "access/subtrans.h"
#include "access/syncscan.h"
+#include "access/sysattr.h"
+#include "access/tableam.h"
#include "access/valid.h"
#include "access/visibilitymap.h"
#include "access/xloginsert.h"
#include "catalog/pg_database.h"
#include "catalog/pg_database_d.h"
#include "commands/vacuum.h"
+#include "executor/tuptable.h"
+#include "nodes/lockoptions.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
+#include "storage/buf.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
#include "storage/proc.h"
@@ -52,6 +57,7 @@
#include "utils/datum.h"
#include "utils/injection_point.h"
#include "utils/inval.h"
+#include "utils/relcache.h"
#include "utils/spccache.h"
#include "utils/syscache.h"
@@ -68,11 +74,8 @@ static void check_lock_if_inplace_updateable_rel(Relation relation,
HeapTuple newtup);
static void check_inplace_rel_lock(HeapTuple oldtup);
#endif
-static Bitmapset *HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external);
+static Bitmapset *HeapUpdateModIdxAttrs(Relation relation,
+ HeapTuple oldtup, HeapTuple newtup);
static bool heap_acquire_tuplock(Relation relation, const ItemPointerData *tid,
LockTupleMode mode, LockWaitPolicy wait_policy,
bool *have_tuple_lock);
@@ -3302,7 +3305,7 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* heap_update - replace a tuple
*
* See table_tuple_update() for an explanation of the parameters, except that
- * this routine directly takes a tuple rather than a slot.
+ * this routine directly takes a heap tuple rather than a slot.
*
* In the failure cases, the routine fills *tmfd with the tuple's t_ctid,
* t_xmax (resolving a possible MultiXact, if necessary), and t_cmax (the last
@@ -3312,17 +3315,13 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
TM_Result
heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ TM_FailureData *tmfd, const LockTupleMode lockmode,
+ const Bitmapset *mix_attrs, const bool hot_allowed)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
- Bitmapset *hot_attrs;
- Bitmapset *sum_attrs;
- Bitmapset *key_attrs;
- Bitmapset *id_attrs;
- Bitmapset *interesting_attrs;
- Bitmapset *modified_attrs;
+ Bitmapset *idx_attrs,
+ *rid_attrs;
ItemId lp;
HeapTupleData oldtup;
HeapTuple heaptup;
@@ -3341,13 +3340,12 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
bool have_tuple_lock = false;
bool iscombo;
bool use_hot_update = false;
- bool summarized_update = false;
bool key_intact;
bool all_visible_cleared = false;
bool all_visible_cleared_new = false;
bool checked_lockers;
bool locker_remains;
- bool id_has_external = false;
+ bool rep_id_key_required = false;
TransactionId xmax_new_tuple,
xmax_old_tuple;
uint16 infomask_old_tuple,
@@ -3378,33 +3376,14 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
#endif
/*
- * Fetch the list of attributes to be checked for various operations.
- *
- * For HOT considerations, this is wasted effort if we fail to update or
- * have to put the new tuple on a different page. But we must compute the
- * list before obtaining buffer lock --- in the worst case, if we are
- * doing an update on one of the relevant system catalogs, we could
- * deadlock if we try to fetch the list later. In any case, the relcache
- * caches the data so this is usually pretty cheap.
- *
- * We also need columns used by the replica identity and columns that are
- * considered the "key" of rows in the table.
+ * Fetch the attributes used across all indexes on this relation as well
+ * as the replica identity and columns.
*
- * Note that we get copies of each bitmap, so we need not worry about
- * relcache flush happening midway through.
- */
- hot_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_HOT_BLOCKING);
- sum_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_SUMMARIZED);
- key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
- id_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
- interesting_attrs = NULL;
- interesting_attrs = bms_add_members(interesting_attrs, hot_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, sum_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, key_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, id_attrs);
+ * NOTE: relcache returns copies of each bitmap, so we need not worry
+ * about relcache flush happening midway through.
+ */
+ idx_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+ rid_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_IDENTITY_KEY);
block = ItemPointerGetBlockNumber(otid);
INJECTION_POINT("heap_update-before-pin", NULL);
@@ -3458,20 +3437,17 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
tmfd->ctid = *otid;
tmfd->xmax = InvalidTransactionId;
tmfd->cmax = InvalidCommandId;
- *update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- /* modified_attrs not yet initialized */
- bms_free(interesting_attrs);
+ bms_free(rid_attrs);
+ bms_free(idx_attrs);
+ /* mix_attrs is owned by the caller, don't free it */
+
return TM_Deleted;
}
/*
- * Fill in enough data in oldtup for HeapDetermineColumnsInfo to work
- * properly.
+ * Fill in enough data in oldtup to determine replica identity attribute
+ * requirements.
*/
oldtup.t_tableOid = RelationGetRelid(relation);
oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
@@ -3482,16 +3458,59 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
newtup->t_tableOid = RelationGetRelid(relation);
/*
- * Determine columns modified by the update. Additionally, identify
- * whether any of the unmodified replica identity key attributes in the
- * old tuple is externally stored or not. This is required because for
- * such attributes the flattened value won't be WAL logged as part of the
- * new tuple so we must include it as part of the old_key_tuple. See
- * ExtractReplicaIdentity.
+ * ExtractReplicaIdentity() needs to know if a modified indexed attrbute
+ * is used as a replica indentity or if any of the replica identity
+ * attributes are referenced in an index, unmodified, and are stored
+ * externally in the old tuple being replaced. In those cases it may be
+ * necessary to WAL log them to so they are available to replicas.
*/
- modified_attrs = HeapDetermineColumnsInfo(relation, interesting_attrs,
- id_attrs, &oldtup,
- newtup, &id_has_external);
+ rep_id_key_required = bms_overlap(mix_attrs, rid_attrs);
+ if (!rep_id_key_required)
+ {
+ Bitmapset *attrs;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ int attidx = -1;
+
+ /*
+ * Reduce the set under review to only the unmodified indexed replica
+ * identity key attributes. idx_attrs is copied (by bms_difference())
+ * not modified here.
+ */
+ attrs = bms_difference(idx_attrs, mix_attrs);
+ attrs = bms_int_members(attrs, rid_attrs);
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /*
+ * attidx is zero-based, attrnum is the normal attribute number
+ */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value;
+ bool isnull;
+
+ /*
+ * System attributes are not added into INDEX_ATTR_BITMAP_INDEXED
+ * bitmap by relcache.
+ */
+ Assert(attrnum > 0);
+
+ value = heap_getattr(&oldtup, attrnum, tupdesc, &isnull);
+
+ /* No need to check attributes that can't be stored externally */
+ if (isnull ||
+ TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
+ continue;
+
+ /* Check if the old tuple's attribute is stored externally */
+ if (VARATT_IS_EXTERNAL((struct varlena *) DatumGetPointer(value)))
+ {
+ rep_id_key_required = true;
+ break;
+ }
+ }
+
+ bms_free(attrs);
+ }
/*
* If we're not updating any "key" column, we can grab a weaker lock type.
@@ -3504,9 +3523,8 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
* is updates that don't manipulate key columns, not those that
* serendipitously arrive at the same key values.
*/
- if (!bms_overlap(modified_attrs, key_attrs))
+ if (lockmode == LockTupleNoKeyExclusive)
{
- *lockmode = LockTupleNoKeyExclusive;
mxact_status = MultiXactStatusNoKeyUpdate;
key_intact = true;
@@ -3523,7 +3541,7 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
}
else
{
- *lockmode = LockTupleExclusive;
+ Assert(lockmode == LockTupleExclusive);
mxact_status = MultiXactStatusUpdate;
key_intact = false;
}
@@ -3534,7 +3552,6 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
* with the new tuple's location, so there's great risk of confusion if we
* use otid anymore.
*/
-
l2:
checked_lockers = false;
locker_remains = false;
@@ -3602,7 +3619,7 @@ l2:
bool current_is_member = false;
if (DoesMultiXactIdConflict((MultiXactId) xwait, infomask,
- *lockmode, ¤t_is_member))
+ lockmode, ¤t_is_member))
{
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
@@ -3611,7 +3628,7 @@ l2:
* requesting a lock and already have one; avoids deadlock).
*/
if (!current_is_member)
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &(oldtup.t_self), lockmode,
LockWaitBlock, &have_tuple_lock);
/* wait for multixact */
@@ -3696,7 +3713,7 @@ l2:
* lock.
*/
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &(oldtup.t_self), lockmode,
LockWaitBlock, &have_tuple_lock);
XactLockTableWait(xwait, relation, &oldtup.t_self,
XLTW_Update);
@@ -3756,17 +3773,14 @@ l2:
tmfd->cmax = InvalidCommandId;
UnlockReleaseBuffer(buffer);
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &(oldtup.t_self), lockmode);
if (vmbuffer != InvalidBuffer)
ReleaseBuffer(vmbuffer);
- *update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
+ bms_free(rid_attrs);
+ bms_free(idx_attrs);
+ /* mix_attrs is owned by the caller, don't free it */
+
return result;
}
@@ -3796,7 +3810,7 @@ l2:
compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
oldtup.t_data->t_infomask,
oldtup.t_data->t_infomask2,
- xid, *lockmode, true,
+ xid, lockmode, true,
&xmax_old_tuple, &infomask_old_tuple,
&infomask2_old_tuple);
@@ -3913,7 +3927,7 @@ l2:
compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
oldtup.t_data->t_infomask,
oldtup.t_data->t_infomask2,
- xid, *lockmode, false,
+ xid, lockmode, false,
&xmax_lock_old_tuple, &infomask_lock_old_tuple,
&infomask2_lock_old_tuple);
@@ -4073,37 +4087,16 @@ l2:
/*
* At this point newbuf and buffer are both pinned and locked, and newbuf
- * has enough space for the new tuple. If they are the same buffer, only
- * one pin is held.
+ * has enough space for the new tuple so we can use the HOT update path if
+ * the caller determined that it is allowable.
+ *
+ * NOTE: If newbuf == buffer then only one pin is held.
*/
-
- if (newbuf == buffer)
- {
- /*
- * Since the new tuple is going into the same page, we might be able
- * to do a HOT update. Check if any of the index columns have been
- * changed.
- */
- if (!bms_overlap(modified_attrs, hot_attrs))
- {
- use_hot_update = true;
-
- /*
- * If none of the columns that are used in hot-blocking indexes
- * were updated, we can apply HOT, but we do still need to check
- * if we need to update the summarizing indexes, and update those
- * indexes if the columns were updated, or we may fail to detect
- * e.g. value bound changes in BRIN minmax indexes.
- */
- if (bms_overlap(modified_attrs, sum_attrs))
- summarized_update = true;
- }
- }
+ if ((newbuf == buffer) && hot_allowed)
+ use_hot_update = true;
else
- {
/* Set a hint that the old page could use prune/defrag */
PageSetFull(page);
- }
/*
* Compute replica identity tuple before entering the critical section so
@@ -4113,8 +4106,7 @@ l2:
* columns are modified or it has external data.
*/
old_key_tuple = ExtractReplicaIdentity(relation, &oldtup,
- bms_overlap(modified_attrs, id_attrs) ||
- id_has_external,
+ rep_id_key_required,
&old_key_copied);
/* NO EREPORT(ERROR) from here till changes are logged */
@@ -4243,7 +4235,7 @@ l2:
* Release the lmgr tuple lock, if we had it.
*/
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &(oldtup.t_self), lockmode);
pgstat_count_heap_update(relation, use_hot_update, newbuf != buffer);
@@ -4257,31 +4249,12 @@ l2:
heap_freetuple(heaptup);
}
- /*
- * If it is a HOT update, the update may still need to update summarized
- * indexes, lest we fail to update those summaries and get incorrect
- * results (for example, minmax bounds of the block may change with this
- * update).
- */
- if (use_hot_update)
- {
- if (summarized_update)
- *update_indexes = TU_Summarizing;
- else
- *update_indexes = TU_None;
- }
- else
- *update_indexes = TU_All;
-
if (old_key_tuple != NULL && old_key_copied)
heap_freetuple(old_key_tuple);
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
+ bms_free(rid_attrs);
+ bms_free(idx_attrs);
+ /* mix_attrs is owned by the caller, don't free it */
return TM_Ok;
}
@@ -4454,28 +4427,113 @@ heap_attr_equals(TupleDesc tupdesc, int attrnum, Datum value1, Datum value2,
}
/*
- * Check which columns are being updated.
- *
- * Given an updated tuple, determine (and return into the output bitmapset),
- * from those listed as interesting, the set of columns that changed.
- *
- * has_external indicates if any of the unmodified attributes (from those
- * listed as interesting) of the old tuple is a member of external_cols and is
- * stored externally.
+ * HOT updates are possible when either: a) there are no modified indexed
+ * attributes, or b) the modified attributes are all on summarizing indexes.
+ * Later, in heap_update(), we can choose to perform a HOT update if there is
+ * space on the page for the new tuple and the following code has determined
+ * that HOT is allowed.
+ */
+bool
+HeapUpdateHotAllowable(Relation relation, const Bitmapset *mix_attrs,
+ bool *summarized_only)
+{
+ bool hot_allowed;
+
+ /*
+ * Let's be optimistic and start off by assuming the best case, no indexes
+ * need updating and HOT is allowable.
+ */
+ hot_allowed = true;
+ *summarized_only = false;
+
+ /*
+ * Check for case (a); when there are no modified index attributes HOT is
+ * allowed.
+ */
+ if (bms_is_empty(mix_attrs))
+ hot_allowed = true;
+ else
+ {
+ Bitmapset *sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+
+ /*
+ * At least one index attribute was modified, but is this case (b)
+ * where all the modified index attributes are only used by
+ * summarizing indexes? If that's the case we need to update those
+ * indexes, but this can be a HOT update.
+ */
+ if (bms_is_subset(mix_attrs, sum_attrs))
+ {
+ hot_allowed = true;
+ *summarized_only = true;
+ }
+ else
+ {
+ /*
+ * Now we know that one or more indexed attribute were updated and
+ * that there was at least one of those attributes were referenced
+ * by a non-summarizing index. HOT is not allowed.
+ */
+ hot_allowed = false;
+ }
+
+ bms_free(sum_attrs);
+ }
+
+ return hot_allowed;
+}
+
+/*
+ * If we're not updating any "key" attributes, we can grab a weaker lock type.
+ * This allows for more concurrency when we are running simultaneously with
+ * foreign key checks.
+ */
+LockTupleMode
+HeapUpdateDetermineLockmode(Relation relation, const Bitmapset *mix_attrs)
+{
+ LockTupleMode lockmode = LockTupleExclusive;
+
+ Bitmapset *key_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_KEY);
+
+ if (!bms_overlap(mix_attrs, key_attrs))
+ lockmode = LockTupleNoKeyExclusive;
+
+ bms_free(key_attrs);
+
+ return lockmode;
+}
+
+/*
+ * Return a Bitmapset that contains the set of modified (changed) indexed
+ * attributes between oldtup and newtup.
*/
static Bitmapset *
-HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external)
+HeapUpdateModIdxAttrs(Relation relation, HeapTuple oldtup, HeapTuple newtup)
{
int attidx;
- Bitmapset *modified = NULL;
+ Bitmapset *attrs,
+ *mix_attrs = NULL;
TupleDesc tupdesc = RelationGetDescr(relation);
+ /* Get the set of all attributes across all indexes for this relation */
+ attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+
+ /* No indexed attributes, we're done */
+ if (bms_is_empty(attrs))
+ return NULL;
+
+ /*
+ * This heap update function is used outside the executor and so unlike
+ * heapam_tuple_update() where there is ResultRelInfo and EState to
+ * provide the concise set of attributes that might have been modified
+ * (via ExecGetAllUpdatedCols()) we simply check all indexed attributes to
+ * find the subset that changed value. That's the "modified indexed
+ * attributes" or "mix_attrs".
+ */
attidx = -1;
- while ((attidx = bms_next_member(interesting_cols, attidx)) >= 0)
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
{
/* attidx is zero-based, attrnum is the normal attribute number */
AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
@@ -4491,7 +4549,7 @@ HeapDetermineColumnsInfo(Relation relation,
*/
if (attrnum == 0)
{
- modified = bms_add_member(modified, attidx);
+ mix_attrs = bms_add_member(mix_attrs, attidx);
continue;
}
@@ -4504,7 +4562,7 @@ HeapDetermineColumnsInfo(Relation relation,
{
if (attrnum != TableOidAttributeNumber)
{
- modified = bms_add_member(modified, attidx);
+ mix_attrs = bms_add_member(mix_attrs, attidx);
continue;
}
}
@@ -4520,29 +4578,12 @@ HeapDetermineColumnsInfo(Relation relation,
if (!heap_attr_equals(tupdesc, attrnum, value1,
value2, isnull1, isnull2))
- {
- modified = bms_add_member(modified, attidx);
- continue;
- }
-
- /*
- * No need to check attributes that can't be stored externally. Note
- * that system attributes can't be stored externally.
- */
- if (attrnum < 0 || isnull1 ||
- TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
- continue;
-
- /*
- * Check if the old tuple's attribute is stored externally and is a
- * member of external_cols.
- */
- if (VARATT_IS_EXTERNAL((varlena *) DatumGetPointer(value1)) &&
- bms_is_member(attidx, external_cols))
- *has_external = true;
+ mix_attrs = bms_add_member(mix_attrs, attidx);
}
- return modified;
+ bms_free(attrs);
+
+ return mix_attrs;
}
/*
@@ -4554,17 +4595,108 @@ HeapDetermineColumnsInfo(Relation relation,
* via ereport().
*/
void
-simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup,
+simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TM_FailureData tmfd;
LockTupleMode lockmode;
+ TupleTableSlot *slot;
+ BufferHeapTupleTableSlot *bslot;
+ HeapTuple oldtup;
+ bool shouldFree = true;
+ Bitmapset *idx_attrs,
+ *mix_attrs;
+ bool hot_allowed,
+ summarized_only;
+ Buffer buffer;
- result = heap_update(relation, otid, tup,
- GetCurrentCommandId(true), InvalidSnapshot,
- true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ Assert(ItemPointerIsValid(otid));
+
+ /*
+ * Fetch this bitmap of interesting attributes from relcache before
+ * obtaining a buffer lock because if we are doing an update on one of the
+ * relevant system catalogs we could deadlock if we try to fetch them
+ * later on. Relcache will return copies of each bitmap, so we need not
+ * worry about relcache flush happening midway through this operation.
+ */
+ idx_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+
+ INJECTION_POINT("heap_update-before-pin", NULL);
+
+ /*
+ * To update a heap tuple we need to find the set of modified indexed
+ * attributes ("mix_attrs") so as to see if a HOT update is allowable or
+ * not. When updating heap tuples via execution of UPDATE statements this
+ * set is constructed before calling into the table AM's tuple_update()
+ * function by the function ExecUpdateModIdxAttrs() which compares the
+ * old/new TupleTableSlots. However, here we have the old TID and the new
+ * tuple, not two TupleTableSlots, but we still need to construct a simlar
+ * bitmap so as to be able to know if HOT updates are allowed or not. To
+ * do that we first have to fetch the old tuple itself. Because
+ * heapam_fetch_row_version() is static, we have to replicate that code
+ * here. This is a bit repetitive because heap_update() will again find
+ * and form the old HeapTuple from the old TID and in most cases the
+ * callers (ignoring extensions, always catalog tuple updates) already had
+ * the set of changed attributes (e.g. the "replaces" array), but for now
+ * this minor repetition of work is necessary.
+ */
+
+ slot = MakeTupleTableSlot(RelationGetDescr(relation), &TTSOpsBufferHeapTuple);
+ bslot = (BufferHeapTupleTableSlot *) slot;
+
+ /*
+ * Set the TID in the slot and then fetch the old tuple so we can examine
+ * it
+ */
+ bslot->base.tupdata.t_self = *otid;
+ if (!heap_fetch(relation, SnapshotAny, &bslot->base.tupdata, &buffer, false))
+ {
+ /*
+ * heap_update() checks for !ItemIdIsNormal(lp) and will return false
+ * in those cases.
+ */
+ Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
+
+ *update_indexes = TU_None;
+
+ /* mix_attrs not yet initialized */
+ bms_free(idx_attrs);
+ ExecDropSingleTupleTableSlot(slot);
+
+ elog(ERROR, "tuple concurrently deleted");
+
+ return;
+ }
+
+ Assert(buffer != InvalidBuffer);
+
+ /* Store in slot, transferring existing pin */
+ ExecStorePinnedBufferHeapTuple(&bslot->base.tupdata, slot, buffer);
+ oldtup = ExecFetchSlotHeapTuple(slot, false, &shouldFree);
+
+ mix_attrs = HeapUpdateModIdxAttrs(relation, oldtup, tuple);
+ lockmode = HeapUpdateDetermineLockmode(relation, mix_attrs);
+ hot_allowed = HeapUpdateHotAllowable(relation, mix_attrs, &summarized_only);
+
+ result = heap_update(relation, otid, tuple, GetCurrentCommandId(true),
+ InvalidSnapshot, true /* wait for commit */ ,
+ &tmfd, lockmode, mix_attrs, hot_allowed);
+
+ if (shouldFree)
+ heap_freetuple(oldtup);
+
+ ExecDropSingleTupleTableSlot(slot);
+ bms_free(idx_attrs);
+
+ /*
+ * Decide whether new index entries are needed for the tuple
+ *
+ * If the update is not HOT, we must update all indexes. If the update is
+ * HOT, it could be that we updated summarized columns, so we either
+ * update only summarized indexes, or none at all.
+ */
+ *update_indexes = TU_None;
switch (result)
{
case TM_SelfModified:
@@ -4574,6 +4706,10 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
case TM_Ok:
/* done successfully */
+ if (!HeapTupleIsHeapOnly(tuple))
+ *update_indexes = TU_All;
+ else if (summarized_only)
+ *update_indexes = TU_Summarizing;
break;
case TM_Updated:
@@ -4590,7 +4726,6 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
}
}
-
/*
* Return the MultiXactStatus corresponding to the given tuple lock mode.
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 3ff36f59bf8..4600af61793 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,7 +27,6 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
-#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -44,6 +43,7 @@
#include "storage/procarray.h"
#include "storage/smgr.h"
#include "utils/builtins.h"
+#include "utils/injection_point.h"
#include "utils/rel.h"
static void reform_and_rewrite_tuple(HeapTuple tuple,
@@ -316,19 +316,26 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd,
- LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
+ bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
+ const Bitmapset *mix_attrs, TU_UpdateIndexes *update_indexes)
{
bool shouldFree = true;
HeapTuple tuple = ExecFetchSlotHeapTuple(slot, true, &shouldFree);
+ bool hot_allowed;
+ bool summarized_only;
TM_Result result;
+ Assert(ItemPointerIsValid(otid));
+
+ hot_allowed = HeapUpdateHotAllowable(relation, mix_attrs, &summarized_only);
+ *lockmode = HeapUpdateDetermineLockmode(relation, mix_attrs);
+
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
- tmfd, lockmode, update_indexes);
+ tmfd, *lockmode, mix_attrs, hot_allowed);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
@@ -341,16 +348,17 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
* HOT, it could be that we updated summarized columns, so we either
* update only summarized indexes, or none at all.
*/
- if (result != TM_Ok)
+ *update_indexes = TU_None;
+ if (result == TM_Ok)
{
- Assert(*update_indexes == TU_None);
- *update_indexes = TU_None;
+ if (HeapTupleIsHeapOnly(tuple))
+ {
+ if (summarized_only)
+ *update_indexes = TU_Summarizing;
+ }
+ else
+ *update_indexes = TU_All;
}
- else if (!HeapTupleIsHeapOnly(tuple))
- Assert(*update_indexes == TU_All);
- else
- Assert((*update_indexes == TU_Summarizing) ||
- (*update_indexes == TU_None));
if (shouldFree)
pfree(tuple);
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index dfda1af412e..42acd5b17a9 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -359,6 +359,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -369,7 +370,9 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
GetCurrentCommandId(true),
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, &lockmode,
+ mix_attrs,
+ update_indexes);
switch (result)
{
diff --git a/src/backend/commands/trigger.c b/src/backend/commands/trigger.c
index 98d402c0a3b..64efa55dfe3 100644
--- a/src/backend/commands/trigger.c
+++ b/src/backend/commands/trigger.c
@@ -2978,6 +2978,7 @@ ExecBRUpdateTriggers(EState *estate, EPQState *epqstate,
bool is_merge_update)
{
TriggerDesc *trigdesc = relinfo->ri_TrigDesc;
+ TupleDesc tupdesc = RelationGetDescr(relinfo->ri_RelationDesc);
TupleTableSlot *oldslot = ExecGetTriggerOldSlot(estate, relinfo);
HeapTuple newtuple = NULL;
HeapTuple trigtuple;
@@ -2985,7 +2986,9 @@ ExecBRUpdateTriggers(EState *estate, EPQState *epqstate,
bool should_free_new = false;
TriggerData LocTriggerData = {0};
int i;
- Bitmapset *updatedCols;
+ Bitmapset *updatedCols = NULL;
+ Bitmapset *remainingCols = NULL;
+ Bitmapset *modifiedCols;
LockTupleMode lockmode;
/* Determine lock mode to use */
@@ -3127,6 +3130,21 @@ ExecBRUpdateTriggers(EState *estate, EPQState *epqstate,
if (should_free_trig)
heap_freetuple(trigtuple);
+ /*
+ * Before UPDATE triggers may have updated attributes not known to
+ * ExecGetAllUpdatedColumns() using heap_modify_tuple() or
+ * heap_modifiy_tuple_by_cols(). Find and record those now.
+ */
+ remainingCols = bms_add_range(NULL, 1 - FirstLowInvalidHeapAttributeNumber,
+ tupdesc->natts - FirstLowInvalidHeapAttributeNumber);
+ remainingCols = bms_del_members(remainingCols, updatedCols);
+ modifiedCols = ExecCompareSlotAttrs(tupdesc, remainingCols, oldslot, newslot);
+ relinfo->ri_extraUpdatedCols =
+ bms_add_members(relinfo->ri_extraUpdatedCols, modifiedCols);
+
+ bms_free(remainingCols);
+ bms_free(modifiedCols);
+
return true;
}
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index 2497ee7edc5..c2e77740e76 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -33,6 +33,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
#include "utils/typcache.h"
@@ -906,6 +907,7 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
bool skip_tuple = false;
Relation rel = resultRelInfo->ri_RelationDesc;
ItemPointer tid = &(searchslot->tts_tid);
+ Bitmapset *mix_attrs;
/*
* We support only non-system tables, with
@@ -944,8 +946,11 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ mix_attrs = ExecUpdateModIdxAttrs(resultRelInfo,
+ estate, searchslot, slot);
+
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
- &update_indexes);
+ mix_attrs, &update_indexes);
conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index b768eae9e53..1064ebe845b 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -66,6 +66,7 @@
#include "nodes/nodeFuncs.h"
#include "storage/bufmgr.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/expandeddatum.h"
#include "utils/lsyscache.h"
#include "utils/typcache.h"
@@ -1929,6 +1930,83 @@ ExecFetchSlotHeapTupleDatum(TupleTableSlot *slot)
return ret;
}
+/*
+ * ExecCompareSlotAttrs
+ *
+ * Compare the subset of attributes in attrs bewtween TupleTableSlots to detect
+ * which attributes have changed.
+ *
+ * Returns a Bitmapset of attribute indices (using
+ * FirstLowInvalidHeapAttributeNumber convention) that differ between the two
+ * slots.
+ */
+Bitmapset *
+ExecCompareSlotAttrs(TupleDesc tupdesc, const Bitmapset *attrs,
+ TupleTableSlot *s1, TupleTableSlot *s2)
+{
+ int attidx = -1;
+ Bitmapset *modified = NULL;
+
+ /* XXX what if slots don't share the same tupleDescriptor... */
+ /* Assert(s1->tts_tupleDescriptor == s2->tts_tupleDescriptor); */
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /* attidx is zero-based, attrnum is the normal attribute number */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value1,
+ value2;
+ bool null1,
+ null2;
+ CompactAttribute *att;
+
+ /*
+ * If it's a whole-tuple reference, say "not equal". It's not really
+ * worth supporting this case, since it could only succeed after a
+ * no-op update, which is hardly a case worth optimizing for.
+ */
+ if (attrnum == 0)
+ {
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /*
+ * Likewise, automatically say "not equal" for any system attribute
+ * other than tableOID; we cannot expect these to be consistent in a
+ * HOT chain, or even to be set correctly yet in the new tuple.
+ */
+ if (attrnum < 0)
+ {
+ if (attrnum != TableOidAttributeNumber)
+ {
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+ }
+
+ att = TupleDescCompactAttr(tupdesc, attrnum - 1);
+ value1 = slot_getattr(s1, attrnum, &null1);
+ value2 = slot_getattr(s2, attrnum, &null2);
+
+ /* A change to/from NULL, so not equal */
+ if (null1 != null2)
+ {
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /* Both NULL, no change/unmodified */
+ if (null2)
+ continue;
+
+ if (!datum_image_eq(value1, value2, att->attbyval, att->attlen))
+ modified = bms_add_member(modified, attidx);
+ }
+
+ return modified;
+}
+
/* ----------------------------------------------------------------
* convenience initialization routines
* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 793c76d4f82..4927fc88e61 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -17,6 +17,7 @@
* ExecModifyTable - retrieve the next tuple from the node
* ExecEndModifyTable - shut down the ModifyTable node
* ExecReScanModifyTable - rescan the ModifyTable node
+ * ExecUpdateModIdxAttrs - find set of updated indexed columns
*
* NOTES
* The ModifyTable node receives input from its outerPlan, which is
@@ -54,6 +55,7 @@
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/tupdesc.h"
#include "access/xact.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
@@ -188,6 +190,68 @@ static TupleTableSlot *ExecMergeNotMatched(ModifyTableContext *context,
ResultRelInfo *resultRelInfo,
bool canSetTag);
+/*
+ * ExecUpdateModIdxAttrs
+ *
+ * Find the set of attributes referenced by this relation and used in this
+ * UPDATE that now differ in value. This is done by reviewing slot datum that
+ * are in the UPDATE statment and are known to be referenced by at least one
+ * index in some way. This set is called the "modified indexed attributes" or
+ * "mix_attrs". An overlap of a single index's attributes and this "mix" set
+ * signals that the attributes in the new_tts used to form the index datum have
+ * changed.
+ *
+ * Return a Bitmapset that contains the set of modified (changed) indexed
+ * attributes between oldtup and newtup.
+ *
+ * NOTE: There is a simlar function called HeapUpdateModIDxAttrs() that operates
+ * on the old TID and new HeapTuple rather than the old/new TupleTableSlots as
+ * this function does. These two functions should mirror one another until
+ * someday when catalog tuple updates track their changes avoiding the need to
+ * re-discover them in simple_heap_update().
+ */
+Bitmapset *
+ExecUpdateModIdxAttrs(ResultRelInfo *resultRelInfo,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts)
+{
+ Relation relation = resultRelInfo->ri_RelationDesc;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ Bitmapset *attrs,
+ *mix_attrs = NULL;
+
+ /* If no indexes, we're done */
+ if (resultRelInfo->ri_NumIndices == 0)
+ return NULL;
+
+ /*
+ * Get the set of all attributes across all indexes for this relation from
+ * the relcache, it returns us a copy of the bitmap so we can modify it.
+ */
+ attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+
+ /*
+ * Fetch the set of attributes explicity SET in the UPDATE statement or
+ * set by a before row trigger (even if not mentioned in the SQL) from the
+ * executor state and then find the intersection with the indexed
+ * attributes. Attributes that are SET might not change value, so we have
+ * to examine them for changes.
+ */
+ attrs = bms_int_members(attrs, ExecGetAllUpdatedCols(resultRelInfo, estate));
+
+ /*
+ * When there are indexed attributes mentioned in the UPDATE then we need
+ * to find the subset that changed value. That's the "modified indexed
+ * attributes" or "mix_attrs".
+ */
+ if (!bms_is_empty(attrs))
+ mix_attrs = ExecCompareSlotAttrs(tupdesc, attrs, old_tts, new_tts);
+
+ bms_free(attrs);
+
+ return mix_attrs;
+}
/*
* Verify that the tuples to be produced by INSERT match the
@@ -2195,14 +2259,17 @@ ExecUpdatePrepareSlot(ResultRelInfo *resultRelInfo,
*/
static TM_Result
ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
- ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *slot,
- bool canSetTag, UpdateContext *updateCxt)
+ ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *oldSlot,
+ TupleTableSlot *slot, bool canSetTag, UpdateContext *updateCxt)
{
EState *estate = context->estate;
Relation resultRelationDesc = resultRelInfo->ri_RelationDesc;
bool partition_constraint_failed;
TM_Result result;
+ /* The set of modified indexed attributes that trigger new index entries */
+ Bitmapset *mix_attrs = NULL;
+
updateCxt->crossPartUpdate = false;
/*
@@ -2319,7 +2386,16 @@ lreplace:
ExecConstraints(resultRelInfo, slot, estate);
/*
- * replace the heap tuple
+ * Next up we need to find out the set of indexed attributes that have
+ * changed in value and should trigger a new index tuple. We could start
+ * with the set of updated columns via ExecGetUpdatedCols(), but if we do
+ * we will overlook attributes directly modified by heap_modify_tuple()
+ * which are not known to ExecGetUpdatedCols().
+ */
+ mix_attrs = ExecUpdateModIdxAttrs(resultRelInfo, estate, oldSlot, slot);
+
+ /*
+ * Call into the table AM to update the heap tuple.
*
* Note: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
* the row to be updated is visible to that snapshot, and throw a
@@ -2333,6 +2409,7 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
+ mix_attrs,
&updateCxt->updateIndexes);
return result;
@@ -2555,8 +2632,8 @@ ExecUpdate(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
*/
redo_act:
lockedtid = *tupleid;
- result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, slot,
- canSetTag, &updateCxt);
+ result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, oldSlot,
+ slot, canSetTag, &updateCxt);
/*
* If ExecUpdateAct reports that a cross-partition update was done,
@@ -3406,8 +3483,8 @@ lmerge_matched:
Assert(oldtuple == NULL);
result = ExecUpdateAct(context, resultRelInfo, tupleid,
- NULL, newslot, canSetTag,
- &updateCxt);
+ NULL, resultRelInfo->ri_oldTupleSlot,
+ newslot, canSetTag, &updateCxt);
/*
* As in ExecUpdate(), if ExecUpdateAct() reports that a
@@ -4539,7 +4616,7 @@ ExecModifyTable(PlanState *pstate)
* For UPDATE/DELETE/MERGE, fetch the row identity info for the tuple
* to be updated/deleted/merged. For a heap relation, that's a TID;
* otherwise we may have a wholerow junk attr that carries the old
- * tuple in toto. Keep this in step with the part of
+ * tuple in total. Keep this in step with the part of
* ExecInitModifyTable that sets up ri_RowIdAttNo.
*/
if (operation == CMD_UPDATE || operation == CMD_DELETE ||
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 6b634c9fff1..f30505d8ae3 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -2475,8 +2475,8 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
bms_free(relation->rd_keyattr);
bms_free(relation->rd_pkattr);
bms_free(relation->rd_idattr);
- bms_free(relation->rd_hotblockingattr);
bms_free(relation->rd_summarizedattr);
+ bms_free(relation->rd_indexedattr);
if (relation->rd_pubdesc)
pfree(relation->rd_pubdesc);
if (relation->rd_options)
@@ -5276,8 +5276,8 @@ RelationGetIndexPredicate(Relation relation)
* (beware: even if PK is deferrable!)
* INDEX_ATTR_BITMAP_IDENTITY_KEY Columns in the table's replica identity
* index (empty if FULL)
- * INDEX_ATTR_BITMAP_HOT_BLOCKING Columns that block updates from being HOT
- * INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
+ * INDEX_ATTR_BITMAP_SUMMARIZED Columns only included in summarizing indexes
+ * INDEX_ATTR_BITMAP_INDEXED Columns referenced by indexes
*
* Attribute numbers are offset by FirstLowInvalidHeapAttributeNumber so that
* we can include system attributes (e.g., OID) in the bitmap representation.
@@ -5300,8 +5300,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
Bitmapset *uindexattrs; /* columns in unique indexes */
Bitmapset *pkindexattrs; /* columns in the primary index */
Bitmapset *idindexattrs; /* columns in the replica identity */
- Bitmapset *hotblockingattrs; /* columns with HOT blocking indexes */
- Bitmapset *summarizedattrs; /* columns with summarizing indexes */
+ Bitmapset *summarizedattrs; /* columns only in summarizing indexes */
+ Bitmapset *indexedattrs; /* columns referenced by indexes */
List *indexoidlist;
List *newindexoidlist;
Oid relpkindex;
@@ -5320,10 +5320,10 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
return bms_copy(relation->rd_pkattr);
case INDEX_ATTR_BITMAP_IDENTITY_KEY:
return bms_copy(relation->rd_idattr);
- case INDEX_ATTR_BITMAP_HOT_BLOCKING:
- return bms_copy(relation->rd_hotblockingattr);
case INDEX_ATTR_BITMAP_SUMMARIZED:
return bms_copy(relation->rd_summarizedattr);
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return bms_copy(relation->rd_indexedattr);
default:
elog(ERROR, "unknown attrKind %u", attrKind);
}
@@ -5366,8 +5366,8 @@ restart:
uindexattrs = NULL;
pkindexattrs = NULL;
idindexattrs = NULL;
- hotblockingattrs = NULL;
summarizedattrs = NULL;
+ indexedattrs = NULL;
foreach(l, indexoidlist)
{
Oid indexOid = lfirst_oid(l);
@@ -5426,7 +5426,7 @@ restart:
if (indexDesc->rd_indam->amsummarizing)
attrs = &summarizedattrs;
else
- attrs = &hotblockingattrs;
+ attrs = &indexedattrs;
/* Collect simple attribute references */
for (i = 0; i < indexDesc->rd_index->indnatts; i++)
@@ -5435,9 +5435,9 @@ restart:
/*
* Since we have covering indexes with non-key columns, we must
- * handle them accurately here. non-key columns must be added into
- * hotblockingattrs or summarizedattrs, since they are in index,
- * and update shouldn't miss them.
+ * handle them accurately here. Non-key columns must be added into
+ * indexedattrs or summarizedattrs, since they are in index, and
+ * update shouldn't miss them.
*
* Summarizing indexes do not block HOT, but do need to be updated
* when the column value changes, thus require a separate
@@ -5498,12 +5498,20 @@ restart:
bms_free(uindexattrs);
bms_free(pkindexattrs);
bms_free(idindexattrs);
- bms_free(hotblockingattrs);
bms_free(summarizedattrs);
+ bms_free(indexedattrs);
goto restart;
}
+ /*
+ * Record what attributes are only referenced by summarizing indexes. Then
+ * add that into the other indexed attributes to track all referenced
+ * attributes.
+ */
+ summarizedattrs = bms_del_members(summarizedattrs, indexedattrs);
+ indexedattrs = bms_add_members(indexedattrs, summarizedattrs);
+
/* Don't leak the old values of these bitmaps, if any */
relation->rd_attrsvalid = false;
bms_free(relation->rd_keyattr);
@@ -5512,10 +5520,10 @@ restart:
relation->rd_pkattr = NULL;
bms_free(relation->rd_idattr);
relation->rd_idattr = NULL;
- bms_free(relation->rd_hotblockingattr);
- relation->rd_hotblockingattr = NULL;
bms_free(relation->rd_summarizedattr);
relation->rd_summarizedattr = NULL;
+ bms_free(relation->rd_indexedattr);
+ relation->rd_indexedattr = NULL;
/*
* Now save copies of the bitmaps in the relcache entry. We intentionally
@@ -5528,8 +5536,8 @@ restart:
relation->rd_keyattr = bms_copy(uindexattrs);
relation->rd_pkattr = bms_copy(pkindexattrs);
relation->rd_idattr = bms_copy(idindexattrs);
- relation->rd_hotblockingattr = bms_copy(hotblockingattrs);
relation->rd_summarizedattr = bms_copy(summarizedattrs);
+ relation->rd_indexedattr = bms_copy(indexedattrs);
relation->rd_attrsvalid = true;
MemoryContextSwitchTo(oldcxt);
@@ -5542,10 +5550,10 @@ restart:
return pkindexattrs;
case INDEX_ATTR_BITMAP_IDENTITY_KEY:
return idindexattrs;
- case INDEX_ATTR_BITMAP_HOT_BLOCKING:
- return hotblockingattrs;
case INDEX_ATTR_BITMAP_SUMMARIZED:
return summarizedattrs;
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return indexedattrs;
default:
elog(ERROR, "unknown attrKind %u", attrKind);
return NULL;
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 3c0961ab36b..7abc8e24f21 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -365,10 +365,9 @@ extern TM_Result heap_delete(Relation relation, const ItemPointerData *tid,
extern void heap_finish_speculative(Relation relation, const ItemPointerData *tid);
extern void heap_abort_speculative(Relation relation, const ItemPointerData *tid);
extern TM_Result heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, const LockTupleMode lockmode,
+ const Bitmapset *mix_attrs, const bool hot_allowed);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -430,6 +429,12 @@ extern void log_heap_prune_and_freeze(Relation relation, Buffer buffer,
OffsetNumber *dead, int ndead,
OffsetNumber *unused, int nunused);
+/* in heap/heapam.c */
+extern bool HeapUpdateHotAllowable(Relation relation, const Bitmapset *mix_attrs,
+ bool *summarized_only);
+extern LockTupleMode HeapUpdateDetermineLockmode(Relation relation,
+ const Bitmapset *mix_attrs);
+
/* in heap/vacuumlazy.c */
extern void heap_vacuum_rel(Relation rel,
const VacuumParams params, BufferAccessStrategy bstrategy);
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 06084752245..19c58a76854 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -549,6 +549,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1523,12 +1524,12 @@ static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ const Bitmapset *mix_attrs, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
- wait, tmfd,
- lockmode, update_indexes);
+ wait, tmfd, lockmode,
+ mix_attrs, update_indexes);
}
/*
@@ -2009,6 +2010,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index d46ba59895d..266d5309103 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -17,6 +17,7 @@
#include "datatype/timestamp.h"
#include "executor/execdesc.h"
#include "fmgr.h"
+#include "nodes/execnodes.h"
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
@@ -606,6 +607,10 @@ extern TupleDesc ExecCleanTypeFromTL(List *targetList);
extern TupleDesc ExecTypeFromExprList(List *exprList);
extern void ExecTypeSetColNames(TupleDesc typeInfo, List *namesList);
extern void UpdateChangedParamSet(PlanState *node, Bitmapset *newchg);
+extern Bitmapset *ExecCompareSlotAttrs(TupleDesc tupdesc,
+ const Bitmapset *attrs,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
typedef struct TupOutputState
{
@@ -803,5 +808,9 @@ extern ResultRelInfo *ExecLookupResultRelByOid(ModifyTableState *node,
Oid resultoid,
bool missing_ok,
bool update_cache);
+extern Bitmapset *ExecUpdateModIdxAttrs(ResultRelInfo *relinfo,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
#endif /* EXECUTOR_H */
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 236830f6b93..10e5e9044ee 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -162,8 +162,8 @@ typedef struct RelationData
Bitmapset *rd_keyattr; /* cols that can be ref'd by foreign keys */
Bitmapset *rd_pkattr; /* cols included in primary key */
Bitmapset *rd_idattr; /* included in replica identity index */
- Bitmapset *rd_hotblockingattr; /* cols blocking HOT update */
Bitmapset *rd_summarizedattr; /* cols indexed by summarizing indexes */
+ Bitmapset *rd_indexedattr; /* all cols referenced by indexes */
PublicationDesc *rd_pubdesc; /* publication descriptor, or NULL */
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2700224939a..57b46ee54e5 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -69,8 +69,8 @@ typedef enum IndexAttrBitmapKind
INDEX_ATTR_BITMAP_KEY,
INDEX_ATTR_BITMAP_PRIMARY_KEY,
INDEX_ATTR_BITMAP_IDENTITY_KEY,
- INDEX_ATTR_BITMAP_HOT_BLOCKING,
INDEX_ATTR_BITMAP_SUMMARIZED,
+ INDEX_ATTR_BITMAP_INDEXED,
} IndexAttrBitmapKind;
extern Bitmapset *RelationGetIndexAttrBitmap(Relation relation,
diff --git a/src/test/regress/expected/generated_virtual.out b/src/test/regress/expected/generated_virtual.out
index 6dab60c937b..7ebb7890d96 100644
--- a/src/test/regress/expected/generated_virtual.out
+++ b/src/test/regress/expected/generated_virtual.out
@@ -287,7 +287,7 @@ DETAIL: Column "b" is a generated column.
INSERT INTO gtest1v VALUES (8, DEFAULT), (9, DEFAULT); -- error
ERROR: cannot insert a non-DEFAULT value into column "b"
DETAIL: Column "b" is a generated column.
-SELECT * FROM gtest1v;
+SELECT * FROM gtest1v ORDER BY a;
a | b
---+----
3 | 6
diff --git a/src/test/regress/expected/triggers.out b/src/test/regress/expected/triggers.out
index 98dee63b50a..ef98fd0cccf 100644
--- a/src/test/regress/expected/triggers.out
+++ b/src/test/regress/expected/triggers.out
@@ -959,16 +959,24 @@ NOTICE: main_view BEFORE UPDATE STATEMENT (before_view_upd_stmt)
NOTICE: main_view AFTER UPDATE STATEMENT (after_view_upd_stmt)
UPDATE 0
-- Delete from view using trigger
-DELETE FROM main_view WHERE a IN (20,21);
+DELETE FROM main_view WHERE a = 20 AND b = 31;
NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
-NOTICE: OLD: (21,10)
-NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
NOTICE: OLD: (20,31)
+NOTICE: main_view AFTER DELETE STATEMENT (after_view_del_stmt)
+DELETE 1
+DELETE FROM main_view WHERE a = 21 AND b = 10;
+NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
+NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
+NOTICE: OLD: (21,10)
+NOTICE: main_view AFTER DELETE STATEMENT (after_view_del_stmt)
+DELETE 1
+DELETE FROM main_view WHERE a = 21 AND b = 32;
+NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
NOTICE: OLD: (21,32)
NOTICE: main_view AFTER DELETE STATEMENT (after_view_del_stmt)
-DELETE 3
+DELETE 1
DELETE FROM main_view WHERE a = 31 RETURNING a, b;
NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
diff --git a/src/test/regress/expected/updatable_views.out b/src/test/regress/expected/updatable_views.out
index 9cea538b8e8..4877a1ddce9 100644
--- a/src/test/regress/expected/updatable_views.out
+++ b/src/test/regress/expected/updatable_views.out
@@ -372,15 +372,15 @@ INSERT INTO rw_view16 (a, b) VALUES (3, 'Row 3'); -- should be OK
UPDATE rw_view16 SET a=3, aa=-3 WHERE a=3; -- should fail
ERROR: multiple assignments to same column "a"
UPDATE rw_view16 SET aa=-3 WHERE a=3; -- should be OK
-SELECT * FROM base_tbl;
+SELECT * FROM base_tbl ORDER BY a;
a | b
----+--------
+ -3 | Row 3
-2 | Row -2
-1 | Row -1
0 | Row 0
1 | Row 1
2 | Row 2
- -3 | Row 3
(6 rows)
DELETE FROM rw_view16 WHERE a=-3; -- should be OK
diff --git a/src/test/regress/sql/generated_virtual.sql b/src/test/regress/sql/generated_virtual.sql
index e750866d2d8..877152d6d69 100644
--- a/src/test/regress/sql/generated_virtual.sql
+++ b/src/test/regress/sql/generated_virtual.sql
@@ -127,7 +127,7 @@ ALTER VIEW gtest1v ALTER COLUMN b SET DEFAULT 100;
INSERT INTO gtest1v VALUES (8, DEFAULT); -- error
INSERT INTO gtest1v VALUES (8, DEFAULT), (9, DEFAULT); -- error
-SELECT * FROM gtest1v;
+SELECT * FROM gtest1v ORDER BY a;
DELETE FROM gtest1v WHERE a >= 5;
DROP VIEW gtest1v;
diff --git a/src/test/regress/sql/triggers.sql b/src/test/regress/sql/triggers.sql
index ea39817ee3d..6ceb61608ae 100644
--- a/src/test/regress/sql/triggers.sql
+++ b/src/test/regress/sql/triggers.sql
@@ -660,7 +660,9 @@ UPDATE main_view SET b = 32 WHERE a = 21 AND b = 31 RETURNING a, b;
UPDATE main_view SET b = 0 WHERE false;
-- Delete from view using trigger
-DELETE FROM main_view WHERE a IN (20,21);
+DELETE FROM main_view WHERE a = 20 AND b = 31;
+DELETE FROM main_view WHERE a = 21 AND b = 10;
+DELETE FROM main_view WHERE a = 21 AND b = 32;
DELETE FROM main_view WHERE a = 31 RETURNING a, b;
\set QUIET true
diff --git a/src/test/regress/sql/updatable_views.sql b/src/test/regress/sql/updatable_views.sql
index 1635adde2d4..160e7799715 100644
--- a/src/test/regress/sql/updatable_views.sql
+++ b/src/test/regress/sql/updatable_views.sql
@@ -125,7 +125,7 @@ INSERT INTO rw_view16 VALUES (3, 'Row 3', 3); -- should fail
INSERT INTO rw_view16 (a, b) VALUES (3, 'Row 3'); -- should be OK
UPDATE rw_view16 SET a=3, aa=-3 WHERE a=3; -- should fail
UPDATE rw_view16 SET aa=-3 WHERE a=3; -- should be OK
-SELECT * FROM base_tbl;
+SELECT * FROM base_tbl ORDER BY a;
DELETE FROM rw_view16 WHERE a=-3; -- should be OK
-- Read-only views
INSERT INTO ro_view17 VALUES (3, 'ROW 3');
--
2.51.2
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2026-03-02 19:08 Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
@ 2026-03-11 15:51 ` Greg Burd <[email protected]>
2026-03-12 20:33 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Greg Burd @ 2026-03-11 15:51 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Jeff Davis <[email protected]>; Nathan Bossart <[email protected]>
Hello again,
Attached is v35 (master@f4a4ce52c0d) where I've separated out changes into three patches. Still nothing related to $subject directly, but foundational for that work (coming soon). I'd like to get these into v19 if at all possible and then target the rest of $subject for v20 so that it has more time to soak.
0001 - This patch adds tests to validate and capture the expected behavior of Heap-only tuple (HOT) updates. This also serves as a foundation that will aide in documenting what exactly changed in the commits implementing $subject at some later date. This patch isn't required, but it does a good job of demonstrating that a) the changes in 0002 don't impact HOT decisions (as intended) and b) that future patches which change HOT behavior have a very obvious record of what changed because they update these test results (not tests) to illustrate that. That said, if the next two patches are merged without this one I'd be just as happy as if all 3 made it into v19.
0002 - This patch plugs a hole (bug?) in ExecGetAllUpdatedCols() which is triggered by an existing test in tsearch.sql and the tsvector_update_trigger(). That trigger uses heap_modify_tuple() to change an indexed attribute that is not discovered by ExecGetAllUpdatedCols(), which seems odd to me at best and at worst wrong (or even a potential security issue). This patch finds and adds columns that are updated into the Bitmapset returned by ExecGetAllUpdatedCols(). The patch includes a helper function ExecCompareSlotAttrs() that will be used in follow-on patches as well.
0003 - This patch moves the logic for HeapDetermineColumnsInfo() into the executor while preserving the functionality of simple_heap_update(). A few helper functions are created to better illustrate HOT and lock mode decision making and are reused when possible. The portion of HeapDetermineColumnsInfo() related to replica identity key WAL logging is now in-line in heap_update().
These commits maintain 100% identical logic for HOT, lockmode, and replica identity decisions (or there's a flaw and that should be fixed so let me know) They simply juggle the logic into places where I think they fit better and provide for future work in this area.
I appreciate your time and effort considering these changes.
best.
-greg
Attachments:
[text/x-patch] v35-0001-Add-tests-to-cover-a-variety-of-heap-HOT-update-.patch (89.5K, 2-v35-0001-Add-tests-to-cover-a-variety-of-heap-HOT-update-.patch)
download | inline diff:
From c09fd7d1825965db3698fad8b8b32b625155e45a Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Tue, 10 Mar 2026 09:28:15 -0400
Subject: [PATCH v35 1/3] Add tests to cover a variety of heap HOT update
behaviors
This commit introduces test infrastructure for verifying Heap-Only Tuple
(HOT) update functionality in PostgreSQL. It provides a baseline for
demonstrating and validating HOT update behavior.
Regression tests:
- Basic HOT vs non-HOT update decisions
- All-or-none property for multiple indexes
- Partial indexes and predicate handling
- BRIN (summarizing) indexes allowing HOT updates
- TOAST column handling with HOT
- Unique constraints behavior
- Multi-column indexes
- Partitioned table HOT updates
Isolation tests:
- HOT chain formation and maintenance
- Concurrent HOT update scenarios
- Index scan behavior with HOT chains
---
.../isolation/expected/hot_updates_chain.out | 144 +++
.../expected/hot_updates_concurrent.out | 143 +++
.../expected/hot_updates_index_scan.out | 132 +++
src/test/isolation/isolation_schedule | 3 +
.../isolation/specs/hot_updates_chain.spec | 110 ++
.../specs/hot_updates_concurrent.spec | 107 ++
.../specs/hot_updates_index_scan.spec | 94 ++
src/test/regress/expected/hot_updates.out | 950 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
src/test/regress/sql/hot_updates.sql | 692 +++++++++++++
10 files changed, 2380 insertions(+)
create mode 100644 src/test/isolation/expected/hot_updates_chain.out
create mode 100644 src/test/isolation/expected/hot_updates_concurrent.out
create mode 100644 src/test/isolation/expected/hot_updates_index_scan.out
create mode 100644 src/test/isolation/specs/hot_updates_chain.spec
create mode 100644 src/test/isolation/specs/hot_updates_concurrent.spec
create mode 100644 src/test/isolation/specs/hot_updates_index_scan.spec
create mode 100644 src/test/regress/expected/hot_updates.out
create mode 100644 src/test/regress/sql/hot_updates.sql
diff --git a/src/test/isolation/expected/hot_updates_chain.out b/src/test/isolation/expected/hot_updates_chain.out
new file mode 100644
index 00000000000..503252009ea
--- /dev/null
+++ b/src/test/isolation/expected/hot_updates_chain.out
@@ -0,0 +1,144 @@
+Parsed test spec with 5 sessions
+
+starting permutation: s1_begin s1_hot_update1 s1_hot_update2 s1_hot_update3 s1_commit s1_select s1_verify_hot
+step s1_begin: BEGIN;
+step s1_hot_update1: UPDATE hot_test SET non_indexed_col = 'update1' WHERE id = 1;
+step s1_hot_update2: UPDATE hot_test SET non_indexed_col = 'update2' WHERE id = 1;
+step s1_hot_update3: UPDATE hot_test SET non_indexed_col = 'update3' WHERE id = 1;
+step s1_commit: COMMIT;
+step s1_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 100|update3
+(1 row)
+
+step s1_verify_hot:
+ -- Check for HOT chain: LP_REDIRECT or tuple with t_ctid pointing to same page
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2 -- LP_REDIRECT indicates HOT chain
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0 -- same page
+ AND t_ctid != ('(0,' || lp || ')')::tid); -- different offset
+
+has_hot_chain
+-------------
+t
+(1 row)
+
+
+starting permutation: s2_begin s2_select_before s1_begin s1_hot_update1 s1_hot_update2 s1_commit s2_select_after s2_commit
+step s2_begin: BEGIN ISOLATION LEVEL REPEATABLE READ;
+step s2_select_before: SELECT non_indexed_col FROM hot_test WHERE id = 1;
+non_indexed_col
+---------------
+initial
+(1 row)
+
+step s1_begin: BEGIN;
+step s1_hot_update1: UPDATE hot_test SET non_indexed_col = 'update1' WHERE id = 1;
+step s1_hot_update2: UPDATE hot_test SET non_indexed_col = 'update2' WHERE id = 1;
+step s1_commit: COMMIT;
+step s2_select_after: SELECT non_indexed_col FROM hot_test WHERE id = 1;
+non_indexed_col
+---------------
+initial
+(1 row)
+
+step s2_commit: COMMIT;
+
+starting permutation: s1_begin s1_hot_update1 s1_hot_update2 s1_commit s3_begin s3_non_hot_update s3_commit s1_select
+step s1_begin: BEGIN;
+step s1_hot_update1: UPDATE hot_test SET non_indexed_col = 'update1' WHERE id = 1;
+step s1_hot_update2: UPDATE hot_test SET non_indexed_col = 'update2' WHERE id = 1;
+step s1_commit: COMMIT;
+step s3_begin: BEGIN;
+step s3_non_hot_update: UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+step s3_commit: COMMIT;
+step s1_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 150|update2
+(1 row)
+
+
+starting permutation: s1_begin s1_hot_update1 s1_commit s3_begin s3_non_hot_update s3_commit s4_begin s4_hot_after_non_hot s4_commit s4_select s4_verify_hot
+step s1_begin: BEGIN;
+step s1_hot_update1: UPDATE hot_test SET non_indexed_col = 'update1' WHERE id = 1;
+step s1_commit: COMMIT;
+step s3_begin: BEGIN;
+step s3_non_hot_update: UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+step s3_commit: COMMIT;
+step s4_begin: BEGIN;
+step s4_hot_after_non_hot: UPDATE hot_test SET non_indexed_col = 'after_non_hot' WHERE id = 1;
+step s4_commit: COMMIT;
+step s4_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 150|after_non_hot
+(1 row)
+
+step s4_verify_hot:
+ -- Check for new HOT chain after non-HOT update broke the previous chain
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0
+ AND t_ctid != ('(0,' || lp || ')')::tid);
+
+has_hot_chain
+-------------
+t
+(1 row)
+
+
+starting permutation: s1_begin s1_hot_update1 s1_hot_update2 s5_begin s5_hot_update_row2_1 s5_hot_update_row2_2 s1_commit s5_commit s1_select s5_select s1_verify_hot s5_verify_hot
+step s1_begin: BEGIN;
+step s1_hot_update1: UPDATE hot_test SET non_indexed_col = 'update1' WHERE id = 1;
+step s1_hot_update2: UPDATE hot_test SET non_indexed_col = 'update2' WHERE id = 1;
+step s5_begin: BEGIN;
+step s5_hot_update_row2_1: UPDATE hot_test SET non_indexed_col = 'row2_update1' WHERE id = 2;
+step s5_hot_update_row2_2: UPDATE hot_test SET non_indexed_col = 'row2_update2' WHERE id = 2;
+step s1_commit: COMMIT;
+step s5_commit: COMMIT;
+step s1_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 100|update2
+(1 row)
+
+step s5_select: SELECT * FROM hot_test WHERE id = 2;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 2| 200|row2_update2
+(1 row)
+
+step s1_verify_hot:
+ -- Check for HOT chain: LP_REDIRECT or tuple with t_ctid pointing to same page
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2 -- LP_REDIRECT indicates HOT chain
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0 -- same page
+ AND t_ctid != ('(0,' || lp || ')')::tid); -- different offset
+
+has_hot_chain
+-------------
+t
+(1 row)
+
+step s5_verify_hot:
+ -- Check for HOT chain on page 0
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0
+ AND t_ctid != ('(0,' || lp || ')')::tid);
+
+has_hot_chain
+-------------
+t
+(1 row)
+
diff --git a/src/test/isolation/expected/hot_updates_concurrent.out b/src/test/isolation/expected/hot_updates_concurrent.out
new file mode 100644
index 00000000000..b1a8b0cb7b2
--- /dev/null
+++ b/src/test/isolation/expected/hot_updates_concurrent.out
@@ -0,0 +1,143 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s1_begin s1_hot_update s2_begin s2_hot_update s1_commit s2_commit s1_select s2_select s2_verify_hot
+step s1_begin: BEGIN;
+step s1_hot_update: UPDATE hot_test SET non_indexed_col = 'updated_s1' WHERE id = 1;
+step s2_begin: BEGIN;
+step s2_hot_update: UPDATE hot_test SET non_indexed_col = 'updated_s2' WHERE id = 1; <waiting ...>
+step s1_commit: COMMIT;
+step s2_hot_update: <... completed>
+step s2_commit: COMMIT;
+step s1_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 100|updated_s2
+(1 row)
+
+step s2_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 100|updated_s2
+(1 row)
+
+step s2_verify_hot:
+ -- Check for HOT chain: look for LP_REDIRECT (lp_flags=2) or tuple with t_ctid pointing to same page
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2 -- LP_REDIRECT indicates HOT chain
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0 -- same page
+ AND t_ctid != ('(0,' || lp || ')')::tid); -- different offset
+
+has_hot_chain
+-------------
+t
+(1 row)
+
+
+starting permutation: s1_begin s1_hot_update s3_begin s3_non_hot_update s1_commit s3_commit s3_select s3_verify_index
+step s1_begin: BEGIN;
+step s1_hot_update: UPDATE hot_test SET non_indexed_col = 'updated_s1' WHERE id = 1;
+step s3_begin: BEGIN;
+step s3_non_hot_update: UPDATE hot_test SET indexed_col = 150 WHERE id = 1; <waiting ...>
+step s1_commit: COMMIT;
+step s3_non_hot_update: <... completed>
+step s3_commit: COMMIT;
+step s3_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 150|updated_s1
+(1 row)
+
+step s3_verify_index:
+ -- Verify index was updated (proves non-HOT)
+ SELECT COUNT(*) = 1 AS index_updated FROM hot_test WHERE indexed_col = 150;
+ SELECT COUNT(*) = 0 AS old_value_gone FROM hot_test WHERE indexed_col = 100;
+
+index_updated
+-------------
+t
+(1 row)
+
+old_value_gone
+--------------
+t
+(1 row)
+
+
+starting permutation: s3_begin s3_non_hot_update s1_begin s1_hot_update s3_commit s1_commit s1_select s1_verify_hot
+step s3_begin: BEGIN;
+step s3_non_hot_update: UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+step s1_begin: BEGIN;
+step s1_hot_update: UPDATE hot_test SET non_indexed_col = 'updated_s1' WHERE id = 1; <waiting ...>
+step s3_commit: COMMIT;
+step s1_hot_update: <... completed>
+step s1_commit: COMMIT;
+step s1_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 150|updated_s1
+(1 row)
+
+step s1_verify_hot:
+ -- Check for HOT chain: look for LP_REDIRECT (lp_flags=2) or tuple with t_ctid pointing to same page
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2 -- LP_REDIRECT indicates HOT chain
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0 -- same page
+ AND t_ctid != ('(0,' || lp || ')')::tid); -- different offset
+
+has_hot_chain
+-------------
+t
+(1 row)
+
+
+starting permutation: s1_begin s1_hot_update s4_begin s4_hot_update_row2 s1_commit s4_commit s1_select s4_select s1_verify_hot s4_verify_hot
+step s1_begin: BEGIN;
+step s1_hot_update: UPDATE hot_test SET non_indexed_col = 'updated_s1' WHERE id = 1;
+step s4_begin: BEGIN;
+step s4_hot_update_row2: UPDATE hot_test SET non_indexed_col = 'updated_s4' WHERE id = 2;
+step s1_commit: COMMIT;
+step s4_commit: COMMIT;
+step s1_select: SELECT * FROM hot_test WHERE id = 1;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 1| 100|updated_s1
+(1 row)
+
+step s4_select: SELECT * FROM hot_test WHERE id = 2;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+ 2| 200|updated_s4
+(1 row)
+
+step s1_verify_hot:
+ -- Check for HOT chain: look for LP_REDIRECT (lp_flags=2) or tuple with t_ctid pointing to same page
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2 -- LP_REDIRECT indicates HOT chain
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0 -- same page
+ AND t_ctid != ('(0,' || lp || ')')::tid); -- different offset
+
+has_hot_chain
+-------------
+t
+(1 row)
+
+step s4_verify_hot:
+ -- Check for HOT chain on page 0
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0
+ AND t_ctid != ('(0,' || lp || ')')::tid);
+
+has_hot_chain
+-------------
+t
+(1 row)
+
diff --git a/src/test/isolation/expected/hot_updates_index_scan.out b/src/test/isolation/expected/hot_updates_index_scan.out
new file mode 100644
index 00000000000..7d8e9ff8857
--- /dev/null
+++ b/src/test/isolation/expected/hot_updates_index_scan.out
@@ -0,0 +1,132 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s1_begin s1_hot_update s2_begin s2_index_scan s1_commit s2_commit
+step s1_begin: BEGIN;
+step s1_hot_update: UPDATE hot_test SET non_indexed_col = 'hot_updated' WHERE id = 50;
+step s2_begin: BEGIN;
+step s2_index_scan: SELECT * FROM hot_test WHERE indexed_col = 500;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+50| 500|initial50
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_commit: COMMIT;
+
+starting permutation: s1_begin s1_non_hot_update s1_commit s2_begin s2_index_scan_new s2_commit s2_verify_index
+step s1_begin: BEGIN;
+step s1_non_hot_update: UPDATE hot_test SET indexed_col = 555 WHERE id = 50;
+step s1_commit: COMMIT;
+step s2_begin: BEGIN;
+step s2_index_scan_new: SELECT * FROM hot_test WHERE indexed_col = 555;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+50| 555|initial50
+(1 row)
+
+step s2_commit: COMMIT;
+step s2_verify_index:
+ -- After non-HOT update, verify index reflects the change
+ SELECT COUNT(*) = 1 AS found_new_value FROM hot_test WHERE indexed_col = 555;
+ SELECT COUNT(*) = 0 AS old_value_gone FROM hot_test WHERE indexed_col = 500;
+
+found_new_value
+---------------
+t
+(1 row)
+
+old_value_gone
+--------------
+t
+(1 row)
+
+
+starting permutation: s3_begin s3_select_for_update s1_begin s1_hot_update s3_commit s1_commit s1_verify_hot
+step s3_begin: BEGIN;
+step s3_select_for_update: SELECT * FROM hot_test WHERE id = 50 FOR UPDATE;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+50| 500|initial50
+(1 row)
+
+step s1_begin: BEGIN;
+step s1_hot_update: UPDATE hot_test SET non_indexed_col = 'hot_updated' WHERE id = 50; <waiting ...>
+step s3_commit: COMMIT;
+step s1_hot_update: <... completed>
+step s1_commit: COMMIT;
+step s1_verify_hot:
+ -- Verify HOT chain exists for row with id=50
+ -- Use actual ctid to find the correct page
+ SELECT EXISTS (
+ SELECT 1 FROM heap_page_items(
+ get_raw_page('hot_test', (SELECT (ctid::text::point)[0]::int FROM hot_test WHERE id = 50))
+ )
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND t_ctid != ('(' || (SELECT (ctid::text::point)[0]::int FROM hot_test WHERE id = 50) || ',' || lp || ')')::tid
+ AND (t_ctid::text::point)[0]::int = (SELECT (ctid::text::point)[0]::int FROM hot_test WHERE id = 50))
+ ) AS has_hot_chain;
+
+has_hot_chain
+-------------
+t
+(1 row)
+
+
+starting permutation: s1_begin s1_hot_update s3_begin s3_select_for_update s1_commit s3_commit
+step s1_begin: BEGIN;
+step s1_hot_update: UPDATE hot_test SET non_indexed_col = 'hot_updated' WHERE id = 50;
+step s3_begin: BEGIN;
+step s3_select_for_update: SELECT * FROM hot_test WHERE id = 50 FOR UPDATE; <waiting ...>
+step s1_commit: COMMIT;
+step s3_select_for_update: <... completed>
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+50| 500|hot_updated
+(1 row)
+
+step s3_commit: COMMIT;
+
+starting permutation: s4_begin s4_select_for_key_share s1_begin s1_hot_update s4_commit s1_commit s1_verify_hot
+step s4_begin: BEGIN;
+step s4_select_for_key_share: SELECT * FROM hot_test WHERE id = 50 FOR KEY SHARE;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+50| 500|initial50
+(1 row)
+
+step s1_begin: BEGIN;
+step s1_hot_update: UPDATE hot_test SET non_indexed_col = 'hot_updated' WHERE id = 50;
+step s4_commit: COMMIT;
+step s1_commit: COMMIT;
+step s1_verify_hot:
+ -- Verify HOT chain exists for row with id=50
+ -- Use actual ctid to find the correct page
+ SELECT EXISTS (
+ SELECT 1 FROM heap_page_items(
+ get_raw_page('hot_test', (SELECT (ctid::text::point)[0]::int FROM hot_test WHERE id = 50))
+ )
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND t_ctid != ('(' || (SELECT (ctid::text::point)[0]::int FROM hot_test WHERE id = 50) || ',' || lp || ')')::tid
+ AND (t_ctid::text::point)[0]::int = (SELECT (ctid::text::point)[0]::int FROM hot_test WHERE id = 50))
+ ) AS has_hot_chain;
+
+has_hot_chain
+-------------
+t
+(1 row)
+
+
+starting permutation: s4_begin s4_select_for_key_share s1_begin s1_non_hot_update s4_commit s1_commit
+step s4_begin: BEGIN;
+step s4_select_for_key_share: SELECT * FROM hot_test WHERE id = 50 FOR KEY SHARE;
+id|indexed_col|non_indexed_col
+--+-----------+---------------
+50| 500|initial50
+(1 row)
+
+step s1_begin: BEGIN;
+step s1_non_hot_update: UPDATE hot_test SET indexed_col = 555 WHERE id = 50;
+step s4_commit: COMMIT;
+step s1_commit: COMMIT;
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 4e466580cd4..46525b0a62a 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -19,6 +19,9 @@ test: multiple-row-versions
test: index-only-scan
test: index-only-bitmapscan
test: predicate-lock-hot-tuple
+test: hot_updates_concurrent
+test: hot_updates_index_scan
+test: hot_updates_chain
test: update-conflict-out
test: deadlock-simple
test: deadlock-hard
diff --git a/src/test/isolation/specs/hot_updates_chain.spec b/src/test/isolation/specs/hot_updates_chain.spec
new file mode 100644
index 00000000000..85cd2176133
--- /dev/null
+++ b/src/test/isolation/specs/hot_updates_chain.spec
@@ -0,0 +1,110 @@
+# Test HOT update chains and their interaction with VACUUM and page pruning
+#
+# This test verifies that HOT update chains are correctly maintained when
+# multiple HOT updates occur on the same row, and that VACUUM correctly
+# handles HOT chains.
+
+setup
+{
+ CREATE EXTENSION IF NOT EXISTS pageinspect;
+
+ CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+ );
+
+ CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
+
+ INSERT INTO hot_test VALUES (1, 100, 'initial');
+ INSERT INTO hot_test VALUES (2, 200, 'initial');
+}
+
+teardown
+{
+ DROP TABLE hot_test;
+ DROP EXTENSION pageinspect;
+}
+
+# Session 1: Create HOT chain with multiple updates
+session s1
+step s1_begin { BEGIN; }
+step s1_hot_update1 { UPDATE hot_test SET non_indexed_col = 'update1' WHERE id = 1; }
+step s1_hot_update2 { UPDATE hot_test SET non_indexed_col = 'update2' WHERE id = 1; }
+step s1_hot_update3 { UPDATE hot_test SET non_indexed_col = 'update3' WHERE id = 1; }
+step s1_commit { COMMIT; }
+step s1_select { SELECT * FROM hot_test WHERE id = 1; }
+step s1_verify_hot {
+ -- Check for HOT chain: LP_REDIRECT or tuple with t_ctid pointing to same page
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2 -- LP_REDIRECT indicates HOT chain
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0 -- same page
+ AND t_ctid != ('(0,' || lp || ')')::tid); -- different offset
+}
+
+# Session 2: Read while HOT chain is being built
+session s2
+step s2_begin { BEGIN ISOLATION LEVEL REPEATABLE READ; }
+step s2_select_before { SELECT non_indexed_col FROM hot_test WHERE id = 1; }
+step s2_select_after { SELECT non_indexed_col FROM hot_test WHERE id = 1; }
+step s2_commit { COMMIT; }
+
+# Session 3: Break HOT chain with non-HOT update
+session s3
+step s3_begin { BEGIN; }
+step s3_non_hot_update { UPDATE hot_test SET indexed_col = 150 WHERE id = 1; }
+step s3_commit { COMMIT; }
+
+# Session 4: Try to build HOT chain after non-HOT update
+session s4
+step s4_begin { BEGIN; }
+step s4_hot_after_non_hot { UPDATE hot_test SET non_indexed_col = 'after_non_hot' WHERE id = 1; }
+step s4_commit { COMMIT; }
+step s4_select { SELECT * FROM hot_test WHERE id = 1; }
+step s4_verify_hot {
+ -- Check for new HOT chain after non-HOT update broke the previous chain
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0
+ AND t_ctid != ('(0,' || lp || ')')::tid);
+}
+
+# Session 5: Multiple sessions building separate HOT chains on different rows
+session s5
+step s5_begin { BEGIN; }
+step s5_hot_update_row2_1 { UPDATE hot_test SET non_indexed_col = 'row2_update1' WHERE id = 2; }
+step s5_hot_update_row2_2 { UPDATE hot_test SET non_indexed_col = 'row2_update2' WHERE id = 2; }
+step s5_commit { COMMIT; }
+step s5_select { SELECT * FROM hot_test WHERE id = 2; }
+step s5_verify_hot {
+ -- Check for HOT chain on page 0
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0
+ AND t_ctid != ('(0,' || lp || ')')::tid);
+}
+
+# Build HOT chain within single transaction
+# All updates should form a HOT chain
+permutation s1_begin s1_hot_update1 s1_hot_update2 s1_hot_update3 s1_commit s1_select s1_verify_hot
+
+# REPEATABLE READ should see consistent snapshot across HOT chain updates
+# Session 2 starts before updates, should see 'initial' throughout
+permutation s2_begin s2_select_before s1_begin s1_hot_update1 s1_hot_update2 s1_commit s2_select_after s2_commit
+
+# HOT chain followed by non-HOT update
+# Non-HOT update breaks the HOT chain
+permutation s1_begin s1_hot_update1 s1_hot_update2 s1_commit s3_begin s3_non_hot_update s3_commit s1_select
+
+# HOT update after non-HOT update can start new HOT chain
+# After breaking chain with indexed column update, new HOT updates can start fresh chain
+permutation s1_begin s1_hot_update1 s1_commit s3_begin s3_non_hot_update s3_commit s4_begin s4_hot_after_non_hot s4_commit s4_select s4_verify_hot
+
+# Multiple sessions building separate HOT chains on different rows
+permutation s1_begin s1_hot_update1 s1_hot_update2 s5_begin s5_hot_update_row2_1 s5_hot_update_row2_2 s1_commit s5_commit s1_select s5_select s1_verify_hot s5_verify_hot
diff --git a/src/test/isolation/specs/hot_updates_concurrent.spec b/src/test/isolation/specs/hot_updates_concurrent.spec
new file mode 100644
index 00000000000..eac78d62ac5
--- /dev/null
+++ b/src/test/isolation/specs/hot_updates_concurrent.spec
@@ -0,0 +1,107 @@
+# Test concurrent HOT updates and validate HOT chains
+#
+# This test verifies that HOT updates work correctly when multiple sessions
+# are updating the same table concurrently, and validates that HOT chains
+# are actually created using heap_page_items().
+
+setup
+{
+ CREATE EXTENSION IF NOT EXISTS pageinspect;
+
+ CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+ );
+
+ CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
+
+ INSERT INTO hot_test VALUES (1, 100, 'initial1');
+ INSERT INTO hot_test VALUES (2, 200, 'initial2');
+ INSERT INTO hot_test VALUES (3, 300, 'initial3');
+}
+
+teardown
+{
+ DROP TABLE hot_test;
+ DROP EXTENSION pageinspect;
+}
+
+# Session 1: HOT update (modify non-indexed column)
+session s1
+step s1_begin { BEGIN; }
+step s1_hot_update { UPDATE hot_test SET non_indexed_col = 'updated_s1' WHERE id = 1; }
+step s1_commit { COMMIT; }
+step s1_select { SELECT * FROM hot_test WHERE id = 1; }
+step s1_verify_hot {
+ -- Check for HOT chain: look for LP_REDIRECT (lp_flags=2) or tuple with t_ctid pointing to same page
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2 -- LP_REDIRECT indicates HOT chain
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0 -- same page
+ AND t_ctid != ('(0,' || lp || ')')::tid); -- different offset
+}
+
+# Session 2: HOT update (modify non-indexed column on same row)
+session s2
+step s2_begin { BEGIN; }
+step s2_hot_update { UPDATE hot_test SET non_indexed_col = 'updated_s2' WHERE id = 1; }
+step s2_commit { COMMIT; }
+step s2_select { SELECT * FROM hot_test WHERE id = 1; }
+step s2_verify_hot {
+ -- Check for HOT chain: look for LP_REDIRECT (lp_flags=2) or tuple with t_ctid pointing to same page
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2 -- LP_REDIRECT indicates HOT chain
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0 -- same page
+ AND t_ctid != ('(0,' || lp || ')')::tid); -- different offset
+}
+
+# Session 3: Non-HOT update (modify indexed column)
+session s3
+step s3_begin { BEGIN; }
+step s3_non_hot_update { UPDATE hot_test SET indexed_col = 150 WHERE id = 1; }
+step s3_commit { COMMIT; }
+step s3_select { SELECT * FROM hot_test WHERE id = 1; }
+step s3_verify_index {
+ -- Verify index was updated (proves non-HOT)
+ SELECT COUNT(*) = 1 AS index_updated FROM hot_test WHERE indexed_col = 150;
+ SELECT COUNT(*) = 0 AS old_value_gone FROM hot_test WHERE indexed_col = 100;
+}
+
+# Session 4: Concurrent HOT updates on different rows
+session s4
+step s4_begin { BEGIN; }
+step s4_hot_update_row2 { UPDATE hot_test SET non_indexed_col = 'updated_s4' WHERE id = 2; }
+step s4_commit { COMMIT; }
+step s4_select { SELECT * FROM hot_test WHERE id = 2; }
+step s4_verify_hot {
+ -- Check for HOT chain on page 0
+ SELECT COUNT(*) > 0 AS has_hot_chain
+ FROM heap_page_items(get_raw_page('hot_test', 0))
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND (t_ctid::text::point)[0]::int = 0
+ AND t_ctid != ('(0,' || lp || ')')::tid);
+}
+
+# Two sessions both doing HOT updates on same row
+# Second session should block until first commits
+# Both should create HOT chains
+permutation s1_begin s1_hot_update s2_begin s2_hot_update s1_commit s2_commit s1_select s2_select s2_verify_hot
+
+# HOT update followed by non-HOT update
+# Non-HOT update should wait for HOT update to commit
+# First update is HOT, second is non-HOT (index updated)
+permutation s1_begin s1_hot_update s3_begin s3_non_hot_update s1_commit s3_commit s3_select s3_verify_index
+
+# Non-HOT update followed by HOT update
+# HOT update should wait for non-HOT update to commit
+# First update is non-HOT (index), second is HOT
+permutation s3_begin s3_non_hot_update s1_begin s1_hot_update s3_commit s1_commit s1_select s1_verify_hot
+
+# Concurrent HOT updates on different rows (should not block)
+# Both sessions should be able to create HOT chains independently
+permutation s1_begin s1_hot_update s4_begin s4_hot_update_row2 s1_commit s4_commit s1_select s4_select s1_verify_hot s4_verify_hot
diff --git a/src/test/isolation/specs/hot_updates_index_scan.spec b/src/test/isolation/specs/hot_updates_index_scan.spec
new file mode 100644
index 00000000000..70c3dae5166
--- /dev/null
+++ b/src/test/isolation/specs/hot_updates_index_scan.spec
@@ -0,0 +1,94 @@
+# Test HOT updates interaction with index scans and SELECT FOR UPDATE
+#
+# This test verifies that HOT updates are correctly handled when concurrent
+# sessions are performing index scans, using SELECT FOR UPDATE, and validates
+# HOT chains using heap_page_items().
+
+setup
+{
+ CREATE EXTENSION IF NOT EXISTS pageinspect;
+
+ CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+ );
+
+ CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
+
+ INSERT INTO hot_test SELECT i, i * 10, 'initial' || i FROM generate_series(1, 100) i;
+}
+
+teardown
+{
+ DROP TABLE hot_test;
+ DROP EXTENSION pageinspect;
+}
+
+# Session 1: Perform HOT update
+session s1
+step s1_begin { BEGIN; }
+step s1_hot_update { UPDATE hot_test SET non_indexed_col = 'hot_updated' WHERE id = 50; }
+step s1_non_hot_update { UPDATE hot_test SET indexed_col = 555 WHERE id = 50; }
+step s1_commit { COMMIT; }
+step s1_verify_hot {
+ -- Verify HOT chain exists for row with id=50
+ -- Use actual ctid to find the correct page
+ SELECT EXISTS (
+ SELECT 1 FROM heap_page_items(
+ get_raw_page('hot_test', (SELECT (ctid::text::point)[0]::int FROM hot_test WHERE id = 50))
+ )
+ WHERE lp_flags = 2
+ OR (t_ctid IS NOT NULL
+ AND t_ctid != ('(' || (SELECT (ctid::text::point)[0]::int FROM hot_test WHERE id = 50) || ',' || lp || ')')::tid
+ AND (t_ctid::text::point)[0]::int = (SELECT (ctid::text::point)[0]::int FROM hot_test WHERE id = 50))
+ ) AS has_hot_chain;
+}
+
+# Session 2: Index scan while HOT update in progress
+session s2
+step s2_begin { BEGIN; }
+step s2_index_scan { SELECT * FROM hot_test WHERE indexed_col = 500; }
+step s2_index_scan_new { SELECT * FROM hot_test WHERE indexed_col = 555; }
+step s2_commit { COMMIT; }
+step s2_verify_index {
+ -- After non-HOT update, verify index reflects the change
+ SELECT COUNT(*) = 1 AS found_new_value FROM hot_test WHERE indexed_col = 555;
+ SELECT COUNT(*) = 0 AS old_value_gone FROM hot_test WHERE indexed_col = 500;
+}
+
+# Session 3: SELECT FOR UPDATE
+session s3
+step s3_begin { BEGIN; }
+step s3_select_for_update { SELECT * FROM hot_test WHERE id = 50 FOR UPDATE; }
+step s3_commit { COMMIT; }
+
+# Session 4: SELECT FOR KEY SHARE (should not block HOT update of non-key column)
+session s4
+step s4_begin { BEGIN; }
+step s4_select_for_key_share { SELECT * FROM hot_test WHERE id = 50 FOR KEY SHARE; }
+step s4_commit { COMMIT; }
+
+# Index scan should see consistent snapshot during HOT update
+# Index scan starts before HOT update commits
+permutation s1_begin s1_hot_update s2_begin s2_index_scan s1_commit s2_commit
+
+# Index scan after non-HOT update should see new index entry
+# Index scan starts after non-HOT update commits
+permutation s1_begin s1_non_hot_update s1_commit s2_begin s2_index_scan_new s2_commit s2_verify_index
+
+# SELECT FOR UPDATE blocks HOT update
+# FOR UPDATE should block the UPDATE until SELECT commits
+permutation s3_begin s3_select_for_update s1_begin s1_hot_update s3_commit s1_commit s1_verify_hot
+
+# HOT update blocks SELECT FOR UPDATE
+# SELECT FOR UPDATE should wait for HOT update to commit
+permutation s1_begin s1_hot_update s3_begin s3_select_for_update s1_commit s3_commit
+
+# SELECT FOR KEY SHARE should not block HOT update (non-key column)
+# HOT update of non-indexed column should not conflict with FOR KEY SHARE
+permutation s4_begin s4_select_for_key_share s1_begin s1_hot_update s4_commit s1_commit s1_verify_hot
+
+# Non-HOT update (key column) should block after FOR KEY SHARE
+# Non-HOT update of indexed column should wait for FOR KEY SHARE
+permutation s4_begin s4_select_for_key_share s1_begin s1_non_hot_update s4_commit s1_commit
diff --git a/src/test/regress/expected/hot_updates.out b/src/test/regress/expected/hot_updates.out
new file mode 100644
index 00000000000..e99a51966ce
--- /dev/null
+++ b/src/test/regress/expected/hot_updates.out
@@ -0,0 +1,950 @@
+-- Load required extensions
+CREATE EXTENSION IF NOT EXISTS pageinspect;
+-- Function to get HOT update count
+CREATE OR REPLACE FUNCTION get_hot_count(rel_name text)
+RETURNS TABLE (
+ updates BIGINT,
+ hot BIGINT
+) AS $$
+DECLARE
+ rel_oid oid;
+BEGIN
+ rel_oid := rel_name::regclass::oid;
+
+ -- Read both committed and transaction-local stats
+ -- In autocommit mode (default for regression tests), this works correctly
+ -- Note: In explicit transactions (BEGIN/COMMIT), committed stats already
+ -- include flushed updates, so this would double-count. For explicit
+ -- transaction testing, call pg_stat_force_next_flush() before this function.
+ updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
+ hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
+
+ RETURN NEXT;
+END;
+$$ LANGUAGE plpgsql;
+-- Check if a tuple is part of a HOT chain (has a predecessor on same page)
+CREATE OR REPLACE FUNCTION has_hot_chain(rel_name text, target_ctid tid)
+RETURNS boolean AS $$
+DECLARE
+ block_num int;
+ page_item record;
+BEGIN
+ block_num := (target_ctid::text::point)[0]::int;
+
+ -- Look for a different tuple on the same page that points to our target tuple
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid IS NOT NULL
+ AND t_ctid = target_ctid
+ AND ('(' || block_num::text || ',' || lp::text || ')')::tid != target_ctid
+ LOOP
+ RETURN true;
+ END LOOP;
+
+ RETURN false;
+END;
+$$ LANGUAGE plpgsql;
+-- Print the HOT chain starting from a given tuple
+CREATE OR REPLACE FUNCTION print_hot_chain(rel_name text, start_ctid tid)
+RETURNS TABLE(chain_position int, ctid tid, lp_flags text, t_ctid tid, chain_end boolean) AS
+$$
+#variable_conflict use_column
+DECLARE
+ block_num int;
+ line_ptr int;
+ current_ctid tid := start_ctid;
+ next_ctid tid;
+ position int := 0;
+ max_iterations int := 100;
+ page_item record;
+ found_predecessor boolean := false;
+ flags_name text;
+BEGIN
+ block_num := (start_ctid::text::point)[0]::int;
+
+ -- Find the predecessor (old tuple pointing to our start_ctid)
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid = start_ctid
+ LOOP
+ current_ctid := ('(' || block_num::text || ',' || page_item.lp::text || ')')::tid;
+ found_predecessor := true;
+ EXIT;
+ END LOOP;
+
+ -- If no predecessor found, start with the given ctid
+ IF NOT found_predecessor THEN
+ current_ctid := start_ctid;
+ END IF;
+
+ -- Follow the chain forward
+ WHILE position < max_iterations LOOP
+ line_ptr := (current_ctid::text::point)[1]::int;
+
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp = line_ptr
+ LOOP
+ -- Map lp_flags to names
+ flags_name := CASE page_item.lp_flags
+ WHEN 0 THEN 'unused (0)'
+ WHEN 1 THEN 'normal (1)'
+ WHEN 2 THEN 'redirect (2)'
+ WHEN 3 THEN 'dead (3)'
+ ELSE 'unknown (' || page_item.lp_flags::text || ')'
+ END;
+
+ RETURN QUERY SELECT
+ position,
+ current_ctid,
+ flags_name,
+ page_item.t_ctid,
+ (page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid)::boolean
+ ;
+
+ IF page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid THEN
+ RETURN;
+ END IF;
+
+ next_ctid := page_item.t_ctid;
+
+ IF (next_ctid::text::point)[0]::int != block_num THEN
+ RETURN;
+ END IF;
+
+ current_ctid := next_ctid;
+ position := position + 1;
+ END LOOP;
+
+ IF position = 0 THEN
+ RETURN;
+ END IF;
+ END LOOP;
+END;
+$$ LANGUAGE plpgsql;
+-- Basic HOT update functionality
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+) USING heap WITH (fillfactor = 50);
+CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
+INSERT INTO hot_test VALUES (1, 100, 'initial');
+INSERT INTO hot_test VALUES (2, 200, 'initial');
+INSERT INTO hot_test VALUES (3, 300, 'initial');
+-- Get baseline
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Should be HOT updates (only non-indexed column modified)
+UPDATE hot_test SET non_indexed_col = 'updated1' WHERE id = 1;
+UPDATE hot_test SET non_indexed_col = 'updated2' WHERE id = 2;
+UPDATE hot_test SET non_indexed_col = 'updated3' WHERE id = 3;
+-- Verify HOT updates occurred
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 3 | 3
+(1 row)
+
+-- Dump the HOT chain for tuple with id == 1
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+ has_chain | chain_position | ctid | lp_flags | t_ctid
+-----------+----------------+-------+------------+--------
+ t | 0 | (0,1) | normal (1) | (0,4)
+ t | 1 | (0,4) | normal (1) | (0,4)
+(2 rows)
+
+-- Trigger optimistic heap page pruning
+SELECT ctid, * FROM hot_test;
+ ctid | id | indexed_col | non_indexed_col
+-------+----+-------------+-----------------
+ (0,4) | 1 | 100 | updated1
+ (0,5) | 2 | 200 | updated2
+ (0,6) | 3 | 300 | updated3
+(3 rows)
+
+-- Dump the HOT chain after prune
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+ has_chain | chain_position | ctid | lp_flags | t_ctid
+-----------+----------------+-------+------------+--------
+ t | 0 | (0,1) | normal (1) | (0,4)
+ t | 1 | (0,4) | normal (1) | (0,4)
+(2 rows)
+
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+-- Verify indexes still work
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 100;
+ QUERY PLAN
+---------------------------------------------------
+ Index Scan using hot_test_indexed_idx on hot_test
+ Index Cond: (indexed_col = 100)
+(2 rows)
+
+SELECT id, indexed_col FROM hot_test WHERE indexed_col = 100;
+ id | indexed_col
+----+-------------
+ 1 | 100
+(1 row)
+
+-- Vacuum the relation, expect the HOT chain to collapse
+VACUUM hot_test;
+-- Show that there is no chain after vacuum
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+ has_chain | chain_position | ctid | lp_flags | t_ctid
+-----------+----------------+-------+------------+--------
+ f | 0 | (0,4) | normal (1) | (0,4)
+(1 row)
+
+-- Non-HOT update (update indexed column)
+UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 4 | 3
+(1 row)
+
+-- Verify index was updated (new value findable)
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+ QUERY PLAN
+---------------------------------------------------
+ Index Scan using hot_test_indexed_idx on hot_test
+ Index Cond: (indexed_col = 150)
+(2 rows)
+
+SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+ id | indexed_col
+----+-------------
+ 1 | 150
+(1 row)
+
+-- Verify old value no longer in index
+EXPLAIN (COSTS OFF) SELECT id FROM hot_test WHERE indexed_col = 100;
+ QUERY PLAN
+---------------------------------------------------
+ Index Scan using hot_test_indexed_idx on hot_test
+ Index Cond: (indexed_col = 100)
+(2 rows)
+
+SELECT id FROM hot_test WHERE indexed_col = 100;
+ id
+----
+(0 rows)
+
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+-- All-or-none property: updating one indexed column requires ALL index updates
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ non_indexed text
+) USING heap WITH (fillfactor = 50);
+CREATE INDEX hot_test_a_idx ON hot_test(col_a);
+CREATE INDEX hot_test_b_idx ON hot_test(col_b);
+CREATE INDEX hot_test_c_idx ON hot_test(col_c);
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'initial');
+-- Update only col_a - should NOT be HOT because an indexed column changed
+-- This means ALL indexes must be updated (all-or-none property)
+UPDATE hot_test SET col_a = 15 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Verify all three indexes still work correctly
+SELECT id, col_a FROM hot_test WHERE col_a = 15; -- updated index
+ id | col_a
+----+-------
+ 1 | 15
+(1 row)
+
+SELECT id, col_b FROM hot_test WHERE col_b = 20; -- unchanged index
+ id | col_b
+----+-------
+ 1 | 20
+(1 row)
+
+SELECT id, col_c FROM hot_test WHERE col_c = 30; -- unchanged index
+ id | col_c
+----+-------
+ 1 | 30
+(1 row)
+
+-- Now update only non-indexed column - should be HOT
+UPDATE hot_test SET non_indexed = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 1
+(1 row)
+
+-- Verify all indexes still work
+SELECT id FROM hot_test WHERE col_a = 15 AND col_b = 20 AND col_c = 30;
+ id
+----
+ 1
+(1 row)
+
+-- Partial index: both old and new outside predicate (conservative = non-HOT)
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ status text,
+ data text
+) WITH (fillfactor = 50);
+-- Partial index only covers status = 'active'
+CREATE INDEX hot_test_active_idx ON hot_test(status) WHERE status = 'active';
+INSERT INTO hot_test VALUES (1, 'active', 'data1');
+INSERT INTO hot_test VALUES (2, 'inactive', 'data2');
+INSERT INTO hot_test VALUES (3, 'deleted', 'data3');
+-- Update non-indexed column on 'active' row (in predicate, status unchanged)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated1' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+-- Update non-indexed column on 'inactive' row (outside predicate)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated2' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 2
+(1 row)
+
+-- Update status from 'inactive' to 'deleted' (both outside predicate)
+-- PostgreSQL is conservative: heap insert happens before predicate check
+-- So this is NON-HOT even though both values are outside predicate
+UPDATE hot_test SET status = 'deleted' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 3 | 2
+(1 row)
+
+-- Verify index still works for 'active' rows
+SELECT id, status FROM hot_test WHERE status = 'active';
+ id | status
+----+--------
+ 1 | active
+(1 row)
+
+-- Only BRIN (summarizing) indexes on non-PK columns
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ ts timestamp,
+ value int,
+ brin_col int
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_ts_brin ON hot_test USING brin(ts);
+CREATE INDEX hot_test_brin_col_brin ON hot_test USING brin(brin_col);
+INSERT INTO hot_test VALUES (1, '2024-01-01', 100, 1000);
+-- Update both BRIN columns - should still be HOT (only summarizing indexes)
+UPDATE hot_test SET ts = '2024-01-02', brin_col = 2000 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+-- Verify BRIN indexes work
+SELECT id FROM hot_test WHERE ts >= '2024-01-02';
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hot_test WHERE brin_col >= 2000;
+ id
+----
+ 1
+(1 row)
+
+-- Update non-indexed column - should also be HOT
+UPDATE hot_test SET value = 200 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 2
+(1 row)
+
+-- Unique constraint (unique index) behaves like regular index
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ unique_col int UNIQUE,
+ data text
+) WITH (fillfactor = 50);
+INSERT INTO hot_test VALUES (1, 100, 'data1');
+INSERT INTO hot_test VALUES (2, 200, 'data2');
+-- Update data (non-indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 2
+(1 row)
+
+-- Verify unique constraint still enforced
+SELECT id, unique_col, data FROM hot_test ORDER BY id;
+ id | unique_col | data
+----+------------+---------
+ 1 | 100 | updated
+ 2 | 200 | updated
+(2 rows)
+
+-- This should fail (unique violation)
+UPDATE hot_test SET unique_col = 100 WHERE id = 2;
+ERROR: duplicate key value violates unique constraint "hot_test_unique_col_key"
+DETAIL: Key (unique_col)=(100) already exists.
+-- Multi-column index: any column change = non-HOT
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ col_d int
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_ab_idx ON hot_test(col_a, col_b);
+CREATE INDEX hot_test_ab_inc_c_idx ON hot_test(col_a, col_b) INCLUDE(col_c);
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 40);
+-- Update col_a (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_a = 15;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Update col_b (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_b = 25;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 0
+(1 row)
+
+-- Update col_c (not indexed, but included) - should NOT be HOT
+UPDATE hot_test SET col_c = 35;
+-- Verify multi-column index-only scan for included columns works
+EXPLAIN (COSTS OFF) SELECT col_c FROM hot_test WHERE col_a = 15 AND col_b = 25;
+ QUERY PLAN
+---------------------------------------------------------
+ Index Only Scan using hot_test_ab_inc_c_idx on hot_test
+ Index Cond: ((col_a = 15) AND (col_b = 25))
+(2 rows)
+
+SELECT col_c FROM hot_test WHERE col_a = 15 AND col_b = 25;
+ col_c
+-------
+ 35
+(1 row)
+
+-- ============================================================================
+-- Expression indexes with JSONB
+-- ============================================================================
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ data jsonb
+) USING heap WITH(fillfactor = 50);
+-- Indexes on specific JSONB paths
+CREATE INDEX hot_test_status_idx ON hot_test((data->'status'));
+CREATE INDEX hot_test_user_id_idx ON hot_test((data->'user'->'id'));
+INSERT INTO hot_test VALUES (
+ 1,
+ '{"status": "active", "user": {"id": 123, "name": "Alice"}, "count": 0}'::jsonb
+);
+-- Baseline
+SELECT 'Baseline' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+----------+---------+-----
+ Baseline | 0 | 0
+(1 row)
+
+-- Update non-indexed path {count} - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{count}', '1') WHERE id = 1;
+SELECT 'After updating count (non-indexed)' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+------------------------------------+---------+-----
+ After updating count (non-indexed) | 1 | 0
+(1 row)
+
+-- Update different non-indexed path {user,name} - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{user,name}', '"Bob"') WHERE id = 1;
+SELECT 'After updating user.name (non-indexed)' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+----------------------------------------+---------+-----
+ After updating user.name (non-indexed) | 2 | 0
+(1 row)
+
+-- Update indexed path {status} - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{status}', '"inactive"') WHERE id = 1;
+SELECT 'After updating status (indexed)' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+---------------------------------+---------+-----
+ After updating status (indexed) | 3 | 0
+(1 row)
+
+-- Update indexed path {user,id} - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{user,id}', '456') WHERE id = 1;
+SELECT 'After updating user.id (indexed)' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+----------------------------------+---------+-----
+ After updating user.id (indexed) | 4 | 0
+(1 row)
+
+-- Verify indexes still work correctly
+SELECT id FROM hot_test WHERE data->'status' = '"inactive"'::jsonb;
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hot_test WHERE data->'user'->'id' = '456'::jsonb;
+ id
+----
+ 1
+(1 row)
+
+-- ============================================================================
+-- Nested paths and path intersection
+-- ============================================================================
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ data jsonb
+) USING heap WITH(fillfactor = 50);
+CREATE INDEX hot_test_deep_idx ON hot_test((data->'a'->'b'->'c'));
+INSERT INTO hot_test VALUES (
+ 1,
+ '{"a": {"b": {"c": "indexed", "d": "not-indexed"}}, "x": "other"}'::jsonb
+);
+SELECT 'Baseline' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+----------+---------+-----
+ Baseline | 0 | 0
+(1 row)
+
+-- Update sibling of indexed path {a,b,d} - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{a,b,d}', '"updated"') WHERE id = 1;
+SELECT 'After updating a.b.d (sibling, non-indexed)' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+---------------------------------------------+---------+-----
+ After updating a.b.d (sibling, non-indexed) | 1 | 0
+(1 row)
+
+-- Update unrelated path {x} - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{x}', '"modified"') WHERE id = 1;
+SELECT 'After updating x (unrelated path)' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+-----------------------------------+---------+-----
+ After updating x (unrelated path) | 2 | 0
+(1 row)
+
+-- Update parent of indexed path {a,b} - should NOT be HOT (affects child)
+UPDATE hot_test SET data = jsonb_set(data, '{a,b}', '{"c": "new", "d": "data"}') WHERE id = 1;
+SELECT 'After updating a.b (parent of indexed)' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+----------------------------------------+---------+-----
+ After updating a.b (parent of indexed) | 3 | 0
+(1 row)
+
+-- ============================================================================
+-- Multiple JSONB mutation functions
+-- ============================================================================
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ data jsonb
+) USING heap WITH(fillfactor = 50);
+CREATE INDEX hot_test_keep_idx ON hot_test((data->'keep'));
+INSERT INTO hot_test VALUES (
+ 1,
+ '{"keep": "important", "remove": "unimportant", "extra": "data"}'::jsonb
+);
+SELECT 'Baseline' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+----------+---------+-----
+ Baseline | 0 | 0
+(1 row)
+
+-- jsonb_delete on non-indexed key - should NOT be HOT
+UPDATE hot_test SET data = data - 'remove' WHERE id = 1;
+SELECT 'After deleting non-indexed key' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+--------------------------------+---------+-----
+ After deleting non-indexed key | 1 | 0
+(1 row)
+
+-- jsonb_set on non-indexed key - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{extra}', '"modified"') WHERE id = 1;
+SELECT 'After modifying non-indexed key' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+---------------------------------+---------+-----
+ After modifying non-indexed key | 2 | 0
+(1 row)
+
+-- jsonb_delete on indexed key - should NOT be HOT
+UPDATE hot_test SET data = data - 'keep' WHERE id = 1;
+SELECT 'After deleting indexed key' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+----------------------------+---------+-----
+ After deleting indexed key | 3 | 0
+(1 row)
+
+-- ============================================================================
+-- Array operations
+-- ============================================================================
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ data jsonb
+) USING heap WITH(fillfactor = 50);
+-- Index on array element
+CREATE INDEX hot_test_tags_idx ON hot_test((data->'tags'->0));
+INSERT INTO hot_test VALUES (
+ 1,
+ '{"tags": ["indexed", "second", "third"], "other": "data"}'::jsonb
+);
+SELECT 'Baseline' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+----------+---------+-----
+ Baseline | 0 | 0
+(1 row)
+
+-- Update non-indexed array element - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{tags,1}', '"modified"') WHERE id = 1;
+SELECT 'After updating tags[1]' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+------------------------+---------+-----
+ After updating tags[1] | 1 | 0
+(1 row)
+
+-- Update indexed array element - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{tags,0}', '"changed"') WHERE id = 1;
+SELECT 'After updating tags[0] (indexed)' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+----------------------------------+---------+-----
+ After updating tags[0] (indexed) | 2 | 0
+(1 row)
+
+-- ============================================================================
+-- Whole column index
+-- ============================================================================
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ data jsonb
+) USING heap WITH(fillfactor = 50);
+-- Index on entire JSONB column, and a path extraction
+CREATE INDEX hot_test_whole_idx ON hot_test(data);
+CREATE INDEX hot_test_tags_idx ON hot_test((data->'a'));
+INSERT INTO hot_test VALUES (1, '{"a": 1, "b": 1}'::jsonb);
+SELECT 'Baseline' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+----------+---------+-----
+ Baseline | 0 | 0
+(1 row)
+
+-- Any modification to data - should NOT be HOT (whole column indexed)
+UPDATE hot_test SET data = jsonb_set(data, '{b}', '2') WHERE id = 1;
+SELECT 'After modifying any field (whole column indexed)' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+--------------------------------------------------+---------+-----
+ After modifying any field (whole column indexed) | 1 | 0
+(1 row)
+
+-- ============================================================================
+-- Performance at scale
+-- ============================================================================
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ data jsonb
+) USING heap WITH(fillfactor=50);
+CREATE INDEX hot_test_status_idx ON hot_test((data->'status'));
+CREATE INDEX hot_test_priority_idx ON hot_test((data->'priority'));
+-- Insert 10000 rows
+INSERT INTO hot_test
+SELECT i, jsonb_build_object(
+ 'status', 'active',
+ 'priority', 1,
+ 'count', 0,
+ 'data', 'value_' || i
+)
+FROM generate_series(1, 10000) i;
+SELECT 'Baseline (10000 rows)' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+-----------------------+---------+-----
+ Baseline (10000 rows) | 0 | 0
+(1 row)
+
+-- Update non-indexed fields on all rows - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{count}', to_jsonb((data->>'count')::int + 1));
+SELECT 'After updating 10000 rows (non-indexed)' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+-----------------------------------------+---------+-----
+ After updating 10000 rows (non-indexed) | 10000 | 0
+(1 row)
+
+-- Verify correctness
+SELECT COUNT(*) AS rows_with_count_1 FROM hot_test WHERE (data->>'count')::int = 1;
+ rows_with_count_1
+-------------------
+ 10000
+(1 row)
+
+-- Update indexed field on subset - should NOT be HOT for those rows
+UPDATE hot_test SET data = jsonb_set(data, '{status}', '"inactive"')
+WHERE id <= 10;
+SELECT 'After updating 10 rows (indexed)' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+----------------------------------+---------+-----
+ After updating 10 rows (indexed) | 10010 | 0
+(1 row)
+
+-- Verify indexes work
+SELECT COUNT(*) FROM hot_test WHERE data->>'status' = 'inactive';
+ count
+-------
+ 10
+(1 row)
+
+SELECT COUNT(*) FROM hot_test WHERE data->>'status' = 'active';
+ count
+-------
+ 9990
+(1 row)
+
+-- Only BRIN (summarizing) indexes on non-PK columns
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ ts timestamp,
+ value int,
+ brin_col int
+) USING heap WITH(fillfactor = 50);
+CREATE INDEX hot_test_ts_brin ON hot_test USING brin(ts);
+CREATE INDEX hot_test_brin_col_brin ON hot_test USING brin(brin_col);
+INSERT INTO hot_test VALUES (1, '2024-01-01', 100, 1000);
+-- Update both BRIN columns - should still be HOT (only summarizing indexes)
+UPDATE hot_test SET ts = '2024-01-02', brin_col = 2000 WHERE id = 1;
+SELECT 'After updating ts, brin_col (summarizing-only)' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+------------------------------------------------+---------+-----
+ After updating ts, brin_col (summarizing-only) | 1 | 1
+(1 row)
+
+-- Verify BRIN indexes work
+SELECT id FROM hot_test WHERE ts >= '2024-01-02';
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hot_test WHERE brin_col >= 2000;
+ id
+----
+ 1
+(1 row)
+
+-- TOASTed columns can participate in HOT
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ large_text text
+) USING heap WITH(fillfactor = 50);
+CREATE INDEX hot_test_idx ON hot_test(large_text);
+-- Insert row with TOASTed column (> 2KB)
+INSERT INTO hot_test VALUES (1, repeat('x', 3000));
+-- Update TOASTed column - should NOT be HOT
+UPDATE hot_test SET large_text = repeat('y', 3000);
+SELECT 'After updating large_text (TOASTed)' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+-------------------------------------+---------+-----
+ After updating large_text (TOASTed) | 1 | 0
+(1 row)
+
+-- Partitioned tables: HOT works within partitions
+CREATE TABLE hot_test_partitioned (
+ id int,
+ partition_key int,
+ indexed_col int,
+ data text,
+ PRIMARY KEY (id, partition_key)
+) PARTITION BY RANGE (partition_key);
+CREATE TABLE hot_test_part1 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (1) TO (100);
+CREATE TABLE hot_test_part2 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (100) TO (200);
+CREATE INDEX hot_test_partitioned_idx ON hot_test_partitioned(indexed_col);
+CREATE INDEX hot_test_part2_data ON hot_test_part2(data);
+INSERT INTO hot_test_partitioned VALUES (1, 50, 100, 'initial1');
+INSERT INTO hot_test_partitioned VALUES (2, 150, 200, 'initial2');
+-- Update in partition 1 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'UPDATED' WHERE id = 1;
+SELECT 'After updating partition 1 data' AS test, * FROM get_hot_count('hot_test_part1');
+ test | updates | hot
+---------------------------------+---------+-----
+ After updating partition 1 data | 1 | 1
+(1 row)
+
+-- Update in partition 2 (indexed column) - should NOT be HOT
+UPDATE hot_test_partitioned SET data = 'UPDATED' WHERE id = 2;
+SELECT 'After updating large_text (TOASTed)' AS test, * FROM get_hot_count('hot_test_part2');
+ test | updates | hot
+-------------------------------------+---------+-----
+ After updating large_text (TOASTed) | 1 | 0
+(1 row)
+
+-- Verify indexes work on partitions
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 100;
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 200;
+ id
+----
+ 2
+(1 row)
+
+-- Update indexed column in partition - should NOT be HOT
+-- Partition 1 previously had 1 update and 1 HOT update, this should
+-- change that to 2 updates and 1 HOT update.
+UPDATE hot_test_partitioned SET indexed_col = 150 WHERE id = 1;
+SELECT 'After updating indexed_col' AS test, * FROM get_hot_count('hot_test_part1');
+ test | updates | hot
+----------------------------+---------+-----
+ After updating indexed_col | 2 | 1
+(1 row)
+
+-- ============================================================================
+-- Partial indexes with complex predicates on JSONB
+-- ============================================================================
+-- Test partial indexes with WHERE clauses on JSONB expressions.
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ data jsonb
+) USING heap WITH(fillfactor = 50);
+-- Partial index: only index status when priority > 5
+CREATE INDEX hot_test_partial_idx ON hot_test((data->'status'))
+ WHERE (data->>'priority')::int > 5;
+INSERT INTO hot_test VALUES (
+ 1,
+ '{"status": "active", "priority": 10, "count": 0}'::jsonb
+);
+INSERT INTO hot_test VALUES (
+ 2,
+ '{"status": "active", "priority": 3, "count": 0}'::jsonb
+);
+SELECT 'Partial Index Test: Baseline' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+------------------------------+---------+-----
+ Partial Index Test: Baseline | 0 | 0
+(1 row)
+
+-- Update non-indexed path on row inside predicate (priority=10 > 5)
+-- Should NOT be HOT despite {count} is not indexed
+UPDATE hot_test SET data = jsonb_set(data, '{count}', '1') WHERE id = 1;
+SELECT 'Partial Index Test: count update, inside predicate' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+----------------------------------------------------+---------+-----
+ Partial Index Test: count update, inside predicate | 1 | 0
+(1 row)
+
+-- Update non-indexed path on row outside predicate (priority=3 <= 5)
+-- Should NOT be HOT dispite {count} is not indexed
+UPDATE hot_test SET data = jsonb_set(data, '{count}', '1') WHERE id = 2;
+SELECT 'Partial Index Test: count update, outside predicate' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+-----------------------------------------------------+---------+-----
+ Partial Index Test: count update, outside predicate | 2 | 0
+(1 row)
+
+-- Update indexed path on row inside predicate (priority=10 > 5)
+-- Should NOT be HOT indexed portion is updated
+UPDATE hot_test SET data = jsonb_set(data, '{status}', '"inactive"') WHERE id = 1;
+SELECT 'Partial Index Test: status update, inside predicate' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+-----------------------------------------------------+---------+-----
+ Partial Index Test: status update, inside predicate | 3 | 0
+(1 row)
+
+-- Update indexed path on row outside predicate (priority=3 <= 5)
+-- PostgreSQL makes a conservative choice and treats it as non-HOT because the
+-- indexed column changed, even though the before/after rows are outside the predicate
+UPDATE hot_test SET data = jsonb_set(data, '{status}', '"inactive"') WHERE id = 2;
+SELECT 'Partial Index Test: status update, outside predicate' AS test, * FROM get_hot_count('hot_test');
+ test | updates | hot
+------------------------------------------------------+---------+-----
+ Partial Index Test: status update, outside predicate | 4 | 0
+(1 row)
+
+-- Verify index works
+SELECT id FROM hot_test WHERE data->'status' = '"inactive"'::jsonb AND (data->>'priority')::int > 5;
+ id
+----
+ 1
+(1 row)
+
+-- ============================================================================
+DROP TABLE IF EXISTS hot_test;
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+DROP FUNCTION IF EXISTS has_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS print_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS get_hot_count(text);
+DROP EXTENSION pageinspect;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 549e9b2d7be..e06247ef7ea 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -137,6 +137,11 @@ test: event_trigger_login
# this test also uses event triggers, so likewise run it by itself
test: fast_default
+# ----------
+# HOT updates tests
+# ----------
+test: hot_updates
+
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
diff --git a/src/test/regress/sql/hot_updates.sql b/src/test/regress/sql/hot_updates.sql
new file mode 100644
index 00000000000..34da4552d4f
--- /dev/null
+++ b/src/test/regress/sql/hot_updates.sql
@@ -0,0 +1,692 @@
+-- Load required extensions
+CREATE EXTENSION IF NOT EXISTS pageinspect;
+
+-- Function to get HOT update count
+CREATE OR REPLACE FUNCTION get_hot_count(rel_name text)
+RETURNS TABLE (
+ updates BIGINT,
+ hot BIGINT
+) AS $$
+DECLARE
+ rel_oid oid;
+BEGIN
+ rel_oid := rel_name::regclass::oid;
+
+ -- Read both committed and transaction-local stats
+ -- In autocommit mode (default for regression tests), this works correctly
+ -- Note: In explicit transactions (BEGIN/COMMIT), committed stats already
+ -- include flushed updates, so this would double-count. For explicit
+ -- transaction testing, call pg_stat_force_next_flush() before this function.
+ updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
+ hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
+
+ RETURN NEXT;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Check if a tuple is part of a HOT chain (has a predecessor on same page)
+CREATE OR REPLACE FUNCTION has_hot_chain(rel_name text, target_ctid tid)
+RETURNS boolean AS $$
+DECLARE
+ block_num int;
+ page_item record;
+BEGIN
+ block_num := (target_ctid::text::point)[0]::int;
+
+ -- Look for a different tuple on the same page that points to our target tuple
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid IS NOT NULL
+ AND t_ctid = target_ctid
+ AND ('(' || block_num::text || ',' || lp::text || ')')::tid != target_ctid
+ LOOP
+ RETURN true;
+ END LOOP;
+
+ RETURN false;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Print the HOT chain starting from a given tuple
+CREATE OR REPLACE FUNCTION print_hot_chain(rel_name text, start_ctid tid)
+RETURNS TABLE(chain_position int, ctid tid, lp_flags text, t_ctid tid, chain_end boolean) AS
+$$
+#variable_conflict use_column
+DECLARE
+ block_num int;
+ line_ptr int;
+ current_ctid tid := start_ctid;
+ next_ctid tid;
+ position int := 0;
+ max_iterations int := 100;
+ page_item record;
+ found_predecessor boolean := false;
+ flags_name text;
+BEGIN
+ block_num := (start_ctid::text::point)[0]::int;
+
+ -- Find the predecessor (old tuple pointing to our start_ctid)
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid = start_ctid
+ LOOP
+ current_ctid := ('(' || block_num::text || ',' || page_item.lp::text || ')')::tid;
+ found_predecessor := true;
+ EXIT;
+ END LOOP;
+
+ -- If no predecessor found, start with the given ctid
+ IF NOT found_predecessor THEN
+ current_ctid := start_ctid;
+ END IF;
+
+ -- Follow the chain forward
+ WHILE position < max_iterations LOOP
+ line_ptr := (current_ctid::text::point)[1]::int;
+
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp = line_ptr
+ LOOP
+ -- Map lp_flags to names
+ flags_name := CASE page_item.lp_flags
+ WHEN 0 THEN 'unused (0)'
+ WHEN 1 THEN 'normal (1)'
+ WHEN 2 THEN 'redirect (2)'
+ WHEN 3 THEN 'dead (3)'
+ ELSE 'unknown (' || page_item.lp_flags::text || ')'
+ END;
+
+ RETURN QUERY SELECT
+ position,
+ current_ctid,
+ flags_name,
+ page_item.t_ctid,
+ (page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid)::boolean
+ ;
+
+ IF page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid THEN
+ RETURN;
+ END IF;
+
+ next_ctid := page_item.t_ctid;
+
+ IF (next_ctid::text::point)[0]::int != block_num THEN
+ RETURN;
+ END IF;
+
+ current_ctid := next_ctid;
+ position := position + 1;
+ END LOOP;
+
+ IF position = 0 THEN
+ RETURN;
+ END IF;
+ END LOOP;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Basic HOT update functionality
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+) USING heap WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
+
+INSERT INTO hot_test VALUES (1, 100, 'initial');
+INSERT INTO hot_test VALUES (2, 200, 'initial');
+INSERT INTO hot_test VALUES (3, 300, 'initial');
+
+-- Get baseline
+SELECT * FROM get_hot_count('hot_test');
+
+-- Should be HOT updates (only non-indexed column modified)
+UPDATE hot_test SET non_indexed_col = 'updated1' WHERE id = 1;
+UPDATE hot_test SET non_indexed_col = 'updated2' WHERE id = 2;
+UPDATE hot_test SET non_indexed_col = 'updated3' WHERE id = 3;
+
+-- Verify HOT updates occurred
+SELECT * FROM get_hot_count('hot_test');
+
+-- Dump the HOT chain for tuple with id == 1
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+
+-- Trigger optimistic heap page pruning
+SELECT ctid, * FROM hot_test;
+
+-- Dump the HOT chain after prune
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+
+SET SESSION enable_seqscan = OFF;
+SET SESSION enable_bitmapscan = OFF;
+
+-- Verify indexes still work
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 100;
+SELECT id, indexed_col FROM hot_test WHERE indexed_col = 100;
+
+-- Vacuum the relation, expect the HOT chain to collapse
+VACUUM hot_test;
+
+-- Show that there is no chain after vacuum
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+
+-- Non-HOT update (update indexed column)
+UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Verify index was updated (new value findable)
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+
+-- Verify old value no longer in index
+EXPLAIN (COSTS OFF) SELECT id FROM hot_test WHERE indexed_col = 100;
+SELECT id FROM hot_test WHERE indexed_col = 100;
+
+SET SESSION enable_seqscan = ON;
+SET SESSION enable_bitmapscan = ON;
+
+-- All-or-none property: updating one indexed column requires ALL index updates
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ non_indexed text
+) USING heap WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_a_idx ON hot_test(col_a);
+CREATE INDEX hot_test_b_idx ON hot_test(col_b);
+CREATE INDEX hot_test_c_idx ON hot_test(col_c);
+
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'initial');
+
+-- Update only col_a - should NOT be HOT because an indexed column changed
+-- This means ALL indexes must be updated (all-or-none property)
+UPDATE hot_test SET col_a = 15 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Verify all three indexes still work correctly
+SELECT id, col_a FROM hot_test WHERE col_a = 15; -- updated index
+SELECT id, col_b FROM hot_test WHERE col_b = 20; -- unchanged index
+SELECT id, col_c FROM hot_test WHERE col_c = 30; -- unchanged index
+
+-- Now update only non-indexed column - should be HOT
+UPDATE hot_test SET non_indexed = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+
+-- Verify all indexes still work
+SELECT id FROM hot_test WHERE col_a = 15 AND col_b = 20 AND col_c = 30;
+
+-- Partial index: both old and new outside predicate (conservative = non-HOT)
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ status text,
+ data text
+) WITH (fillfactor = 50);
+
+-- Partial index only covers status = 'active'
+CREATE INDEX hot_test_active_idx ON hot_test(status) WHERE status = 'active';
+
+INSERT INTO hot_test VALUES (1, 'active', 'data1');
+INSERT INTO hot_test VALUES (2, 'inactive', 'data2');
+INSERT INTO hot_test VALUES (3, 'deleted', 'data3');
+
+-- Update non-indexed column on 'active' row (in predicate, status unchanged)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated1' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update non-indexed column on 'inactive' row (outside predicate)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated2' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update status from 'inactive' to 'deleted' (both outside predicate)
+-- PostgreSQL is conservative: heap insert happens before predicate check
+-- So this is NON-HOT even though both values are outside predicate
+UPDATE hot_test SET status = 'deleted' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Verify index still works for 'active' rows
+SELECT id, status FROM hot_test WHERE status = 'active';
+
+-- Only BRIN (summarizing) indexes on non-PK columns
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ ts timestamp,
+ value int,
+ brin_col int
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_ts_brin ON hot_test USING brin(ts);
+CREATE INDEX hot_test_brin_col_brin ON hot_test USING brin(brin_col);
+
+INSERT INTO hot_test VALUES (1, '2024-01-01', 100, 1000);
+
+-- Update both BRIN columns - should still be HOT (only summarizing indexes)
+UPDATE hot_test SET ts = '2024-01-02', brin_col = 2000 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Verify BRIN indexes work
+SELECT id FROM hot_test WHERE ts >= '2024-01-02';
+SELECT id FROM hot_test WHERE brin_col >= 2000;
+
+-- Update non-indexed column - should also be HOT
+UPDATE hot_test SET value = 200 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Unique constraint (unique index) behaves like regular index
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ unique_col int UNIQUE,
+ data text
+) WITH (fillfactor = 50);
+
+INSERT INTO hot_test VALUES (1, 100, 'data1');
+INSERT INTO hot_test VALUES (2, 200, 'data2');
+
+-- Update data (non-indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+
+-- Verify unique constraint still enforced
+SELECT id, unique_col, data FROM hot_test ORDER BY id;
+
+-- This should fail (unique violation)
+UPDATE hot_test SET unique_col = 100 WHERE id = 2;
+
+-- Multi-column index: any column change = non-HOT
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ col_d int
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_ab_idx ON hot_test(col_a, col_b);
+CREATE INDEX hot_test_ab_inc_c_idx ON hot_test(col_a, col_b) INCLUDE(col_c);
+
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 40);
+
+-- Update col_a (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_a = 15;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update col_b (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_b = 25;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update col_c (not indexed, but included) - should NOT be HOT
+UPDATE hot_test SET col_c = 35;
+
+-- Verify multi-column index-only scan for included columns works
+EXPLAIN (COSTS OFF) SELECT col_c FROM hot_test WHERE col_a = 15 AND col_b = 25;
+SELECT col_c FROM hot_test WHERE col_a = 15 AND col_b = 25;
+
+-- ============================================================================
+-- Expression indexes with JSONB
+-- ============================================================================
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ data jsonb
+) USING heap WITH(fillfactor = 50);
+
+-- Indexes on specific JSONB paths
+CREATE INDEX hot_test_status_idx ON hot_test((data->'status'));
+CREATE INDEX hot_test_user_id_idx ON hot_test((data->'user'->'id'));
+
+INSERT INTO hot_test VALUES (
+ 1,
+ '{"status": "active", "user": {"id": 123, "name": "Alice"}, "count": 0}'::jsonb
+);
+
+-- Baseline
+SELECT 'Baseline' AS test, * FROM get_hot_count('hot_test');
+
+-- Update non-indexed path {count} - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{count}', '1') WHERE id = 1;
+SELECT 'After updating count (non-indexed)' AS test, * FROM get_hot_count('hot_test');
+
+-- Update different non-indexed path {user,name} - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{user,name}', '"Bob"') WHERE id = 1;
+SELECT 'After updating user.name (non-indexed)' AS test, * FROM get_hot_count('hot_test');
+
+-- Update indexed path {status} - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{status}', '"inactive"') WHERE id = 1;
+SELECT 'After updating status (indexed)' AS test, * FROM get_hot_count('hot_test');
+
+-- Update indexed path {user,id} - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{user,id}', '456') WHERE id = 1;
+SELECT 'After updating user.id (indexed)' AS test, * FROM get_hot_count('hot_test');
+
+-- Verify indexes still work correctly
+SELECT id FROM hot_test WHERE data->'status' = '"inactive"'::jsonb;
+SELECT id FROM hot_test WHERE data->'user'->'id' = '456'::jsonb;
+
+-- ============================================================================
+-- Nested paths and path intersection
+-- ============================================================================
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ data jsonb
+) USING heap WITH(fillfactor = 50);
+
+CREATE INDEX hot_test_deep_idx ON hot_test((data->'a'->'b'->'c'));
+
+INSERT INTO hot_test VALUES (
+ 1,
+ '{"a": {"b": {"c": "indexed", "d": "not-indexed"}}, "x": "other"}'::jsonb
+);
+
+SELECT 'Baseline' AS test, * FROM get_hot_count('hot_test');
+
+-- Update sibling of indexed path {a,b,d} - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{a,b,d}', '"updated"') WHERE id = 1;
+SELECT 'After updating a.b.d (sibling, non-indexed)' AS test, * FROM get_hot_count('hot_test');
+
+-- Update unrelated path {x} - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{x}', '"modified"') WHERE id = 1;
+SELECT 'After updating x (unrelated path)' AS test, * FROM get_hot_count('hot_test');
+
+-- Update parent of indexed path {a,b} - should NOT be HOT (affects child)
+UPDATE hot_test SET data = jsonb_set(data, '{a,b}', '{"c": "new", "d": "data"}') WHERE id = 1;
+SELECT 'After updating a.b (parent of indexed)' AS test, * FROM get_hot_count('hot_test');
+
+-- ============================================================================
+-- Multiple JSONB mutation functions
+-- ============================================================================
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ data jsonb
+) USING heap WITH(fillfactor = 50);
+
+CREATE INDEX hot_test_keep_idx ON hot_test((data->'keep'));
+
+INSERT INTO hot_test VALUES (
+ 1,
+ '{"keep": "important", "remove": "unimportant", "extra": "data"}'::jsonb
+);
+
+SELECT 'Baseline' AS test, * FROM get_hot_count('hot_test');
+
+-- jsonb_delete on non-indexed key - should NOT be HOT
+UPDATE hot_test SET data = data - 'remove' WHERE id = 1;
+SELECT 'After deleting non-indexed key' AS test, * FROM get_hot_count('hot_test');
+
+-- jsonb_set on non-indexed key - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{extra}', '"modified"') WHERE id = 1;
+SELECT 'After modifying non-indexed key' AS test, * FROM get_hot_count('hot_test');
+
+-- jsonb_delete on indexed key - should NOT be HOT
+UPDATE hot_test SET data = data - 'keep' WHERE id = 1;
+SELECT 'After deleting indexed key' AS test, * FROM get_hot_count('hot_test');
+
+-- ============================================================================
+-- Array operations
+-- ============================================================================
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ data jsonb
+) USING heap WITH(fillfactor = 50);
+
+-- Index on array element
+CREATE INDEX hot_test_tags_idx ON hot_test((data->'tags'->0));
+
+INSERT INTO hot_test VALUES (
+ 1,
+ '{"tags": ["indexed", "second", "third"], "other": "data"}'::jsonb
+);
+
+SELECT 'Baseline' AS test, * FROM get_hot_count('hot_test');
+
+-- Update non-indexed array element - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{tags,1}', '"modified"') WHERE id = 1;
+SELECT 'After updating tags[1]' AS test, * FROM get_hot_count('hot_test');
+
+-- Update indexed array element - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{tags,0}', '"changed"') WHERE id = 1;
+SELECT 'After updating tags[0] (indexed)' AS test, * FROM get_hot_count('hot_test');
+
+-- ============================================================================
+-- Whole column index
+-- ============================================================================
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ data jsonb
+) USING heap WITH(fillfactor = 50);
+
+-- Index on entire JSONB column, and a path extraction
+CREATE INDEX hot_test_whole_idx ON hot_test(data);
+CREATE INDEX hot_test_tags_idx ON hot_test((data->'a'));
+
+INSERT INTO hot_test VALUES (1, '{"a": 1, "b": 1}'::jsonb);
+
+SELECT 'Baseline' AS test, * FROM get_hot_count('hot_test');
+
+-- Any modification to data - should NOT be HOT (whole column indexed)
+UPDATE hot_test SET data = jsonb_set(data, '{b}', '2') WHERE id = 1;
+SELECT 'After modifying any field (whole column indexed)' AS test, * FROM get_hot_count('hot_test');
+
+-- ============================================================================
+-- Performance at scale
+-- ============================================================================
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ data jsonb
+) USING heap WITH(fillfactor=50);
+
+CREATE INDEX hot_test_status_idx ON hot_test((data->'status'));
+CREATE INDEX hot_test_priority_idx ON hot_test((data->'priority'));
+
+-- Insert 10000 rows
+INSERT INTO hot_test
+SELECT i, jsonb_build_object(
+ 'status', 'active',
+ 'priority', 1,
+ 'count', 0,
+ 'data', 'value_' || i
+)
+FROM generate_series(1, 10000) i;
+
+SELECT 'Baseline (10000 rows)' AS test, * FROM get_hot_count('hot_test');
+
+-- Update non-indexed fields on all rows - should NOT be HOT
+UPDATE hot_test SET data = jsonb_set(data, '{count}', to_jsonb((data->>'count')::int + 1));
+
+SELECT 'After updating 10000 rows (non-indexed)' AS test, * FROM get_hot_count('hot_test');
+
+-- Verify correctness
+SELECT COUNT(*) AS rows_with_count_1 FROM hot_test WHERE (data->>'count')::int = 1;
+
+-- Update indexed field on subset - should NOT be HOT for those rows
+UPDATE hot_test SET data = jsonb_set(data, '{status}', '"inactive"')
+WHERE id <= 10;
+
+SELECT 'After updating 10 rows (indexed)' AS test, * FROM get_hot_count('hot_test');
+
+-- Verify indexes work
+SELECT COUNT(*) FROM hot_test WHERE data->>'status' = 'inactive';
+SELECT COUNT(*) FROM hot_test WHERE data->>'status' = 'active';
+
+-- Only BRIN (summarizing) indexes on non-PK columns
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ ts timestamp,
+ value int,
+ brin_col int
+) USING heap WITH(fillfactor = 50);
+
+CREATE INDEX hot_test_ts_brin ON hot_test USING brin(ts);
+CREATE INDEX hot_test_brin_col_brin ON hot_test USING brin(brin_col);
+
+INSERT INTO hot_test VALUES (1, '2024-01-01', 100, 1000);
+
+-- Update both BRIN columns - should still be HOT (only summarizing indexes)
+UPDATE hot_test SET ts = '2024-01-02', brin_col = 2000 WHERE id = 1;
+SELECT 'After updating ts, brin_col (summarizing-only)' AS test, * FROM get_hot_count('hot_test');
+
+-- Verify BRIN indexes work
+SELECT id FROM hot_test WHERE ts >= '2024-01-02';
+SELECT id FROM hot_test WHERE brin_col >= 2000;
+
+-- TOASTed columns can participate in HOT
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ large_text text
+) USING heap WITH(fillfactor = 50);
+
+CREATE INDEX hot_test_idx ON hot_test(large_text);
+
+-- Insert row with TOASTed column (> 2KB)
+INSERT INTO hot_test VALUES (1, repeat('x', 3000));
+
+-- Update TOASTed column - should NOT be HOT
+UPDATE hot_test SET large_text = repeat('y', 3000);
+SELECT 'After updating large_text (TOASTed)' AS test, * FROM get_hot_count('hot_test');
+
+-- Partitioned tables: HOT works within partitions
+CREATE TABLE hot_test_partitioned (
+ id int,
+ partition_key int,
+ indexed_col int,
+ data text,
+ PRIMARY KEY (id, partition_key)
+) PARTITION BY RANGE (partition_key);
+
+CREATE TABLE hot_test_part1 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (1) TO (100);
+CREATE TABLE hot_test_part2 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (100) TO (200);
+
+CREATE INDEX hot_test_partitioned_idx ON hot_test_partitioned(indexed_col);
+CREATE INDEX hot_test_part2_data ON hot_test_part2(data);
+
+INSERT INTO hot_test_partitioned VALUES (1, 50, 100, 'initial1');
+INSERT INTO hot_test_partitioned VALUES (2, 150, 200, 'initial2');
+
+-- Update in partition 1 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'UPDATED' WHERE id = 1;
+SELECT 'After updating partition 1 data' AS test, * FROM get_hot_count('hot_test_part1');
+
+-- Update in partition 2 (indexed column) - should NOT be HOT
+UPDATE hot_test_partitioned SET data = 'UPDATED' WHERE id = 2;
+SELECT 'After updating large_text (TOASTed)' AS test, * FROM get_hot_count('hot_test_part2');
+
+-- Verify indexes work on partitions
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 100;
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 200;
+
+-- Update indexed column in partition - should NOT be HOT
+-- Partition 1 previously had 1 update and 1 HOT update, this should
+-- change that to 2 updates and 1 HOT update.
+UPDATE hot_test_partitioned SET indexed_col = 150 WHERE id = 1;
+SELECT 'After updating indexed_col' AS test, * FROM get_hot_count('hot_test_part1');
+
+-- ============================================================================
+-- Partial indexes with complex predicates on JSONB
+-- ============================================================================
+-- Test partial indexes with WHERE clauses on JSONB expressions.
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ data jsonb
+) USING heap WITH(fillfactor = 50);
+
+-- Partial index: only index status when priority > 5
+CREATE INDEX hot_test_partial_idx ON hot_test((data->'status'))
+ WHERE (data->>'priority')::int > 5;
+
+INSERT INTO hot_test VALUES (
+ 1,
+ '{"status": "active", "priority": 10, "count": 0}'::jsonb
+);
+INSERT INTO hot_test VALUES (
+ 2,
+ '{"status": "active", "priority": 3, "count": 0}'::jsonb
+);
+
+SELECT 'Partial Index Test: Baseline' AS test, * FROM get_hot_count('hot_test');
+
+-- Update non-indexed path on row inside predicate (priority=10 > 5)
+-- Should NOT be HOT despite {count} is not indexed
+UPDATE hot_test SET data = jsonb_set(data, '{count}', '1') WHERE id = 1;
+SELECT 'Partial Index Test: count update, inside predicate' AS test, * FROM get_hot_count('hot_test');
+
+-- Update non-indexed path on row outside predicate (priority=3 <= 5)
+-- Should NOT be HOT dispite {count} is not indexed
+UPDATE hot_test SET data = jsonb_set(data, '{count}', '1') WHERE id = 2;
+SELECT 'Partial Index Test: count update, outside predicate' AS test, * FROM get_hot_count('hot_test');
+
+-- Update indexed path on row inside predicate (priority=10 > 5)
+-- Should NOT be HOT indexed portion is updated
+UPDATE hot_test SET data = jsonb_set(data, '{status}', '"inactive"') WHERE id = 1;
+SELECT 'Partial Index Test: status update, inside predicate' AS test, * FROM get_hot_count('hot_test');
+
+-- Update indexed path on row outside predicate (priority=3 <= 5)
+-- PostgreSQL makes a conservative choice and treats it as non-HOT because the
+-- indexed column changed, even though the before/after rows are outside the predicate
+UPDATE hot_test SET data = jsonb_set(data, '{status}', '"inactive"') WHERE id = 2;
+SELECT 'Partial Index Test: status update, outside predicate' AS test, * FROM get_hot_count('hot_test');
+
+-- Verify index works
+SELECT id FROM hot_test WHERE data->'status' = '"inactive"'::jsonb AND (data->>'priority')::int > 5;
+-- ============================================================================
+DROP TABLE IF EXISTS hot_test;
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+DROP FUNCTION IF EXISTS has_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS print_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS get_hot_count(text);
+DROP EXTENSION pageinspect;
--
2.51.2
[text/x-patch] v35-0002-Identify-and-track-columns-modified-by-heap_modi.patch (7.0K, 3-v35-0002-Identify-and-track-columns-modified-by-heap_modi.patch)
download | inline diff:
From ff260840eadfd1cc41528fc503435c04be421083 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Tue, 10 Mar 2026 08:17:31 -0400
Subject: [PATCH v35 2/3] Identify and track columns modified by
heap_modifiy_tuple() on update
ExecGetAllUpdatedCols() misses attributes modified using
heap_modify_tuple() that are not explictly SET in the UPDATE or by
triggers. This happens in one test (tsearch.sql) when the
tsvector_update_trigger() is invoked and modifies an indexed attribute
that isn't referenced in any SQL.
The net is that the functions like HeapDetermineColumnsInfo() have to
scan all indexed attributes for changes rather than being able to first
reduce the indexed set by intersecting it with the set of attributes
known to be potentially updated.
While this isn't so bad, it is an oversight should someone in the future
build some security related feature using that incomplete result. It
also might save a fraction of overhead calculating modified index
attributes in heap_update().
This commit adds to ExecBRUpdateTriggers() code that identify changes to
indexed columns not found by ExecGetAllUpdatedCols() and adds those
attributes to ri_extraUpdatedCols.
This commit introduces ExecCompareSlotAttrs() as a utility function to
identify those attributes that have changed. It compares a subset of
attributes between two TupleTableSlots and returns a Bitmapset of
attributes that differ.
It would be nice to integrate this into HeapDetermineColumnsInfo(),
however it would be a layering violation given that it is within
heap_update().
---
src/backend/commands/trigger.c | 20 +++++++-
src/backend/executor/execTuples.c | 78 +++++++++++++++++++++++++++++++
src/include/executor/executor.h | 5 ++
3 files changed, 102 insertions(+), 1 deletion(-)
diff --git a/src/backend/commands/trigger.c b/src/backend/commands/trigger.c
index 98d402c0a3b..bbe077a9ca9 100644
--- a/src/backend/commands/trigger.c
+++ b/src/backend/commands/trigger.c
@@ -2978,6 +2978,7 @@ ExecBRUpdateTriggers(EState *estate, EPQState *epqstate,
bool is_merge_update)
{
TriggerDesc *trigdesc = relinfo->ri_TrigDesc;
+ TupleDesc tupdesc = RelationGetDescr(relinfo->ri_RelationDesc);
TupleTableSlot *oldslot = ExecGetTriggerOldSlot(estate, relinfo);
HeapTuple newtuple = NULL;
HeapTuple trigtuple;
@@ -2985,7 +2986,9 @@ ExecBRUpdateTriggers(EState *estate, EPQState *epqstate,
bool should_free_new = false;
TriggerData LocTriggerData = {0};
int i;
- Bitmapset *updatedCols;
+ Bitmapset *updatedCols = NULL;
+ Bitmapset *remainingCols = NULL;
+ Bitmapset *modifiedCols;
LockTupleMode lockmode;
/* Determine lock mode to use */
@@ -3127,6 +3130,21 @@ ExecBRUpdateTriggers(EState *estate, EPQState *epqstate,
if (should_free_trig)
heap_freetuple(trigtuple);
+ /*
+ * Before UPDATE triggers may have updated attributes not known to
+ * ExecGetAllUpdatedColumns() using heap_modify_tuple() or
+ * heap_modifiy_tuple_by_cols(). Find and record those now.
+ */
+ remainingCols = bms_add_range(NULL, 1 - FirstLowInvalidHeapAttributeNumber,
+ tupdesc->natts - FirstLowInvalidHeapAttributeNumber);
+ remainingCols = bms_del_members(remainingCols, updatedCols);
+ modifiedCols = ExecCompareSlotAttrs(tupdesc, remainingCols, oldslot, newslot);
+ relinfo->ri_extraUpdatedCols =
+ bms_add_members(relinfo->ri_extraUpdatedCols, modifiedCols);
+
+ bms_free(remainingCols);
+ bms_free(modifiedCols);
+
return true;
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index b768eae9e53..1064ebe845b 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -66,6 +66,7 @@
#include "nodes/nodeFuncs.h"
#include "storage/bufmgr.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/expandeddatum.h"
#include "utils/lsyscache.h"
#include "utils/typcache.h"
@@ -1929,6 +1930,83 @@ ExecFetchSlotHeapTupleDatum(TupleTableSlot *slot)
return ret;
}
+/*
+ * ExecCompareSlotAttrs
+ *
+ * Compare the subset of attributes in attrs bewtween TupleTableSlots to detect
+ * which attributes have changed.
+ *
+ * Returns a Bitmapset of attribute indices (using
+ * FirstLowInvalidHeapAttributeNumber convention) that differ between the two
+ * slots.
+ */
+Bitmapset *
+ExecCompareSlotAttrs(TupleDesc tupdesc, const Bitmapset *attrs,
+ TupleTableSlot *s1, TupleTableSlot *s2)
+{
+ int attidx = -1;
+ Bitmapset *modified = NULL;
+
+ /* XXX what if slots don't share the same tupleDescriptor... */
+ /* Assert(s1->tts_tupleDescriptor == s2->tts_tupleDescriptor); */
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /* attidx is zero-based, attrnum is the normal attribute number */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value1,
+ value2;
+ bool null1,
+ null2;
+ CompactAttribute *att;
+
+ /*
+ * If it's a whole-tuple reference, say "not equal". It's not really
+ * worth supporting this case, since it could only succeed after a
+ * no-op update, which is hardly a case worth optimizing for.
+ */
+ if (attrnum == 0)
+ {
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /*
+ * Likewise, automatically say "not equal" for any system attribute
+ * other than tableOID; we cannot expect these to be consistent in a
+ * HOT chain, or even to be set correctly yet in the new tuple.
+ */
+ if (attrnum < 0)
+ {
+ if (attrnum != TableOidAttributeNumber)
+ {
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+ }
+
+ att = TupleDescCompactAttr(tupdesc, attrnum - 1);
+ value1 = slot_getattr(s1, attrnum, &null1);
+ value2 = slot_getattr(s2, attrnum, &null2);
+
+ /* A change to/from NULL, so not equal */
+ if (null1 != null2)
+ {
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /* Both NULL, no change/unmodified */
+ if (null2)
+ continue;
+
+ if (!datum_image_eq(value1, value2, att->attbyval, att->attlen))
+ modified = bms_add_member(modified, attidx);
+ }
+
+ return modified;
+}
+
/* ----------------------------------------------------------------
* convenience initialization routines
* ----------------------------------------------------------------
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index d46ba59895d..5dcfaa2027f 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -17,6 +17,7 @@
#include "datatype/timestamp.h"
#include "executor/execdesc.h"
#include "fmgr.h"
+#include "nodes/execnodes.h"
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
@@ -606,6 +607,10 @@ extern TupleDesc ExecCleanTypeFromTL(List *targetList);
extern TupleDesc ExecTypeFromExprList(List *exprList);
extern void ExecTypeSetColNames(TupleDesc typeInfo, List *namesList);
extern void UpdateChangedParamSet(PlanState *node, Bitmapset *newchg);
+extern Bitmapset *ExecCompareSlotAttrs(TupleDesc tupdesc,
+ const Bitmapset *attrs,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
typedef struct TupOutputState
{
--
2.51.2
[text/x-patch] v35-0003-Identify-modified-indexed-attributes-in-the-exec.patch (54.4K, 4-v35-0003-Identify-modified-indexed-attributes-in-the-exec.patch)
download | inline diff:
From 0208756d9666cb3b30b5b85a443a4df65463cb38 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Tue, 10 Mar 2026 08:18:23 -0400
Subject: [PATCH v35 3/3] Identify modified indexed attributes in the executor
on UPDATE
Refactor executor update logic to determine which indexed columns have
actually changed during an UPDATE operation rather than leaving this up
to HeapDetermineColumnsInfo() in heap_update(). Finding this set of
attributes is not heap-specific, but more general to all table AMs and
having this information in the executor could inform other decisions
about when index inserts are required and when they are not regardless
of the table AM's MVCC implementation strategy.
The heap-only tuple decision (HOT) in heap functions as it always has,
but the determination of the "modified indexed attributes"
(modified_idx_attrs, formerly known as modified_attrs).
ExecUpdateModifiedIdxAttrs() replaces HeapDetermineColumnsInfo() and is
called before table_tuple_update() crucially without the need for an
exclusive buffer lock on the page that holds the tuple being updated.
This reduces the time the buffer lock is held later within
heapam_tuple_update() and heap_update().
ExecUpdateModifiedIdxAttrs() uses the previously-introduced
ExecCompareSlotAttrs() function to identify which attributes have
changed and then intersects that with the set of indexed attributes to
identify the modified indexed set, the modified_idx_attrs.
Besides identifying the set of modified indexed attributes
HeapDetermineColumnsInfo() was also responsible for part of the logic
involved in the decision about what to WAL log for the replica identity
key. This logic moved into heap_update() and out of the replacement
named HeapUpdateModifiedIdxAttrs(). Doing this allows for
simple_heap_update() and heapam_tuple_update() to share the same logic
as they both call into heap_update().
Updates stemming from logical replication also use the new
ExecUpdateModifiedIdxAttrs() in ExecSimpleRelationUpdate().
This patch introduces a few helper functions to reduce code duplication
and increase readability: HeapUpdateHotAllowable(),
HeapUpdateDetermineLockmode(). These are used in both heap_update() and
simple_heap_update().
The heap_update() function is called now with lockmode pre-determined
and a boolean indicating if the update allows HOT updates or not, both
const. If during heap_update() the new tuple will fit on the same page
and that boolean is true, the update is HOT. This means that although
the functions and timing of the code involed in HOT decisions have
changed, none of the logic related to when HOT is allowed has changed.
Development of this feature exposed nondeterministic behavior in three
existing tests which have been adjusted to avoid inconsistent test
results due to tuple ordering during heap page scans.
---
src/backend/access/heap/heapam.c | 478 +++++++++++-------
src/backend/access/heap/heapam_handler.c | 32 +-
src/backend/access/table/tableam.c | 5 +-
src/backend/executor/execReplication.c | 9 +-
src/backend/executor/nodeModifyTable.c | 93 +++-
src/backend/utils/cache/relcache.c | 44 +-
src/include/access/heapam.h | 13 +-
src/include/access/tableam.h | 8 +-
src/include/executor/executor.h | 4 +
src/include/utils/rel.h | 2 +-
src/include/utils/relcache.h | 2 +-
.../regress/expected/generated_virtual.out | 2 +-
src/test/regress/expected/triggers.out | 16 +-
src/test/regress/expected/updatable_views.out | 4 +-
src/test/regress/sql/generated_virtual.sql | 2 +-
src/test/regress/sql/triggers.sql | 4 +-
src/test/regress/sql/updatable_views.sql | 2 +-
17 files changed, 492 insertions(+), 228 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 1ecc8330851..997dc9642d8 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -37,14 +37,20 @@
#include "access/multixact.h"
#include "access/subtrans.h"
#include "access/syncscan.h"
+#include "access/sysattr.h"
+#include "access/tableam.h"
#include "access/valid.h"
#include "access/visibilitymap.h"
#include "access/xloginsert.h"
#include "catalog/pg_database.h"
#include "catalog/pg_database_d.h"
#include "commands/vacuum.h"
+#include "executor/tuptable.h"
+#include "optimizer/cost.h"
+#include "nodes/lockoptions.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
+#include "storage/buf.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
#include "storage/proc.h"
@@ -52,6 +58,7 @@
#include "utils/datum.h"
#include "utils/injection_point.h"
#include "utils/inval.h"
+#include "utils/relcache.h"
#include "utils/spccache.h"
#include "utils/syscache.h"
@@ -68,11 +75,8 @@ static void check_lock_if_inplace_updateable_rel(Relation relation,
HeapTuple newtup);
static void check_inplace_rel_lock(HeapTuple oldtup);
#endif
-static Bitmapset *HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external);
+static Bitmapset *HeapUpdateModifiedIdxAttrs(Relation relation,
+ HeapTuple oldtup, HeapTuple newtup);
static bool heap_acquire_tuplock(Relation relation, const ItemPointerData *tid,
LockTupleMode mode, LockWaitPolicy wait_policy,
bool *have_tuple_lock);
@@ -3302,7 +3306,7 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* heap_update - replace a tuple
*
* See table_tuple_update() for an explanation of the parameters, except that
- * this routine directly takes a tuple rather than a slot.
+ * this routine directly takes a heap tuple rather than a slot.
*
* In the failure cases, the routine fills *tmfd with the tuple's t_ctid,
* t_xmax (resolving a possible MultiXact, if necessary), and t_cmax (the last
@@ -3312,17 +3316,13 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
TM_Result
heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ TM_FailureData *tmfd, const LockTupleMode lockmode,
+ const Bitmapset *modified_idx_attrs, const bool hot_allowed)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
- Bitmapset *hot_attrs;
- Bitmapset *sum_attrs;
- Bitmapset *key_attrs;
- Bitmapset *id_attrs;
- Bitmapset *interesting_attrs;
- Bitmapset *modified_attrs;
+ Bitmapset *idx_attrs,
+ *rid_attrs;
ItemId lp;
HeapTupleData oldtup;
HeapTuple heaptup;
@@ -3341,13 +3341,12 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
bool have_tuple_lock = false;
bool iscombo;
bool use_hot_update = false;
- bool summarized_update = false;
bool key_intact;
bool all_visible_cleared = false;
bool all_visible_cleared_new = false;
bool checked_lockers;
bool locker_remains;
- bool id_has_external = false;
+ bool rep_id_key_required = false;
TransactionId xmax_new_tuple,
xmax_old_tuple;
uint16 infomask_old_tuple,
@@ -3378,33 +3377,14 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
#endif
/*
- * Fetch the list of attributes to be checked for various operations.
- *
- * For HOT considerations, this is wasted effort if we fail to update or
- * have to put the new tuple on a different page. But we must compute the
- * list before obtaining buffer lock --- in the worst case, if we are
- * doing an update on one of the relevant system catalogs, we could
- * deadlock if we try to fetch the list later. In any case, the relcache
- * caches the data so this is usually pretty cheap.
- *
- * We also need columns used by the replica identity and columns that are
- * considered the "key" of rows in the table.
+ * Fetch the attributes used across all indexes on this relation as well
+ * as the replica identity and columns.
*
- * Note that we get copies of each bitmap, so we need not worry about
- * relcache flush happening midway through.
- */
- hot_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_HOT_BLOCKING);
- sum_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_SUMMARIZED);
- key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
- id_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
- interesting_attrs = NULL;
- interesting_attrs = bms_add_members(interesting_attrs, hot_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, sum_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, key_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, id_attrs);
+ * NOTE: relcache returns copies of each bitmap, so we need not worry
+ * about relcache flush happening midway through.
+ */
+ idx_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+ rid_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_IDENTITY_KEY);
block = ItemPointerGetBlockNumber(otid);
INJECTION_POINT("heap_update-before-pin", NULL);
@@ -3458,20 +3438,17 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
tmfd->ctid = *otid;
tmfd->xmax = InvalidTransactionId;
tmfd->cmax = InvalidCommandId;
- *update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- /* modified_attrs not yet initialized */
- bms_free(interesting_attrs);
+ bms_free(rid_attrs);
+ bms_free(idx_attrs);
+ /* modified_idx_attrs is owned by the caller, don't free it */
+
return TM_Deleted;
}
/*
- * Fill in enough data in oldtup for HeapDetermineColumnsInfo to work
- * properly.
+ * Fill in enough data in oldtup to determine replica identity attribute
+ * requirements.
*/
oldtup.t_tableOid = RelationGetRelid(relation);
oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
@@ -3482,16 +3459,59 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
newtup->t_tableOid = RelationGetRelid(relation);
/*
- * Determine columns modified by the update. Additionally, identify
- * whether any of the unmodified replica identity key attributes in the
- * old tuple is externally stored or not. This is required because for
- * such attributes the flattened value won't be WAL logged as part of the
- * new tuple so we must include it as part of the old_key_tuple. See
- * ExtractReplicaIdentity.
+ * ExtractReplicaIdentity() needs to know if a modified indexed attrbute
+ * is used as a replica indentity or if any of the replica identity
+ * attributes are referenced in an index, unmodified, and are stored
+ * externally in the old tuple being replaced. In those cases it may be
+ * necessary to WAL log them to so they are available to replicas.
*/
- modified_attrs = HeapDetermineColumnsInfo(relation, interesting_attrs,
- id_attrs, &oldtup,
- newtup, &id_has_external);
+ rep_id_key_required = bms_overlap(modified_idx_attrs, rid_attrs);
+ if (!rep_id_key_required)
+ {
+ Bitmapset *attrs;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ int attidx = -1;
+
+ /*
+ * Reduce the set under review to only the unmodified indexed replica
+ * identity key attributes. idx_attrs is copied (by bms_difference())
+ * not modified here.
+ */
+ attrs = bms_difference(idx_attrs, modified_idx_attrs);
+ attrs = bms_int_members(attrs, rid_attrs);
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /*
+ * attidx is zero-based, attrnum is the normal attribute number
+ */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value;
+ bool isnull;
+
+ /*
+ * System attributes are not added into INDEX_ATTR_BITMAP_INDEXED
+ * bitmap by relcache.
+ */
+ Assert(attrnum > 0);
+
+ value = heap_getattr(&oldtup, attrnum, tupdesc, &isnull);
+
+ /* No need to check attributes that can't be stored externally */
+ if (isnull ||
+ TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
+ continue;
+
+ /* Check if the old tuple's attribute is stored externally */
+ if (VARATT_IS_EXTERNAL((struct varlena *) DatumGetPointer(value)))
+ {
+ rep_id_key_required = true;
+ break;
+ }
+ }
+
+ bms_free(attrs);
+ }
/*
* If we're not updating any "key" column, we can grab a weaker lock type.
@@ -3504,9 +3524,8 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
* is updates that don't manipulate key columns, not those that
* serendipitously arrive at the same key values.
*/
- if (!bms_overlap(modified_attrs, key_attrs))
+ if (lockmode == LockTupleNoKeyExclusive)
{
- *lockmode = LockTupleNoKeyExclusive;
mxact_status = MultiXactStatusNoKeyUpdate;
key_intact = true;
@@ -3523,7 +3542,7 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
}
else
{
- *lockmode = LockTupleExclusive;
+ Assert(lockmode == LockTupleExclusive);
mxact_status = MultiXactStatusUpdate;
key_intact = false;
}
@@ -3534,7 +3553,6 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
* with the new tuple's location, so there's great risk of confusion if we
* use otid anymore.
*/
-
l2:
checked_lockers = false;
locker_remains = false;
@@ -3602,7 +3620,7 @@ l2:
bool current_is_member = false;
if (DoesMultiXactIdConflict((MultiXactId) xwait, infomask,
- *lockmode, ¤t_is_member))
+ lockmode, ¤t_is_member))
{
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
@@ -3611,7 +3629,7 @@ l2:
* requesting a lock and already have one; avoids deadlock).
*/
if (!current_is_member)
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &(oldtup.t_self), lockmode,
LockWaitBlock, &have_tuple_lock);
/* wait for multixact */
@@ -3696,7 +3714,7 @@ l2:
* lock.
*/
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &(oldtup.t_self), lockmode,
LockWaitBlock, &have_tuple_lock);
XactLockTableWait(xwait, relation, &oldtup.t_self,
XLTW_Update);
@@ -3756,17 +3774,14 @@ l2:
tmfd->cmax = InvalidCommandId;
UnlockReleaseBuffer(buffer);
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &(oldtup.t_self), lockmode);
if (vmbuffer != InvalidBuffer)
ReleaseBuffer(vmbuffer);
- *update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
+ bms_free(rid_attrs);
+ bms_free(idx_attrs);
+ /* modified_idx_attrs is owned by the caller, don't free it */
+
return result;
}
@@ -3796,7 +3811,7 @@ l2:
compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
oldtup.t_data->t_infomask,
oldtup.t_data->t_infomask2,
- xid, *lockmode, true,
+ xid, lockmode, true,
&xmax_old_tuple, &infomask_old_tuple,
&infomask2_old_tuple);
@@ -3913,7 +3928,7 @@ l2:
compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
oldtup.t_data->t_infomask,
oldtup.t_data->t_infomask2,
- xid, *lockmode, false,
+ xid, lockmode, false,
&xmax_lock_old_tuple, &infomask_lock_old_tuple,
&infomask2_lock_old_tuple);
@@ -4073,37 +4088,19 @@ l2:
/*
* At this point newbuf and buffer are both pinned and locked, and newbuf
- * has enough space for the new tuple. If they are the same buffer, only
- * one pin is held.
+ * has enough space for the new tuple so we can use the HOT update path if
+ * the caller determined that it is allowable.
+ *
+ * NOTE: If newbuf == buffer then only one pin is held.
*/
-
if (newbuf == buffer)
{
- /*
- * Since the new tuple is going into the same page, we might be able
- * to do a HOT update. Check if any of the index columns have been
- * changed.
- */
- if (!bms_overlap(modified_attrs, hot_attrs))
- {
+ if (hot_allowed)
use_hot_update = true;
-
- /*
- * If none of the columns that are used in hot-blocking indexes
- * were updated, we can apply HOT, but we do still need to check
- * if we need to update the summarizing indexes, and update those
- * indexes if the columns were updated, or we may fail to detect
- * e.g. value bound changes in BRIN minmax indexes.
- */
- if (bms_overlap(modified_attrs, sum_attrs))
- summarized_update = true;
- }
}
else
- {
/* Set a hint that the old page could use prune/defrag */
PageSetFull(page);
- }
/*
* Compute replica identity tuple before entering the critical section so
@@ -4113,8 +4110,7 @@ l2:
* columns are modified or it has external data.
*/
old_key_tuple = ExtractReplicaIdentity(relation, &oldtup,
- bms_overlap(modified_attrs, id_attrs) ||
- id_has_external,
+ rep_id_key_required,
&old_key_copied);
/* NO EREPORT(ERROR) from here till changes are logged */
@@ -4243,7 +4239,7 @@ l2:
* Release the lmgr tuple lock, if we had it.
*/
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &(oldtup.t_self), lockmode);
pgstat_count_heap_update(relation, use_hot_update, newbuf != buffer);
@@ -4257,31 +4253,12 @@ l2:
heap_freetuple(heaptup);
}
- /*
- * If it is a HOT update, the update may still need to update summarized
- * indexes, lest we fail to update those summaries and get incorrect
- * results (for example, minmax bounds of the block may change with this
- * update).
- */
- if (use_hot_update)
- {
- if (summarized_update)
- *update_indexes = TU_Summarizing;
- else
- *update_indexes = TU_None;
- }
- else
- *update_indexes = TU_All;
-
if (old_key_tuple != NULL && old_key_copied)
heap_freetuple(old_key_tuple);
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
+ bms_free(rid_attrs);
+ bms_free(idx_attrs);
+ /* modified_idx_attrs is owned by the caller, don't free it */
return TM_Ok;
}
@@ -4454,28 +4431,113 @@ heap_attr_equals(TupleDesc tupdesc, int attrnum, Datum value1, Datum value2,
}
/*
- * Check which columns are being updated.
- *
- * Given an updated tuple, determine (and return into the output bitmapset),
- * from those listed as interesting, the set of columns that changed.
- *
- * has_external indicates if any of the unmodified attributes (from those
- * listed as interesting) of the old tuple is a member of external_cols and is
- * stored externally.
+ * HOT updates are possible when either: a) there are no modified indexed
+ * attributes, or b) the modified attributes are all on summarizing indexes.
+ * Later, in heap_update(), we can choose to perform a HOT update if there is
+ * space on the page for the new tuple and the following code has determined
+ * that HOT is allowed.
+ */
+bool
+HeapUpdateHotAllowable(Relation relation, const Bitmapset *modified_idx_attrs,
+ bool *summarized_only)
+{
+ bool hot_allowed;
+
+ /*
+ * Let's be optimistic and start off by assuming the best case, no indexes
+ * need updating and HOT is allowable.
+ */
+ hot_allowed = true;
+ *summarized_only = false;
+
+ /*
+ * Check for case (a); when there are no modified index attributes HOT is
+ * allowed.
+ */
+ if (bms_is_empty(modified_idx_attrs))
+ hot_allowed = true;
+ else
+ {
+ Bitmapset *sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+
+ /*
+ * At least one index attribute was modified, but is this case (b)
+ * where all the modified index attributes are only used by
+ * summarizing indexes? If that's the case we need to update those
+ * indexes, but this can be a HOT update.
+ */
+ if (bms_is_subset(modified_idx_attrs, sum_attrs))
+ {
+ hot_allowed = true;
+ *summarized_only = true;
+ }
+ else
+ {
+ /*
+ * Now we know that one or more indexed attribute were updated and
+ * that there was at least one of those attributes were referenced
+ * by a non-summarizing index. HOT is not allowed.
+ */
+ hot_allowed = false;
+ }
+
+ bms_free(sum_attrs);
+ }
+
+ return hot_allowed;
+}
+
+/*
+ * If we're not updating any "key" attributes, we can grab a weaker lock type.
+ * This allows for more concurrency when we are running simultaneously with
+ * foreign key checks.
+ */
+LockTupleMode
+HeapUpdateDetermineLockmode(Relation relation, const Bitmapset *modified_idx_attrs)
+{
+ LockTupleMode lockmode = LockTupleExclusive;
+
+ Bitmapset *key_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_KEY);
+
+ if (!bms_overlap(modified_idx_attrs, key_attrs))
+ lockmode = LockTupleNoKeyExclusive;
+
+ bms_free(key_attrs);
+
+ return lockmode;
+}
+
+/*
+ * Return a Bitmapset that contains the set of modified (changed) indexed
+ * attributes between oldtup and newtup.
*/
static Bitmapset *
-HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external)
+HeapUpdateModifiedIdxAttrs(Relation relation, HeapTuple oldtup, HeapTuple newtup)
{
int attidx;
- Bitmapset *modified = NULL;
+ Bitmapset *attrs,
+ *modified_idx_attrs = NULL;
TupleDesc tupdesc = RelationGetDescr(relation);
+ /* Get the set of all attributes across all indexes for this relation */
+ attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+
+ /* No indexed attributes, we're done */
+ if (bms_is_empty(attrs))
+ return NULL;
+
+ /*
+ * This heap update function is used outside the executor and so unlike
+ * heapam_tuple_update() where there is ResultRelInfo and EState to
+ * provide the concise set of attributes that might have been modified
+ * (via ExecGetAllUpdatedCols()) we simply check all indexed attributes to
+ * find the subset that changed value. That's the "modified indexed
+ * attributes" or "modified_idx_attrs".
+ */
attidx = -1;
- while ((attidx = bms_next_member(interesting_cols, attidx)) >= 0)
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
{
/* attidx is zero-based, attrnum is the normal attribute number */
AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
@@ -4491,7 +4553,7 @@ HeapDetermineColumnsInfo(Relation relation,
*/
if (attrnum == 0)
{
- modified = bms_add_member(modified, attidx);
+ modified_idx_attrs = bms_add_member(modified_idx_attrs, attidx);
continue;
}
@@ -4504,7 +4566,7 @@ HeapDetermineColumnsInfo(Relation relation,
{
if (attrnum != TableOidAttributeNumber)
{
- modified = bms_add_member(modified, attidx);
+ modified_idx_attrs = bms_add_member(modified_idx_attrs, attidx);
continue;
}
}
@@ -4520,29 +4582,12 @@ HeapDetermineColumnsInfo(Relation relation,
if (!heap_attr_equals(tupdesc, attrnum, value1,
value2, isnull1, isnull2))
- {
- modified = bms_add_member(modified, attidx);
- continue;
- }
-
- /*
- * No need to check attributes that can't be stored externally. Note
- * that system attributes can't be stored externally.
- */
- if (attrnum < 0 || isnull1 ||
- TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
- continue;
-
- /*
- * Check if the old tuple's attribute is stored externally and is a
- * member of external_cols.
- */
- if (VARATT_IS_EXTERNAL((varlena *) DatumGetPointer(value1)) &&
- bms_is_member(attidx, external_cols))
- *has_external = true;
+ modified_idx_attrs = bms_add_member(modified_idx_attrs, attidx);
}
- return modified;
+ bms_free(attrs);
+
+ return modified_idx_attrs;
}
/*
@@ -4554,17 +4599,109 @@ HeapDetermineColumnsInfo(Relation relation,
* via ereport().
*/
void
-simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup,
+simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TM_FailureData tmfd;
LockTupleMode lockmode;
+ TupleTableSlot *slot;
+ BufferHeapTupleTableSlot *bslot;
+ HeapTuple oldtup;
+ bool shouldFree = true;
+ Bitmapset *idx_attrs,
+ *modified_idx_attrs;
+ bool hot_allowed,
+ summarized_only;
+ Buffer buffer;
- result = heap_update(relation, otid, tup,
- GetCurrentCommandId(true), InvalidSnapshot,
- true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ Assert(ItemPointerIsValid(otid));
+
+ /*
+ * Fetch this bitmap of interesting attributes from relcache before
+ * obtaining a buffer lock because if we are doing an update on one of the
+ * relevant system catalogs we could deadlock if we try to fetch them
+ * later on. Relcache will return copies of each bitmap, so we need not
+ * worry about relcache flush happening midway through this operation.
+ */
+ idx_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+
+ INJECTION_POINT("heap_update-before-pin", NULL);
+
+ /*
+ * To update a heap tuple we need to find the set of modified indexed
+ * attributes ("modified_idx_attrs") so as to see if a HOT update is
+ * allowable or not. When updating heap tuples via execution of UPDATE
+ * statements this set is constructed before calling into the table AM's
+ * tuple_update() function by the function ExecUpdateModifiedIdxAttrs()
+ * which compares the old/new TupleTableSlots. However, here we have the
+ * old TID and the new tuple, not two TupleTableSlots, but we still need
+ * to construct a similar bitmap so as to be able to know if HOT updates
+ * are allowed or not. To do that we first have to fetch the old tuple
+ * itself. Because heapam_fetch_row_version() is static, we have to
+ * replicate that code here. This is a bit repetitive because
+ * heap_update() will again find and form the old HeapTuple from the old
+ * TID and in most cases the callers (ignoring extensions, always catalog
+ * tuple updates) already had the set of changed attributes (e.g. the
+ * "replaces" array), but for now this minor repetition of work is
+ * necessary.
+ */
+
+ slot = MakeTupleTableSlot(RelationGetDescr(relation), &TTSOpsBufferHeapTuple);
+ bslot = (BufferHeapTupleTableSlot *) slot;
+
+ /*
+ * Set the TID in the slot and then fetch the old tuple so we can examine
+ * it
+ */
+ bslot->base.tupdata.t_self = *otid;
+ if (!heap_fetch(relation, SnapshotAny, &bslot->base.tupdata, &buffer, false))
+ {
+ /*
+ * heap_update() checks for !ItemIdIsNormal(lp) and will return false
+ * in those cases.
+ */
+ Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
+
+ *update_indexes = TU_None;
+
+ /* modified_idx_attrs not yet initialized */
+ bms_free(idx_attrs);
+ ExecDropSingleTupleTableSlot(slot);
+
+ elog(ERROR, "tuple concurrently deleted");
+
+ return;
+ }
+
+ Assert(buffer != InvalidBuffer);
+
+ /* Store in slot, transferring existing pin */
+ ExecStorePinnedBufferHeapTuple(&bslot->base.tupdata, slot, buffer);
+ oldtup = ExecFetchSlotHeapTuple(slot, false, &shouldFree);
+
+ modified_idx_attrs = HeapUpdateModifiedIdxAttrs(relation, oldtup, tuple);
+ lockmode = HeapUpdateDetermineLockmode(relation, modified_idx_attrs);
+ hot_allowed = HeapUpdateHotAllowable(relation, modified_idx_attrs, &summarized_only);
+
+ result = heap_update(relation, otid, tuple, GetCurrentCommandId(true),
+ InvalidSnapshot, true /* wait for commit */ ,
+ &tmfd, lockmode, modified_idx_attrs, hot_allowed);
+
+ if (shouldFree)
+ heap_freetuple(oldtup);
+
+ ExecDropSingleTupleTableSlot(slot);
+ bms_free(idx_attrs);
+
+ /*
+ * Decide whether new index entries are needed for the tuple
+ *
+ * If the update is not HOT, we must update all indexes. If the update is
+ * HOT, it could be that we updated summarized columns, so we either
+ * update only summarized indexes, or none at all.
+ */
+ *update_indexes = TU_None;
switch (result)
{
case TM_SelfModified:
@@ -4574,6 +4711,10 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
case TM_Ok:
/* done successfully */
+ if (!HeapTupleIsHeapOnly(tuple))
+ *update_indexes = TU_All;
+ else if (summarized_only)
+ *update_indexes = TU_Summarizing;
break;
case TM_Updated:
@@ -4590,7 +4731,6 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
}
}
-
/*
* Return the MultiXactStatus corresponding to the given tuple lock mode.
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 3ff36f59bf8..bbdb732c001 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,7 +27,6 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
-#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -44,6 +43,7 @@
#include "storage/procarray.h"
#include "storage/smgr.h"
#include "utils/builtins.h"
+#include "utils/injection_point.h"
#include "utils/rel.h"
static void reform_and_rewrite_tuple(HeapTuple tuple,
@@ -316,19 +316,26 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd,
- LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
+ bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
+ const Bitmapset *modified_idx_attrs, TU_UpdateIndexes *update_indexes)
{
bool shouldFree = true;
HeapTuple tuple = ExecFetchSlotHeapTuple(slot, true, &shouldFree);
+ bool hot_allowed;
+ bool summarized_only;
TM_Result result;
+ Assert(ItemPointerIsValid(otid));
+
+ hot_allowed = HeapUpdateHotAllowable(relation, modified_idx_attrs, &summarized_only);
+ *lockmode = HeapUpdateDetermineLockmode(relation, modified_idx_attrs);
+
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
- tmfd, lockmode, update_indexes);
+ tmfd, *lockmode, modified_idx_attrs, hot_allowed);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
@@ -341,16 +348,17 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
* HOT, it could be that we updated summarized columns, so we either
* update only summarized indexes, or none at all.
*/
- if (result != TM_Ok)
+ *update_indexes = TU_None;
+ if (result == TM_Ok)
{
- Assert(*update_indexes == TU_None);
- *update_indexes = TU_None;
+ if (HeapTupleIsHeapOnly(tuple))
+ {
+ if (summarized_only)
+ *update_indexes = TU_Summarizing;
+ }
+ else
+ *update_indexes = TU_All;
}
- else if (!HeapTupleIsHeapOnly(tuple))
- Assert(*update_indexes == TU_All);
- else
- Assert((*update_indexes == TU_Summarizing) ||
- (*update_indexes == TU_None));
if (shouldFree)
pfree(tuple);
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index dfda1af412e..9ba72d51dfa 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -359,6 +359,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
+ const Bitmapset *modified_idx_attrs,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -369,7 +370,9 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
GetCurrentCommandId(true),
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, &lockmode,
+ modified_idx_attrs,
+ update_indexes);
switch (result)
{
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index 2497ee7edc5..74a7379186b 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -33,6 +33,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
#include "utils/typcache.h"
@@ -906,6 +907,7 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
bool skip_tuple = false;
Relation rel = resultRelInfo->ri_RelationDesc;
ItemPointer tid = &(searchslot->tts_tid);
+ Bitmapset *modified_idx_attrs;
/*
* We support only non-system tables, with
@@ -944,8 +946,13 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ modified_idx_attrs = ExecUpdateModifiedIdxAttrs(resultRelInfo,
+ estate, searchslot, slot);
+
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
- &update_indexes);
+ modified_idx_attrs, &update_indexes);
+ bms_free(modified_idx_attrs);
+
conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 327c27abff9..cca834a7359 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -17,6 +17,7 @@
* ExecModifyTable - retrieve the next tuple from the node
* ExecEndModifyTable - shut down the ModifyTable node
* ExecReScanModifyTable - rescan the ModifyTable node
+ * ExecUpdateModifiedIdxAttrs - find set of updated indexed columns
*
* NOTES
* The ModifyTable node receives input from its outerPlan, which is
@@ -54,6 +55,7 @@
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/tupdesc.h"
#include "access/xact.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
@@ -188,6 +190,68 @@ static TupleTableSlot *ExecMergeNotMatched(ModifyTableContext *context,
ResultRelInfo *resultRelInfo,
bool canSetTag);
+/*
+ * ExecUpdateModifiedIdxAttrs
+ *
+ * Find the set of attributes referenced by this relation and used in this
+ * UPDATE that now differ in value. This is done by reviewing slot datum that
+ * are in the UPDATE statment and are known to be referenced by at least one
+ * index in some way. This set is called the "modified indexed attributes" or
+ * "modified_idx_attrs". An overlap of a single index's attributes and this "mix" set
+ * signals that the attributes in the new_tts used to form the index datum have
+ * changed.
+ *
+ * Return a Bitmapset that contains the set of modified (changed) indexed
+ * attributes between oldtup and newtup.
+ *
+ * NOTE: There is a similar function called HeapUpdateModifiedIdxAttrs() that operates
+ * on the old TID and new HeapTuple rather than the old/new TupleTableSlots as
+ * this function does. These two functions should mirror one another until
+ * someday when catalog tuple updates track their changes avoiding the need to
+ * re-discover them in simple_heap_update().
+ */
+Bitmapset *
+ExecUpdateModifiedIdxAttrs(ResultRelInfo *resultRelInfo,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts)
+{
+ Relation relation = resultRelInfo->ri_RelationDesc;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ Bitmapset *attrs,
+ *modified_idx_attrs = NULL;
+
+ /* If no indexes, we're done */
+ if (resultRelInfo->ri_NumIndices == 0)
+ return NULL;
+
+ /*
+ * Get the set of all attributes across all indexes for this relation from
+ * the relcache, it returns us a copy of the bitmap so we can modify it.
+ */
+ attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+
+ /*
+ * Fetch the set of attributes explicity SET in the UPDATE statement or
+ * set by a before row trigger (even if not mentioned in the SQL) from the
+ * executor state and then find the intersection with the indexed
+ * attributes. Attributes that are SET might not change value, so we have
+ * to examine them for changes.
+ */
+ attrs = bms_int_members(attrs, ExecGetAllUpdatedCols(resultRelInfo, estate));
+
+ /*
+ * When there are indexed attributes mentioned in the UPDATE then we need
+ * to find the subset that changed value. That's the "modified indexed
+ * attributes" or "modified_idx_attrs".
+ */
+ if (!bms_is_empty(attrs))
+ modified_idx_attrs = ExecCompareSlotAttrs(tupdesc, attrs, old_tts, new_tts);
+
+ bms_free(attrs);
+
+ return modified_idx_attrs;
+}
/*
* Verify that the tuples to be produced by INSERT match the
@@ -2195,14 +2259,17 @@ ExecUpdatePrepareSlot(ResultRelInfo *resultRelInfo,
*/
static TM_Result
ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
- ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *slot,
- bool canSetTag, UpdateContext *updateCxt)
+ ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *oldSlot,
+ TupleTableSlot *slot, bool canSetTag, UpdateContext *updateCxt)
{
EState *estate = context->estate;
Relation resultRelationDesc = resultRelInfo->ri_RelationDesc;
bool partition_constraint_failed;
TM_Result result;
+ /* The set of modified indexed attributes that trigger new index entries */
+ Bitmapset *modified_idx_attrs = NULL;
+
updateCxt->crossPartUpdate = false;
/*
@@ -2319,7 +2386,16 @@ lreplace:
ExecConstraints(resultRelInfo, slot, estate);
/*
- * replace the heap tuple
+ * Next up we need to find out the set of indexed attributes that have
+ * changed in value and should trigger a new index tuple. We could start
+ * with the set of updated columns via ExecGetUpdatedCols(), but if we do
+ * we will overlook attributes directly modified by heap_modify_tuple()
+ * which are not known to ExecGetUpdatedCols().
+ */
+ modified_idx_attrs = ExecUpdateModifiedIdxAttrs(resultRelInfo, estate, oldSlot, slot);
+
+ /*
+ * Call into the table AM to update the heap tuple.
*
* Note: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
* the row to be updated is visible to that snapshot, and throw a
@@ -2333,6 +2409,7 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
+ modified_idx_attrs,
&updateCxt->updateIndexes);
return result;
@@ -2555,8 +2632,8 @@ ExecUpdate(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
*/
redo_act:
lockedtid = *tupleid;
- result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, slot,
- canSetTag, &updateCxt);
+ result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, oldSlot,
+ slot, canSetTag, &updateCxt);
/*
* If ExecUpdateAct reports that a cross-partition update was done,
@@ -3406,8 +3483,8 @@ lmerge_matched:
Assert(oldtuple == NULL);
result = ExecUpdateAct(context, resultRelInfo, tupleid,
- NULL, newslot, canSetTag,
- &updateCxt);
+ NULL, resultRelInfo->ri_oldTupleSlot,
+ newslot, canSetTag, &updateCxt);
/*
* As in ExecUpdate(), if ExecUpdateAct() reports that a
@@ -4544,7 +4621,7 @@ ExecModifyTable(PlanState *pstate)
* For UPDATE/DELETE/MERGE, fetch the row identity info for the tuple
* to be updated/deleted/merged. For a heap relation, that's a TID;
* otherwise we may have a wholerow junk attr that carries the old
- * tuple in toto. Keep this in step with the part of
+ * tuple in total. Keep this in step with the part of
* ExecInitModifyTable that sets up ri_RowIdAttNo.
*/
if (operation == CMD_UPDATE || operation == CMD_DELETE ||
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a1c88c6b1b6..4303108565f 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -2475,8 +2475,8 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
bms_free(relation->rd_keyattr);
bms_free(relation->rd_pkattr);
bms_free(relation->rd_idattr);
- bms_free(relation->rd_hotblockingattr);
bms_free(relation->rd_summarizedattr);
+ bms_free(relation->rd_indexedattr);
if (relation->rd_pubdesc)
pfree(relation->rd_pubdesc);
if (relation->rd_options)
@@ -5276,8 +5276,8 @@ RelationGetIndexPredicate(Relation relation)
* (beware: even if PK is deferrable!)
* INDEX_ATTR_BITMAP_IDENTITY_KEY Columns in the table's replica identity
* index (empty if FULL)
- * INDEX_ATTR_BITMAP_HOT_BLOCKING Columns that block updates from being HOT
- * INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
+ * INDEX_ATTR_BITMAP_SUMMARIZED Columns only included in summarizing indexes
+ * INDEX_ATTR_BITMAP_INDEXED Columns referenced by indexes
*
* Attribute numbers are offset by FirstLowInvalidHeapAttributeNumber so that
* we can include system attributes (e.g., OID) in the bitmap representation.
@@ -5300,8 +5300,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
Bitmapset *uindexattrs; /* columns in unique indexes */
Bitmapset *pkindexattrs; /* columns in the primary index */
Bitmapset *idindexattrs; /* columns in the replica identity */
- Bitmapset *hotblockingattrs; /* columns with HOT blocking indexes */
- Bitmapset *summarizedattrs; /* columns with summarizing indexes */
+ Bitmapset *summarizedattrs; /* columns only in summarizing indexes */
+ Bitmapset *indexedattrs; /* columns referenced by indexes */
List *indexoidlist;
List *newindexoidlist;
Oid relpkindex;
@@ -5320,10 +5320,10 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
return bms_copy(relation->rd_pkattr);
case INDEX_ATTR_BITMAP_IDENTITY_KEY:
return bms_copy(relation->rd_idattr);
- case INDEX_ATTR_BITMAP_HOT_BLOCKING:
- return bms_copy(relation->rd_hotblockingattr);
case INDEX_ATTR_BITMAP_SUMMARIZED:
return bms_copy(relation->rd_summarizedattr);
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return bms_copy(relation->rd_indexedattr);
default:
elog(ERROR, "unknown attrKind %u", attrKind);
}
@@ -5366,8 +5366,8 @@ restart:
uindexattrs = NULL;
pkindexattrs = NULL;
idindexattrs = NULL;
- hotblockingattrs = NULL;
summarizedattrs = NULL;
+ indexedattrs = NULL;
foreach(l, indexoidlist)
{
Oid indexOid = lfirst_oid(l);
@@ -5426,7 +5426,7 @@ restart:
if (indexDesc->rd_indam->amsummarizing)
attrs = &summarizedattrs;
else
- attrs = &hotblockingattrs;
+ attrs = &indexedattrs;
/* Collect simple attribute references */
for (i = 0; i < indexDesc->rd_index->indnatts; i++)
@@ -5435,9 +5435,9 @@ restart:
/*
* Since we have covering indexes with non-key columns, we must
- * handle them accurately here. non-key columns must be added into
- * hotblockingattrs or summarizedattrs, since they are in index,
- * and update shouldn't miss them.
+ * handle them accurately here. Non-key columns must be added into
+ * indexedattrs or summarizedattrs, since they are in index, and
+ * update shouldn't miss them.
*
* Summarizing indexes do not block HOT, but do need to be updated
* when the column value changes, thus require a separate
@@ -5498,12 +5498,20 @@ restart:
bms_free(uindexattrs);
bms_free(pkindexattrs);
bms_free(idindexattrs);
- bms_free(hotblockingattrs);
bms_free(summarizedattrs);
+ bms_free(indexedattrs);
goto restart;
}
+ /*
+ * Record what attributes are only referenced by summarizing indexes. Then
+ * add that into the other indexed attributes to track all referenced
+ * attributes.
+ */
+ summarizedattrs = bms_del_members(summarizedattrs, indexedattrs);
+ indexedattrs = bms_add_members(indexedattrs, summarizedattrs);
+
/* Don't leak the old values of these bitmaps, if any */
relation->rd_attrsvalid = false;
bms_free(relation->rd_keyattr);
@@ -5512,10 +5520,10 @@ restart:
relation->rd_pkattr = NULL;
bms_free(relation->rd_idattr);
relation->rd_idattr = NULL;
- bms_free(relation->rd_hotblockingattr);
- relation->rd_hotblockingattr = NULL;
bms_free(relation->rd_summarizedattr);
relation->rd_summarizedattr = NULL;
+ bms_free(relation->rd_indexedattr);
+ relation->rd_indexedattr = NULL;
/*
* Now save copies of the bitmaps in the relcache entry. We intentionally
@@ -5528,8 +5536,8 @@ restart:
relation->rd_keyattr = bms_copy(uindexattrs);
relation->rd_pkattr = bms_copy(pkindexattrs);
relation->rd_idattr = bms_copy(idindexattrs);
- relation->rd_hotblockingattr = bms_copy(hotblockingattrs);
relation->rd_summarizedattr = bms_copy(summarizedattrs);
+ relation->rd_indexedattr = bms_copy(indexedattrs);
relation->rd_attrsvalid = true;
MemoryContextSwitchTo(oldcxt);
@@ -5542,10 +5550,10 @@ restart:
return pkindexattrs;
case INDEX_ATTR_BITMAP_IDENTITY_KEY:
return idindexattrs;
- case INDEX_ATTR_BITMAP_HOT_BLOCKING:
- return hotblockingattrs;
case INDEX_ATTR_BITMAP_SUMMARIZED:
return summarizedattrs;
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return indexedattrs;
default:
elog(ERROR, "unknown attrKind %u", attrKind);
return NULL;
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 24a27cc043a..909b4fad7c2 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -366,10 +366,9 @@ extern TM_Result heap_delete(Relation relation, const ItemPointerData *tid,
extern void heap_finish_speculative(Relation relation, const ItemPointerData *tid);
extern void heap_abort_speculative(Relation relation, const ItemPointerData *tid);
extern TM_Result heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, const LockTupleMode lockmode,
+ const Bitmapset *modified_idx_attrs, const bool hot_allowed);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -431,6 +430,12 @@ extern void log_heap_prune_and_freeze(Relation relation, Buffer buffer,
OffsetNumber *dead, int ndead,
OffsetNumber *unused, int nunused);
+/* in heap/heapam.c */
+extern bool HeapUpdateHotAllowable(Relation relation, const Bitmapset *modified_idx_attrs,
+ bool *summarized_only);
+extern LockTupleMode HeapUpdateDetermineLockmode(Relation relation,
+ const Bitmapset *modified_idx_attrs);
+
/* in heap/vacuumlazy.c */
extern void heap_vacuum_rel(Relation rel,
const VacuumParams params, BufferAccessStrategy bstrategy);
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 06084752245..8ec20dcfc11 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -549,6 +549,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
+ const Bitmapset *modified_idx_attrs,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1523,12 +1524,12 @@ static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ const Bitmapset *modified_idx_attrs, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
- wait, tmfd,
- lockmode, update_indexes);
+ wait, tmfd, lockmode,
+ modified_idx_attrs, update_indexes);
}
/*
@@ -2009,6 +2010,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
+ const Bitmapset *modified_idx_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 5dcfaa2027f..24ec43c35a9 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -808,5 +808,9 @@ extern ResultRelInfo *ExecLookupResultRelByOid(ModifyTableState *node,
Oid resultoid,
bool missing_ok,
bool update_cache);
+extern Bitmapset *ExecUpdateModifiedIdxAttrs(ResultRelInfo *relinfo,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
#endif /* EXECUTOR_H */
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 236830f6b93..10e5e9044ee 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -162,8 +162,8 @@ typedef struct RelationData
Bitmapset *rd_keyattr; /* cols that can be ref'd by foreign keys */
Bitmapset *rd_pkattr; /* cols included in primary key */
Bitmapset *rd_idattr; /* included in replica identity index */
- Bitmapset *rd_hotblockingattr; /* cols blocking HOT update */
Bitmapset *rd_summarizedattr; /* cols indexed by summarizing indexes */
+ Bitmapset *rd_indexedattr; /* all cols referenced by indexes */
PublicationDesc *rd_pubdesc; /* publication descriptor, or NULL */
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2700224939a..57b46ee54e5 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -69,8 +69,8 @@ typedef enum IndexAttrBitmapKind
INDEX_ATTR_BITMAP_KEY,
INDEX_ATTR_BITMAP_PRIMARY_KEY,
INDEX_ATTR_BITMAP_IDENTITY_KEY,
- INDEX_ATTR_BITMAP_HOT_BLOCKING,
INDEX_ATTR_BITMAP_SUMMARIZED,
+ INDEX_ATTR_BITMAP_INDEXED,
} IndexAttrBitmapKind;
extern Bitmapset *RelationGetIndexAttrBitmap(Relation relation,
diff --git a/src/test/regress/expected/generated_virtual.out b/src/test/regress/expected/generated_virtual.out
index 6dab60c937b..7ebb7890d96 100644
--- a/src/test/regress/expected/generated_virtual.out
+++ b/src/test/regress/expected/generated_virtual.out
@@ -287,7 +287,7 @@ DETAIL: Column "b" is a generated column.
INSERT INTO gtest1v VALUES (8, DEFAULT), (9, DEFAULT); -- error
ERROR: cannot insert a non-DEFAULT value into column "b"
DETAIL: Column "b" is a generated column.
-SELECT * FROM gtest1v;
+SELECT * FROM gtest1v ORDER BY a;
a | b
---+----
3 | 6
diff --git a/src/test/regress/expected/triggers.out b/src/test/regress/expected/triggers.out
index 98dee63b50a..ef98fd0cccf 100644
--- a/src/test/regress/expected/triggers.out
+++ b/src/test/regress/expected/triggers.out
@@ -959,16 +959,24 @@ NOTICE: main_view BEFORE UPDATE STATEMENT (before_view_upd_stmt)
NOTICE: main_view AFTER UPDATE STATEMENT (after_view_upd_stmt)
UPDATE 0
-- Delete from view using trigger
-DELETE FROM main_view WHERE a IN (20,21);
+DELETE FROM main_view WHERE a = 20 AND b = 31;
NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
-NOTICE: OLD: (21,10)
-NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
NOTICE: OLD: (20,31)
+NOTICE: main_view AFTER DELETE STATEMENT (after_view_del_stmt)
+DELETE 1
+DELETE FROM main_view WHERE a = 21 AND b = 10;
+NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
+NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
+NOTICE: OLD: (21,10)
+NOTICE: main_view AFTER DELETE STATEMENT (after_view_del_stmt)
+DELETE 1
+DELETE FROM main_view WHERE a = 21 AND b = 32;
+NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
NOTICE: OLD: (21,32)
NOTICE: main_view AFTER DELETE STATEMENT (after_view_del_stmt)
-DELETE 3
+DELETE 1
DELETE FROM main_view WHERE a = 31 RETURNING a, b;
NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
diff --git a/src/test/regress/expected/updatable_views.out b/src/test/regress/expected/updatable_views.out
index 9cea538b8e8..4877a1ddce9 100644
--- a/src/test/regress/expected/updatable_views.out
+++ b/src/test/regress/expected/updatable_views.out
@@ -372,15 +372,15 @@ INSERT INTO rw_view16 (a, b) VALUES (3, 'Row 3'); -- should be OK
UPDATE rw_view16 SET a=3, aa=-3 WHERE a=3; -- should fail
ERROR: multiple assignments to same column "a"
UPDATE rw_view16 SET aa=-3 WHERE a=3; -- should be OK
-SELECT * FROM base_tbl;
+SELECT * FROM base_tbl ORDER BY a;
a | b
----+--------
+ -3 | Row 3
-2 | Row -2
-1 | Row -1
0 | Row 0
1 | Row 1
2 | Row 2
- -3 | Row 3
(6 rows)
DELETE FROM rw_view16 WHERE a=-3; -- should be OK
diff --git a/src/test/regress/sql/generated_virtual.sql b/src/test/regress/sql/generated_virtual.sql
index e750866d2d8..877152d6d69 100644
--- a/src/test/regress/sql/generated_virtual.sql
+++ b/src/test/regress/sql/generated_virtual.sql
@@ -127,7 +127,7 @@ ALTER VIEW gtest1v ALTER COLUMN b SET DEFAULT 100;
INSERT INTO gtest1v VALUES (8, DEFAULT); -- error
INSERT INTO gtest1v VALUES (8, DEFAULT), (9, DEFAULT); -- error
-SELECT * FROM gtest1v;
+SELECT * FROM gtest1v ORDER BY a;
DELETE FROM gtest1v WHERE a >= 5;
DROP VIEW gtest1v;
diff --git a/src/test/regress/sql/triggers.sql b/src/test/regress/sql/triggers.sql
index ea39817ee3d..6ceb61608ae 100644
--- a/src/test/regress/sql/triggers.sql
+++ b/src/test/regress/sql/triggers.sql
@@ -660,7 +660,9 @@ UPDATE main_view SET b = 32 WHERE a = 21 AND b = 31 RETURNING a, b;
UPDATE main_view SET b = 0 WHERE false;
-- Delete from view using trigger
-DELETE FROM main_view WHERE a IN (20,21);
+DELETE FROM main_view WHERE a = 20 AND b = 31;
+DELETE FROM main_view WHERE a = 21 AND b = 10;
+DELETE FROM main_view WHERE a = 21 AND b = 32;
DELETE FROM main_view WHERE a = 31 RETURNING a, b;
\set QUIET true
diff --git a/src/test/regress/sql/updatable_views.sql b/src/test/regress/sql/updatable_views.sql
index 1635adde2d4..160e7799715 100644
--- a/src/test/regress/sql/updatable_views.sql
+++ b/src/test/regress/sql/updatable_views.sql
@@ -125,7 +125,7 @@ INSERT INTO rw_view16 VALUES (3, 'Row 3', 3); -- should fail
INSERT INTO rw_view16 (a, b) VALUES (3, 'Row 3'); -- should be OK
UPDATE rw_view16 SET a=3, aa=-3 WHERE a=3; -- should fail
UPDATE rw_view16 SET aa=-3 WHERE a=3; -- should be OK
-SELECT * FROM base_tbl;
+SELECT * FROM base_tbl ORDER BY a;
DELETE FROM rw_view16 WHERE a=-3; -- should be OK
-- Read-only views
INSERT INTO ro_view17 VALUES (3, 'ROW 3');
--
2.51.2
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2026-03-02 19:08 Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-11 15:51 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
@ 2026-03-12 20:33 ` Nathan Bossart <[email protected]>
2026-03-12 21:31 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Nathan Bossart @ 2026-03-12 20:33 UTC (permalink / raw)
To: Greg Burd <[email protected]>; +Cc: pgsql-hackers; Jeff Davis <[email protected]>
On Wed, Mar 11, 2026 at 11:51:03AM -0400, Greg Burd wrote:
> 0002 - This patch plugs a hole (bug?) in ExecGetAllUpdatedCols() which is
> triggered by an existing test in tsearch.sql and the
> tsvector_update_trigger(). That trigger uses heap_modify_tuple() to
> change an indexed attribute that is not discovered by
> ExecGetAllUpdatedCols(), which seems odd to me at best and at worst wrong
> (or even a potential security issue). This patch finds and adds columns
> that are updated into the Bitmapset returned by ExecGetAllUpdatedCols().
> The patch includes a helper function ExecCompareSlotAttrs() that will be
> used in follow-on patches as well.
I just looked at this one for now.
> The net is that the functions like HeapDetermineColumnsInfo() have to
> scan all indexed attributes for changes rather than being able to first
> reduce the indexed set by intersecting it with the set of attributes
> known to be potentially updated.
I noticed the patch doesn't update HeapDetermineColumnsInfo() accordingly.
Is that intended?
> This commit introduces ExecCompareSlotAttrs() as a utility function to
> identify those attributes that have changed. It compares a subset of
> attributes between two TupleTableSlots and returns a Bitmapset of
> attributes that differ.
Hm. Most of this new function looks duplicated from
HeapDetermineColumnsInfo(), so IIUC this commit effectively adds another
scan through all the attributes. Does this produce noticeably more
overhead?
> It would be nice to integrate this into HeapDetermineColumnsInfo(),
> however it would be a layering violation given that it is within
> heap_update().
It'd be good to understand whether the current behavior is intentional or
just a happy accident. I found commit 2fd8685e7f, which looks like it was
intended as a prerequisite for the WARM feature (which I don't think was
ever committed). And it seems to have scanned through all indexed columns
when HOT was first introduced in commit 282d2a03dd.
I'm also curious whether anything else could modify columns that won't be
discovered by ExecGetAllUpdatedCols(). Having HeapDetermineColumnsInfo()
scan everything seems like a defense against such things, which is perhaps
why you've left it unchanged in the patch. I haven't looked into 0003 yet.
Is 0002 a prerequisite for that or a separate fix?
--
nathan
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2026-03-02 19:08 Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-11 15:51 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-12 20:33 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
@ 2026-03-12 21:31 ` Greg Burd <[email protected]>
2026-03-15 21:11 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Greg Burd @ 2026-03-12 21:31 UTC (permalink / raw)
To: Nathan Bossart <[email protected]>; +Cc: pgsql-hackers; Jeff Davis <[email protected]>
On Thu, Mar 12, 2026, at 4:33 PM, Nathan Bossart wrote:
> On Wed, Mar 11, 2026 at 11:51:03AM -0400, Greg Burd wrote:
>> 0002 - This patch plugs a hole (bug?) in ExecGetAllUpdatedCols() which is
>> triggered by an existing test in tsearch.sql and the
>> tsvector_update_trigger(). That trigger uses heap_modify_tuple() to
>> change an indexed attribute that is not discovered by
>> ExecGetAllUpdatedCols(), which seems odd to me at best and at worst wrong
>> (or even a potential security issue). This patch finds and adds columns
>> that are updated into the Bitmapset returned by ExecGetAllUpdatedCols().
>> The patch includes a helper function ExecCompareSlotAttrs() that will be
>> used in follow-on patches as well.
>
> I just looked at this one for now.
Hey Nathan!
Thanks for taking the time to review 0002.
>> The net is that the functions like HeapDetermineColumnsInfo() have to
>> scan all indexed attributes for changes rather than being able to first
>> reduce the indexed set by intersecting it with the set of attributes
>> known to be potentially updated.
>
> I noticed the patch doesn't update HeapDetermineColumnsInfo() accordingly.
> Is that intended?
Yes, that is intended. The 0002 patch is bug fix that I'd hidden along with what is now 0003, I pulled it out for clarity and to discuss independent of the other changes.
>> This commit introduces ExecCompareSlotAttrs() as a utility function to
>> identify those attributes that have changed. It compares a subset of
>> attributes between two TupleTableSlots and returns a Bitmapset of
>> attributes that differ.
>
> Hm. Most of this new function looks duplicated from
> HeapDetermineColumnsInfo(), so IIUC this commit effectively adds another
> scan through all the attributes. Does this produce noticeably more
> overhead?
Yes, it appears similar to that for a reason but it differs in one key way. It compares TupleTableSlots, not HeapTuples.
The commit doesn't add another scan, the new code only scans the attributes that ExecGetAllUpdatedCols() didn't pick up earlier and have cached for us at this point. The intersection between that set and what is indexed is almost always the NULL set because most UPDATEs don't invoke functions via triggers that modify indexed columns using heap_modify_tuple() directly. But, notably there is the case in tsearch.sql that does.
This introduces almost no net new overhead and when it does in fact do some work it's doing no more than what was done before in HeapDetermineColumnsInfo().
>> It would be nice to integrate this into HeapDetermineColumnsInfo(),
>> however it would be a layering violation given that it is within
>> heap_update().
>
> It'd be good to understand whether the current behavior is intentional or
> just a happy accident. I found commit 2fd8685e7f, which looks like it was
> intended as a prerequisite for the WARM feature (which I don't think was
> ever committed). And it seems to have scanned through all indexed columns
> when HOT was first introduced in commit 282d2a03dd.
Hard to tell if it was accidental or intentional, more digging required, but I'd bet that others poking in this area noticed the test failure and didn't connect the dots fully and just assumed best practice was to scan all indexed columns, even ones that could not have been updated at all.
Honestly, if we wrote this section from scratch again today I'm better it'd be closer to where my patch takes us than not.
> I'm also curious whether anything else could modify columns that won't be
> discovered by ExecGetAllUpdatedCols(). Having HeapDetermineColumnsInfo()
> scan everything seems like a defense against such things, which is perhaps
> why you've left it unchanged in the patch. I haven't looked into 0003 yet.
> Is 0002 a prerequisite for that or a separate fix?
Other than the heap_modify_tuple() calls I don't know of something that allows for direct changes but that doesn't matter, 0002 will scan and pick up those attributes even if we introduce a new modification path in the future (as intended).
HeapDetermineColumnsInfo() can't call ExecGetAllUpdatedCols() because that function needs resultRelInfo/EState both not available inside heap (table AM) calls. Also, the new helper compares TTS, not HeapTuples, which is what we have in heapam_tuple_update(), so not an option
0002 is a both a bug fix (IMO) and a pre-req for 0003 because in the next patch we use the new ExecCompareSlotAttrs() function from within the executor ahead of calling into ExecUpdate().
> --
> nathan
Thanks for your time and comments, let me know if you have more. :)
best.
-greg
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2026-03-02 19:08 Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-11 15:51 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-12 20:33 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2026-03-12 21:31 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
@ 2026-03-15 21:11 ` Jeff Davis <[email protected]>
2026-03-16 16:23 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Jeff Davis @ 2026-03-15 21:11 UTC (permalink / raw)
To: Greg Burd <[email protected]>; Nathan Bossart <[email protected]>; +Cc: pgsql-hackers
On Thu, 2026-03-12 at 17:31 -0400, Greg Burd wrote:
> Other than the heap_modify_tuple() calls I don't know of something
> that allows for direct changes but that doesn't matter, 0002 will
> scan and pick up those attributes even if we introduce a new
> modification path in the future (as intended).
Why do extra work in ExecBRUpdateTriggers() to eliminate the false
negative case if we don't rely on it anyway? If we do need to rely on
it in subsequent patches, then we need to be sure, right?
I guess I'm confused about whether 0002 is introducing a new guarantee
or if it's just a convenient place to eliminate one source of false
negatives.
Regards,
Jeff Davis
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2026-03-02 19:08 Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-11 15:51 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-12 20:33 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2026-03-12 21:31 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-15 21:11 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
@ 2026-03-16 16:23 ` Greg Burd <[email protected]>
2026-03-16 17:29 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2026-03-16 17:55 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
0 siblings, 2 replies; 37+ messages in thread
From: Greg Burd @ 2026-03-16 16:23 UTC (permalink / raw)
To: Jeff Davis <[email protected]>; Nathan Bossart <[email protected]>; +Cc: pgsql-hackers
On Sun, Mar 15, 2026, at 5:11 PM, Jeff Davis wrote:
> On Thu, 2026-03-12 at 17:31 -0400, Greg Burd wrote:
>> Other than the heap_modify_tuple() calls I don't know of something
>> that allows for direct changes but that doesn't matter, 0002 will
>> scan and pick up those attributes even if we introduce a new
>> modification path in the future (as intended).
Hello Jeff, thanks for taking a look! :)
> Why do extra work in ExecBRUpdateTriggers() to eliminate the false
> negative case if we don't rely on it anyway? If we do need to rely on
> it in subsequent patches, then we need to be sure, right?
Later commits do currently rely on it, ExecUpdateModifiedIdxAttrs() uses it in the next commit (0003) to avoid reviewing indexed attributes that could not have possibly changed. Imagine a table with a lot of indexes where updates only modify one or two at a time. Why are we testing indexed attributes for changes in HeapDeterminColumnsInfo() that couldn't have changed? The answer is that a) HeapDeterminColumnsInfo() lives in heap, not the executor (see patch 0003) so it has no ability to call ExecGetAllUpdatedCols(), and b) the set returned by ExecGetAllUpdatedCols() is sometimes incomplete.
I see (a) as something I fix in patch 0003 and (b) as an oversight (or bug). I'll also argue that the overhead of checking for additional attributes in ExecBRUpdateTriggers() vs the overhead of checking all indexed attributes in HeapDeterminColumnsInfo() is net zero once patch 0003 is applied.
The argument to keep 0002 is both performance as much as correctness. After 0002 and 0003 ExecUpdateModifiedIdxAttrs() replaces HeapDeterminColumnsInfo() and doesn't have to scan all indexed attributes anymore. Relations with lots of indexed attributes but update patterns that only focus on subsets of those attributes will benefit as there will be fewer memcmp() calls when comparing datums.
What do we "need to be sure" of? That ExecGetAllUpdatedCols() not really contains all attributes that its name implies? I think it now does that after 0002, do you disagree?
> I guess I'm confused about whether 0002 is introducing a new guarantee
> or if it's just a convenient place to eliminate one source of false
> negatives.
I think it is a new guarantee that was implied before now but not required until 0003. I think this change reduces overhead and helps to avoid some future security feature that depends on ExecGetAllUpdatedCols() to provide that guarantee.
Does that make sense?
> Regards,
> Jeff Davis
best.
-greg
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2026-03-02 19:08 Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-11 15:51 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-12 20:33 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2026-03-12 21:31 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-15 21:11 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-16 16:23 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
@ 2026-03-16 17:29 ` Nathan Bossart <[email protected]>
1 sibling, 0 replies; 37+ messages in thread
From: Nathan Bossart @ 2026-03-16 17:29 UTC (permalink / raw)
To: Greg Burd <[email protected]>; +Cc: Jeff Davis <[email protected]>; pgsql-hackers
On Mon, Mar 16, 2026 at 12:23:04PM -0400, Greg Burd wrote:
> On Sun, Mar 15, 2026, at 5:11 PM, Jeff Davis wrote:
>> Why do extra work in ExecBRUpdateTriggers() to eliminate the false
>> negative case if we don't rely on it anyway? If we do need to rely on
>> it in subsequent patches, then we need to be sure, right?
>
> [...]
>
> What do we "need to be sure" of? That ExecGetAllUpdatedCols() not really
> contains all attributes that its name implies? I think it now does that
> after 0002, do you disagree?
I'm admittedly still digging into the details, but the main question on my
mind is whether there are other cases lurking that our in-tree tests aren't
catching or that only exist in extensions. Will there be some sort of
check or assertion to catch those?
--
nathan
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2026-03-02 19:08 Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-11 15:51 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-12 20:33 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2026-03-12 21:31 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-15 21:11 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-16 16:23 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
@ 2026-03-16 17:55 ` Jeff Davis <[email protected]>
2026-03-17 15:22 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2026-03-17 16:38 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
1 sibling, 2 replies; 37+ messages in thread
From: Jeff Davis @ 2026-03-16 17:55 UTC (permalink / raw)
To: Greg Burd <[email protected]>; Nathan Bossart <[email protected]>; +Cc: pgsql-hackers
On Mon, 2026-03-16 at 12:23 -0400, Greg Burd wrote:
> Hello Jeff, thanks for taking a look! :)
Hi, thank you for working on this problem!
> > Why do extra work in ExecBRUpdateTriggers() to eliminate the false
> > negative case if we don't rely on it anyway? If we do need to rely
> > on
> > it in subsequent patches, then we need to be sure, right?
>
> Later commits do currently rely on it, ExecUpdateModifiedIdxAttrs()
> uses it in the next commit (0003) to avoid reviewing indexed
> attributes that could not have possibly changed.
OK. The first half of the commit message for 0002 is slightly confusing
because it's referring to pre-existing behavior, behavior changed by
the commit, and also future work. It might help to clarify the tenses
like:
- Previously, ExecGetAllUpdatedCols() had gaps ..., but not a real bug
because ...
- This commit closes those gaps by updating ri_extraUpdatedCols in
ExecBRUpdateTriggers(), making ExecGetAllUpdatedCols() reliable.
- We know there are no other gaps because ...
- Useful to fix because later work will rely on it for [very brief
reason]
> Imagine a table with a lot of indexes where updates only modify one
> or two at a time. Why are we testing indexed attributes for changes
> in HeapDeterminColumnsInfo() that couldn't have changed? The answer
> is that a) HeapDeterminColumnsInfo() lives in heap, not the executor
> (see patch 0003) so it has no ability to call
> ExecGetAllUpdatedCols(), and b) the set returned by
> ExecGetAllUpdatedCols() is sometimes incomplete.
That's helpful, thank you.
> What do we "need to be sure" of? That ExecGetAllUpdatedCols() not
> really contains all attributes that its name implies? I think it now
> does that after 0002, do you disagree?
I don't disagree, but I think we need some kind statement that we
believe that it's true, and a brief explanation why. (I don't have much
of an opinion about whether it's in this thread, the commit message, or
the code.)
>
> I think it is a new guarantee that was implied before now but not
> required until 0003. I think this change reduces overhead and helps
> to avoid some future security feature that depends on
> ExecGetAllUpdatedCols() to provide that guarantee.
>
> Does that make sense?
A subtlety here is that perhaps ExecGetAllUpdatedCols() already *was*
correct, and it just meant something different than we thought: the
*targeted* columns of an update, instead of the actually-updated
values.
If so we should think about whether that distinction should be
preserved. For instance, column filtering for triggers should be based
on the targeted columns (rather than actually-updated values) because,
semantically, it should still fire even for a no-op update. Perhaps
similar for choosing the lock mode?
>
Regards,
Jeff Davis
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2026-03-02 19:08 Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-11 15:51 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-12 20:33 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2026-03-12 21:31 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-15 21:11 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-16 16:23 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-16 17:55 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
@ 2026-03-17 15:22 ` Nathan Bossart <[email protected]>
1 sibling, 0 replies; 37+ messages in thread
From: Nathan Bossart @ 2026-03-17 15:22 UTC (permalink / raw)
To: Greg Burd <[email protected]>; +Cc: Jeff Davis <[email protected]>; pgsql-hackers
Catching up here. I see that you dropped 0002. Can you explain why that's
no longer needed?
On Mon, Mar 16, 2026 at 04:51:31PM -0400, Greg Burd wrote:
> Refactor executor update logic to determine which indexed columns have
> actually changed during an UPDATE operation rather than leaving this up
> to HeapDetermineColumnsInfo() in heap_update(). Finding this set of
> attributes is not heap-specific, but more general to all table AMs and
> having this information in the executor could inform other decisions
> about when index inserts are required and when they are not regardless
> of the table AM's MVCC implementation strategy.
Nice, this is a crisp motivation statement.
> Development of this feature exposed nondeterministic behavior in three
> existing tests which have been adjusted to avoid inconsistent test
> results due to tuple ordering during heap page scans.
Logistically speaking, these could be nice to get out of the way early as a
prerequisite patch so we can focus on the substance of this patch.
The rest of my comments are from a relatively quick skim. Deeper review to
follow...
> + /*
> + * Reduce the set under review to only the unmodified indexed replica
> + * identity key attributes. idx_attrs is copied (by bms_difference())
> + * not modified here.
> + */
> + attrs = bms_difference(idx_attrs, modified_idx_attrs);
> + attrs = bms_int_members(attrs, rid_attrs);
> +
> + while ((attidx = bms_next_member(attrs, attidx)) >= 0)
Could it be worth moving this loop (and some surrounding code) to a helper
function?
> - * Note: beyond this point, use oldtup not otid to refer to old tuple.
> + * NOTE: beyond this point, use oldtup not otid to refer to old tuple.
nitpick: Please remove unnecessary changes.
> @@ -5269,10 +5269,10 @@ RelationGetIndexPredicate(Relation relation)
> * in expressions (i.e., usable for FKs)
> * INDEX_ATTR_BITMAP_PRIMARY_KEY Columns in the table's primary key
> * (beware: even if PK is deferrable!)
> + * INDEX_ATTR_BITMAP_SUMMARIZED Columns only included in summarizing indexes
> * INDEX_ATTR_BITMAP_IDENTITY_KEY Columns in the table's replica identity
> * index (empty if FULL)
> - * INDEX_ATTR_BITMAP_HOT_BLOCKING Columns that block updates from being HOT
> - * INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
> + * INDEX_ATTR_BITMAP_INDEXED Columns referenced by indexes
Is the meaning of INDEX_ATTR_BITMAP_SUMMARIZED changing in this patch? I
see you moved it and dropped the "only".
> - Bitmapset *summarizedattrs; /* columns with summarizing indexes */
> + Bitmapset *indexedattrs; /* columns referenced by indexes */
> + Bitmapset *summarizedattrs; /* columns only in summarizing indexes */
But you added an "only" here...
--
nathan
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2026-03-02 19:08 Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-11 15:51 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-12 20:33 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2026-03-12 21:31 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-15 21:11 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-16 16:23 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-16 17:55 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
@ 2026-03-17 16:38 ` Jeff Davis <[email protected]>
2026-03-17 18:04 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
1 sibling, 1 reply; 37+ messages in thread
From: Jeff Davis @ 2026-03-17 16:38 UTC (permalink / raw)
To: Greg Burd <[email protected]>; Nathan Bossart <[email protected]>; +Cc: pgsql-hackers
On Mon, 2026-03-16 at 16:51 -0400, Greg Burd wrote:
> > Also, the "actually changed values" is only valid for a single
> > tuple,
> > and it would be good to clarify that and make sure there's not a
> > lot of
> > room for confusion there.
>
> Yes, that's true... too much confusion and not enough juice for the
> squeeze. I'm dropping that.
That is an interesting case you found in that the columns targeted by
an update are not a superset of the columns with actually changed
values. But I'm not sure exactly what to make of that fact, and if it's
not important for your other changes then I agree that we should drop
it.
However, it might be good to comment somewhere that your changes (which
are based on values in specific tuples) cannot rely on
ExecGetAllUpdatedCols(), to avoid confusion in the future.
Regards,
Jeff Davis
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2026-03-02 19:08 Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-11 15:51 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-12 20:33 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2026-03-12 21:31 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-15 21:11 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-16 16:23 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-16 17:55 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-17 16:38 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
@ 2026-03-17 18:04 ` Greg Burd <[email protected]>
2026-03-23 18:39 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Greg Burd @ 2026-03-17 18:04 UTC (permalink / raw)
To: Jeff Davis <[email protected]>; Nathan Bossart <[email protected]>; +Cc: pgsql-hackers
On Tue, Mar 17, 2026, at 12:38 PM, Jeff Davis wrote:
> On Mon, 2026-03-16 at 16:51 -0400, Greg Burd wrote:
>> > Also, the "actually changed values" is only valid for a single
>> > tuple,
>> > and it would be good to clarify that and make sure there's not a
>> > lot of
>> > room for confusion there.
>>
>> Yes, that's true... too much confusion and not enough juice for the
>> squeeze. I'm dropping that.
>
> That is an interesting case you found in that the columns targeted by
> an update are not a superset of the columns with actually changed
> values. But I'm not sure exactly what to make of that fact, and if it's
> not important for your other changes then I agree that we should drop
> it.
>
> However, it might be good to comment somewhere that your changes (which
> are based on values in specific tuples) cannot rely on
> ExecGetAllUpdatedCols(), to avoid confusion in the future.
Fair point, I'll do that.
> Regards,
> Jeff Davis
v37 attached with changes you and Nathan asked for so far. More please! :)
thanks Jeff and Nathan!
best.
-greg
Attachments:
[text/x-patch] v37-0001-Add-tests-to-cover-a-variety-of-heap-HOT-update-.patch (45.3K, 2-v37-0001-Add-tests-to-cover-a-variety-of-heap-HOT-update-.patch)
download | inline diff:
From 6553faa775465e0d525450b6b3cf84f95a02e033 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Tue, 10 Mar 2026 09:28:15 -0400
Subject: [PATCH v37 1/4] Add tests to cover a variety of heap HOT update
behaviors
This commit introduces test infrastructure for verifying Heap-Only Tuple
(HOT) update functionality in PostgreSQL. It provides a baseline for
demonstrating and validating HOT update behavior.
Regression tests:
- Basic HOT vs non-HOT update decisions
- All-or-none property for multiple indexes
- Partial indexes and predicate handling
- BRIN (summarizing) indexes allowing HOT updates
- TOAST column handling with HOT
- Unique constraints behavior
- Multi-column indexes
- Partitioned table HOT updates
Isolation tests:
- HOT chain formation and maintenance
- Concurrent HOT update scenarios
- Index scan behavior with HOT chains
---
src/test/regress/expected/hot_updates.out | 745 ++++++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
src/test/regress/sql/hot_updates.sql | 605 ++++++++++++++++++
3 files changed, 1355 insertions(+)
create mode 100644 src/test/regress/expected/hot_updates.out
create mode 100644 src/test/regress/sql/hot_updates.sql
diff --git a/src/test/regress/expected/hot_updates.out b/src/test/regress/expected/hot_updates.out
new file mode 100644
index 00000000000..273fe3310da
--- /dev/null
+++ b/src/test/regress/expected/hot_updates.out
@@ -0,0 +1,745 @@
+--
+-- HOT_UPDATES
+-- Test Heap-Only Tuple (HOT) update decisions
+--
+-- This test systematically verifies that HOT updates are used when appropriate
+-- and avoided when necessary (e.g., when indexed columns are modified).
+--
+-- We use multiple validation methods:
+-- 1. Statistics functions (pg_stat_get_tuples_hot_updated)
+-- 2. pageinspect extension for HOT chain examination
+-- 3. EXPLAIN to verify index usage after updates
+--
+-- Load required extensions
+CREATE EXTENSION IF NOT EXISTS pageinspect;
+-- Function to get HOT update count
+CREATE OR REPLACE FUNCTION get_hot_count(rel_name text)
+RETURNS TABLE (
+ updates BIGINT,
+ hot BIGINT
+) AS $$
+DECLARE
+ rel_oid oid;
+BEGIN
+ rel_oid := rel_name::regclass::oid;
+
+ -- Read both committed and transaction-local stats
+ -- In autocommit mode (default for regression tests), this works correctly
+ -- Note: In explicit transactions (BEGIN/COMMIT), committed stats already
+ -- include flushed updates, so this would double-count. For explicit
+ -- transaction testing, call pg_stat_force_next_flush() before this function.
+ updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
+ hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
+
+ RETURN NEXT;
+END;
+$$ LANGUAGE plpgsql;
+-- Check if a tuple is part of a HOT chain (has a predecessor on same page)
+CREATE OR REPLACE FUNCTION has_hot_chain(rel_name text, target_ctid tid)
+RETURNS boolean AS $$
+DECLARE
+ block_num int;
+ page_item record;
+BEGIN
+ block_num := (target_ctid::text::point)[0]::int;
+
+ -- Look for a different tuple on the same page that points to our target tuple
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid IS NOT NULL
+ AND t_ctid = target_ctid
+ AND ('(' || block_num::text || ',' || lp::text || ')')::tid != target_ctid
+ LOOP
+ RETURN true;
+ END LOOP;
+
+ RETURN false;
+END;
+$$ LANGUAGE plpgsql;
+-- Print the HOT chain starting from a given tuple
+CREATE OR REPLACE FUNCTION print_hot_chain(rel_name text, start_ctid tid)
+RETURNS TABLE(chain_position int, ctid tid, lp_flags text, t_ctid tid, chain_end boolean) AS
+$$
+#variable_conflict use_column
+DECLARE
+ block_num int;
+ line_ptr int;
+ current_ctid tid := start_ctid;
+ next_ctid tid;
+ position int := 0;
+ max_iterations int := 100;
+ page_item record;
+ found_predecessor boolean := false;
+ flags_name text;
+BEGIN
+ block_num := (start_ctid::text::point)[0]::int;
+
+ -- Find the predecessor (old tuple pointing to our start_ctid)
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid = start_ctid
+ LOOP
+ current_ctid := ('(' || block_num::text || ',' || page_item.lp::text || ')')::tid;
+ found_predecessor := true;
+ EXIT;
+ END LOOP;
+
+ -- If no predecessor found, start with the given ctid
+ IF NOT found_predecessor THEN
+ current_ctid := start_ctid;
+ END IF;
+
+ -- Follow the chain forward
+ WHILE position < max_iterations LOOP
+ line_ptr := (current_ctid::text::point)[1]::int;
+
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp = line_ptr
+ LOOP
+ -- Map lp_flags to names
+ flags_name := CASE page_item.lp_flags
+ WHEN 0 THEN 'unused (0)'
+ WHEN 1 THEN 'normal (1)'
+ WHEN 2 THEN 'redirect (2)'
+ WHEN 3 THEN 'dead (3)'
+ ELSE 'unknown (' || page_item.lp_flags::text || ')'
+ END;
+
+ RETURN QUERY SELECT
+ position,
+ current_ctid,
+ flags_name,
+ page_item.t_ctid,
+ (page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid)::boolean
+ ;
+
+ IF page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid THEN
+ RETURN;
+ END IF;
+
+ next_ctid := page_item.t_ctid;
+
+ IF (next_ctid::text::point)[0]::int != block_num THEN
+ RETURN;
+ END IF;
+
+ current_ctid := next_ctid;
+ position := position + 1;
+ END LOOP;
+
+ IF position = 0 THEN
+ RETURN;
+ END IF;
+ END LOOP;
+END;
+$$ LANGUAGE plpgsql;
+-- Basic HOT update (update non-indexed column)
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
+INSERT INTO hot_test VALUES (1, 100, 'initial');
+INSERT INTO hot_test VALUES (2, 200, 'initial');
+INSERT INTO hot_test VALUES (3, 300, 'initial');
+-- Get baseline
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Should be HOT updates (only non-indexed column modified)
+UPDATE hot_test SET non_indexed_col = 'updated1' WHERE id = 1;
+UPDATE hot_test SET non_indexed_col = 'updated2' WHERE id = 2;
+UPDATE hot_test SET non_indexed_col = 'updated3' WHERE id = 3;
+-- Verify HOT updates occurred
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 3 | 3
+(1 row)
+
+-- Dump the HOT chain before VACUUMing
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+ has_chain | chain_position | ctid | lp_flags | t_ctid
+-----------+----------------+-------+------------+--------
+ t | 0 | (0,1) | normal (1) | (0,4)
+ t | 1 | (0,4) | normal (1) | (0,4)
+(2 rows)
+
+-- Vacuum the relation, expect the HOT chain to collapse
+VACUUM hot_test;
+-- Show that there is no chain after vacuum
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+ has_chain | chain_position | ctid | lp_flags | t_ctid
+-----------+----------------+-------+------------+--------
+ f | 0 | (0,4) | normal (1) | (0,4)
+(1 row)
+
+-- Non-HOT update (update indexed column)
+UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 4 | 3
+(1 row)
+
+-- Verify index was updated (new value findable)
+SET enable_seqscan = off;
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+ QUERY PLAN
+---------------------------------------------------
+ Index Scan using hot_test_indexed_idx on hot_test
+ Index Cond: (indexed_col = 150)
+(2 rows)
+
+SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+ id | indexed_col
+----+-------------
+ 1 | 150
+(1 row)
+
+-- Verify old value no longer in index
+EXPLAIN (COSTS OFF) SELECT id FROM hot_test WHERE indexed_col = 100;
+ QUERY PLAN
+---------------------------------------------------
+ Index Scan using hot_test_indexed_idx on hot_test
+ Index Cond: (indexed_col = 100)
+(2 rows)
+
+SELECT id FROM hot_test WHERE indexed_col = 100;
+ id
+----
+(0 rows)
+
+RESET enable_seqscan;
+-- All-or-none property: updating one indexed column requires ALL index updates
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ non_indexed text
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_a_idx ON hot_test(col_a);
+CREATE INDEX hot_test_b_idx ON hot_test(col_b);
+CREATE INDEX hot_test_c_idx ON hot_test(col_c);
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'initial');
+-- Update only col_a - should NOT be HOT because an indexed column changed
+-- This means ALL indexes must be updated (all-or-none property)
+UPDATE hot_test SET col_a = 15 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Now update only non-indexed column - should be HOT
+UPDATE hot_test SET non_indexed = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 1
+(1 row)
+
+-- Partial index: both old and new outside predicate (conservative = non-HOT)
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ status text,
+ data text
+) WITH (fillfactor = 50);
+-- Partial index only covers status = 'active'
+CREATE INDEX hot_test_active_idx ON hot_test(status) WHERE status = 'active';
+INSERT INTO hot_test VALUES (1, 'active', 'data1');
+INSERT INTO hot_test VALUES (2, 'inactive', 'data2');
+INSERT INTO hot_test VALUES (3, 'deleted', 'data3');
+-- Update non-indexed column on 'active' row (in predicate, status unchanged)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated1' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+-- Update non-indexed column on 'inactive' row (outside predicate)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated2' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 2
+(1 row)
+
+-- Update status from 'inactive' to 'deleted' (both outside predicate)
+-- PostgreSQL is conservative: heap insert happens before predicate check
+-- So this is NON-HOT even though both values are outside predicate
+UPDATE hot_test SET status = 'deleted' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 3 | 2
+(1 row)
+
+-- Verify index still works for 'active' rows
+SELECT id, status FROM hot_test WHERE status = 'active';
+ id | status
+----+--------
+ 1 | active
+(1 row)
+
+-- Only BRIN (summarizing) indexes on non-PK columns
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ ts timestamp,
+ value int,
+ brin_col int
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_ts_brin ON hot_test USING brin(ts);
+CREATE INDEX hot_test_brin_col_brin ON hot_test USING brin(brin_col);
+INSERT INTO hot_test VALUES (1, '2024-01-01', 100, 1000);
+-- Update both BRIN columns - should still be HOT (only summarizing indexes)
+UPDATE hot_test SET ts = '2024-01-02', brin_col = 2000 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+-- Update non-indexed column - should also be HOT
+UPDATE hot_test SET value = 200 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 2
+(1 row)
+
+-- TOAST and HOT: TOASTed columns can participate in HOT
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ large_text text,
+ small_text text
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_idx ON hot_test(indexed_col);
+-- Insert row with TOASTed column (> 2KB)
+INSERT INTO hot_test VALUES (1, 100, repeat('x', 3000), 'small');
+-- Update non-indexed, non-TOASTed column - should be HOT
+UPDATE hot_test SET small_text = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+-- Update TOASTed column - should be HOT if indexed column unchanged
+UPDATE hot_test SET large_text = repeat('y', 3000);
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 2
+(1 row)
+
+-- Update indexed column - should NOT be HOT
+UPDATE hot_test SET indexed_col = 200;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 3 | 2
+(1 row)
+
+-- Unique constraint (unique index) behaves like regular index
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ unique_col int UNIQUE,
+ data text
+) WITH (fillfactor = 50);
+INSERT INTO hot_test VALUES (1, 100, 'data1');
+INSERT INTO hot_test VALUES (2, 200, 'data2');
+-- Update data (non-indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 2
+(1 row)
+
+-- Verify unique constraint still enforced
+SELECT id, unique_col, data FROM hot_test ORDER BY id;
+ id | unique_col | data
+----+------------+---------
+ 1 | 100 | updated
+ 2 | 200 | updated
+(2 rows)
+
+-- This should fail (unique violation)
+UPDATE hot_test SET unique_col = 100 WHERE id = 2;
+ERROR: duplicate key value violates unique constraint "hot_test_unique_col_key"
+DETAIL: Key (unique_col)=(100) already exists.
+-- Multi-column index: any column change = non-HOT
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ data text
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_ab_idx ON hot_test(col_a, col_b);
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'data');
+-- Update col_a (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_a = 15;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Reset
+UPDATE hot_test SET col_a = 10;
+-- Update col_b (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_b = 25;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 3 | 0
+(1 row)
+
+-- Reset
+UPDATE hot_test SET col_b = 20;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 4 | 0
+(1 row)
+
+-- Update col_c (not indexed) - should be HOT
+UPDATE hot_test SET col_c = 35;
+-- Update data (not indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 6 | 2
+(1 row)
+
+-- Partitioned tables: HOT works within partitions
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+NOTICE: table "hot_test_partitioned" does not exist, skipping
+CREATE TABLE hot_test_partitioned (
+ id int,
+ partition_key int,
+ indexed_col int,
+ data text,
+ PRIMARY KEY (id, partition_key)
+) PARTITION BY RANGE (partition_key);
+CREATE TABLE hot_test_part1 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (1) TO (100) WITH (fillfactor = 50);
+CREATE TABLE hot_test_part2 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (100) TO (200) WITH (fillfactor = 50);
+CREATE INDEX hot_test_part_idx ON hot_test_partitioned(indexed_col);
+INSERT INTO hot_test_partitioned VALUES (1, 50, 100, 'initial1');
+INSERT INTO hot_test_partitioned VALUES (2, 150, 200, 'initial2');
+-- Update in partition 1 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'updated1' WHERE id = 1;
+-- Update in partition 2 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'updated2' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test_part1');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+SELECT * FROM get_hot_count('hot_test_part2');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+-- Verify indexes work on partitions
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 100;
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 200;
+ id
+----
+ 2
+(1 row)
+
+-- Update indexed column in partition - should NOT be HOT
+UPDATE hot_test_partitioned SET indexed_col = 150 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test_part1');
+ updates | hot
+---------+-----
+ 2 | 1
+(1 row)
+
+-- Verify index was updated
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 150;
+ id
+----
+ 1
+(1 row)
+
+-- ============================================================================
+-- Trigger modifications: heap_modify_tuple() and HOT
+-- ============================================================================
+-- Test that we correctly detect when triggers modify indexed columns via
+-- heap_modify_tuple(), even when those columns aren't in the UPDATE's SET clause
+CREATE TABLE hot_trigger_test (
+ id int PRIMARY KEY,
+ triggered_col int,
+ data text
+) WITH (fillfactor = 50);
+CREATE INDEX hot_trigger_idx ON hot_trigger_test(triggered_col);
+-- Create a trigger that modifies an indexed column
+CREATE OR REPLACE FUNCTION modify_triggered_col()
+RETURNS TRIGGER AS $$
+BEGIN
+ NEW.triggered_col = NEW.triggered_col + 1;
+ RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+CREATE TRIGGER before_update_modify
+ BEFORE UPDATE ON hot_trigger_test
+ FOR EACH ROW
+ EXECUTE FUNCTION modify_triggered_col();
+INSERT INTO hot_trigger_test VALUES (1, 100, 'initial');
+SELECT * FROM get_hot_count('hot_trigger_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Update only data column, but trigger modifies indexed column
+-- Should NOT be HOT because trigger modified an indexed column
+UPDATE hot_trigger_test SET data = 'updated' WHERE id = 1;
+-- Verify it was NOT a HOT update (indexed column was modified by trigger)
+SELECT * FROM get_hot_count('hot_trigger_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Verify the triggered column was actually modified
+SELECT triggered_col FROM hot_trigger_test WHERE id = 1;
+ triggered_col
+---------------
+ 101
+(1 row)
+
+DROP TABLE hot_trigger_test CASCADE;
+DROP FUNCTION modify_triggered_col();
+-- ============================================================================
+-- JSONB expression indexes and sub-attribute tracking
+-- ============================================================================
+-- Test that updates to non-indexed JSONB paths can be HOT updates
+CREATE TABLE hot_jsonb_test (
+ id int PRIMARY KEY,
+ data jsonb
+) WITH (fillfactor = 50);
+-- Create expression index on a specific JSON path
+CREATE INDEX hot_jsonb_name_idx ON hot_jsonb_test ((data->>'name'));
+INSERT INTO hot_jsonb_test VALUES
+ (1, '{"name":"Alice","age":30,"city":"NYC"}'),
+ (2, '{"name":"Bob","age":25,"city":"LA"}');
+SELECT * FROM get_hot_count('hot_jsonb_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Update non-indexed JSON path (age) - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = jsonb_set(data, '{age}', '31') WHERE id = 1;
+SELECT * FROM get_hot_count('hot_jsonb_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Update indexed JSON path (name) - should NOT be HOT
+UPDATE hot_jsonb_test SET data = jsonb_set(data, '{name}', '"Alice2"') WHERE id = 1;
+SELECT * FROM get_hot_count('hot_jsonb_test');
+ updates | hot
+---------+-----
+ 2 | 0
+(1 row)
+
+-- Verify index works
+SELECT id FROM hot_jsonb_test WHERE data->>'name' = 'Alice2';
+ id
+----
+ 1
+(1 row)
+
+-- Test jsonb_delete on non-indexed path - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = data - 'city' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_jsonb_test');
+ updates | hot
+---------+-----
+ 3 | 0
+(1 row)
+
+-- Test jsonb_insert on non-indexed path - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = jsonb_insert(data, '{country}', '"USA"') WHERE id = 2;
+SELECT * FROM get_hot_count('hot_jsonb_test');
+ updates | hot
+---------+-----
+ 4 | 0
+(1 row)
+
+DROP TABLE hot_jsonb_test;
+-- ============================================================================
+-- XML expression indexes and sub-attribute tracking
+-- ============================================================================
+-- Test that updates to non-indexed XML paths can be HOT updates
+CREATE TABLE hot_xml_test (
+ id int PRIMARY KEY,
+ doc xml
+) WITH (fillfactor = 50);
+-- Create expression index on a specific XPath
+CREATE INDEX hot_xml_name_idx ON hot_xml_test ((xpath('/person/name/text()', doc)));
+INSERT INTO hot_xml_test VALUES
+ (1, '<person><name>Alice</name><age>30</age></person>'),
+ (2, '<person><name>Bob</name><age>25</age></person>');
+ERROR: could not identify a comparison function for type xml
+SELECT * FROM get_hot_count('hot_xml_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Update non-indexed XPath (age) - behavior depends on XML comparison fallback
+-- Full XML value replacement means non-indexed path updates still require index comparison
+UPDATE hot_xml_test SET doc = '<person><name>Alice</name><age>31</age></person>' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_xml_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Update indexed XPath (name) - should NOT be HOT
+UPDATE hot_xml_test SET doc = '<person><name>Alice2</name><age>31</age></person>' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_xml_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Verify index works
+SELECT id FROM hot_xml_test WHERE xpath('/person/name/text()', doc) = ARRAY['Alice2'::text];
+ERROR: operator does not exist: xml[] = text[]
+LINE 1: ..._xml_test WHERE xpath('/person/name/text()', doc) = ARRAY['A...
+ ^
+DETAIL: No operator of that name accepts the given argument types.
+HINT: You might need to add explicit type casts.
+DROP TABLE hot_xml_test;
+-- ============================================================================
+-- GIN indexes and amcomparedatums for JSONB
+-- ============================================================================
+-- Test that GIN indexes can use amcomparedatums to enable HOT when extracted keys match
+CREATE TABLE hot_gin_test (
+ id int PRIMARY KEY,
+ tags text[],
+ properties jsonb
+) WITH (fillfactor = 50);
+-- GIN index on text array
+CREATE INDEX hot_gin_tags_idx ON hot_gin_test USING gin (tags);
+-- GIN index on JSONB (jsonb_ops - keys and values)
+CREATE INDEX hot_gin_props_idx ON hot_gin_test USING gin (properties);
+INSERT INTO hot_gin_test VALUES
+ (1, ARRAY['tag1', 'tag2'], '{"key1":"val1","key2":"val2"}'),
+ (2, ARRAY['tag3', 'tag4'], '{"key3":"val3","key4":"val4"}');
+SELECT * FROM get_hot_count('hot_gin_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Update that changes tag order but not content - after amcomparedatums should be HOT
+-- (GIN extracts same keys, just different order)
+UPDATE hot_gin_test SET tags = ARRAY['tag2', 'tag1'] WHERE id = 1;
+SELECT * FROM get_hot_count('hot_gin_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Update JSONB value (not key) - after amcomparedatums may be HOT or non-HOT
+-- depending on GIN operator class (jsonb_ops indexes both keys and values)
+UPDATE hot_gin_test SET properties = '{"key1":"val1_new","key2":"val2"}' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_gin_test');
+ updates | hot
+---------+-----
+ 2 | 0
+(1 row)
+
+-- Add new tag - should NOT be HOT (different extracted keys)
+UPDATE hot_gin_test SET tags = ARRAY['tag2', 'tag1', 'tag5'] WHERE id = 1;
+SELECT * FROM get_hot_count('hot_gin_test');
+ updates | hot
+---------+-----
+ 3 | 0
+(1 row)
+
+-- Verify GIN indexes work
+SELECT id FROM hot_gin_test WHERE tags @> ARRAY['tag5'];
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hot_gin_test WHERE properties @> '{"key1":"val1_new"}';
+ id
+----
+ 1
+(1 row)
+
+DROP TABLE hot_gin_test;
+-- ============================================================================
+-- Cleanup
+-- ============================================================================
+DROP TABLE IF EXISTS hot_test;
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+DROP FUNCTION IF EXISTS has_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS print_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS get_hot_count(text);
+DROP EXTENSION pageinspect;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e779ada70cb..05e63a5d76f 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -137,6 +137,11 @@ test: event_trigger_login
# this test also uses event triggers, so likewise run it by itself
test: fast_default
+# ----------
+# HOT updates tests
+# ----------
+test: hot_updates
+
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
diff --git a/src/test/regress/sql/hot_updates.sql b/src/test/regress/sql/hot_updates.sql
new file mode 100644
index 00000000000..a8894006177
--- /dev/null
+++ b/src/test/regress/sql/hot_updates.sql
@@ -0,0 +1,605 @@
+--
+-- HOT_UPDATES
+-- Test Heap-Only Tuple (HOT) update decisions
+--
+-- This test systematically verifies that HOT updates are used when appropriate
+-- and avoided when necessary (e.g., when indexed columns are modified).
+--
+-- We use multiple validation methods:
+-- 1. Statistics functions (pg_stat_get_tuples_hot_updated)
+-- 2. pageinspect extension for HOT chain examination
+-- 3. EXPLAIN to verify index usage after updates
+--
+
+-- Load required extensions
+CREATE EXTENSION IF NOT EXISTS pageinspect;
+
+-- Function to get HOT update count
+CREATE OR REPLACE FUNCTION get_hot_count(rel_name text)
+RETURNS TABLE (
+ updates BIGINT,
+ hot BIGINT
+) AS $$
+DECLARE
+ rel_oid oid;
+BEGIN
+ rel_oid := rel_name::regclass::oid;
+
+ -- Read both committed and transaction-local stats
+ -- In autocommit mode (default for regression tests), this works correctly
+ -- Note: In explicit transactions (BEGIN/COMMIT), committed stats already
+ -- include flushed updates, so this would double-count. For explicit
+ -- transaction testing, call pg_stat_force_next_flush() before this function.
+ updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
+ hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
+
+ RETURN NEXT;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Check if a tuple is part of a HOT chain (has a predecessor on same page)
+CREATE OR REPLACE FUNCTION has_hot_chain(rel_name text, target_ctid tid)
+RETURNS boolean AS $$
+DECLARE
+ block_num int;
+ page_item record;
+BEGIN
+ block_num := (target_ctid::text::point)[0]::int;
+
+ -- Look for a different tuple on the same page that points to our target tuple
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid IS NOT NULL
+ AND t_ctid = target_ctid
+ AND ('(' || block_num::text || ',' || lp::text || ')')::tid != target_ctid
+ LOOP
+ RETURN true;
+ END LOOP;
+
+ RETURN false;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Print the HOT chain starting from a given tuple
+CREATE OR REPLACE FUNCTION print_hot_chain(rel_name text, start_ctid tid)
+RETURNS TABLE(chain_position int, ctid tid, lp_flags text, t_ctid tid, chain_end boolean) AS
+$$
+#variable_conflict use_column
+DECLARE
+ block_num int;
+ line_ptr int;
+ current_ctid tid := start_ctid;
+ next_ctid tid;
+ position int := 0;
+ max_iterations int := 100;
+ page_item record;
+ found_predecessor boolean := false;
+ flags_name text;
+BEGIN
+ block_num := (start_ctid::text::point)[0]::int;
+
+ -- Find the predecessor (old tuple pointing to our start_ctid)
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid = start_ctid
+ LOOP
+ current_ctid := ('(' || block_num::text || ',' || page_item.lp::text || ')')::tid;
+ found_predecessor := true;
+ EXIT;
+ END LOOP;
+
+ -- If no predecessor found, start with the given ctid
+ IF NOT found_predecessor THEN
+ current_ctid := start_ctid;
+ END IF;
+
+ -- Follow the chain forward
+ WHILE position < max_iterations LOOP
+ line_ptr := (current_ctid::text::point)[1]::int;
+
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp = line_ptr
+ LOOP
+ -- Map lp_flags to names
+ flags_name := CASE page_item.lp_flags
+ WHEN 0 THEN 'unused (0)'
+ WHEN 1 THEN 'normal (1)'
+ WHEN 2 THEN 'redirect (2)'
+ WHEN 3 THEN 'dead (3)'
+ ELSE 'unknown (' || page_item.lp_flags::text || ')'
+ END;
+
+ RETURN QUERY SELECT
+ position,
+ current_ctid,
+ flags_name,
+ page_item.t_ctid,
+ (page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid)::boolean
+ ;
+
+ IF page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid THEN
+ RETURN;
+ END IF;
+
+ next_ctid := page_item.t_ctid;
+
+ IF (next_ctid::text::point)[0]::int != block_num THEN
+ RETURN;
+ END IF;
+
+ current_ctid := next_ctid;
+ position := position + 1;
+ END LOOP;
+
+ IF position = 0 THEN
+ RETURN;
+ END IF;
+ END LOOP;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Basic HOT update (update non-indexed column)
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
+
+INSERT INTO hot_test VALUES (1, 100, 'initial');
+INSERT INTO hot_test VALUES (2, 200, 'initial');
+INSERT INTO hot_test VALUES (3, 300, 'initial');
+
+-- Get baseline
+SELECT * FROM get_hot_count('hot_test');
+
+-- Should be HOT updates (only non-indexed column modified)
+UPDATE hot_test SET non_indexed_col = 'updated1' WHERE id = 1;
+UPDATE hot_test SET non_indexed_col = 'updated2' WHERE id = 2;
+UPDATE hot_test SET non_indexed_col = 'updated3' WHERE id = 3;
+
+-- Verify HOT updates occurred
+SELECT * FROM get_hot_count('hot_test');
+
+-- Dump the HOT chain before VACUUMing
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+
+-- Vacuum the relation, expect the HOT chain to collapse
+VACUUM hot_test;
+
+-- Show that there is no chain after vacuum
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+
+-- Non-HOT update (update indexed column)
+UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Verify index was updated (new value findable)
+SET enable_seqscan = off;
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+
+-- Verify old value no longer in index
+EXPLAIN (COSTS OFF) SELECT id FROM hot_test WHERE indexed_col = 100;
+SELECT id FROM hot_test WHERE indexed_col = 100;
+RESET enable_seqscan;
+
+-- All-or-none property: updating one indexed column requires ALL index updates
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ non_indexed text
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_a_idx ON hot_test(col_a);
+CREATE INDEX hot_test_b_idx ON hot_test(col_b);
+CREATE INDEX hot_test_c_idx ON hot_test(col_c);
+
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'initial');
+
+-- Update only col_a - should NOT be HOT because an indexed column changed
+-- This means ALL indexes must be updated (all-or-none property)
+UPDATE hot_test SET col_a = 15 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Now update only non-indexed column - should be HOT
+UPDATE hot_test SET non_indexed = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+
+-- Partial index: both old and new outside predicate (conservative = non-HOT)
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ status text,
+ data text
+) WITH (fillfactor = 50);
+
+-- Partial index only covers status = 'active'
+CREATE INDEX hot_test_active_idx ON hot_test(status) WHERE status = 'active';
+
+INSERT INTO hot_test VALUES (1, 'active', 'data1');
+INSERT INTO hot_test VALUES (2, 'inactive', 'data2');
+INSERT INTO hot_test VALUES (3, 'deleted', 'data3');
+
+-- Update non-indexed column on 'active' row (in predicate, status unchanged)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated1' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update non-indexed column on 'inactive' row (outside predicate)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated2' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update status from 'inactive' to 'deleted' (both outside predicate)
+-- PostgreSQL is conservative: heap insert happens before predicate check
+-- So this is NON-HOT even though both values are outside predicate
+UPDATE hot_test SET status = 'deleted' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Verify index still works for 'active' rows
+SELECT id, status FROM hot_test WHERE status = 'active';
+
+-- Only BRIN (summarizing) indexes on non-PK columns
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ ts timestamp,
+ value int,
+ brin_col int
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_ts_brin ON hot_test USING brin(ts);
+CREATE INDEX hot_test_brin_col_brin ON hot_test USING brin(brin_col);
+
+INSERT INTO hot_test VALUES (1, '2024-01-01', 100, 1000);
+
+-- Update both BRIN columns - should still be HOT (only summarizing indexes)
+UPDATE hot_test SET ts = '2024-01-02', brin_col = 2000 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update non-indexed column - should also be HOT
+UPDATE hot_test SET value = 200 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- TOAST and HOT: TOASTed columns can participate in HOT
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ large_text text,
+ small_text text
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_idx ON hot_test(indexed_col);
+
+-- Insert row with TOASTed column (> 2KB)
+INSERT INTO hot_test VALUES (1, 100, repeat('x', 3000), 'small');
+
+-- Update non-indexed, non-TOASTed column - should be HOT
+UPDATE hot_test SET small_text = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update TOASTed column - should be HOT if indexed column unchanged
+UPDATE hot_test SET large_text = repeat('y', 3000);
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update indexed column - should NOT be HOT
+UPDATE hot_test SET indexed_col = 200;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Unique constraint (unique index) behaves like regular index
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ unique_col int UNIQUE,
+ data text
+) WITH (fillfactor = 50);
+
+INSERT INTO hot_test VALUES (1, 100, 'data1');
+INSERT INTO hot_test VALUES (2, 200, 'data2');
+
+-- Update data (non-indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+
+-- Verify unique constraint still enforced
+SELECT id, unique_col, data FROM hot_test ORDER BY id;
+
+-- This should fail (unique violation)
+UPDATE hot_test SET unique_col = 100 WHERE id = 2;
+
+-- Multi-column index: any column change = non-HOT
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ data text
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_ab_idx ON hot_test(col_a, col_b);
+
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'data');
+
+-- Update col_a (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_a = 15;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Reset
+UPDATE hot_test SET col_a = 10;
+
+-- Update col_b (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_b = 25;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Reset
+UPDATE hot_test SET col_b = 20;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update col_c (not indexed) - should be HOT
+UPDATE hot_test SET col_c = 35;
+
+-- Update data (not indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+
+-- Partitioned tables: HOT works within partitions
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+
+CREATE TABLE hot_test_partitioned (
+ id int,
+ partition_key int,
+ indexed_col int,
+ data text,
+ PRIMARY KEY (id, partition_key)
+) PARTITION BY RANGE (partition_key);
+
+CREATE TABLE hot_test_part1 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (1) TO (100) WITH (fillfactor = 50);
+CREATE TABLE hot_test_part2 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (100) TO (200) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_part_idx ON hot_test_partitioned(indexed_col);
+
+INSERT INTO hot_test_partitioned VALUES (1, 50, 100, 'initial1');
+INSERT INTO hot_test_partitioned VALUES (2, 150, 200, 'initial2');
+
+-- Update in partition 1 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'updated1' WHERE id = 1;
+
+-- Update in partition 2 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'updated2' WHERE id = 2;
+
+SELECT * FROM get_hot_count('hot_test_part1');
+SELECT * FROM get_hot_count('hot_test_part2');
+
+-- Verify indexes work on partitions
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 100;
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 200;
+
+-- Update indexed column in partition - should NOT be HOT
+UPDATE hot_test_partitioned SET indexed_col = 150 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test_part1');
+
+-- Verify index was updated
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 150;
+
+-- ============================================================================
+-- Trigger modifications: heap_modify_tuple() and HOT
+-- ============================================================================
+-- Test that we correctly detect when triggers modify indexed columns via
+-- heap_modify_tuple(), even when those columns aren't in the UPDATE's SET clause
+
+CREATE TABLE hot_trigger_test (
+ id int PRIMARY KEY,
+ triggered_col int,
+ data text
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_trigger_idx ON hot_trigger_test(triggered_col);
+
+-- Create a trigger that modifies an indexed column
+CREATE OR REPLACE FUNCTION modify_triggered_col()
+RETURNS TRIGGER AS $$
+BEGIN
+ NEW.triggered_col = NEW.triggered_col + 1;
+ RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE TRIGGER before_update_modify
+ BEFORE UPDATE ON hot_trigger_test
+ FOR EACH ROW
+ EXECUTE FUNCTION modify_triggered_col();
+
+INSERT INTO hot_trigger_test VALUES (1, 100, 'initial');
+
+SELECT * FROM get_hot_count('hot_trigger_test');
+
+-- Update only data column, but trigger modifies indexed column
+-- Should NOT be HOT because trigger modified an indexed column
+UPDATE hot_trigger_test SET data = 'updated' WHERE id = 1;
+
+-- Verify it was NOT a HOT update (indexed column was modified by trigger)
+SELECT * FROM get_hot_count('hot_trigger_test');
+
+-- Verify the triggered column was actually modified
+SELECT triggered_col FROM hot_trigger_test WHERE id = 1;
+
+DROP TABLE hot_trigger_test CASCADE;
+DROP FUNCTION modify_triggered_col();
+
+-- ============================================================================
+-- JSONB expression indexes and sub-attribute tracking
+-- ============================================================================
+-- Test that updates to non-indexed JSONB paths can be HOT updates
+
+CREATE TABLE hot_jsonb_test (
+ id int PRIMARY KEY,
+ data jsonb
+) WITH (fillfactor = 50);
+
+-- Create expression index on a specific JSON path
+CREATE INDEX hot_jsonb_name_idx ON hot_jsonb_test ((data->>'name'));
+
+INSERT INTO hot_jsonb_test VALUES
+ (1, '{"name":"Alice","age":30,"city":"NYC"}'),
+ (2, '{"name":"Bob","age":25,"city":"LA"}');
+
+SELECT * FROM get_hot_count('hot_jsonb_test');
+
+-- Update non-indexed JSON path (age) - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = jsonb_set(data, '{age}', '31') WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_jsonb_test');
+
+-- Update indexed JSON path (name) - should NOT be HOT
+UPDATE hot_jsonb_test SET data = jsonb_set(data, '{name}', '"Alice2"') WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_jsonb_test');
+
+-- Verify index works
+SELECT id FROM hot_jsonb_test WHERE data->>'name' = 'Alice2';
+
+-- Test jsonb_delete on non-indexed path - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = data - 'city' WHERE id = 2;
+
+SELECT * FROM get_hot_count('hot_jsonb_test');
+
+-- Test jsonb_insert on non-indexed path - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = jsonb_insert(data, '{country}', '"USA"') WHERE id = 2;
+
+SELECT * FROM get_hot_count('hot_jsonb_test');
+
+DROP TABLE hot_jsonb_test;
+
+-- ============================================================================
+-- XML expression indexes and sub-attribute tracking
+-- ============================================================================
+-- Test that updates to non-indexed XML paths can be HOT updates
+
+CREATE TABLE hot_xml_test (
+ id int PRIMARY KEY,
+ doc xml
+) WITH (fillfactor = 50);
+
+-- Create expression index on a specific XPath
+CREATE INDEX hot_xml_name_idx ON hot_xml_test ((xpath('/person/name/text()', doc)));
+
+INSERT INTO hot_xml_test VALUES
+ (1, '<person><name>Alice</name><age>30</age></person>'),
+ (2, '<person><name>Bob</name><age>25</age></person>');
+
+SELECT * FROM get_hot_count('hot_xml_test');
+
+-- Update non-indexed XPath (age) - behavior depends on XML comparison fallback
+-- Full XML value replacement means non-indexed path updates still require index comparison
+UPDATE hot_xml_test SET doc = '<person><name>Alice</name><age>31</age></person>' WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_xml_test');
+
+-- Update indexed XPath (name) - should NOT be HOT
+UPDATE hot_xml_test SET doc = '<person><name>Alice2</name><age>31</age></person>' WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_xml_test');
+
+-- Verify index works
+SELECT id FROM hot_xml_test WHERE xpath('/person/name/text()', doc) = ARRAY['Alice2'::text];
+
+DROP TABLE hot_xml_test;
+
+-- ============================================================================
+-- GIN indexes and amcomparedatums for JSONB
+-- ============================================================================
+-- Test that GIN indexes can use amcomparedatums to enable HOT when extracted keys match
+
+CREATE TABLE hot_gin_test (
+ id int PRIMARY KEY,
+ tags text[],
+ properties jsonb
+) WITH (fillfactor = 50);
+
+-- GIN index on text array
+CREATE INDEX hot_gin_tags_idx ON hot_gin_test USING gin (tags);
+
+-- GIN index on JSONB (jsonb_ops - keys and values)
+CREATE INDEX hot_gin_props_idx ON hot_gin_test USING gin (properties);
+
+INSERT INTO hot_gin_test VALUES
+ (1, ARRAY['tag1', 'tag2'], '{"key1":"val1","key2":"val2"}'),
+ (2, ARRAY['tag3', 'tag4'], '{"key3":"val3","key4":"val4"}');
+
+SELECT * FROM get_hot_count('hot_gin_test');
+
+-- Update that changes tag order but not content - after amcomparedatums should be HOT
+-- (GIN extracts same keys, just different order)
+UPDATE hot_gin_test SET tags = ARRAY['tag2', 'tag1'] WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_gin_test');
+
+-- Update JSONB value (not key) - after amcomparedatums may be HOT or non-HOT
+-- depending on GIN operator class (jsonb_ops indexes both keys and values)
+UPDATE hot_gin_test SET properties = '{"key1":"val1_new","key2":"val2"}' WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_gin_test');
+
+-- Add new tag - should NOT be HOT (different extracted keys)
+UPDATE hot_gin_test SET tags = ARRAY['tag2', 'tag1', 'tag5'] WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_gin_test');
+
+-- Verify GIN indexes work
+SELECT id FROM hot_gin_test WHERE tags @> ARRAY['tag5'];
+SELECT id FROM hot_gin_test WHERE properties @> '{"key1":"val1_new"}';
+
+DROP TABLE hot_gin_test;
+
+-- ============================================================================
+-- Cleanup
+-- ============================================================================
+DROP TABLE IF EXISTS hot_test;
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+DROP FUNCTION IF EXISTS has_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS print_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS get_hot_count(text);
+DROP EXTENSION pageinspect;
--
2.51.2
[text/x-patch] v37-0002-Identify-modified-indexed-attributes-in-the-exec.patch (61.4K, 3-v37-0002-Identify-modified-indexed-attributes-in-the-exec.patch)
download | inline diff:
From 06ea9702713f4852c18b1e726ad35e3ff80a56c7 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Tue, 10 Mar 2026 08:17:31 -0400
Subject: [PATCH v37 2/4] Identify modified indexed attributes in the executor
on UPDATE
Refactor executor update logic to determine which indexed columns have
actually changed during an UPDATE operation rather than leaving this up
to HeapDetermineColumnsInfo() in heap_update(). Finding this set of
attributes is not heap-specific, but more general to all table AMs and
having this information in the executor could inform other decisions
about when index inserts are required and when they are not regardless
of the table AM's MVCC implementation strategy.
The heap-only tuple decision (HOT) in heap functions as it always has,
but the determination of the "modified indexed attributes"
(modified_idx_attrs, formerly known as modified_attrs).
ExecUpdateModifiedIdxAttrs() replaces HeapDetermineColumnsInfo() and is
called before table_tuple_update() crucially without the need for an
exclusive buffer lock on the page that holds the tuple being updated.
This reduces the time the buffer lock is held later within
heapam_tuple_update() and heap_update().
Besides identifying the set of modified indexed attributes
HeapDetermineColumnsInfo() was also partially responsible for the
decision about what to WAL log for the replica identity key. This logic
moved into heap_update() and out of the replacement named
HeapUpdateModifiedIdxAttrs(). Doing this allows for
simple_heap_update() and heapam_tuple_update() to share the same logic
as they both call into heap_update().
Updates stemming from logical replication also use the new
ExecUpdateModifiedIdxAttrs() in ExecSimpleRelationUpdate().
ExecUpdateModifiedIdxAttrs() uses ExecCompareSlotAttrs() to identify
which attributes have changed and then intersects that with the set of
indexed attributes to identify the modified indexed set, the
modified_idx_attrs.
This patch introduces a few helper functions to reduce code duplication
and increase readability: HeapUpdateHotAllowable(),
HeapUpdateDetermineLockmode(). These are used in both heap_update() and
simple_heap_update().
The heap_update() function is called now with lockmode pre-determined
and a boolean indicating if the update allows HOT updates or not, both
const. If during heap_update() the new tuple will fit on the same page
and that boolean is true, the update is HOT. This means that although
the functions and timing of the code involed in HOT decisions have
changed, none of the logic related to when HOT is allowed has changed.
Development of this feature exposed nondeterministic behavior in three
existing tests which have been adjusted to avoid inconsistent test
results due to tuple ordering during heap page scans.
---
src/backend/access/heap/heapam.c | 480 ++++++++++++------
src/backend/access/heap/heapam_handler.c | 31 +-
src/backend/access/table/tableam.c | 5 +-
src/backend/executor/execReplication.c | 9 +-
src/backend/executor/execTuples.c | 70 +++
src/backend/executor/nodeModifyTable.c | 88 +++-
src/backend/utils/cache/relcache.c | 44 +-
src/include/access/heapam.h | 13 +-
src/include/access/tableam.h | 8 +-
src/include/executor/executor.h | 9 +
src/include/utils/rel.h | 2 +-
src/include/utils/relcache.h | 2 +-
.../expected/syscache-update-pruned.out | 12 +-
.../specs/syscache-update-pruned.spec | 6 +-
.../regress/expected/generated_virtual.out | 2 +-
src/test/regress/expected/triggers.out | 16 +-
src/test/regress/expected/tsearch.out | 3 +-
src/test/regress/expected/updatable_views.out | 4 +-
src/test/regress/sql/generated_virtual.sql | 2 +-
src/test/regress/sql/triggers.sql | 4 +-
src/test/regress/sql/tsearch.sql | 3 +-
src/test/regress/sql/updatable_views.sql | 2 +-
22 files changed, 583 insertions(+), 232 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e5bd062de77..307855fdd67 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -37,21 +37,26 @@
#include "access/multixact.h"
#include "access/subtrans.h"
#include "access/syncscan.h"
+#include "access/sysattr.h"
+#include "access/tableam.h"
#include "access/valid.h"
#include "access/visibilitymap.h"
#include "access/xloginsert.h"
#include "catalog/pg_database.h"
#include "catalog/pg_database_d.h"
#include "commands/vacuum.h"
+#include "executor/tuptable.h"
+#include "nodes/lockoptions.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
+#include "storage/buf.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
-#include "storage/proc.h"
#include "storage/procarray.h"
#include "utils/datum.h"
#include "utils/injection_point.h"
#include "utils/inval.h"
+#include "utils/relcache.h"
#include "utils/spccache.h"
#include "utils/syscache.h"
@@ -68,11 +73,8 @@ static void check_lock_if_inplace_updateable_rel(Relation relation,
HeapTuple newtup);
static void check_inplace_rel_lock(HeapTuple oldtup);
#endif
-static Bitmapset *HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external);
+static Bitmapset *HeapUpdateModifiedIdxAttrs(Relation relation,
+ HeapTuple oldtup, HeapTuple newtup);
static bool heap_acquire_tuplock(Relation relation, const ItemPointerData *tid,
LockTupleMode mode, LockWaitPolicy wait_policy,
bool *have_tuple_lock);
@@ -3312,7 +3314,7 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* heap_update - replace a tuple
*
* See table_tuple_update() for an explanation of the parameters, except that
- * this routine directly takes a tuple rather than a slot.
+ * this routine directly takes a heap tuple rather than a slot.
*
* In the failure cases, the routine fills *tmfd with the tuple's t_ctid,
* t_xmax (resolving a possible MultiXact, if necessary), and t_cmax (the last
@@ -3322,17 +3324,13 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
TM_Result
heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ TM_FailureData *tmfd, const LockTupleMode lockmode,
+ const Bitmapset *modified_idx_attrs, const bool hot_allowed)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
- Bitmapset *hot_attrs;
- Bitmapset *sum_attrs;
- Bitmapset *key_attrs;
- Bitmapset *id_attrs;
- Bitmapset *interesting_attrs;
- Bitmapset *modified_attrs;
+ Bitmapset *idx_attrs,
+ *rid_attrs;
ItemId lp;
HeapTupleData oldtup;
HeapTuple heaptup;
@@ -3352,13 +3350,12 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
bool have_tuple_lock = false;
bool iscombo;
bool use_hot_update = false;
- bool summarized_update = false;
bool key_intact;
bool all_visible_cleared = false;
bool all_visible_cleared_new = false;
bool checked_lockers;
bool locker_remains;
- bool id_has_external = false;
+ bool rep_id_key_required = false;
TransactionId xmax_new_tuple,
xmax_old_tuple;
uint16 infomask_old_tuple,
@@ -3389,36 +3386,21 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
#endif
/*
- * Fetch the list of attributes to be checked for various operations.
- *
- * For HOT considerations, this is wasted effort if we fail to update or
- * have to put the new tuple on a different page. But we must compute the
- * list before obtaining buffer lock --- in the worst case, if we are
- * doing an update on one of the relevant system catalogs, we could
- * deadlock if we try to fetch the list later. In any case, the relcache
- * caches the data so this is usually pretty cheap.
- *
- * We also need columns used by the replica identity and columns that are
- * considered the "key" of rows in the table.
+ * Fetch the attributes used across all indexes on this relation as well as
+ * the replica identity and columns.
*
- * Note that we get copies of each bitmap, so we need not worry about
- * relcache flush happening midway through.
- */
- hot_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_HOT_BLOCKING);
- sum_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_SUMMARIZED);
- key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
- id_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
- interesting_attrs = NULL;
- interesting_attrs = bms_add_members(interesting_attrs, hot_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, sum_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, key_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, id_attrs);
+ * NOTE: We must compute the list before obtaining buffer lock. In the
+ * worst case, if we are doing an update on one of the relevant system
+ * catalogs, we could deadlock if we try to fetch the list later. Keep in
+ * mind that relcache returns copies of each bitmap, so we need not worry
+ * about relcache flush happening midway through, but we do need to free
+ * them.
+ */
+ idx_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+ rid_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_IDENTITY_KEY);
block = ItemPointerGetBlockNumber(otid);
- INJECTION_POINT("heap_update-before-pin", NULL);
+ INJECTION_POINT("simple_heap_update-before-pin", NULL);
buffer = ReadBuffer(relation, block);
page = BufferGetPage(buffer);
@@ -3469,20 +3451,17 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
tmfd->ctid = *otid;
tmfd->xmax = InvalidTransactionId;
tmfd->cmax = InvalidCommandId;
- *update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- /* modified_attrs not yet initialized */
- bms_free(interesting_attrs);
+ bms_free(rid_attrs);
+ bms_free(idx_attrs);
+ /* modified_idx_attrs is owned by the caller, don't free it */
+
return TM_Deleted;
}
/*
- * Fill in enough data in oldtup for HeapDetermineColumnsInfo to work
- * properly.
+ * Fill in enough data in oldtup to determine replica identity attribute
+ * requirements.
*/
oldtup.t_tableOid = RelationGetRelid(relation);
oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
@@ -3493,16 +3472,59 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
newtup->t_tableOid = RelationGetRelid(relation);
/*
- * Determine columns modified by the update. Additionally, identify
- * whether any of the unmodified replica identity key attributes in the
- * old tuple is externally stored or not. This is required because for
- * such attributes the flattened value won't be WAL logged as part of the
- * new tuple so we must include it as part of the old_key_tuple. See
- * ExtractReplicaIdentity.
+ * ExtractReplicaIdentity() needs to know if a modified indexed attrbute
+ * is used as a replica indentity or if any of the replica identity
+ * attributes are referenced in an index, unmodified, and are stored
+ * externally in the old tuple being replaced. In those cases it may be
+ * necessary to WAL log them to so they are available to replicas.
*/
- modified_attrs = HeapDetermineColumnsInfo(relation, interesting_attrs,
- id_attrs, &oldtup,
- newtup, &id_has_external);
+ rep_id_key_required = bms_overlap(modified_idx_attrs, rid_attrs);
+ if (!rep_id_key_required)
+ {
+ Bitmapset *attrs;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ int attidx = -1;
+
+ /*
+ * Reduce the set under review to only the unmodified indexed replica
+ * identity key attributes. idx_attrs is copied (by bms_difference())
+ * not modified here.
+ */
+ attrs = bms_difference(idx_attrs, modified_idx_attrs);
+ attrs = bms_int_members(attrs, rid_attrs);
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /*
+ * attidx is zero-based, attrnum is the normal attribute number
+ */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value;
+ bool isnull;
+
+ /*
+ * System attributes are not added into INDEX_ATTR_BITMAP_INDEXED
+ * bitmap by relcache.
+ */
+ Assert(attrnum > 0);
+
+ value = heap_getattr(&oldtup, attrnum, tupdesc, &isnull);
+
+ /* No need to check attributes that can't be stored externally */
+ if (isnull ||
+ TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
+ continue;
+
+ /* Check if the old tuple's attribute is stored externally */
+ if (VARATT_IS_EXTERNAL((struct varlena *) DatumGetPointer(value)))
+ {
+ rep_id_key_required = true;
+ break;
+ }
+ }
+
+ bms_free(attrs);
+ }
/*
* If we're not updating any "key" column, we can grab a weaker lock type.
@@ -3515,9 +3537,8 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
* is updates that don't manipulate key columns, not those that
* serendipitously arrive at the same key values.
*/
- if (!bms_overlap(modified_attrs, key_attrs))
+ if (lockmode == LockTupleNoKeyExclusive)
{
- *lockmode = LockTupleNoKeyExclusive;
mxact_status = MultiXactStatusNoKeyUpdate;
key_intact = true;
@@ -3534,7 +3555,7 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
}
else
{
- *lockmode = LockTupleExclusive;
+ Assert(lockmode == LockTupleExclusive);
mxact_status = MultiXactStatusUpdate;
key_intact = false;
}
@@ -3613,7 +3634,7 @@ l2:
bool current_is_member = false;
if (DoesMultiXactIdConflict((MultiXactId) xwait, infomask,
- *lockmode, ¤t_is_member))
+ lockmode, ¤t_is_member))
{
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
@@ -3622,7 +3643,7 @@ l2:
* requesting a lock and already have one; avoids deadlock).
*/
if (!current_is_member)
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &(oldtup.t_self), lockmode,
LockWaitBlock, &have_tuple_lock);
/* wait for multixact */
@@ -3707,7 +3728,7 @@ l2:
* lock.
*/
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &(oldtup.t_self), lockmode,
LockWaitBlock, &have_tuple_lock);
XactLockTableWait(xwait, relation, &oldtup.t_self,
XLTW_Update);
@@ -3767,17 +3788,14 @@ l2:
tmfd->cmax = InvalidCommandId;
UnlockReleaseBuffer(buffer);
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &(oldtup.t_self), lockmode);
if (vmbuffer != InvalidBuffer)
ReleaseBuffer(vmbuffer);
- *update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
+ bms_free(rid_attrs);
+ bms_free(idx_attrs);
+ /* modified_idx_attrs is owned by the caller, don't free it */
+
return result;
}
@@ -3807,7 +3825,7 @@ l2:
compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
oldtup.t_data->t_infomask,
oldtup.t_data->t_infomask2,
- xid, *lockmode, true,
+ xid, lockmode, true,
&xmax_old_tuple, &infomask_old_tuple,
&infomask2_old_tuple);
@@ -3924,7 +3942,7 @@ l2:
compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
oldtup.t_data->t_infomask,
oldtup.t_data->t_infomask2,
- xid, *lockmode, false,
+ xid, lockmode, false,
&xmax_lock_old_tuple, &infomask_lock_old_tuple,
&infomask2_lock_old_tuple);
@@ -4086,10 +4104,11 @@ l2:
/*
* At this point newbuf and buffer are both pinned and locked, and newbuf
- * has enough space for the new tuple. If they are the same buffer, only
- * one pin is held.
+ * has enough space for the new tuple so we can use the HOT update path if
+ * the caller determined that it is allowable.
+ *
+ * NOTE: If newbuf == buffer then only one pin is held.
*/
-
if (newbuf == buffer)
{
/*
@@ -4097,20 +4116,8 @@ l2:
* to do a HOT update. Check if any of the index columns have been
* changed.
*/
- if (!bms_overlap(modified_attrs, hot_attrs))
- {
+ if (hot_allowed)
use_hot_update = true;
-
- /*
- * If none of the columns that are used in hot-blocking indexes
- * were updated, we can apply HOT, but we do still need to check
- * if we need to update the summarizing indexes, and update those
- * indexes if the columns were updated, or we may fail to detect
- * e.g. value bound changes in BRIN minmax indexes.
- */
- if (bms_overlap(modified_attrs, sum_attrs))
- summarized_update = true;
- }
}
else
{
@@ -4126,8 +4133,7 @@ l2:
* columns are modified or it has external data.
*/
old_key_tuple = ExtractReplicaIdentity(relation, &oldtup,
- bms_overlap(modified_attrs, id_attrs) ||
- id_has_external,
+ rep_id_key_required,
&old_key_copied);
/* NO EREPORT(ERROR) from here till changes are logged */
@@ -4256,7 +4262,7 @@ l2:
* Release the lmgr tuple lock, if we had it.
*/
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &(oldtup.t_self), lockmode);
pgstat_count_heap_update(relation, use_hot_update, newbuf != buffer);
@@ -4270,31 +4276,12 @@ l2:
heap_freetuple(heaptup);
}
- /*
- * If it is a HOT update, the update may still need to update summarized
- * indexes, lest we fail to update those summaries and get incorrect
- * results (for example, minmax bounds of the block may change with this
- * update).
- */
- if (use_hot_update)
- {
- if (summarized_update)
- *update_indexes = TU_Summarizing;
- else
- *update_indexes = TU_None;
- }
- else
- *update_indexes = TU_All;
-
if (old_key_tuple != NULL && old_key_copied)
heap_freetuple(old_key_tuple);
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
+ bms_free(rid_attrs);
+ bms_free(idx_attrs);
+ /* modified_idx_attrs is owned by the caller, don't free it */
return TM_Ok;
}
@@ -4467,28 +4454,115 @@ heap_attr_equals(TupleDesc tupdesc, int attrnum, Datum value1, Datum value2,
}
/*
- * Check which columns are being updated.
- *
- * Given an updated tuple, determine (and return into the output bitmapset),
- * from those listed as interesting, the set of columns that changed.
- *
- * has_external indicates if any of the unmodified attributes (from those
- * listed as interesting) of the old tuple is a member of external_cols and is
- * stored externally.
+ * HOT updates are possible when either: a) there are no modified indexed
+ * attributes, or b) the modified attributes are all on summarizing indexes.
+ * Later, in heap_update(), we can choose to perform a HOT update if there is
+ * space on the page for the new tuple and the following code has determined
+ * that HOT is allowed.
+ */
+bool
+HeapUpdateHotAllowable(Relation relation, const Bitmapset *modified_idx_attrs,
+ bool *summarized_only)
+{
+ bool hot_allowed;
+
+ /*
+ * Let's be optimistic and start off by assuming the best case, no indexes
+ * need updating and HOT is allowable.
+ */
+ hot_allowed = true;
+ *summarized_only = false;
+
+ /*
+ * Check for case (a); when there are no modified index attributes HOT is
+ * allowed.
+ */
+ if (bms_is_empty(modified_idx_attrs))
+ hot_allowed = true;
+ else
+ {
+ Bitmapset *sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+
+ /*
+ * At least one index attribute was modified, but is this case (b)
+ * where all the modified index attributes are only used by summarizing
+ * indexes? If it is, then we need to update those indexes, but this
+ * update can still be considered heap-only (HOT) and avoid updating
+ * any non-summarizing indexes on the relation.
+ */
+ if (bms_is_subset(modified_idx_attrs, sum_attrs))
+ {
+ hot_allowed = true;
+ *summarized_only = true;
+ }
+ else
+ {
+ /*
+ * Now we know a) one or more indexed attributes were modified
+ * (changed value, not just referenced within the UPDATE) and that
+ * b) at least one of those attributes is used by a non-summarizing
+ * index. HOT is not allowed.
+ */
+ hot_allowed = false;
+ }
+
+ bms_free(sum_attrs);
+ }
+
+ return hot_allowed;
+}
+
+/*
+ * If we're not updating any attributes used when forming the index keys we can
+ * grab a weaker lock type. This allows for more concurrency when we are
+ * running simultaneously with foreign key checks.
+ */
+LockTupleMode
+HeapUpdateDetermineLockmode(Relation relation, const Bitmapset *modified_idx_attrs)
+{
+ LockTupleMode lockmode = LockTupleExclusive;
+
+ Bitmapset *key_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_KEY);
+
+ if (!bms_overlap(modified_idx_attrs, key_attrs))
+ lockmode = LockTupleNoKeyExclusive;
+
+ bms_free(key_attrs);
+
+ return lockmode;
+}
+
+/*
+ * Return a Bitmapset that contains the set of modified (changed) indexed
+ * attributes between oldtup and newtup.
*/
static Bitmapset *
-HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external)
+HeapUpdateModifiedIdxAttrs(Relation relation, HeapTuple oldtup, HeapTuple newtup)
{
int attidx;
- Bitmapset *modified = NULL;
+ Bitmapset *attrs,
+ *modified_idx_attrs = NULL;
TupleDesc tupdesc = RelationGetDescr(relation);
+ /* Get the set of all attributes across all indexes for this relation */
+ attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+
+ /* No indexed attributes, we're done */
+ if (bms_is_empty(attrs))
+ return NULL;
+
+ /*
+ * This heap update function is used outside the executor and so unlike
+ * heapam_tuple_update() where there is ResultRelInfo and EState to
+ * provide the concise set of attributes that might have been modified
+ * (via ExecGetAllUpdatedCols()) we simply check all indexed attributes to
+ * find the subset that changed value. That's the "modified indexed
+ * attributes" or "modified_idx_attrs".
+ */
attidx = -1;
- while ((attidx = bms_next_member(interesting_cols, attidx)) >= 0)
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
{
/* attidx is zero-based, attrnum is the normal attribute number */
AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
@@ -4504,7 +4578,7 @@ HeapDetermineColumnsInfo(Relation relation,
*/
if (attrnum == 0)
{
- modified = bms_add_member(modified, attidx);
+ modified_idx_attrs = bms_add_member(modified_idx_attrs, attidx);
continue;
}
@@ -4517,7 +4591,7 @@ HeapDetermineColumnsInfo(Relation relation,
{
if (attrnum != TableOidAttributeNumber)
{
- modified = bms_add_member(modified, attidx);
+ modified_idx_attrs = bms_add_member(modified_idx_attrs, attidx);
continue;
}
}
@@ -4533,29 +4607,12 @@ HeapDetermineColumnsInfo(Relation relation,
if (!heap_attr_equals(tupdesc, attrnum, value1,
value2, isnull1, isnull2))
- {
- modified = bms_add_member(modified, attidx);
- continue;
- }
-
- /*
- * No need to check attributes that can't be stored externally. Note
- * that system attributes can't be stored externally.
- */
- if (attrnum < 0 || isnull1 ||
- TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
- continue;
-
- /*
- * Check if the old tuple's attribute is stored externally and is a
- * member of external_cols.
- */
- if (VARATT_IS_EXTERNAL((varlena *) DatumGetPointer(value1)) &&
- bms_is_member(attidx, external_cols))
- *has_external = true;
+ modified_idx_attrs = bms_add_member(modified_idx_attrs, attidx);
}
- return modified;
+ bms_free(attrs);
+
+ return modified_idx_attrs;
}
/*
@@ -4567,17 +4624,112 @@ HeapDetermineColumnsInfo(Relation relation,
* via ereport().
*/
void
-simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup,
+simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TM_FailureData tmfd;
LockTupleMode lockmode;
+ TupleTableSlot *slot;
+ BufferHeapTupleTableSlot *bslot;
+ HeapTuple oldtup;
+ bool shouldFree = true;
+ Bitmapset *idx_attrs,
+ *modified_idx_attrs;
+ bool hot_allowed,
+ summarized_only;
+ Buffer buffer;
- result = heap_update(relation, otid, tup,
- GetCurrentCommandId(true), InvalidSnapshot,
- true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ Assert(ItemPointerIsValid(otid));
+
+ /*
+ * Fetch this bitmap of interesting attributes from relcache before
+ * obtaining a buffer lock because if we are doing an update on one of the
+ * relevant system catalogs we could deadlock if we try to fetch them
+ * later on. Relcache will return copies of each bitmap, so we need not
+ * worry about relcache flush happening midway through this operation.
+ */
+ idx_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+
+ INJECTION_POINT("simple_heap_update-before-pin", NULL);
+
+ /*
+ * To update a heap tuple we need to find the set of modified indexed
+ * attributes ("modified_idx_attrs") and use that to determine if a HOT
+ * update is allowable or not. When updating heap tuples via execution of
+ * UPDATE statements this set is constructed before calling into the table
+ * AM's update function by ExecUpdateModifiedIdxAttrs() which compares the
+ * old/new TupleTableSlots.
+ *
+ * Here things are a bit different, we have the old TID and the new tuple,
+ * not two TupleTableSlots, but we still need to construct a similar bitmap
+ * so as to be able to know if HOT updates are allowed or not.
+ *
+ * To do that we first have to fetch the old tuple itself, but because
+ * heapam_fetch_row_version() is static, we replicate in part that code
+ * here.
+ *
+ * This is a bit repetitive because heap_update() will again find and form
+ * the old HeapTuple from the old TID and in most cases the callers
+ * (ignoring extensions, are always catalog tuple updates) already had the
+ * set of changed attributes (the "replaces" array), but for now this minor
+ * repetition of work is necessary.
+ */
+ slot = MakeTupleTableSlot(RelationGetDescr(relation), &TTSOpsBufferHeapTuple, 0);
+ bslot = (BufferHeapTupleTableSlot *) slot;
+
+ /*
+ * Set the TID in the slot and then fetch the old tuple so we can examine
+ * it
+ */
+ bslot->base.tupdata.t_self = *otid;
+ if (!heap_fetch(relation, SnapshotAny, &bslot->base.tupdata, &buffer, false))
+ {
+ /*
+ * heap_update() checks for !ItemIdIsNormal(lp) and will return false
+ * in those cases.
+ */
+ Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
+
+ *update_indexes = TU_None;
+
+ /* modified_idx_attrs not yet initialized */
+ bms_free(idx_attrs);
+ ExecDropSingleTupleTableSlot(slot);
+
+ elog(ERROR, "tuple concurrently deleted");
+
+ return;
+ }
+
+ Assert(buffer != InvalidBuffer);
+
+ /* Store in slot, transferring existing pin */
+ ExecStorePinnedBufferHeapTuple(&bslot->base.tupdata, slot, buffer);
+ oldtup = ExecFetchSlotHeapTuple(slot, false, &shouldFree);
+
+ modified_idx_attrs = HeapUpdateModifiedIdxAttrs(relation, oldtup, tuple);
+ lockmode = HeapUpdateDetermineLockmode(relation, modified_idx_attrs);
+ hot_allowed = HeapUpdateHotAllowable(relation, modified_idx_attrs, &summarized_only);
+
+ result = heap_update(relation, otid, tuple, GetCurrentCommandId(true),
+ InvalidSnapshot, true /* wait for commit */ ,
+ &tmfd, lockmode, modified_idx_attrs, hot_allowed);
+
+ if (shouldFree)
+ heap_freetuple(oldtup);
+
+ ExecDropSingleTupleTableSlot(slot);
+ bms_free(idx_attrs);
+
+ /*
+ * Decide whether new index entries are needed for the tuple
+ *
+ * If the update is not HOT, we must update all indexes. If the update is
+ * HOT, it could be that we updated summarized columns, so we either
+ * update only summarized indexes, or none at all.
+ */
+ *update_indexes = TU_None;
switch (result)
{
case TM_SelfModified:
@@ -4587,6 +4739,10 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
case TM_Ok:
/* done successfully */
+ if (!HeapTupleIsHeapOnly(tuple))
+ *update_indexes = TU_All;
+ else if (summarized_only)
+ *update_indexes = TU_Summarizing;
break;
case TM_Updated:
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 253a735b6c1..3726c867c65 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,7 +27,6 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
-#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -325,19 +324,26 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd,
- LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
+ bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
+ const Bitmapset *modified_idx_attrs, TU_UpdateIndexes *update_indexes)
{
bool shouldFree = true;
HeapTuple tuple = ExecFetchSlotHeapTuple(slot, true, &shouldFree);
+ bool hot_allowed;
+ bool summarized_only;
TM_Result result;
+ Assert(ItemPointerIsValid(otid));
+
+ hot_allowed = HeapUpdateHotAllowable(relation, modified_idx_attrs, &summarized_only);
+ *lockmode = HeapUpdateDetermineLockmode(relation, modified_idx_attrs);
+
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
- tmfd, lockmode, update_indexes);
+ tmfd, *lockmode, modified_idx_attrs, hot_allowed);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
@@ -350,16 +356,17 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
* HOT, it could be that we updated summarized columns, so we either
* update only summarized indexes, or none at all.
*/
- if (result != TM_Ok)
+ *update_indexes = TU_None;
+ if (result == TM_Ok)
{
- Assert(*update_indexes == TU_None);
- *update_indexes = TU_None;
+ if (HeapTupleIsHeapOnly(tuple))
+ {
+ if (summarized_only)
+ *update_indexes = TU_Summarizing;
+ }
+ else
+ *update_indexes = TU_All;
}
- else if (!HeapTupleIsHeapOnly(tuple))
- Assert(*update_indexes == TU_All);
- else
- Assert((*update_indexes == TU_Summarizing) ||
- (*update_indexes == TU_None));
if (shouldFree)
pfree(tuple);
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index dfda1af412e..9ba72d51dfa 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -359,6 +359,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
+ const Bitmapset *modified_idx_attrs,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -369,7 +370,9 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
GetCurrentCommandId(true),
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, &lockmode,
+ modified_idx_attrs,
+ update_indexes);
switch (result)
{
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index 2497ee7edc5..74a7379186b 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -33,6 +33,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
#include "utils/typcache.h"
@@ -906,6 +907,7 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
bool skip_tuple = false;
Relation rel = resultRelInfo->ri_RelationDesc;
ItemPointer tid = &(searchslot->tts_tid);
+ Bitmapset *modified_idx_attrs;
/*
* We support only non-system tables, with
@@ -944,8 +946,13 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ modified_idx_attrs = ExecUpdateModifiedIdxAttrs(resultRelInfo,
+ estate, searchslot, slot);
+
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
- &update_indexes);
+ modified_idx_attrs, &update_indexes);
+ bms_free(modified_idx_attrs);
+
conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index b717b03b3d2..e8c5639b61e 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -66,6 +66,7 @@
#include "nodes/nodeFuncs.h"
#include "storage/bufmgr.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/expandeddatum.h"
#include "utils/lsyscache.h"
#include "utils/typcache.h"
@@ -1999,6 +2000,75 @@ ExecFetchSlotHeapTupleDatum(TupleTableSlot *slot)
return ret;
}
+/*
+ * ExecCompareSlotAttrs
+ *
+ * Compare the subset of attributes in attrs bewtween TupleTableSlots to detect
+ * which attributes have changed.
+ *
+ * Returns a reused when possible Bitmapset of attribute indices (using
+ * FirstLowInvalidHeapAttributeNumber convention) that differ between the two
+ * slots.
+ */
+Bitmapset *
+ExecCompareSlotAttrs(Bitmapset *attrs, TupleDesc tupdesc,
+ TupleTableSlot *s1, TupleTableSlot *s2)
+{
+ int attidx = -1;
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /* attidx is zero-based, attrnum is the normal attribute number */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value1,
+ value2;
+ bool null1,
+ null2;
+ CompactAttribute *att;
+
+ /*
+ * If it's a whole-tuple reference, say "not equal". It's not really
+ * worth supporting this case, since it could only succeed after a
+ * no-op update, which is hardly a case worth optimizing for.
+ */
+ if (attrnum == 0)
+ continue;
+
+ /*
+ * Likewise, automatically say "not equal" for any system attribute
+ * other than tableOID; we cannot expect these to be consistent in a
+ * HOT chain, or even to be set correctly yet in the new tuple.
+ */
+ if (attrnum < 0)
+ {
+ if (attrnum == TableOidAttributeNumber)
+ attrs = bms_del_member(attrs, attidx);
+ else
+ continue;
+ }
+
+ att = TupleDescCompactAttr(tupdesc, attrnum - 1);
+ value1 = slot_getattr(s1, attrnum, &null1);
+ value2 = slot_getattr(s2, attrnum, &null2);
+
+ /* A change to/from NULL, so not equal */
+ if (null1 != null2)
+ continue;
+
+ /* Both NULL, no change/unmodified */
+ if (null2)
+ {
+ attrs = bms_del_member(attrs, attidx);
+ continue;
+ }
+
+ if (datum_image_eq(value1, value2, att->attbyval, att->attlen))
+ attrs = bms_del_member(attrs, attidx);
+ }
+
+ return attrs;
+}
+
/* ----------------------------------------------------------------
* convenience initialization routines
* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 4cd5e262e0f..ea5058c7a37 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -17,6 +17,7 @@
* ExecModifyTable - retrieve the next tuple from the node
* ExecEndModifyTable - shut down the ModifyTable node
* ExecReScanModifyTable - rescan the ModifyTable node
+ * ExecUpdateModifiedIdxAttrs - find set of updated indexed columns
*
* NOTES
* The ModifyTable node receives input from its outerPlan, which is
@@ -55,6 +56,7 @@
#include "access/htup_details.h"
#include "access/tableam.h"
#include "access/tupconvert.h"
+#include "access/tupdesc.h"
#include "access/xact.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
@@ -190,6 +192,63 @@ static TupleTableSlot *ExecMergeNotMatched(ModifyTableContext *context,
ResultRelInfo *resultRelInfo,
bool canSetTag);
+/*
+ * ExecUpdateModifiedIdxAttrs
+ *
+ * Find the set of attributes referenced by this relation and used in this
+ * UPDATE that now differ in value. This is done by reviewing slot datum that
+ * are in the UPDATE statment and are known to be referenced by at least one
+ * index in some way. This set is called the "modified indexed attributes" or
+ * "modified_idx_attrs". An overlap of a single index's attributes and this
+ * modified_idx_attrs set signals that the attributes in the new_tts used to
+ * form the index datum have changed.
+ *
+ * Return a Bitmapset that contains the set of modified (changed) indexed
+ * attributes between oldtup and newtup.
+ *
+ * Note: There is a similar function called HeapUpdateModifiedIdxAttrs() that operates
+ * on the old TID and new HeapTuple rather than the old/new TupleTableSlots as
+ * this function does. These two functions should mirror one another until
+ * someday when catalog tuple updates track their changes avoiding the need to
+ * re-discover them in simple_heap_update().
+ */
+Bitmapset *
+ExecUpdateModifiedIdxAttrs(ResultRelInfo *resultRelInfo,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts)
+{
+ Relation relation = resultRelInfo->ri_RelationDesc;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ Bitmapset *attrs;
+
+ /* If no indexes, we're done */
+ if (resultRelInfo->ri_NumIndices == 0)
+ return NULL;
+
+ /*
+ * Get the set of all attributes across all indexes for this relation from
+ * the relcache, it returns us a copy of the bitmap so we can modify it.
+ *
+ * Note: We intentionally scan all indexed columns when looking for changes
+ * rather than reduce that set by intersecting it with
+ * ExecGetAllUpdatedCols(). Desipte the name it provides the set of
+ * targeted attributes in the SQL used for the UPDATE and any triggers, but
+ * that doesn't include any attributes updated using heap_modifiy_tuple().
+ * There is one test in tsearch.sql that does just that, modifies an
+ * indexed attribute that isn't specified in the SQL and so isn't present
+ * in that bitmapset.
+ */
+ attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+
+ /*
+ * When there are indexed attributes mentioned in the UPDATE then we need
+ * to find the subset that changed value. That's the "modified_idx_attrs".
+ */
+ attrs = ExecCompareSlotAttrs(attrs, tupdesc, old_tts, new_tts);
+
+ return attrs;
+}
/*
* Verify that the tuples to be produced by INSERT match the
@@ -2197,14 +2256,17 @@ ExecUpdatePrepareSlot(ResultRelInfo *resultRelInfo,
*/
static TM_Result
ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
- ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *slot,
- bool canSetTag, UpdateContext *updateCxt)
+ ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *oldSlot,
+ TupleTableSlot *slot, bool canSetTag, UpdateContext *updateCxt)
{
EState *estate = context->estate;
Relation resultRelationDesc = resultRelInfo->ri_RelationDesc;
bool partition_constraint_failed;
TM_Result result;
+ /* The set of modified indexed attributes that trigger new index entries */
+ Bitmapset *modified_idx_attrs = NULL;
+
updateCxt->crossPartUpdate = false;
/*
@@ -2321,7 +2383,16 @@ lreplace:
ExecConstraints(resultRelInfo, slot, estate);
/*
- * replace the heap tuple
+ * Next up we need to find out the set of indexed attributes that have
+ * changed in value and should trigger a new index tuple. We could start
+ * with the set of updated columns via ExecGetUpdatedCols(), but if we do
+ * we will overlook attributes directly modified by heap_modify_tuple()
+ * which are not known to ExecGetUpdatedCols().
+ */
+ modified_idx_attrs = ExecUpdateModifiedIdxAttrs(resultRelInfo, estate, oldSlot, slot);
+
+ /*
+ * Call into the table AM to update the heap tuple.
*
* Note: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
* the row to be updated is visible to that snapshot, and throw a
@@ -2335,6 +2406,7 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
+ modified_idx_attrs,
&updateCxt->updateIndexes);
return result;
@@ -2557,8 +2629,8 @@ ExecUpdate(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
*/
redo_act:
lockedtid = *tupleid;
- result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, slot,
- canSetTag, &updateCxt);
+ result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, oldSlot,
+ slot, canSetTag, &updateCxt);
/*
* If ExecUpdateAct reports that a cross-partition update was done,
@@ -3408,8 +3480,8 @@ lmerge_matched:
Assert(oldtuple == NULL);
result = ExecUpdateAct(context, resultRelInfo, tupleid,
- NULL, newslot, canSetTag,
- &updateCxt);
+ NULL, resultRelInfo->ri_oldTupleSlot,
+ newslot, canSetTag, &updateCxt);
/*
* As in ExecUpdate(), if ExecUpdateAct() reports that a
@@ -4546,7 +4618,7 @@ ExecModifyTable(PlanState *pstate)
* For UPDATE/DELETE/MERGE, fetch the row identity info for the tuple
* to be updated/deleted/merged. For a heap relation, that's a TID;
* otherwise we may have a wholerow junk attr that carries the old
- * tuple in toto. Keep this in step with the part of
+ * tuple in total. Keep this in step with the part of
* ExecInitModifyTable that sets up ri_RowIdAttNo.
*/
if (operation == CMD_UPDATE || operation == CMD_DELETE ||
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 3a4f19e8d58..f2b7fb8f444 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -2469,7 +2469,7 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
bms_free(relation->rd_keyattr);
bms_free(relation->rd_pkattr);
bms_free(relation->rd_idattr);
- bms_free(relation->rd_hotblockingattr);
+ bms_free(relation->rd_indexedattr);
bms_free(relation->rd_summarizedattr);
if (relation->rd_pubdesc)
pfree(relation->rd_pubdesc);
@@ -5271,8 +5271,8 @@ RelationGetIndexPredicate(Relation relation)
* (beware: even if PK is deferrable!)
* INDEX_ATTR_BITMAP_IDENTITY_KEY Columns in the table's replica identity
* index (empty if FULL)
- * INDEX_ATTR_BITMAP_HOT_BLOCKING Columns that block updates from being HOT
- * INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
+ * INDEX_ATTR_BITMAP_INDEXED Columns referenced by indexes
+ * INDEX_ATTR_BITMAP_SUMMARIZED Columns only included in summarizing indexes
*
* Attribute numbers are offset by FirstLowInvalidHeapAttributeNumber so that
* we can include system attributes (e.g., OID) in the bitmap representation.
@@ -5295,8 +5295,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
Bitmapset *uindexattrs; /* columns in unique indexes */
Bitmapset *pkindexattrs; /* columns in the primary index */
Bitmapset *idindexattrs; /* columns in the replica identity */
- Bitmapset *hotblockingattrs; /* columns with HOT blocking indexes */
- Bitmapset *summarizedattrs; /* columns with summarizing indexes */
+ Bitmapset *indexedattrs; /* columns referenced by indexes */
+ Bitmapset *summarizedattrs; /* columns only in summarizing indexes */
List *indexoidlist;
List *newindexoidlist;
Oid relpkindex;
@@ -5315,8 +5315,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
return bms_copy(relation->rd_pkattr);
case INDEX_ATTR_BITMAP_IDENTITY_KEY:
return bms_copy(relation->rd_idattr);
- case INDEX_ATTR_BITMAP_HOT_BLOCKING:
- return bms_copy(relation->rd_hotblockingattr);
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return bms_copy(relation->rd_indexedattr);
case INDEX_ATTR_BITMAP_SUMMARIZED:
return bms_copy(relation->rd_summarizedattr);
default:
@@ -5361,7 +5361,7 @@ restart:
uindexattrs = NULL;
pkindexattrs = NULL;
idindexattrs = NULL;
- hotblockingattrs = NULL;
+ indexedattrs = NULL;
summarizedattrs = NULL;
foreach(l, indexoidlist)
{
@@ -5421,7 +5421,7 @@ restart:
if (indexDesc->rd_indam->amsummarizing)
attrs = &summarizedattrs;
else
- attrs = &hotblockingattrs;
+ attrs = &indexedattrs;
/* Collect simple attribute references */
for (i = 0; i < indexDesc->rd_index->indnatts; i++)
@@ -5430,9 +5430,9 @@ restart:
/*
* Since we have covering indexes with non-key columns, we must
- * handle them accurately here. non-key columns must be added into
- * hotblockingattrs or summarizedattrs, since they are in index,
- * and update shouldn't miss them.
+ * handle them accurately here. Non-key columns must be added into
+ * indexedattrs or summarizedattrs, since they are in index, and
+ * update shouldn't miss them.
*
* Summarizing indexes do not block HOT, but do need to be updated
* when the column value changes, thus require a separate
@@ -5493,12 +5493,20 @@ restart:
bms_free(uindexattrs);
bms_free(pkindexattrs);
bms_free(idindexattrs);
- bms_free(hotblockingattrs);
+ bms_free(indexedattrs);
bms_free(summarizedattrs);
goto restart;
}
+ /*
+ * Record what attributes are only referenced by summarizing indexes. Then
+ * add that into the other indexed attributes to track all referenced
+ * attributes.
+ */
+ summarizedattrs = bms_del_members(summarizedattrs, indexedattrs);
+ indexedattrs = bms_add_members(indexedattrs, summarizedattrs);
+
/* Don't leak the old values of these bitmaps, if any */
relation->rd_attrsvalid = false;
bms_free(relation->rd_keyattr);
@@ -5507,8 +5515,8 @@ restart:
relation->rd_pkattr = NULL;
bms_free(relation->rd_idattr);
relation->rd_idattr = NULL;
- bms_free(relation->rd_hotblockingattr);
- relation->rd_hotblockingattr = NULL;
+ bms_free(relation->rd_indexedattr);
+ relation->rd_indexedattr = NULL;
bms_free(relation->rd_summarizedattr);
relation->rd_summarizedattr = NULL;
@@ -5523,7 +5531,7 @@ restart:
relation->rd_keyattr = bms_copy(uindexattrs);
relation->rd_pkattr = bms_copy(pkindexattrs);
relation->rd_idattr = bms_copy(idindexattrs);
- relation->rd_hotblockingattr = bms_copy(hotblockingattrs);
+ relation->rd_indexedattr = bms_copy(indexedattrs);
relation->rd_summarizedattr = bms_copy(summarizedattrs);
relation->rd_attrsvalid = true;
MemoryContextSwitchTo(oldcxt);
@@ -5537,8 +5545,8 @@ restart:
return pkindexattrs;
case INDEX_ATTR_BITMAP_IDENTITY_KEY:
return idindexattrs;
- case INDEX_ATTR_BITMAP_HOT_BLOCKING:
- return hotblockingattrs;
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return indexedattrs;
case INDEX_ATTR_BITMAP_SUMMARIZED:
return summarizedattrs;
default:
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 2fdc50b865b..088097a9188 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -390,10 +390,9 @@ extern TM_Result heap_delete(Relation relation, const ItemPointerData *tid,
extern void heap_finish_speculative(Relation relation, const ItemPointerData *tid);
extern void heap_abort_speculative(Relation relation, const ItemPointerData *tid);
extern TM_Result heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, const LockTupleMode lockmode,
+ const Bitmapset *modified_idx_attrs, const bool hot_allowed);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -456,6 +455,12 @@ extern void log_heap_prune_and_freeze(Relation relation, Buffer buffer,
OffsetNumber *dead, int ndead,
OffsetNumber *unused, int nunused);
+/* in heap/heapam.c */
+extern bool HeapUpdateHotAllowable(Relation relation, const Bitmapset *modified_idx_attrs,
+ bool *summarized_only);
+extern LockTupleMode HeapUpdateDetermineLockmode(Relation relation,
+ const Bitmapset *modified_idx_attrs);
+
/* in heap/vacuumlazy.c */
extern void heap_vacuum_rel(Relation rel,
const VacuumParams params, BufferAccessStrategy bstrategy);
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 06084752245..8ec20dcfc11 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -549,6 +549,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
+ const Bitmapset *modified_idx_attrs,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1523,12 +1524,12 @@ static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ const Bitmapset *modified_idx_attrs, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
- wait, tmfd,
- lockmode, update_indexes);
+ wait, tmfd, lockmode,
+ modified_idx_attrs, update_indexes);
}
/*
@@ -2009,6 +2010,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
+ const Bitmapset *modified_idx_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 064df01811e..713ed35d8cf 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -18,6 +18,7 @@
#include "datatype/timestamp.h"
#include "executor/execdesc.h"
#include "fmgr.h"
+#include "nodes/execnodes.h"
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
@@ -610,6 +611,10 @@ extern TupleDesc ExecCleanTypeFromTL(List *targetList);
extern TupleDesc ExecTypeFromExprList(List *exprList);
extern void ExecTypeSetColNames(TupleDesc typeInfo, List *namesList);
extern void UpdateChangedParamSet(PlanState *node, Bitmapset *newchg);
+extern Bitmapset *ExecCompareSlotAttrs(Bitmapset *attrs,
+ TupleDesc tupdesc,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
typedef struct TupOutputState
{
@@ -807,5 +812,9 @@ extern ResultRelInfo *ExecLookupResultRelByOid(ModifyTableState *node,
Oid resultoid,
bool missing_ok,
bool update_cache);
+extern Bitmapset *ExecUpdateModifiedIdxAttrs(ResultRelInfo *relinfo,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
#endif /* EXECUTOR_H */
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 236830f6b93..11460e134f0 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -162,7 +162,7 @@ typedef struct RelationData
Bitmapset *rd_keyattr; /* cols that can be ref'd by foreign keys */
Bitmapset *rd_pkattr; /* cols included in primary key */
Bitmapset *rd_idattr; /* included in replica identity index */
- Bitmapset *rd_hotblockingattr; /* cols blocking HOT update */
+ Bitmapset *rd_indexedattr; /* all cols referenced by indexes */
Bitmapset *rd_summarizedattr; /* cols indexed by summarizing indexes */
PublicationDesc *rd_pubdesc; /* publication descriptor, or NULL */
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2700224939a..d4db82496b4 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -69,7 +69,7 @@ typedef enum IndexAttrBitmapKind
INDEX_ATTR_BITMAP_KEY,
INDEX_ATTR_BITMAP_PRIMARY_KEY,
INDEX_ATTR_BITMAP_IDENTITY_KEY,
- INDEX_ATTR_BITMAP_HOT_BLOCKING,
+ INDEX_ATTR_BITMAP_INDEXED,
INDEX_ATTR_BITMAP_SUMMARIZED,
} IndexAttrBitmapKind;
diff --git a/src/test/modules/injection_points/expected/syscache-update-pruned.out b/src/test/modules/injection_points/expected/syscache-update-pruned.out
index a6a4e8db996..07ef67a1eb4 100644
--- a/src/test/modules/injection_points/expected/syscache-update-pruned.out
+++ b/src/test/modules/injection_points/expected/syscache-update-pruned.out
@@ -16,8 +16,8 @@ step wakeinval4:
step at2: <... completed>
step wakeinval4: <... completed>
step wakegrant4:
- SELECT FROM injection_points_detach('heap_update-before-pin');
- SELECT FROM injection_points_wakeup('heap_update-before-pin');
+ SELECT FROM injection_points_detach('simple_heap_update-before-pin');
+ SELECT FROM injection_points_wakeup('simple_heap_update-before-pin');
<waiting ...>
step grant1: <... completed>
ERROR: tuple concurrently deleted
@@ -42,8 +42,8 @@ step mkrels4:
SELECT FROM vactest.mkrels('intruder', 1, 100); -- repopulate LP_UNUSED
step wakegrant4:
- SELECT FROM injection_points_detach('heap_update-before-pin');
- SELECT FROM injection_points_wakeup('heap_update-before-pin');
+ SELECT FROM injection_points_detach('simple_heap_update-before-pin');
+ SELECT FROM injection_points_wakeup('simple_heap_update-before-pin');
<waiting ...>
step grant1: <... completed>
ERROR: duplicate key value violates unique constraint "pg_class_oid_index"
@@ -71,8 +71,8 @@ step at2: <... completed>
step wakeinval4: <... completed>
step at4: ALTER TABLE vactest.child50 INHERIT vactest.orig50;
step wakegrant4:
- SELECT FROM injection_points_detach('heap_update-before-pin');
- SELECT FROM injection_points_wakeup('heap_update-before-pin');
+ SELECT FROM injection_points_detach('simple_heap_update-before-pin');
+ SELECT FROM injection_points_wakeup('simple_heap_update-before-pin');
<waiting ...>
step grant1: <... completed>
step wakegrant4: <... completed>
diff --git a/src/test/modules/injection_points/specs/syscache-update-pruned.spec b/src/test/modules/injection_points/specs/syscache-update-pruned.spec
index e3a4295bd12..fef9ac895a1 100644
--- a/src/test/modules/injection_points/specs/syscache-update-pruned.spec
+++ b/src/test/modules/injection_points/specs/syscache-update-pruned.spec
@@ -103,7 +103,7 @@ session s1
setup {
SET debug_discard_caches = 0;
SELECT FROM injection_points_set_local();
- SELECT FROM injection_points_attach('heap_update-before-pin', 'wait');
+ SELECT FROM injection_points_attach('simple_heap_update-before-pin', 'wait');
}
step cachefill1 { SELECT FROM vactest.reloid_catcache_set('vactest.orig50'); }
step grant1 { GRANT SELECT ON vactest.orig50 TO PUBLIC; }
@@ -140,8 +140,8 @@ step mkrels4 {
SELECT FROM vactest.mkrels('intruder', 1, 100); -- repopulate LP_UNUSED
}
step wakegrant4 {
- SELECT FROM injection_points_detach('heap_update-before-pin');
- SELECT FROM injection_points_wakeup('heap_update-before-pin');
+ SELECT FROM injection_points_detach('simple_heap_update-before-pin');
+ SELECT FROM injection_points_wakeup('simple_heap_update-before-pin');
}
step at4 { ALTER TABLE vactest.child50 INHERIT vactest.orig50; }
step wakeinval4 {
diff --git a/src/test/regress/expected/generated_virtual.out b/src/test/regress/expected/generated_virtual.out
index 6dab60c937b..7ebb7890d96 100644
--- a/src/test/regress/expected/generated_virtual.out
+++ b/src/test/regress/expected/generated_virtual.out
@@ -287,7 +287,7 @@ DETAIL: Column "b" is a generated column.
INSERT INTO gtest1v VALUES (8, DEFAULT), (9, DEFAULT); -- error
ERROR: cannot insert a non-DEFAULT value into column "b"
DETAIL: Column "b" is a generated column.
-SELECT * FROM gtest1v;
+SELECT * FROM gtest1v ORDER BY a;
a | b
---+----
3 | 6
diff --git a/src/test/regress/expected/triggers.out b/src/test/regress/expected/triggers.out
index 98dee63b50a..ef98fd0cccf 100644
--- a/src/test/regress/expected/triggers.out
+++ b/src/test/regress/expected/triggers.out
@@ -959,16 +959,24 @@ NOTICE: main_view BEFORE UPDATE STATEMENT (before_view_upd_stmt)
NOTICE: main_view AFTER UPDATE STATEMENT (after_view_upd_stmt)
UPDATE 0
-- Delete from view using trigger
-DELETE FROM main_view WHERE a IN (20,21);
+DELETE FROM main_view WHERE a = 20 AND b = 31;
NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
-NOTICE: OLD: (21,10)
-NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
NOTICE: OLD: (20,31)
+NOTICE: main_view AFTER DELETE STATEMENT (after_view_del_stmt)
+DELETE 1
+DELETE FROM main_view WHERE a = 21 AND b = 10;
+NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
+NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
+NOTICE: OLD: (21,10)
+NOTICE: main_view AFTER DELETE STATEMENT (after_view_del_stmt)
+DELETE 1
+DELETE FROM main_view WHERE a = 21 AND b = 32;
+NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
NOTICE: OLD: (21,32)
NOTICE: main_view AFTER DELETE STATEMENT (after_view_del_stmt)
-DELETE 3
+DELETE 1
DELETE FROM main_view WHERE a = 31 RETURNING a, b;
NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
diff --git a/src/test/regress/expected/tsearch.out b/src/test/regress/expected/tsearch.out
index 9287c440709..c604ec35fa5 100644
--- a/src/test/regress/expected/tsearch.out
+++ b/src/test/regress/expected/tsearch.out
@@ -2483,7 +2483,8 @@ SELECT to_tsquery('SKIES & My | booKs');
'sky' | 'book'
(1 row)
---trigger
+-- tsvector_update_trigger() uses heap_modify_tuple() to set column 'a'
+-- without going through the executor's SET-clause tracking.
CREATE TRIGGER tsvectorupdate
BEFORE UPDATE OR INSERT ON test_tsvector
FOR EACH ROW EXECUTE PROCEDURE tsvector_update_trigger(a, 'pg_catalog.english', t);
diff --git a/src/test/regress/expected/updatable_views.out b/src/test/regress/expected/updatable_views.out
index 9cea538b8e8..4877a1ddce9 100644
--- a/src/test/regress/expected/updatable_views.out
+++ b/src/test/regress/expected/updatable_views.out
@@ -372,15 +372,15 @@ INSERT INTO rw_view16 (a, b) VALUES (3, 'Row 3'); -- should be OK
UPDATE rw_view16 SET a=3, aa=-3 WHERE a=3; -- should fail
ERROR: multiple assignments to same column "a"
UPDATE rw_view16 SET aa=-3 WHERE a=3; -- should be OK
-SELECT * FROM base_tbl;
+SELECT * FROM base_tbl ORDER BY a;
a | b
----+--------
+ -3 | Row 3
-2 | Row -2
-1 | Row -1
0 | Row 0
1 | Row 1
2 | Row 2
- -3 | Row 3
(6 rows)
DELETE FROM rw_view16 WHERE a=-3; -- should be OK
diff --git a/src/test/regress/sql/generated_virtual.sql b/src/test/regress/sql/generated_virtual.sql
index e750866d2d8..877152d6d69 100644
--- a/src/test/regress/sql/generated_virtual.sql
+++ b/src/test/regress/sql/generated_virtual.sql
@@ -127,7 +127,7 @@ ALTER VIEW gtest1v ALTER COLUMN b SET DEFAULT 100;
INSERT INTO gtest1v VALUES (8, DEFAULT); -- error
INSERT INTO gtest1v VALUES (8, DEFAULT), (9, DEFAULT); -- error
-SELECT * FROM gtest1v;
+SELECT * FROM gtest1v ORDER BY a;
DELETE FROM gtest1v WHERE a >= 5;
DROP VIEW gtest1v;
diff --git a/src/test/regress/sql/triggers.sql b/src/test/regress/sql/triggers.sql
index ea39817ee3d..6ceb61608ae 100644
--- a/src/test/regress/sql/triggers.sql
+++ b/src/test/regress/sql/triggers.sql
@@ -660,7 +660,9 @@ UPDATE main_view SET b = 32 WHERE a = 21 AND b = 31 RETURNING a, b;
UPDATE main_view SET b = 0 WHERE false;
-- Delete from view using trigger
-DELETE FROM main_view WHERE a IN (20,21);
+DELETE FROM main_view WHERE a = 20 AND b = 31;
+DELETE FROM main_view WHERE a = 21 AND b = 10;
+DELETE FROM main_view WHERE a = 21 AND b = 32;
DELETE FROM main_view WHERE a = 31 RETURNING a, b;
\set QUIET true
diff --git a/src/test/regress/sql/tsearch.sql b/src/test/regress/sql/tsearch.sql
index dc74aa0c889..77ac5fd3c5a 100644
--- a/src/test/regress/sql/tsearch.sql
+++ b/src/test/regress/sql/tsearch.sql
@@ -752,7 +752,8 @@ SELECT to_tsvector('SKIES My booKs');
SELECT plainto_tsquery('SKIES My booKs');
SELECT to_tsquery('SKIES & My | booKs');
---trigger
+-- tsvector_update_trigger() uses heap_modify_tuple() to set column 'a'
+-- without going through the executor's SET-clause tracking.
CREATE TRIGGER tsvectorupdate
BEFORE UPDATE OR INSERT ON test_tsvector
FOR EACH ROW EXECUTE PROCEDURE tsvector_update_trigger(a, 'pg_catalog.english', t);
diff --git a/src/test/regress/sql/updatable_views.sql b/src/test/regress/sql/updatable_views.sql
index 1635adde2d4..160e7799715 100644
--- a/src/test/regress/sql/updatable_views.sql
+++ b/src/test/regress/sql/updatable_views.sql
@@ -125,7 +125,7 @@ INSERT INTO rw_view16 VALUES (3, 'Row 3', 3); -- should fail
INSERT INTO rw_view16 (a, b) VALUES (3, 'Row 3'); -- should be OK
UPDATE rw_view16 SET a=3, aa=-3 WHERE a=3; -- should fail
UPDATE rw_view16 SET aa=-3 WHERE a=3; -- should be OK
-SELECT * FROM base_tbl;
+SELECT * FROM base_tbl ORDER BY a;
DELETE FROM rw_view16 WHERE a=-3; -- should be OK
-- Read-only views
INSERT INTO ro_view17 VALUES (3, 'ROW 3');
--
2.51.2
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2026-03-02 19:08 Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-11 15:51 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-12 20:33 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2026-03-12 21:31 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-15 21:11 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-16 16:23 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-16 17:55 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-17 16:38 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-17 18:04 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
@ 2026-03-23 18:39 ` Nathan Bossart <[email protected]>
2026-03-24 18:02 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Nathan Bossart @ 2026-03-23 18:39 UTC (permalink / raw)
To: Greg Burd <[email protected]>; +Cc: Jeff Davis <[email protected]>; pgsql-hackers
Thanks for the new patch. As a general note, please be sure to run
pgindent on patches. My review is still rather surface-level, sorry.
On Tue, Mar 17, 2026 at 02:04:11PM -0400, Greg Burd wrote:
> - id_attrs = RelationGetIndexAttrBitmap(relation,
> - INDEX_ATTR_BITMAP_IDENTITY_KEY);
> [...]
> + rid_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_IDENTITY_KEY);
I'm nitpicking, but it took me a while to parse the
replica-identity-related code in heap_update() until I discovered that this
variable was renamed. I think we ought to leave the name alone.
> /*
> * At this point newbuf and buffer are both pinned and locked, and newbuf
> - * has enough space for the new tuple. If they are the same buffer, only
> - * one pin is held.
> + * has enough space for the new tuple so we can use the HOT update path if
> + * the caller determined that it is allowable.
> + *
> + * NOTE: If newbuf == buffer then only one pin is held.
> */
> -
> if (newbuf == buffer)
Sorry, more nitpicks. In addition to the unnecessary removal of the blank
line, I'm not sure the changes to this comment are needed.
> - /*
> - * If it is a HOT update, the update may still need to update summarized
> - * indexes, lest we fail to update those summaries and get incorrect
> - * results (for example, minmax bounds of the block may change with this
> - * update).
> - */
> - if (use_hot_update)
> - {
> - if (summarized_update)
> - *update_indexes = TU_Summarizing;
> - else
> - *update_indexes = TU_None;
> - }
> - else
> - *update_indexes = TU_All;
So, the "HOT but still need to update summarized indexes" code has been
moved from heap_update() to HeapUpdateHotAllowable(), which is called by
heap_update()'s callers (i.e., simple_heap_update() and
heapam_tuple_update()). That looks correct to me at a glance.
> -simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup,
> +simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
nitpick: This variable name change looks unnecessary.
> @@ -944,8 +946,13 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
> if (rel->rd_rel->relispartition)
> ExecPartitionCheck(resultRelInfo, slot, estate, true);
>
> + modified_idx_attrs = ExecUpdateModifiedIdxAttrs(resultRelInfo,
> + estate, searchslot, slot);
> +
> simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
> - &update_indexes);
> + modified_idx_attrs, &update_indexes);
> + bms_free(modified_idx_attrs);
I don't know how constructive of a comment this is, but this change in
particular seems quite out of place. It feels odd to me that we expect
callers of simple_table_tuple_update() to determine the
modified-index-attributes. I guess I'm confused why this work doesn't
belong one level down, i.e., in the tuple_update function.
> - * INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
> + * INDEX_ATTR_BITMAP_INDEXED Columns referenced by indexes
> + * INDEX_ATTR_BITMAP_SUMMARIZED Columns only included in summarizing indexes
> - Bitmapset *summarizedattrs; /* columns with summarizing indexes */
> + Bitmapset *indexedattrs; /* columns referenced by indexes */
> + Bitmapset *summarizedattrs; /* columns only in summarizing indexes */
As before, the comment changes for the summarized-attr-related stuff seem
unnecessary.
> if (indexDesc->rd_indam->amsummarizing)
> attrs = &summarizedattrs;
> else
> - attrs = &hotblockingattrs;
> + attrs = &indexedattrs;
> + /*
> + * Record what attributes are only referenced by summarizing indexes. Then
> + * add that into the other indexed attributes to track all referenced
> + * attributes.
> + */
> + summarizedattrs = bms_del_members(summarizedattrs, indexedattrs);
> + indexedattrs = bms_add_members(indexedattrs, summarizedattrs);
The difference between hotblockingattrs and indexedattrs seems quite
subtle. Am I understanding correctly that indexedattrs is essentially just
hotblockingattrs + summarizedattrs? And that this is all meant for
INDEX_ATTR_BITMAP_INDEXED?
- INJECTION_POINT("heap_update-before-pin", NULL);
+ INJECTION_POINT("simple_heap_update-before-pin", NULL);
Why was this changed in heap_update()?
--
nathan
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2026-03-02 19:08 Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-11 15:51 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-12 20:33 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2026-03-12 21:31 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-15 21:11 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-16 16:23 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-16 17:55 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-17 16:38 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-17 18:04 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-23 18:39 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
@ 2026-03-24 18:02 ` Greg Burd <[email protected]>
2026-03-24 19:44 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Greg Burd @ 2026-03-24 18:02 UTC (permalink / raw)
To: Nathan Bossart <[email protected]>; +Cc: Jeff Davis <[email protected]>; pgsql-hackers
On Mon, Mar 23, 2026, at 2:39 PM, Nathan Bossart wrote:
> Thanks for the new patch. As a general note, please be sure to run
> pgindent on patches. My review is still rather surface-level, sorry.
Hello Nathan,
Thanks for continuing to review my work. I appreciate your time. I do run pgindent on all patches, maybe something slipped it. Apologies if that's the case. :)
> On Tue, Mar 17, 2026 at 02:04:11PM -0400, Greg Burd wrote:
>> - id_attrs = RelationGetIndexAttrBitmap(relation,
>> - INDEX_ATTR_BITMAP_IDENTITY_KEY);
>> [...]
>> + rid_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_IDENTITY_KEY);
>
> I'm nitpicking, but it took me a while to parse the
> replica-identity-related code in heap_update() until I discovered that this
> variable was renamed. I think we ought to leave the name alone.
Okay, reverted to "id_attrs".
>> /*
>> * At this point newbuf and buffer are both pinned and locked, and newbuf
>> - * has enough space for the new tuple. If they are the same buffer, only
>> - * one pin is held.
>> + * has enough space for the new tuple so we can use the HOT update path if
>> + * the caller determined that it is allowable.
>> + *
>> + * NOTE: If newbuf == buffer then only one pin is held.
>> */
>> -
>> if (newbuf == buffer)
>
> Sorry, more nitpicks. In addition to the unnecessary removal of the blank
> line, I'm not sure the changes to this comment are needed.
Okay, reverted to earlier comment and blank line re-inserted. :)
>> - /*
>> - * If it is a HOT update, the update may still need to update summarized
>> - * indexes, lest we fail to update those summaries and get incorrect
>> - * results (for example, minmax bounds of the block may change with this
>> - * update).
>> - */
>> - if (use_hot_update)
>> - {
>> - if (summarized_update)
>> - *update_indexes = TU_Summarizing;
>> - else
>> - *update_indexes = TU_None;
>> - }
>> - else
>> - *update_indexes = TU_All;
>
> So, the "HOT but still need to update summarized indexes" code has been
> moved from heap_update() to HeapUpdateHotAllowable(), which is called by
> heap_update()'s callers (i.e., simple_heap_update() and
> heapam_tuple_update()). That looks correct to me at a glance.
Yes, that's indeed what that is.
>> -simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup,
>> +simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
>
> nitpick: This variable name change looks unnecessary.
Okay, reverted to "tup".
>> @@ -944,8 +946,13 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
>> if (rel->rd_rel->relispartition)
>> ExecPartitionCheck(resultRelInfo, slot, estate, true);
>>
>> + modified_idx_attrs = ExecUpdateModifiedIdxAttrs(resultRelInfo,
>> + estate, searchslot, slot);
>> +
>> simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
>> - &update_indexes);
>> + modified_idx_attrs, &update_indexes);
>> + bms_free(modified_idx_attrs);
>
> I don't know how constructive of a comment this is, but this change in
> particular seems quite out of place. It feels odd to me that we expect
> callers of simple_table_tuple_update() to determine the
> modified-index-attributes. I guess I'm confused why this work doesn't
> belong one level down, i.e., in the tuple_update function.
Problem is that simple_table_tuple_update() has the old TID and the new slot, but not the old slot (searchslot), so I'd have to change the signature of that function either way. Passing modified_idx_attrs is the new pattern, so I am just reusing that here.
I could replicate what's in simple_heap_update() and call HeapUpdateModifiedIdxAttrs() after re-constructing the HeapTuple, but that feels very ugly/unnecessary to me given that the caller has that information already in slot form.
I've left this as is, but I'm happy to continue discussing options.
>> - * INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
>> + * INDEX_ATTR_BITMAP_INDEXED Columns referenced by indexes
>> + * INDEX_ATTR_BITMAP_SUMMARIZED Columns only included in summarizing indexes
>
>> - Bitmapset *summarizedattrs; /* columns with summarizing indexes */
>> + Bitmapset *indexedattrs; /* columns referenced by indexes */
>> + Bitmapset *summarizedattrs; /* columns only in summarizing indexes */
>
> As before, the comment changes for the summarized-attr-related stuff seem
> unnecessary.
I disagree, the "only" is required to highlight the logic change here. Before this patch summarized attrs could overlap with indexed attrs, now it should not. This makes the logic a bit easier later in HeapUpdateHotAllowable().
>> if (indexDesc->rd_indam->amsummarizing)
>> attrs = &summarizedattrs;
>> else
>> - attrs = &hotblockingattrs;
>> + attrs = &indexedattrs;
>
>> + /*
>> + * Record what attributes are only referenced by summarizing indexes. Then
>> + * add that into the other indexed attributes to track all referenced
>> + * attributes.
>> + */
>> + summarizedattrs = bms_del_members(summarizedattrs, indexedattrs);
>> + indexedattrs = bms_add_members(indexedattrs, summarizedattrs);
>
> The difference between hotblockingattrs and indexedattrs seems quite
> subtle.
I feel it was *much* more subtle before and mis-named ("hot blocking"). But, let's review. On master today in heapam.c heap_update() near the start and before the buffer lock there is the following:
hot_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_HOT_BLOCKING);
sum_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_SUMMARIZED);
key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
id_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_IDENTITY_KEY);
It turns out that hot_attrs includes all INDEX_ATTR_BITMAP_IDENTITY_KEY and INDEX_ATTR_BITMAP_IDENTITY_KEY except for those found when scanning a summarizing index. So, what comes next is a bit wasteful.
interesting_attrs = NULL;
interesting_attrs = bms_add_members(interesting_attrs, hot_attrs);
interesting_attrs = bms_add_members(interesting_attrs, sum_attrs);
interesting_attrs = bms_add_members(interesting_attrs, key_attrs); <- unnecessary
interesting_attrs = bms_add_members(interesting_attrs, id_attrs); <- unnecessary
And that in the end, what's passed to HeapDetermineColumnsInfo() is all indexed attributes, including summarized, and those found in expressions. That function then reduces the set from "interesting" to "modified (and indexed is implied)". It does this by testing before/after datum for equality (memcmp() via datumIsEqual()) and that becomes our "modified_attrs" set used for lockmode and HOT eligibility tests.
When testing later on (on master) for the HOT or NOT decision the code:
if (newbuf == buffer)
{
// first test to see if any modified/indexed attributes are used
// by non-summarizing indexes
if (!bms_overlap(modified_attrs, hot_attrs))
{
// and if not, we're going HOT
use_hot_update = true;
// at this point if there is any overlap it means that the only
// attributes that might be referenced by an index and modified
// are summarizing, there can't be any non-summarizing attributes
// in the modified_attrs set otherwise our first test would have
// failed, so this tests for the "only summarizing" case
if (bms_overlap(modified_attrs, sum_attrs))
only_summarized = true;
}
}
My thinking was, why re-create this every update? Why not have the cached representation of these bitmaps have what's needed?
Now, I've changed the logic in this patch. First in the executor nodeModifyTable.c ExecUpdateModifiedIdxAttrs() identify which indexed attributes were modified (changed value):
// get all attributes indexed on a relation, including summarized
// note how we no longer construct "interesting_attrs" from a number
// of bitmaps, the map we want is the map we cached and the name matches
// the content, *all* indexed attributes (not indexed, but not summarized)
attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
// compare the old/new reducing the set to only those that changed
// as determined by datum_is_equal() to produce the modified/indexed
// attribute set
attrs = ExecCompareSlotAttrs(attrs, tupdesc, old_tts, new_tts);
Then in heapam_handler.c heapam_tuple_update():
// call our helper function
hot_allowed = HeapUpdateHotAllowable(relation, modified_idx_attrs, &summarized_only);
HeapUpdateHotAllowable()
{
// if no indexed attributes were modified, we're done
if (bms_is_empty(modified_idx_attrs))
return true;
else
{
// now we need the *only* summarized attributes
Bitmapset *sum_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_SUMMARIZED);
// if the modified set is a sumset of the summarized,
// we're only updating summarized
if (bms_is_subset(modified_idx_attrs, sum_attrs))
{
hot_allowed = true;
*summarized_only = true;
}
else
// at least one attribute is modified, referenced by an index
// that isn't summarizing, HOT isn't allowed
hot_allowed = false;
bms_free(sum_attrs);
}
}
So, we go from 3 calls to RelationGetIndexAttrBitmap() to 1, or at most 2 when there's a summarizing index (which is frequently the case).
This feels more logical, cleaner, and has less overhead but supports the same HOT logic.
> Am I understanding correctly that indexedattrs is essentially just
> hotblockingattrs + summarizedattrs? And that this is all meant for
> INDEX_ATTR_BITMAP_INDEXED?
>
> - INJECTION_POINT("heap_update-before-pin", NULL);
> + INJECTION_POINT("simple_heap_update-before-pin", NULL);
>
> Why was this changed in heap_update()?
Oops, that's a mistake. Fixed it.
> --
> nathan
Thanks for your review, v38 attached.
best.
-greg
Attachments:
[text/x-patch] v38-0001-Add-tests-to-cover-a-variety-of-heap-HOT-update-.patch (45.3K, 2-v38-0001-Add-tests-to-cover-a-variety-of-heap-HOT-update-.patch)
download | inline diff:
From eb74d10bdb90c35be7c02a7585195af951518ad7 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Tue, 10 Mar 2026 09:28:15 -0400
Subject: [PATCH v38 1/2] Add tests to cover a variety of heap HOT update
behaviors
This commit introduces test infrastructure for verifying Heap-Only Tuple
(HOT) update functionality in PostgreSQL. It provides a baseline for
demonstrating and validating HOT update behavior.
Regression tests:
- Basic HOT vs non-HOT update decisions
- All-or-none property for multiple indexes
- Partial indexes and predicate handling
- BRIN (summarizing) indexes allowing HOT updates
- TOAST column handling with HOT
- Unique constraints behavior
- Multi-column indexes
- Partitioned table HOT updates
Isolation tests:
- HOT chain formation and maintenance
- Concurrent HOT update scenarios
- Index scan behavior with HOT chains
---
src/test/regress/expected/hot_updates.out | 745 ++++++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
src/test/regress/sql/hot_updates.sql | 605 ++++++++++++++++++
3 files changed, 1355 insertions(+)
create mode 100644 src/test/regress/expected/hot_updates.out
create mode 100644 src/test/regress/sql/hot_updates.sql
diff --git a/src/test/regress/expected/hot_updates.out b/src/test/regress/expected/hot_updates.out
new file mode 100644
index 00000000000..273fe3310da
--- /dev/null
+++ b/src/test/regress/expected/hot_updates.out
@@ -0,0 +1,745 @@
+--
+-- HOT_UPDATES
+-- Test Heap-Only Tuple (HOT) update decisions
+--
+-- This test systematically verifies that HOT updates are used when appropriate
+-- and avoided when necessary (e.g., when indexed columns are modified).
+--
+-- We use multiple validation methods:
+-- 1. Statistics functions (pg_stat_get_tuples_hot_updated)
+-- 2. pageinspect extension for HOT chain examination
+-- 3. EXPLAIN to verify index usage after updates
+--
+-- Load required extensions
+CREATE EXTENSION IF NOT EXISTS pageinspect;
+-- Function to get HOT update count
+CREATE OR REPLACE FUNCTION get_hot_count(rel_name text)
+RETURNS TABLE (
+ updates BIGINT,
+ hot BIGINT
+) AS $$
+DECLARE
+ rel_oid oid;
+BEGIN
+ rel_oid := rel_name::regclass::oid;
+
+ -- Read both committed and transaction-local stats
+ -- In autocommit mode (default for regression tests), this works correctly
+ -- Note: In explicit transactions (BEGIN/COMMIT), committed stats already
+ -- include flushed updates, so this would double-count. For explicit
+ -- transaction testing, call pg_stat_force_next_flush() before this function.
+ updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
+ hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
+
+ RETURN NEXT;
+END;
+$$ LANGUAGE plpgsql;
+-- Check if a tuple is part of a HOT chain (has a predecessor on same page)
+CREATE OR REPLACE FUNCTION has_hot_chain(rel_name text, target_ctid tid)
+RETURNS boolean AS $$
+DECLARE
+ block_num int;
+ page_item record;
+BEGIN
+ block_num := (target_ctid::text::point)[0]::int;
+
+ -- Look for a different tuple on the same page that points to our target tuple
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid IS NOT NULL
+ AND t_ctid = target_ctid
+ AND ('(' || block_num::text || ',' || lp::text || ')')::tid != target_ctid
+ LOOP
+ RETURN true;
+ END LOOP;
+
+ RETURN false;
+END;
+$$ LANGUAGE plpgsql;
+-- Print the HOT chain starting from a given tuple
+CREATE OR REPLACE FUNCTION print_hot_chain(rel_name text, start_ctid tid)
+RETURNS TABLE(chain_position int, ctid tid, lp_flags text, t_ctid tid, chain_end boolean) AS
+$$
+#variable_conflict use_column
+DECLARE
+ block_num int;
+ line_ptr int;
+ current_ctid tid := start_ctid;
+ next_ctid tid;
+ position int := 0;
+ max_iterations int := 100;
+ page_item record;
+ found_predecessor boolean := false;
+ flags_name text;
+BEGIN
+ block_num := (start_ctid::text::point)[0]::int;
+
+ -- Find the predecessor (old tuple pointing to our start_ctid)
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid = start_ctid
+ LOOP
+ current_ctid := ('(' || block_num::text || ',' || page_item.lp::text || ')')::tid;
+ found_predecessor := true;
+ EXIT;
+ END LOOP;
+
+ -- If no predecessor found, start with the given ctid
+ IF NOT found_predecessor THEN
+ current_ctid := start_ctid;
+ END IF;
+
+ -- Follow the chain forward
+ WHILE position < max_iterations LOOP
+ line_ptr := (current_ctid::text::point)[1]::int;
+
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp = line_ptr
+ LOOP
+ -- Map lp_flags to names
+ flags_name := CASE page_item.lp_flags
+ WHEN 0 THEN 'unused (0)'
+ WHEN 1 THEN 'normal (1)'
+ WHEN 2 THEN 'redirect (2)'
+ WHEN 3 THEN 'dead (3)'
+ ELSE 'unknown (' || page_item.lp_flags::text || ')'
+ END;
+
+ RETURN QUERY SELECT
+ position,
+ current_ctid,
+ flags_name,
+ page_item.t_ctid,
+ (page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid)::boolean
+ ;
+
+ IF page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid THEN
+ RETURN;
+ END IF;
+
+ next_ctid := page_item.t_ctid;
+
+ IF (next_ctid::text::point)[0]::int != block_num THEN
+ RETURN;
+ END IF;
+
+ current_ctid := next_ctid;
+ position := position + 1;
+ END LOOP;
+
+ IF position = 0 THEN
+ RETURN;
+ END IF;
+ END LOOP;
+END;
+$$ LANGUAGE plpgsql;
+-- Basic HOT update (update non-indexed column)
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
+INSERT INTO hot_test VALUES (1, 100, 'initial');
+INSERT INTO hot_test VALUES (2, 200, 'initial');
+INSERT INTO hot_test VALUES (3, 300, 'initial');
+-- Get baseline
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Should be HOT updates (only non-indexed column modified)
+UPDATE hot_test SET non_indexed_col = 'updated1' WHERE id = 1;
+UPDATE hot_test SET non_indexed_col = 'updated2' WHERE id = 2;
+UPDATE hot_test SET non_indexed_col = 'updated3' WHERE id = 3;
+-- Verify HOT updates occurred
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 3 | 3
+(1 row)
+
+-- Dump the HOT chain before VACUUMing
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+ has_chain | chain_position | ctid | lp_flags | t_ctid
+-----------+----------------+-------+------------+--------
+ t | 0 | (0,1) | normal (1) | (0,4)
+ t | 1 | (0,4) | normal (1) | (0,4)
+(2 rows)
+
+-- Vacuum the relation, expect the HOT chain to collapse
+VACUUM hot_test;
+-- Show that there is no chain after vacuum
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+ has_chain | chain_position | ctid | lp_flags | t_ctid
+-----------+----------------+-------+------------+--------
+ f | 0 | (0,4) | normal (1) | (0,4)
+(1 row)
+
+-- Non-HOT update (update indexed column)
+UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 4 | 3
+(1 row)
+
+-- Verify index was updated (new value findable)
+SET enable_seqscan = off;
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+ QUERY PLAN
+---------------------------------------------------
+ Index Scan using hot_test_indexed_idx on hot_test
+ Index Cond: (indexed_col = 150)
+(2 rows)
+
+SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+ id | indexed_col
+----+-------------
+ 1 | 150
+(1 row)
+
+-- Verify old value no longer in index
+EXPLAIN (COSTS OFF) SELECT id FROM hot_test WHERE indexed_col = 100;
+ QUERY PLAN
+---------------------------------------------------
+ Index Scan using hot_test_indexed_idx on hot_test
+ Index Cond: (indexed_col = 100)
+(2 rows)
+
+SELECT id FROM hot_test WHERE indexed_col = 100;
+ id
+----
+(0 rows)
+
+RESET enable_seqscan;
+-- All-or-none property: updating one indexed column requires ALL index updates
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ non_indexed text
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_a_idx ON hot_test(col_a);
+CREATE INDEX hot_test_b_idx ON hot_test(col_b);
+CREATE INDEX hot_test_c_idx ON hot_test(col_c);
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'initial');
+-- Update only col_a - should NOT be HOT because an indexed column changed
+-- This means ALL indexes must be updated (all-or-none property)
+UPDATE hot_test SET col_a = 15 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Now update only non-indexed column - should be HOT
+UPDATE hot_test SET non_indexed = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 1
+(1 row)
+
+-- Partial index: both old and new outside predicate (conservative = non-HOT)
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ status text,
+ data text
+) WITH (fillfactor = 50);
+-- Partial index only covers status = 'active'
+CREATE INDEX hot_test_active_idx ON hot_test(status) WHERE status = 'active';
+INSERT INTO hot_test VALUES (1, 'active', 'data1');
+INSERT INTO hot_test VALUES (2, 'inactive', 'data2');
+INSERT INTO hot_test VALUES (3, 'deleted', 'data3');
+-- Update non-indexed column on 'active' row (in predicate, status unchanged)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated1' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+-- Update non-indexed column on 'inactive' row (outside predicate)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated2' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 2
+(1 row)
+
+-- Update status from 'inactive' to 'deleted' (both outside predicate)
+-- PostgreSQL is conservative: heap insert happens before predicate check
+-- So this is NON-HOT even though both values are outside predicate
+UPDATE hot_test SET status = 'deleted' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 3 | 2
+(1 row)
+
+-- Verify index still works for 'active' rows
+SELECT id, status FROM hot_test WHERE status = 'active';
+ id | status
+----+--------
+ 1 | active
+(1 row)
+
+-- Only BRIN (summarizing) indexes on non-PK columns
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ ts timestamp,
+ value int,
+ brin_col int
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_ts_brin ON hot_test USING brin(ts);
+CREATE INDEX hot_test_brin_col_brin ON hot_test USING brin(brin_col);
+INSERT INTO hot_test VALUES (1, '2024-01-01', 100, 1000);
+-- Update both BRIN columns - should still be HOT (only summarizing indexes)
+UPDATE hot_test SET ts = '2024-01-02', brin_col = 2000 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+-- Update non-indexed column - should also be HOT
+UPDATE hot_test SET value = 200 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 2
+(1 row)
+
+-- TOAST and HOT: TOASTed columns can participate in HOT
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ large_text text,
+ small_text text
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_idx ON hot_test(indexed_col);
+-- Insert row with TOASTed column (> 2KB)
+INSERT INTO hot_test VALUES (1, 100, repeat('x', 3000), 'small');
+-- Update non-indexed, non-TOASTed column - should be HOT
+UPDATE hot_test SET small_text = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+-- Update TOASTed column - should be HOT if indexed column unchanged
+UPDATE hot_test SET large_text = repeat('y', 3000);
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 2
+(1 row)
+
+-- Update indexed column - should NOT be HOT
+UPDATE hot_test SET indexed_col = 200;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 3 | 2
+(1 row)
+
+-- Unique constraint (unique index) behaves like regular index
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ unique_col int UNIQUE,
+ data text
+) WITH (fillfactor = 50);
+INSERT INTO hot_test VALUES (1, 100, 'data1');
+INSERT INTO hot_test VALUES (2, 200, 'data2');
+-- Update data (non-indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 2
+(1 row)
+
+-- Verify unique constraint still enforced
+SELECT id, unique_col, data FROM hot_test ORDER BY id;
+ id | unique_col | data
+----+------------+---------
+ 1 | 100 | updated
+ 2 | 200 | updated
+(2 rows)
+
+-- This should fail (unique violation)
+UPDATE hot_test SET unique_col = 100 WHERE id = 2;
+ERROR: duplicate key value violates unique constraint "hot_test_unique_col_key"
+DETAIL: Key (unique_col)=(100) already exists.
+-- Multi-column index: any column change = non-HOT
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ data text
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_ab_idx ON hot_test(col_a, col_b);
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'data');
+-- Update col_a (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_a = 15;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Reset
+UPDATE hot_test SET col_a = 10;
+-- Update col_b (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_b = 25;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 3 | 0
+(1 row)
+
+-- Reset
+UPDATE hot_test SET col_b = 20;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 4 | 0
+(1 row)
+
+-- Update col_c (not indexed) - should be HOT
+UPDATE hot_test SET col_c = 35;
+-- Update data (not indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 6 | 2
+(1 row)
+
+-- Partitioned tables: HOT works within partitions
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+NOTICE: table "hot_test_partitioned" does not exist, skipping
+CREATE TABLE hot_test_partitioned (
+ id int,
+ partition_key int,
+ indexed_col int,
+ data text,
+ PRIMARY KEY (id, partition_key)
+) PARTITION BY RANGE (partition_key);
+CREATE TABLE hot_test_part1 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (1) TO (100) WITH (fillfactor = 50);
+CREATE TABLE hot_test_part2 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (100) TO (200) WITH (fillfactor = 50);
+CREATE INDEX hot_test_part_idx ON hot_test_partitioned(indexed_col);
+INSERT INTO hot_test_partitioned VALUES (1, 50, 100, 'initial1');
+INSERT INTO hot_test_partitioned VALUES (2, 150, 200, 'initial2');
+-- Update in partition 1 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'updated1' WHERE id = 1;
+-- Update in partition 2 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'updated2' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test_part1');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+SELECT * FROM get_hot_count('hot_test_part2');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+-- Verify indexes work on partitions
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 100;
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 200;
+ id
+----
+ 2
+(1 row)
+
+-- Update indexed column in partition - should NOT be HOT
+UPDATE hot_test_partitioned SET indexed_col = 150 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test_part1');
+ updates | hot
+---------+-----
+ 2 | 1
+(1 row)
+
+-- Verify index was updated
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 150;
+ id
+----
+ 1
+(1 row)
+
+-- ============================================================================
+-- Trigger modifications: heap_modify_tuple() and HOT
+-- ============================================================================
+-- Test that we correctly detect when triggers modify indexed columns via
+-- heap_modify_tuple(), even when those columns aren't in the UPDATE's SET clause
+CREATE TABLE hot_trigger_test (
+ id int PRIMARY KEY,
+ triggered_col int,
+ data text
+) WITH (fillfactor = 50);
+CREATE INDEX hot_trigger_idx ON hot_trigger_test(triggered_col);
+-- Create a trigger that modifies an indexed column
+CREATE OR REPLACE FUNCTION modify_triggered_col()
+RETURNS TRIGGER AS $$
+BEGIN
+ NEW.triggered_col = NEW.triggered_col + 1;
+ RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+CREATE TRIGGER before_update_modify
+ BEFORE UPDATE ON hot_trigger_test
+ FOR EACH ROW
+ EXECUTE FUNCTION modify_triggered_col();
+INSERT INTO hot_trigger_test VALUES (1, 100, 'initial');
+SELECT * FROM get_hot_count('hot_trigger_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Update only data column, but trigger modifies indexed column
+-- Should NOT be HOT because trigger modified an indexed column
+UPDATE hot_trigger_test SET data = 'updated' WHERE id = 1;
+-- Verify it was NOT a HOT update (indexed column was modified by trigger)
+SELECT * FROM get_hot_count('hot_trigger_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Verify the triggered column was actually modified
+SELECT triggered_col FROM hot_trigger_test WHERE id = 1;
+ triggered_col
+---------------
+ 101
+(1 row)
+
+DROP TABLE hot_trigger_test CASCADE;
+DROP FUNCTION modify_triggered_col();
+-- ============================================================================
+-- JSONB expression indexes and sub-attribute tracking
+-- ============================================================================
+-- Test that updates to non-indexed JSONB paths can be HOT updates
+CREATE TABLE hot_jsonb_test (
+ id int PRIMARY KEY,
+ data jsonb
+) WITH (fillfactor = 50);
+-- Create expression index on a specific JSON path
+CREATE INDEX hot_jsonb_name_idx ON hot_jsonb_test ((data->>'name'));
+INSERT INTO hot_jsonb_test VALUES
+ (1, '{"name":"Alice","age":30,"city":"NYC"}'),
+ (2, '{"name":"Bob","age":25,"city":"LA"}');
+SELECT * FROM get_hot_count('hot_jsonb_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Update non-indexed JSON path (age) - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = jsonb_set(data, '{age}', '31') WHERE id = 1;
+SELECT * FROM get_hot_count('hot_jsonb_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Update indexed JSON path (name) - should NOT be HOT
+UPDATE hot_jsonb_test SET data = jsonb_set(data, '{name}', '"Alice2"') WHERE id = 1;
+SELECT * FROM get_hot_count('hot_jsonb_test');
+ updates | hot
+---------+-----
+ 2 | 0
+(1 row)
+
+-- Verify index works
+SELECT id FROM hot_jsonb_test WHERE data->>'name' = 'Alice2';
+ id
+----
+ 1
+(1 row)
+
+-- Test jsonb_delete on non-indexed path - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = data - 'city' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_jsonb_test');
+ updates | hot
+---------+-----
+ 3 | 0
+(1 row)
+
+-- Test jsonb_insert on non-indexed path - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = jsonb_insert(data, '{country}', '"USA"') WHERE id = 2;
+SELECT * FROM get_hot_count('hot_jsonb_test');
+ updates | hot
+---------+-----
+ 4 | 0
+(1 row)
+
+DROP TABLE hot_jsonb_test;
+-- ============================================================================
+-- XML expression indexes and sub-attribute tracking
+-- ============================================================================
+-- Test that updates to non-indexed XML paths can be HOT updates
+CREATE TABLE hot_xml_test (
+ id int PRIMARY KEY,
+ doc xml
+) WITH (fillfactor = 50);
+-- Create expression index on a specific XPath
+CREATE INDEX hot_xml_name_idx ON hot_xml_test ((xpath('/person/name/text()', doc)));
+INSERT INTO hot_xml_test VALUES
+ (1, '<person><name>Alice</name><age>30</age></person>'),
+ (2, '<person><name>Bob</name><age>25</age></person>');
+ERROR: could not identify a comparison function for type xml
+SELECT * FROM get_hot_count('hot_xml_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Update non-indexed XPath (age) - behavior depends on XML comparison fallback
+-- Full XML value replacement means non-indexed path updates still require index comparison
+UPDATE hot_xml_test SET doc = '<person><name>Alice</name><age>31</age></person>' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_xml_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Update indexed XPath (name) - should NOT be HOT
+UPDATE hot_xml_test SET doc = '<person><name>Alice2</name><age>31</age></person>' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_xml_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Verify index works
+SELECT id FROM hot_xml_test WHERE xpath('/person/name/text()', doc) = ARRAY['Alice2'::text];
+ERROR: operator does not exist: xml[] = text[]
+LINE 1: ..._xml_test WHERE xpath('/person/name/text()', doc) = ARRAY['A...
+ ^
+DETAIL: No operator of that name accepts the given argument types.
+HINT: You might need to add explicit type casts.
+DROP TABLE hot_xml_test;
+-- ============================================================================
+-- GIN indexes and amcomparedatums for JSONB
+-- ============================================================================
+-- Test that GIN indexes can use amcomparedatums to enable HOT when extracted keys match
+CREATE TABLE hot_gin_test (
+ id int PRIMARY KEY,
+ tags text[],
+ properties jsonb
+) WITH (fillfactor = 50);
+-- GIN index on text array
+CREATE INDEX hot_gin_tags_idx ON hot_gin_test USING gin (tags);
+-- GIN index on JSONB (jsonb_ops - keys and values)
+CREATE INDEX hot_gin_props_idx ON hot_gin_test USING gin (properties);
+INSERT INTO hot_gin_test VALUES
+ (1, ARRAY['tag1', 'tag2'], '{"key1":"val1","key2":"val2"}'),
+ (2, ARRAY['tag3', 'tag4'], '{"key3":"val3","key4":"val4"}');
+SELECT * FROM get_hot_count('hot_gin_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Update that changes tag order but not content - after amcomparedatums should be HOT
+-- (GIN extracts same keys, just different order)
+UPDATE hot_gin_test SET tags = ARRAY['tag2', 'tag1'] WHERE id = 1;
+SELECT * FROM get_hot_count('hot_gin_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Update JSONB value (not key) - after amcomparedatums may be HOT or non-HOT
+-- depending on GIN operator class (jsonb_ops indexes both keys and values)
+UPDATE hot_gin_test SET properties = '{"key1":"val1_new","key2":"val2"}' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_gin_test');
+ updates | hot
+---------+-----
+ 2 | 0
+(1 row)
+
+-- Add new tag - should NOT be HOT (different extracted keys)
+UPDATE hot_gin_test SET tags = ARRAY['tag2', 'tag1', 'tag5'] WHERE id = 1;
+SELECT * FROM get_hot_count('hot_gin_test');
+ updates | hot
+---------+-----
+ 3 | 0
+(1 row)
+
+-- Verify GIN indexes work
+SELECT id FROM hot_gin_test WHERE tags @> ARRAY['tag5'];
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hot_gin_test WHERE properties @> '{"key1":"val1_new"}';
+ id
+----
+ 1
+(1 row)
+
+DROP TABLE hot_gin_test;
+-- ============================================================================
+-- Cleanup
+-- ============================================================================
+DROP TABLE IF EXISTS hot_test;
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+DROP FUNCTION IF EXISTS has_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS print_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS get_hot_count(text);
+DROP EXTENSION pageinspect;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 734da057c34..675eb175059 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -137,6 +137,11 @@ test: event_trigger_login
# this test also uses event triggers, so likewise run it by itself
test: fast_default
+# ----------
+# HOT updates tests
+# ----------
+test: hot_updates
+
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
diff --git a/src/test/regress/sql/hot_updates.sql b/src/test/regress/sql/hot_updates.sql
new file mode 100644
index 00000000000..a8894006177
--- /dev/null
+++ b/src/test/regress/sql/hot_updates.sql
@@ -0,0 +1,605 @@
+--
+-- HOT_UPDATES
+-- Test Heap-Only Tuple (HOT) update decisions
+--
+-- This test systematically verifies that HOT updates are used when appropriate
+-- and avoided when necessary (e.g., when indexed columns are modified).
+--
+-- We use multiple validation methods:
+-- 1. Statistics functions (pg_stat_get_tuples_hot_updated)
+-- 2. pageinspect extension for HOT chain examination
+-- 3. EXPLAIN to verify index usage after updates
+--
+
+-- Load required extensions
+CREATE EXTENSION IF NOT EXISTS pageinspect;
+
+-- Function to get HOT update count
+CREATE OR REPLACE FUNCTION get_hot_count(rel_name text)
+RETURNS TABLE (
+ updates BIGINT,
+ hot BIGINT
+) AS $$
+DECLARE
+ rel_oid oid;
+BEGIN
+ rel_oid := rel_name::regclass::oid;
+
+ -- Read both committed and transaction-local stats
+ -- In autocommit mode (default for regression tests), this works correctly
+ -- Note: In explicit transactions (BEGIN/COMMIT), committed stats already
+ -- include flushed updates, so this would double-count. For explicit
+ -- transaction testing, call pg_stat_force_next_flush() before this function.
+ updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
+ hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
+
+ RETURN NEXT;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Check if a tuple is part of a HOT chain (has a predecessor on same page)
+CREATE OR REPLACE FUNCTION has_hot_chain(rel_name text, target_ctid tid)
+RETURNS boolean AS $$
+DECLARE
+ block_num int;
+ page_item record;
+BEGIN
+ block_num := (target_ctid::text::point)[0]::int;
+
+ -- Look for a different tuple on the same page that points to our target tuple
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid IS NOT NULL
+ AND t_ctid = target_ctid
+ AND ('(' || block_num::text || ',' || lp::text || ')')::tid != target_ctid
+ LOOP
+ RETURN true;
+ END LOOP;
+
+ RETURN false;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Print the HOT chain starting from a given tuple
+CREATE OR REPLACE FUNCTION print_hot_chain(rel_name text, start_ctid tid)
+RETURNS TABLE(chain_position int, ctid tid, lp_flags text, t_ctid tid, chain_end boolean) AS
+$$
+#variable_conflict use_column
+DECLARE
+ block_num int;
+ line_ptr int;
+ current_ctid tid := start_ctid;
+ next_ctid tid;
+ position int := 0;
+ max_iterations int := 100;
+ page_item record;
+ found_predecessor boolean := false;
+ flags_name text;
+BEGIN
+ block_num := (start_ctid::text::point)[0]::int;
+
+ -- Find the predecessor (old tuple pointing to our start_ctid)
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid = start_ctid
+ LOOP
+ current_ctid := ('(' || block_num::text || ',' || page_item.lp::text || ')')::tid;
+ found_predecessor := true;
+ EXIT;
+ END LOOP;
+
+ -- If no predecessor found, start with the given ctid
+ IF NOT found_predecessor THEN
+ current_ctid := start_ctid;
+ END IF;
+
+ -- Follow the chain forward
+ WHILE position < max_iterations LOOP
+ line_ptr := (current_ctid::text::point)[1]::int;
+
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp = line_ptr
+ LOOP
+ -- Map lp_flags to names
+ flags_name := CASE page_item.lp_flags
+ WHEN 0 THEN 'unused (0)'
+ WHEN 1 THEN 'normal (1)'
+ WHEN 2 THEN 'redirect (2)'
+ WHEN 3 THEN 'dead (3)'
+ ELSE 'unknown (' || page_item.lp_flags::text || ')'
+ END;
+
+ RETURN QUERY SELECT
+ position,
+ current_ctid,
+ flags_name,
+ page_item.t_ctid,
+ (page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid)::boolean
+ ;
+
+ IF page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid THEN
+ RETURN;
+ END IF;
+
+ next_ctid := page_item.t_ctid;
+
+ IF (next_ctid::text::point)[0]::int != block_num THEN
+ RETURN;
+ END IF;
+
+ current_ctid := next_ctid;
+ position := position + 1;
+ END LOOP;
+
+ IF position = 0 THEN
+ RETURN;
+ END IF;
+ END LOOP;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Basic HOT update (update non-indexed column)
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
+
+INSERT INTO hot_test VALUES (1, 100, 'initial');
+INSERT INTO hot_test VALUES (2, 200, 'initial');
+INSERT INTO hot_test VALUES (3, 300, 'initial');
+
+-- Get baseline
+SELECT * FROM get_hot_count('hot_test');
+
+-- Should be HOT updates (only non-indexed column modified)
+UPDATE hot_test SET non_indexed_col = 'updated1' WHERE id = 1;
+UPDATE hot_test SET non_indexed_col = 'updated2' WHERE id = 2;
+UPDATE hot_test SET non_indexed_col = 'updated3' WHERE id = 3;
+
+-- Verify HOT updates occurred
+SELECT * FROM get_hot_count('hot_test');
+
+-- Dump the HOT chain before VACUUMing
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+
+-- Vacuum the relation, expect the HOT chain to collapse
+VACUUM hot_test;
+
+-- Show that there is no chain after vacuum
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+
+-- Non-HOT update (update indexed column)
+UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Verify index was updated (new value findable)
+SET enable_seqscan = off;
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+
+-- Verify old value no longer in index
+EXPLAIN (COSTS OFF) SELECT id FROM hot_test WHERE indexed_col = 100;
+SELECT id FROM hot_test WHERE indexed_col = 100;
+RESET enable_seqscan;
+
+-- All-or-none property: updating one indexed column requires ALL index updates
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ non_indexed text
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_a_idx ON hot_test(col_a);
+CREATE INDEX hot_test_b_idx ON hot_test(col_b);
+CREATE INDEX hot_test_c_idx ON hot_test(col_c);
+
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'initial');
+
+-- Update only col_a - should NOT be HOT because an indexed column changed
+-- This means ALL indexes must be updated (all-or-none property)
+UPDATE hot_test SET col_a = 15 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Now update only non-indexed column - should be HOT
+UPDATE hot_test SET non_indexed = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+
+-- Partial index: both old and new outside predicate (conservative = non-HOT)
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ status text,
+ data text
+) WITH (fillfactor = 50);
+
+-- Partial index only covers status = 'active'
+CREATE INDEX hot_test_active_idx ON hot_test(status) WHERE status = 'active';
+
+INSERT INTO hot_test VALUES (1, 'active', 'data1');
+INSERT INTO hot_test VALUES (2, 'inactive', 'data2');
+INSERT INTO hot_test VALUES (3, 'deleted', 'data3');
+
+-- Update non-indexed column on 'active' row (in predicate, status unchanged)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated1' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update non-indexed column on 'inactive' row (outside predicate)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated2' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update status from 'inactive' to 'deleted' (both outside predicate)
+-- PostgreSQL is conservative: heap insert happens before predicate check
+-- So this is NON-HOT even though both values are outside predicate
+UPDATE hot_test SET status = 'deleted' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Verify index still works for 'active' rows
+SELECT id, status FROM hot_test WHERE status = 'active';
+
+-- Only BRIN (summarizing) indexes on non-PK columns
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ ts timestamp,
+ value int,
+ brin_col int
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_ts_brin ON hot_test USING brin(ts);
+CREATE INDEX hot_test_brin_col_brin ON hot_test USING brin(brin_col);
+
+INSERT INTO hot_test VALUES (1, '2024-01-01', 100, 1000);
+
+-- Update both BRIN columns - should still be HOT (only summarizing indexes)
+UPDATE hot_test SET ts = '2024-01-02', brin_col = 2000 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update non-indexed column - should also be HOT
+UPDATE hot_test SET value = 200 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- TOAST and HOT: TOASTed columns can participate in HOT
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ large_text text,
+ small_text text
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_idx ON hot_test(indexed_col);
+
+-- Insert row with TOASTed column (> 2KB)
+INSERT INTO hot_test VALUES (1, 100, repeat('x', 3000), 'small');
+
+-- Update non-indexed, non-TOASTed column - should be HOT
+UPDATE hot_test SET small_text = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update TOASTed column - should be HOT if indexed column unchanged
+UPDATE hot_test SET large_text = repeat('y', 3000);
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update indexed column - should NOT be HOT
+UPDATE hot_test SET indexed_col = 200;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Unique constraint (unique index) behaves like regular index
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ unique_col int UNIQUE,
+ data text
+) WITH (fillfactor = 50);
+
+INSERT INTO hot_test VALUES (1, 100, 'data1');
+INSERT INTO hot_test VALUES (2, 200, 'data2');
+
+-- Update data (non-indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+
+-- Verify unique constraint still enforced
+SELECT id, unique_col, data FROM hot_test ORDER BY id;
+
+-- This should fail (unique violation)
+UPDATE hot_test SET unique_col = 100 WHERE id = 2;
+
+-- Multi-column index: any column change = non-HOT
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ data text
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_ab_idx ON hot_test(col_a, col_b);
+
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'data');
+
+-- Update col_a (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_a = 15;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Reset
+UPDATE hot_test SET col_a = 10;
+
+-- Update col_b (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_b = 25;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Reset
+UPDATE hot_test SET col_b = 20;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update col_c (not indexed) - should be HOT
+UPDATE hot_test SET col_c = 35;
+
+-- Update data (not indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+
+-- Partitioned tables: HOT works within partitions
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+
+CREATE TABLE hot_test_partitioned (
+ id int,
+ partition_key int,
+ indexed_col int,
+ data text,
+ PRIMARY KEY (id, partition_key)
+) PARTITION BY RANGE (partition_key);
+
+CREATE TABLE hot_test_part1 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (1) TO (100) WITH (fillfactor = 50);
+CREATE TABLE hot_test_part2 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (100) TO (200) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_part_idx ON hot_test_partitioned(indexed_col);
+
+INSERT INTO hot_test_partitioned VALUES (1, 50, 100, 'initial1');
+INSERT INTO hot_test_partitioned VALUES (2, 150, 200, 'initial2');
+
+-- Update in partition 1 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'updated1' WHERE id = 1;
+
+-- Update in partition 2 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'updated2' WHERE id = 2;
+
+SELECT * FROM get_hot_count('hot_test_part1');
+SELECT * FROM get_hot_count('hot_test_part2');
+
+-- Verify indexes work on partitions
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 100;
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 200;
+
+-- Update indexed column in partition - should NOT be HOT
+UPDATE hot_test_partitioned SET indexed_col = 150 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test_part1');
+
+-- Verify index was updated
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 150;
+
+-- ============================================================================
+-- Trigger modifications: heap_modify_tuple() and HOT
+-- ============================================================================
+-- Test that we correctly detect when triggers modify indexed columns via
+-- heap_modify_tuple(), even when those columns aren't in the UPDATE's SET clause
+
+CREATE TABLE hot_trigger_test (
+ id int PRIMARY KEY,
+ triggered_col int,
+ data text
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_trigger_idx ON hot_trigger_test(triggered_col);
+
+-- Create a trigger that modifies an indexed column
+CREATE OR REPLACE FUNCTION modify_triggered_col()
+RETURNS TRIGGER AS $$
+BEGIN
+ NEW.triggered_col = NEW.triggered_col + 1;
+ RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE TRIGGER before_update_modify
+ BEFORE UPDATE ON hot_trigger_test
+ FOR EACH ROW
+ EXECUTE FUNCTION modify_triggered_col();
+
+INSERT INTO hot_trigger_test VALUES (1, 100, 'initial');
+
+SELECT * FROM get_hot_count('hot_trigger_test');
+
+-- Update only data column, but trigger modifies indexed column
+-- Should NOT be HOT because trigger modified an indexed column
+UPDATE hot_trigger_test SET data = 'updated' WHERE id = 1;
+
+-- Verify it was NOT a HOT update (indexed column was modified by trigger)
+SELECT * FROM get_hot_count('hot_trigger_test');
+
+-- Verify the triggered column was actually modified
+SELECT triggered_col FROM hot_trigger_test WHERE id = 1;
+
+DROP TABLE hot_trigger_test CASCADE;
+DROP FUNCTION modify_triggered_col();
+
+-- ============================================================================
+-- JSONB expression indexes and sub-attribute tracking
+-- ============================================================================
+-- Test that updates to non-indexed JSONB paths can be HOT updates
+
+CREATE TABLE hot_jsonb_test (
+ id int PRIMARY KEY,
+ data jsonb
+) WITH (fillfactor = 50);
+
+-- Create expression index on a specific JSON path
+CREATE INDEX hot_jsonb_name_idx ON hot_jsonb_test ((data->>'name'));
+
+INSERT INTO hot_jsonb_test VALUES
+ (1, '{"name":"Alice","age":30,"city":"NYC"}'),
+ (2, '{"name":"Bob","age":25,"city":"LA"}');
+
+SELECT * FROM get_hot_count('hot_jsonb_test');
+
+-- Update non-indexed JSON path (age) - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = jsonb_set(data, '{age}', '31') WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_jsonb_test');
+
+-- Update indexed JSON path (name) - should NOT be HOT
+UPDATE hot_jsonb_test SET data = jsonb_set(data, '{name}', '"Alice2"') WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_jsonb_test');
+
+-- Verify index works
+SELECT id FROM hot_jsonb_test WHERE data->>'name' = 'Alice2';
+
+-- Test jsonb_delete on non-indexed path - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = data - 'city' WHERE id = 2;
+
+SELECT * FROM get_hot_count('hot_jsonb_test');
+
+-- Test jsonb_insert on non-indexed path - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = jsonb_insert(data, '{country}', '"USA"') WHERE id = 2;
+
+SELECT * FROM get_hot_count('hot_jsonb_test');
+
+DROP TABLE hot_jsonb_test;
+
+-- ============================================================================
+-- XML expression indexes and sub-attribute tracking
+-- ============================================================================
+-- Test that updates to non-indexed XML paths can be HOT updates
+
+CREATE TABLE hot_xml_test (
+ id int PRIMARY KEY,
+ doc xml
+) WITH (fillfactor = 50);
+
+-- Create expression index on a specific XPath
+CREATE INDEX hot_xml_name_idx ON hot_xml_test ((xpath('/person/name/text()', doc)));
+
+INSERT INTO hot_xml_test VALUES
+ (1, '<person><name>Alice</name><age>30</age></person>'),
+ (2, '<person><name>Bob</name><age>25</age></person>');
+
+SELECT * FROM get_hot_count('hot_xml_test');
+
+-- Update non-indexed XPath (age) - behavior depends on XML comparison fallback
+-- Full XML value replacement means non-indexed path updates still require index comparison
+UPDATE hot_xml_test SET doc = '<person><name>Alice</name><age>31</age></person>' WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_xml_test');
+
+-- Update indexed XPath (name) - should NOT be HOT
+UPDATE hot_xml_test SET doc = '<person><name>Alice2</name><age>31</age></person>' WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_xml_test');
+
+-- Verify index works
+SELECT id FROM hot_xml_test WHERE xpath('/person/name/text()', doc) = ARRAY['Alice2'::text];
+
+DROP TABLE hot_xml_test;
+
+-- ============================================================================
+-- GIN indexes and amcomparedatums for JSONB
+-- ============================================================================
+-- Test that GIN indexes can use amcomparedatums to enable HOT when extracted keys match
+
+CREATE TABLE hot_gin_test (
+ id int PRIMARY KEY,
+ tags text[],
+ properties jsonb
+) WITH (fillfactor = 50);
+
+-- GIN index on text array
+CREATE INDEX hot_gin_tags_idx ON hot_gin_test USING gin (tags);
+
+-- GIN index on JSONB (jsonb_ops - keys and values)
+CREATE INDEX hot_gin_props_idx ON hot_gin_test USING gin (properties);
+
+INSERT INTO hot_gin_test VALUES
+ (1, ARRAY['tag1', 'tag2'], '{"key1":"val1","key2":"val2"}'),
+ (2, ARRAY['tag3', 'tag4'], '{"key3":"val3","key4":"val4"}');
+
+SELECT * FROM get_hot_count('hot_gin_test');
+
+-- Update that changes tag order but not content - after amcomparedatums should be HOT
+-- (GIN extracts same keys, just different order)
+UPDATE hot_gin_test SET tags = ARRAY['tag2', 'tag1'] WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_gin_test');
+
+-- Update JSONB value (not key) - after amcomparedatums may be HOT or non-HOT
+-- depending on GIN operator class (jsonb_ops indexes both keys and values)
+UPDATE hot_gin_test SET properties = '{"key1":"val1_new","key2":"val2"}' WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_gin_test');
+
+-- Add new tag - should NOT be HOT (different extracted keys)
+UPDATE hot_gin_test SET tags = ARRAY['tag2', 'tag1', 'tag5'] WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_gin_test');
+
+-- Verify GIN indexes work
+SELECT id FROM hot_gin_test WHERE tags @> ARRAY['tag5'];
+SELECT id FROM hot_gin_test WHERE properties @> '{"key1":"val1_new"}';
+
+DROP TABLE hot_gin_test;
+
+-- ============================================================================
+-- Cleanup
+-- ============================================================================
+DROP TABLE IF EXISTS hot_test;
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+DROP FUNCTION IF EXISTS has_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS print_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS get_hot_count(text);
+DROP EXTENSION pageinspect;
--
2.51.2
[text/x-patch] v38-0002-Identify-modified-indexed-attributes-in-the-exec.patch (60.5K, 3-v38-0002-Identify-modified-indexed-attributes-in-the-exec.patch)
download | inline diff:
From 5e6e0414192294882ccfbd1c731f12cebf2d507f Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Tue, 10 Mar 2026 08:17:31 -0400
Subject: [PATCH v38 2/2] Identify modified indexed attributes in the executor
on UPDATE
Refactor executor update logic to determine which indexed columns have
actually changed during an UPDATE operation rather than leaving this up
to HeapDetermineColumnsInfo() in heap_update(). Finding this set of
attributes is not heap-specific, but more general to all table AMs and
having this information in the executor could inform other decisions
about when index inserts are required and when they are not regardless
of the table AM's MVCC implementation strategy.
The heap-only tuple decision (HOT) in heap functions as it always has,
but the determination of the "modified indexed attributes"
(modified_idx_attrs, formerly known as modified_attrs).
ExecUpdateModifiedIdxAttrs() replaces HeapDetermineColumnsInfo() and is
called before table_tuple_update() crucially without the need for an
exclusive buffer lock on the page that holds the tuple being updated.
This reduces the time the buffer lock is held later within
heapam_tuple_update() and heap_update().
Besides identifying the set of modified indexed attributes
HeapDetermineColumnsInfo() was also partially responsible for the
decision about what to WAL log for the replica identity key. This logic
moved into heap_update() and out of the replacement named
HeapUpdateModifiedIdxAttrs(). Doing this allows for
simple_heap_update() and heapam_tuple_update() to share the same logic
as they both call into heap_update().
Updates stemming from logical replication also use the new
ExecUpdateModifiedIdxAttrs() in ExecSimpleRelationUpdate().
ExecUpdateModifiedIdxAttrs() uses ExecCompareSlotAttrs() to identify
which attributes have changed and then intersects that with the set of
indexed attributes to identify the modified indexed set, the
modified_idx_attrs.
This patch introduces a few helper functions to reduce code duplication
and increase readability: HeapUpdateHotAllowable(),
HeapUpdateDetermineLockmode(). These are used in both heap_update() and
simple_heap_update().
The heap_update() function is called now with lockmode pre-determined
and a boolean indicating if the update allows HOT updates or not, both
const. If during heap_update() the new tuple will fit on the same page
and that boolean is true, the update is HOT. This means that although
the functions and timing of the code involed in HOT decisions have
changed, none of the logic related to when HOT is allowed has changed.
Development of this feature exposed nondeterministic behavior in three
existing tests which have been adjusted to avoid inconsistent test
results due to tuple ordering during heap page scans.
---
src/backend/access/heap/heapam.c | 463 ++++++++++++------
src/backend/access/heap/heapam_handler.c | 31 +-
src/backend/access/table/tableam.c | 5 +-
src/backend/executor/execReplication.c | 9 +-
src/backend/executor/execTuples.c | 70 +++
src/backend/executor/nodeModifyTable.c | 88 +++-
src/backend/utils/cache/relcache.c | 44 +-
src/include/access/heapam.h | 13 +-
src/include/access/tableam.h | 8 +-
src/include/executor/executor.h | 8 +
src/include/utils/rel.h | 2 +-
src/include/utils/relcache.h | 2 +-
.../expected/syscache-update-pruned.out | 12 +-
.../specs/syscache-update-pruned.spec | 6 +-
.../regress/expected/generated_virtual.out | 2 +-
src/test/regress/expected/triggers.out | 16 +-
src/test/regress/expected/tsearch.out | 3 +-
src/test/regress/expected/updatable_views.out | 4 +-
src/test/regress/sql/generated_virtual.sql | 2 +-
src/test/regress/sql/triggers.sql | 4 +-
src/test/regress/sql/tsearch.sql | 3 +-
src/test/regress/sql/updatable_views.sql | 2 +-
22 files changed, 573 insertions(+), 224 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e5bd062de77..dcb8a34b7bf 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -37,21 +37,26 @@
#include "access/multixact.h"
#include "access/subtrans.h"
#include "access/syncscan.h"
+#include "access/sysattr.h"
+#include "access/tableam.h"
#include "access/valid.h"
#include "access/visibilitymap.h"
#include "access/xloginsert.h"
#include "catalog/pg_database.h"
#include "catalog/pg_database_d.h"
#include "commands/vacuum.h"
+#include "executor/tuptable.h"
+#include "nodes/lockoptions.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
+#include "storage/buf.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
-#include "storage/proc.h"
#include "storage/procarray.h"
#include "utils/datum.h"
#include "utils/injection_point.h"
#include "utils/inval.h"
+#include "utils/relcache.h"
#include "utils/spccache.h"
#include "utils/syscache.h"
@@ -68,11 +73,8 @@ static void check_lock_if_inplace_updateable_rel(Relation relation,
HeapTuple newtup);
static void check_inplace_rel_lock(HeapTuple oldtup);
#endif
-static Bitmapset *HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external);
+static Bitmapset *HeapUpdateModifiedIdxAttrs(Relation relation,
+ HeapTuple oldtup, HeapTuple newtup);
static bool heap_acquire_tuplock(Relation relation, const ItemPointerData *tid,
LockTupleMode mode, LockWaitPolicy wait_policy,
bool *have_tuple_lock);
@@ -3312,7 +3314,7 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* heap_update - replace a tuple
*
* See table_tuple_update() for an explanation of the parameters, except that
- * this routine directly takes a tuple rather than a slot.
+ * this routine directly takes a heap tuple rather than a slot.
*
* In the failure cases, the routine fills *tmfd with the tuple's t_ctid,
* t_xmax (resolving a possible MultiXact, if necessary), and t_cmax (the last
@@ -3322,17 +3324,13 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
TM_Result
heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ TM_FailureData *tmfd, const LockTupleMode lockmode,
+ const Bitmapset *modified_idx_attrs, const bool hot_allowed)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
- Bitmapset *hot_attrs;
- Bitmapset *sum_attrs;
- Bitmapset *key_attrs;
- Bitmapset *id_attrs;
- Bitmapset *interesting_attrs;
- Bitmapset *modified_attrs;
+ Bitmapset *idx_attrs,
+ *id_attrs;
ItemId lp;
HeapTupleData oldtup;
HeapTuple heaptup;
@@ -3352,13 +3350,12 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
bool have_tuple_lock = false;
bool iscombo;
bool use_hot_update = false;
- bool summarized_update = false;
bool key_intact;
bool all_visible_cleared = false;
bool all_visible_cleared_new = false;
bool checked_lockers;
bool locker_remains;
- bool id_has_external = false;
+ bool rep_id_key_required = false;
TransactionId xmax_new_tuple,
xmax_old_tuple;
uint16 infomask_old_tuple,
@@ -3389,33 +3386,18 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
#endif
/*
- * Fetch the list of attributes to be checked for various operations.
- *
- * For HOT considerations, this is wasted effort if we fail to update or
- * have to put the new tuple on a different page. But we must compute the
- * list before obtaining buffer lock --- in the worst case, if we are
- * doing an update on one of the relevant system catalogs, we could
- * deadlock if we try to fetch the list later. In any case, the relcache
- * caches the data so this is usually pretty cheap.
- *
- * We also need columns used by the replica identity and columns that are
- * considered the "key" of rows in the table.
+ * Fetch the attributes used across all indexes on this relation as well
+ * as the replica identity and columns.
*
- * Note that we get copies of each bitmap, so we need not worry about
- * relcache flush happening midway through.
- */
- hot_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_HOT_BLOCKING);
- sum_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_SUMMARIZED);
- key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
- id_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
- interesting_attrs = NULL;
- interesting_attrs = bms_add_members(interesting_attrs, hot_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, sum_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, key_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, id_attrs);
+ * Note: We must compute the list before obtaining buffer lock. In the
+ * worst case, if we are doing an update on one of the relevant system
+ * catalogs, we could deadlock if we try to fetch the list later. Keep in
+ * mind that relcache returns copies of each bitmap, so we need not worry
+ * about relcache flush happening midway through, but we do need to free
+ * them.
+ */
+ idx_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+ id_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_IDENTITY_KEY);
block = ItemPointerGetBlockNumber(otid);
INJECTION_POINT("heap_update-before-pin", NULL);
@@ -3469,20 +3451,17 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
tmfd->ctid = *otid;
tmfd->xmax = InvalidTransactionId;
tmfd->cmax = InvalidCommandId;
- *update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
bms_free(id_attrs);
- /* modified_attrs not yet initialized */
- bms_free(interesting_attrs);
+ bms_free(idx_attrs);
+ /* modified_idx_attrs is owned by the caller, don't free it */
+
return TM_Deleted;
}
/*
- * Fill in enough data in oldtup for HeapDetermineColumnsInfo to work
- * properly.
+ * Fill in enough data in oldtup to determine replica identity attribute
+ * requirements.
*/
oldtup.t_tableOid = RelationGetRelid(relation);
oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
@@ -3493,16 +3472,59 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
newtup->t_tableOid = RelationGetRelid(relation);
/*
- * Determine columns modified by the update. Additionally, identify
- * whether any of the unmodified replica identity key attributes in the
- * old tuple is externally stored or not. This is required because for
- * such attributes the flattened value won't be WAL logged as part of the
- * new tuple so we must include it as part of the old_key_tuple. See
- * ExtractReplicaIdentity.
+ * ExtractReplicaIdentity() needs to know if a modified indexed attrbute
+ * is used as a replica indentity or if any of the replica identity
+ * attributes are referenced in an index, unmodified, and are stored
+ * externally in the old tuple being replaced. In those cases it may be
+ * necessary to WAL log them to so they are available to replicas.
*/
- modified_attrs = HeapDetermineColumnsInfo(relation, interesting_attrs,
- id_attrs, &oldtup,
- newtup, &id_has_external);
+ rep_id_key_required = bms_overlap(modified_idx_attrs, id_attrs);
+ if (!rep_id_key_required)
+ {
+ Bitmapset *attrs;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ int attidx = -1;
+
+ /*
+ * Reduce the set under review to only the unmodified indexed replica
+ * identity key attributes. idx_attrs is copied (by bms_difference())
+ * not modified here.
+ */
+ attrs = bms_difference(idx_attrs, modified_idx_attrs);
+ attrs = bms_int_members(attrs, id_attrs);
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /*
+ * attidx is zero-based, attrnum is the normal attribute number
+ */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value;
+ bool isnull;
+
+ /*
+ * System attributes are not added into INDEX_ATTR_BITMAP_INDEXED
+ * bitmap by relcache.
+ */
+ Assert(attrnum > 0);
+
+ value = heap_getattr(&oldtup, attrnum, tupdesc, &isnull);
+
+ /* No need to check attributes that can't be stored externally */
+ if (isnull ||
+ TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
+ continue;
+
+ /* Check if the old tuple's attribute is stored externally */
+ if (VARATT_IS_EXTERNAL((struct varlena *) DatumGetPointer(value)))
+ {
+ rep_id_key_required = true;
+ break;
+ }
+ }
+
+ bms_free(attrs);
+ }
/*
* If we're not updating any "key" column, we can grab a weaker lock type.
@@ -3515,9 +3537,8 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
* is updates that don't manipulate key columns, not those that
* serendipitously arrive at the same key values.
*/
- if (!bms_overlap(modified_attrs, key_attrs))
+ if (lockmode == LockTupleNoKeyExclusive)
{
- *lockmode = LockTupleNoKeyExclusive;
mxact_status = MultiXactStatusNoKeyUpdate;
key_intact = true;
@@ -3534,7 +3555,7 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
}
else
{
- *lockmode = LockTupleExclusive;
+ Assert(lockmode == LockTupleExclusive);
mxact_status = MultiXactStatusUpdate;
key_intact = false;
}
@@ -3613,7 +3634,7 @@ l2:
bool current_is_member = false;
if (DoesMultiXactIdConflict((MultiXactId) xwait, infomask,
- *lockmode, ¤t_is_member))
+ lockmode, ¤t_is_member))
{
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
@@ -3622,7 +3643,7 @@ l2:
* requesting a lock and already have one; avoids deadlock).
*/
if (!current_is_member)
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &(oldtup.t_self), lockmode,
LockWaitBlock, &have_tuple_lock);
/* wait for multixact */
@@ -3707,7 +3728,7 @@ l2:
* lock.
*/
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &(oldtup.t_self), lockmode,
LockWaitBlock, &have_tuple_lock);
XactLockTableWait(xwait, relation, &oldtup.t_self,
XLTW_Update);
@@ -3767,17 +3788,14 @@ l2:
tmfd->cmax = InvalidCommandId;
UnlockReleaseBuffer(buffer);
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &(oldtup.t_self), lockmode);
if (vmbuffer != InvalidBuffer)
ReleaseBuffer(vmbuffer);
- *update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
+ bms_free(idx_attrs);
+ /* modified_idx_attrs is owned by the caller, don't free it */
+
return result;
}
@@ -3807,7 +3825,7 @@ l2:
compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
oldtup.t_data->t_infomask,
oldtup.t_data->t_infomask2,
- xid, *lockmode, true,
+ xid, lockmode, true,
&xmax_old_tuple, &infomask_old_tuple,
&infomask2_old_tuple);
@@ -3924,7 +3942,7 @@ l2:
compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
oldtup.t_data->t_infomask,
oldtup.t_data->t_infomask2,
- xid, *lockmode, false,
+ xid, lockmode, false,
&xmax_lock_old_tuple, &infomask_lock_old_tuple,
&infomask2_lock_old_tuple);
@@ -4097,20 +4115,8 @@ l2:
* to do a HOT update. Check if any of the index columns have been
* changed.
*/
- if (!bms_overlap(modified_attrs, hot_attrs))
- {
+ if (hot_allowed)
use_hot_update = true;
-
- /*
- * If none of the columns that are used in hot-blocking indexes
- * were updated, we can apply HOT, but we do still need to check
- * if we need to update the summarizing indexes, and update those
- * indexes if the columns were updated, or we may fail to detect
- * e.g. value bound changes in BRIN minmax indexes.
- */
- if (bms_overlap(modified_attrs, sum_attrs))
- summarized_update = true;
- }
}
else
{
@@ -4126,8 +4132,7 @@ l2:
* columns are modified or it has external data.
*/
old_key_tuple = ExtractReplicaIdentity(relation, &oldtup,
- bms_overlap(modified_attrs, id_attrs) ||
- id_has_external,
+ rep_id_key_required,
&old_key_copied);
/* NO EREPORT(ERROR) from here till changes are logged */
@@ -4256,7 +4261,7 @@ l2:
* Release the lmgr tuple lock, if we had it.
*/
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &(oldtup.t_self), lockmode);
pgstat_count_heap_update(relation, use_hot_update, newbuf != buffer);
@@ -4270,31 +4275,12 @@ l2:
heap_freetuple(heaptup);
}
- /*
- * If it is a HOT update, the update may still need to update summarized
- * indexes, lest we fail to update those summaries and get incorrect
- * results (for example, minmax bounds of the block may change with this
- * update).
- */
- if (use_hot_update)
- {
- if (summarized_update)
- *update_indexes = TU_Summarizing;
- else
- *update_indexes = TU_None;
- }
- else
- *update_indexes = TU_All;
-
if (old_key_tuple != NULL && old_key_copied)
heap_freetuple(old_key_tuple);
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
+ bms_free(idx_attrs);
+ /* modified_idx_attrs is owned by the caller, don't free it */
return TM_Ok;
}
@@ -4467,28 +4453,115 @@ heap_attr_equals(TupleDesc tupdesc, int attrnum, Datum value1, Datum value2,
}
/*
- * Check which columns are being updated.
- *
- * Given an updated tuple, determine (and return into the output bitmapset),
- * from those listed as interesting, the set of columns that changed.
- *
- * has_external indicates if any of the unmodified attributes (from those
- * listed as interesting) of the old tuple is a member of external_cols and is
- * stored externally.
+ * HOT updates are possible when either: a) there are no modified indexed
+ * attributes, or b) the modified attributes are all on summarizing indexes.
+ * Later, in heap_update(), we can choose to perform a HOT update if there is
+ * space on the page for the new tuple and the following code has determined
+ * that HOT is allowed.
+ */
+bool
+HeapUpdateHotAllowable(Relation relation, const Bitmapset *modified_idx_attrs,
+ bool *summarized_only)
+{
+ bool hot_allowed;
+
+ /*
+ * Let's be optimistic and start off by assuming the best case, no indexes
+ * need updating and HOT is allowable.
+ */
+ hot_allowed = true;
+ *summarized_only = false;
+
+ /*
+ * Check for case (a); when there are no modified index attributes HOT is
+ * allowed.
+ */
+ if (bms_is_empty(modified_idx_attrs))
+ hot_allowed = true;
+ else
+ {
+ Bitmapset *sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+
+ /*
+ * At least one index attribute was modified, but is this case (b)
+ * where all the modified index attributes are only used by
+ * summarizing indexes? If it is, then we need to update those
+ * indexes, but this update can still be considered heap-only (HOT)
+ * and avoid updating any non-summarizing indexes on the relation.
+ */
+ if (bms_is_subset(modified_idx_attrs, sum_attrs))
+ {
+ hot_allowed = true;
+ *summarized_only = true;
+ }
+ else
+ {
+ /*
+ * Now we know a) one or more indexed attributes were modified
+ * (changed value, not just referenced within the UPDATE) and that
+ * b) at least one of those attributes is used by a
+ * non-summarizing index. HOT is not allowed.
+ */
+ hot_allowed = false;
+ }
+
+ bms_free(sum_attrs);
+ }
+
+ return hot_allowed;
+}
+
+/*
+ * If we're not updating any attributes used when forming the index keys we can
+ * grab a weaker lock type. This allows for more concurrency when we are
+ * running simultaneously with foreign key checks.
+ */
+LockTupleMode
+HeapUpdateDetermineLockmode(Relation relation, const Bitmapset *modified_idx_attrs)
+{
+ LockTupleMode lockmode = LockTupleExclusive;
+
+ Bitmapset *key_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_KEY);
+
+ if (!bms_overlap(modified_idx_attrs, key_attrs))
+ lockmode = LockTupleNoKeyExclusive;
+
+ bms_free(key_attrs);
+
+ return lockmode;
+}
+
+/*
+ * Return a Bitmapset that contains the set of modified (changed) indexed
+ * attributes between oldtup and newtup.
*/
static Bitmapset *
-HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external)
+HeapUpdateModifiedIdxAttrs(Relation relation, HeapTuple oldtup, HeapTuple newtup)
{
int attidx;
- Bitmapset *modified = NULL;
+ Bitmapset *attrs,
+ *modified_idx_attrs = NULL;
TupleDesc tupdesc = RelationGetDescr(relation);
+ /* Get the set of all attributes across all indexes for this relation */
+ attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+
+ /* No indexed attributes, we're done */
+ if (bms_is_empty(attrs))
+ return NULL;
+
+ /*
+ * This heap update function is used outside the executor and so unlike
+ * heapam_tuple_update() where there is ResultRelInfo and EState to
+ * provide the concise set of attributes that might have been modified
+ * (via ExecGetAllUpdatedCols()) we simply check all indexed attributes to
+ * find the subset that changed value. That's the "modified indexed
+ * attributes" or "modified_idx_attrs".
+ */
attidx = -1;
- while ((attidx = bms_next_member(interesting_cols, attidx)) >= 0)
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
{
/* attidx is zero-based, attrnum is the normal attribute number */
AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
@@ -4504,7 +4577,7 @@ HeapDetermineColumnsInfo(Relation relation,
*/
if (attrnum == 0)
{
- modified = bms_add_member(modified, attidx);
+ modified_idx_attrs = bms_add_member(modified_idx_attrs, attidx);
continue;
}
@@ -4517,7 +4590,7 @@ HeapDetermineColumnsInfo(Relation relation,
{
if (attrnum != TableOidAttributeNumber)
{
- modified = bms_add_member(modified, attidx);
+ modified_idx_attrs = bms_add_member(modified_idx_attrs, attidx);
continue;
}
}
@@ -4533,29 +4606,12 @@ HeapDetermineColumnsInfo(Relation relation,
if (!heap_attr_equals(tupdesc, attrnum, value1,
value2, isnull1, isnull2))
- {
- modified = bms_add_member(modified, attidx);
- continue;
- }
-
- /*
- * No need to check attributes that can't be stored externally. Note
- * that system attributes can't be stored externally.
- */
- if (attrnum < 0 || isnull1 ||
- TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
- continue;
-
- /*
- * Check if the old tuple's attribute is stored externally and is a
- * member of external_cols.
- */
- if (VARATT_IS_EXTERNAL((varlena *) DatumGetPointer(value1)) &&
- bms_is_member(attidx, external_cols))
- *has_external = true;
+ modified_idx_attrs = bms_add_member(modified_idx_attrs, attidx);
}
- return modified;
+ bms_free(attrs);
+
+ return modified_idx_attrs;
}
/*
@@ -4573,11 +4629,106 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
TM_Result result;
TM_FailureData tmfd;
LockTupleMode lockmode;
+ TupleTableSlot *slot;
+ BufferHeapTupleTableSlot *bslot;
+ HeapTuple oldtup;
+ bool shouldFree = true;
+ Bitmapset *idx_attrs,
+ *modified_idx_attrs;
+ bool hot_allowed,
+ summarized_only;
+ Buffer buffer;
- result = heap_update(relation, otid, tup,
- GetCurrentCommandId(true), InvalidSnapshot,
- true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ Assert(ItemPointerIsValid(otid));
+
+ /*
+ * Fetch this bitmap of interesting attributes from relcache before
+ * obtaining a buffer lock because if we are doing an update on one of the
+ * relevant system catalogs we could deadlock if we try to fetch them
+ * later on. Relcache will return copies of each bitmap, so we need not
+ * worry about relcache flush happening midway through this operation.
+ */
+ idx_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+
+ INJECTION_POINT("simple_heap_update-before-pin", NULL);
+
+ /*
+ * To update a heap tuple we need to find the set of modified indexed
+ * attributes ("modified_idx_attrs") and use that to determine if a HOT
+ * update is allowable or not. When updating heap tuples via execution of
+ * UPDATE statements this set is constructed before calling into the table
+ * AM's update function by ExecUpdateModifiedIdxAttrs() which compares the
+ * old/new TupleTableSlots.
+ *
+ * Here things are a bit different, we have the old TID and the new tuple,
+ * not two TupleTableSlots, but we still need to construct a similar
+ * bitmap so as to be able to know if HOT updates are allowed or not.
+ *
+ * To do that we first have to fetch the old tuple itself, but because
+ * heapam_fetch_row_version() is static, we replicate in part that code
+ * here.
+ *
+ * This is a bit repetitive because heap_update() will again find and form
+ * the old HeapTuple from the old TID and in most cases the callers
+ * (ignoring extensions, are always catalog tuple updates) already had the
+ * set of changed attributes (the "replaces" array), but for now this
+ * minor repetition of work is necessary.
+ */
+ slot = MakeTupleTableSlot(RelationGetDescr(relation), &TTSOpsBufferHeapTuple, 0);
+ bslot = (BufferHeapTupleTableSlot *) slot;
+
+ /*
+ * Set the TID in the slot and then fetch the old tuple so we can examine
+ * it
+ */
+ bslot->base.tupdata.t_self = *otid;
+ if (!heap_fetch(relation, SnapshotAny, &bslot->base.tupdata, &buffer, false))
+ {
+ /*
+ * heap_update() checks for !ItemIdIsNormal(lp) and will return false
+ * in those cases.
+ */
+ Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
+
+ *update_indexes = TU_None;
+
+ /* modified_idx_attrs not yet initialized */
+ bms_free(idx_attrs);
+ ExecDropSingleTupleTableSlot(slot);
+
+ elog(ERROR, "tuple concurrently deleted");
+
+ return;
+ }
+
+ Assert(buffer != InvalidBuffer);
+
+ /* Store in slot, transferring existing pin */
+ ExecStorePinnedBufferHeapTuple(&bslot->base.tupdata, slot, buffer);
+ oldtup = ExecFetchSlotHeapTuple(slot, false, &shouldFree);
+
+ modified_idx_attrs = HeapUpdateModifiedIdxAttrs(relation, oldtup, tup);
+ lockmode = HeapUpdateDetermineLockmode(relation, modified_idx_attrs);
+ hot_allowed = HeapUpdateHotAllowable(relation, modified_idx_attrs, &summarized_only);
+
+ result = heap_update(relation, otid, tup, GetCurrentCommandId(true),
+ InvalidSnapshot, true /* wait for commit */ ,
+ &tmfd, lockmode, modified_idx_attrs, hot_allowed);
+
+ if (shouldFree)
+ heap_freetuple(oldtup);
+
+ ExecDropSingleTupleTableSlot(slot);
+ bms_free(idx_attrs);
+
+ /*
+ * Decide whether new index entries are needed for the tuple
+ *
+ * If the update is not HOT, we must update all indexes. If the update is
+ * HOT, it could be that we updated summarized columns, so we either
+ * update only summarized indexes, or none at all.
+ */
+ *update_indexes = TU_None;
switch (result)
{
case TM_SelfModified:
@@ -4587,6 +4738,10 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
case TM_Ok:
/* done successfully */
+ if (!HeapTupleIsHeapOnly(tup))
+ *update_indexes = TU_All;
+ else if (summarized_only)
+ *update_indexes = TU_Summarizing;
break;
case TM_Updated:
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 253a735b6c1..3726c867c65 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,7 +27,6 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
-#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -325,19 +324,26 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd,
- LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
+ bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
+ const Bitmapset *modified_idx_attrs, TU_UpdateIndexes *update_indexes)
{
bool shouldFree = true;
HeapTuple tuple = ExecFetchSlotHeapTuple(slot, true, &shouldFree);
+ bool hot_allowed;
+ bool summarized_only;
TM_Result result;
+ Assert(ItemPointerIsValid(otid));
+
+ hot_allowed = HeapUpdateHotAllowable(relation, modified_idx_attrs, &summarized_only);
+ *lockmode = HeapUpdateDetermineLockmode(relation, modified_idx_attrs);
+
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
- tmfd, lockmode, update_indexes);
+ tmfd, *lockmode, modified_idx_attrs, hot_allowed);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
@@ -350,16 +356,17 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
* HOT, it could be that we updated summarized columns, so we either
* update only summarized indexes, or none at all.
*/
- if (result != TM_Ok)
+ *update_indexes = TU_None;
+ if (result == TM_Ok)
{
- Assert(*update_indexes == TU_None);
- *update_indexes = TU_None;
+ if (HeapTupleIsHeapOnly(tuple))
+ {
+ if (summarized_only)
+ *update_indexes = TU_Summarizing;
+ }
+ else
+ *update_indexes = TU_All;
}
- else if (!HeapTupleIsHeapOnly(tuple))
- Assert(*update_indexes == TU_All);
- else
- Assert((*update_indexes == TU_Summarizing) ||
- (*update_indexes == TU_None));
if (shouldFree)
pfree(tuple);
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index dfda1af412e..9ba72d51dfa 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -359,6 +359,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
+ const Bitmapset *modified_idx_attrs,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -369,7 +370,9 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
GetCurrentCommandId(true),
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, &lockmode,
+ modified_idx_attrs,
+ update_indexes);
switch (result)
{
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index 2497ee7edc5..8a269dd2f6c 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -33,6 +33,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
#include "utils/typcache.h"
@@ -906,6 +907,7 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
bool skip_tuple = false;
Relation rel = resultRelInfo->ri_RelationDesc;
ItemPointer tid = &(searchslot->tts_tid);
+ Bitmapset *modified_idx_attrs;
/*
* We support only non-system tables, with
@@ -944,8 +946,13 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ modified_idx_attrs = ExecUpdateModifiedIdxAttrs(resultRelInfo,
+ searchslot, slot);
+
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
- &update_indexes);
+ modified_idx_attrs, &update_indexes);
+ bms_free(modified_idx_attrs);
+
conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 9d900147a55..19054109a77 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -66,6 +66,7 @@
#include "nodes/nodeFuncs.h"
#include "storage/bufmgr.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/expandeddatum.h"
#include "utils/lsyscache.h"
#include "utils/typcache.h"
@@ -2005,6 +2006,75 @@ ExecFetchSlotHeapTupleDatum(TupleTableSlot *slot)
return ret;
}
+/*
+ * ExecCompareSlotAttrs
+ *
+ * Compare the subset of attributes in attrs bewtween TupleTableSlots to detect
+ * which attributes have changed.
+ *
+ * Returns a reused when possible Bitmapset of attribute indices (using
+ * FirstLowInvalidHeapAttributeNumber convention) that differ between the two
+ * slots.
+ */
+Bitmapset *
+ExecCompareSlotAttrs(Bitmapset *attrs, TupleDesc tupdesc,
+ TupleTableSlot *s1, TupleTableSlot *s2)
+{
+ int attidx = -1;
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /* attidx is zero-based, attrnum is the normal attribute number */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value1,
+ value2;
+ bool null1,
+ null2;
+ CompactAttribute *att;
+
+ /*
+ * If it's a whole-tuple reference, say "not equal". It's not really
+ * worth supporting this case, since it could only succeed after a
+ * no-op update, which is hardly a case worth optimizing for.
+ */
+ if (attrnum == 0)
+ continue;
+
+ /*
+ * Likewise, automatically say "not equal" for any system attribute
+ * other than tableOID; we cannot expect these to be consistent in a
+ * HOT chain, or even to be set correctly yet in the new tuple.
+ */
+ if (attrnum < 0)
+ {
+ if (attrnum == TableOidAttributeNumber)
+ attrs = bms_del_member(attrs, attidx);
+ else
+ continue;
+ }
+
+ att = TupleDescCompactAttr(tupdesc, attrnum - 1);
+ value1 = slot_getattr(s1, attrnum, &null1);
+ value2 = slot_getattr(s2, attrnum, &null2);
+
+ /* A change to/from NULL, so not equal */
+ if (null1 != null2)
+ continue;
+
+ /* Both NULL, no change/unmodified */
+ if (null2)
+ {
+ attrs = bms_del_member(attrs, attidx);
+ continue;
+ }
+
+ if (datum_image_eq(value1, value2, att->attbyval, att->attlen))
+ attrs = bms_del_member(attrs, attidx);
+ }
+
+ return attrs;
+}
+
/* ----------------------------------------------------------------
* convenience initialization routines
* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 4cd5e262e0f..4c0c5a03026 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -17,6 +17,7 @@
* ExecModifyTable - retrieve the next tuple from the node
* ExecEndModifyTable - shut down the ModifyTable node
* ExecReScanModifyTable - rescan the ModifyTable node
+ * ExecUpdateModifiedIdxAttrs - find set of updated indexed columns
*
* NOTES
* The ModifyTable node receives input from its outerPlan, which is
@@ -55,6 +56,7 @@
#include "access/htup_details.h"
#include "access/tableam.h"
#include "access/tupconvert.h"
+#include "access/tupdesc.h"
#include "access/xact.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
@@ -190,6 +192,63 @@ static TupleTableSlot *ExecMergeNotMatched(ModifyTableContext *context,
ResultRelInfo *resultRelInfo,
bool canSetTag);
+/*
+ * ExecUpdateModifiedIdxAttrs
+ *
+ * Find the set of attributes referenced by this relation and used in this
+ * UPDATE that now differ in value. This is done by reviewing slot datum that
+ * are in the UPDATE statment and are known to be referenced by at least one
+ * index in some way. This set is called the "modified indexed attributes" or
+ * "modified_idx_attrs". An overlap of a single index's attributes and this
+ * modified_idx_attrs set signals that the attributes in the new_tts used to
+ * form the index datum have changed.
+ *
+ * Return a Bitmapset that contains the set of modified (changed) indexed
+ * attributes between oldtup and newtup.
+ *
+ * Note: There is a similar function called HeapUpdateModifiedIdxAttrs() that operates
+ * on the old TID and new HeapTuple rather than the old/new TupleTableSlots as
+ * this function does. These two functions should mirror one another until
+ * someday when catalog tuple updates track their changes avoiding the need to
+ * re-discover them in simple_heap_update().
+ */
+Bitmapset *
+ExecUpdateModifiedIdxAttrs(ResultRelInfo *resultRelInfo,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts)
+{
+ Relation relation = resultRelInfo->ri_RelationDesc;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ Bitmapset *attrs;
+
+ /* If no indexes, we're done */
+ if (resultRelInfo->ri_NumIndices == 0)
+ return NULL;
+
+ /*
+ * Get the set of all attributes across all indexes for this relation from
+ * the relcache, it returns us a copy of the bitmap so we can modify it.
+ *
+ * Note: We intentionally scan all indexed columns when looking for
+ * changes rather than reduce that set by intersecting it with
+ * ExecGetAllUpdatedCols(). Desipte the name it provides the set of
+ * targeted attributes in the SQL used for the UPDATE and any triggers,
+ * but that doesn't include any attributes updated using
+ * heap_modifiy_tuple(). There is one test in tsearch.sql that does just
+ * that, modifies an indexed attribute that isn't specified in the SQL and
+ * so isn't present in that bitmapset.
+ */
+ attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+
+ /*
+ * When there are indexed attributes mentioned in the UPDATE then we need
+ * to find the subset that changed value. That's the
+ * "modified_idx_attrs".
+ */
+ attrs = ExecCompareSlotAttrs(attrs, tupdesc, old_tts, new_tts);
+
+ return attrs;
+}
/*
* Verify that the tuples to be produced by INSERT match the
@@ -2197,14 +2256,17 @@ ExecUpdatePrepareSlot(ResultRelInfo *resultRelInfo,
*/
static TM_Result
ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
- ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *slot,
- bool canSetTag, UpdateContext *updateCxt)
+ ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *oldSlot,
+ TupleTableSlot *slot, bool canSetTag, UpdateContext *updateCxt)
{
EState *estate = context->estate;
Relation resultRelationDesc = resultRelInfo->ri_RelationDesc;
bool partition_constraint_failed;
TM_Result result;
+ /* The set of modified indexed attributes that trigger new index entries */
+ Bitmapset *modified_idx_attrs = NULL;
+
updateCxt->crossPartUpdate = false;
/*
@@ -2321,7 +2383,16 @@ lreplace:
ExecConstraints(resultRelInfo, slot, estate);
/*
- * replace the heap tuple
+ * Next up we need to find out the set of indexed attributes that have
+ * changed in value and should trigger a new index tuple. We could start
+ * with the set of updated columns via ExecGetUpdatedCols(), but if we do
+ * we will overlook attributes directly modified by heap_modify_tuple()
+ * which are not known to ExecGetUpdatedCols().
+ */
+ modified_idx_attrs = ExecUpdateModifiedIdxAttrs(resultRelInfo, oldSlot, slot);
+
+ /*
+ * Call into the table AM to update the heap tuple.
*
* Note: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
* the row to be updated is visible to that snapshot, and throw a
@@ -2335,6 +2406,7 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
+ modified_idx_attrs,
&updateCxt->updateIndexes);
return result;
@@ -2557,8 +2629,8 @@ ExecUpdate(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
*/
redo_act:
lockedtid = *tupleid;
- result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, slot,
- canSetTag, &updateCxt);
+ result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, oldSlot,
+ slot, canSetTag, &updateCxt);
/*
* If ExecUpdateAct reports that a cross-partition update was done,
@@ -3408,8 +3480,8 @@ lmerge_matched:
Assert(oldtuple == NULL);
result = ExecUpdateAct(context, resultRelInfo, tupleid,
- NULL, newslot, canSetTag,
- &updateCxt);
+ NULL, resultRelInfo->ri_oldTupleSlot,
+ newslot, canSetTag, &updateCxt);
/*
* As in ExecUpdate(), if ExecUpdateAct() reports that a
@@ -4546,7 +4618,7 @@ ExecModifyTable(PlanState *pstate)
* For UPDATE/DELETE/MERGE, fetch the row identity info for the tuple
* to be updated/deleted/merged. For a heap relation, that's a TID;
* otherwise we may have a wholerow junk attr that carries the old
- * tuple in toto. Keep this in step with the part of
+ * tuple in total. Keep this in step with the part of
* ExecInitModifyTable that sets up ri_RowIdAttNo.
*/
if (operation == CMD_UPDATE || operation == CMD_DELETE ||
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 3a4f19e8d58..f2b7fb8f444 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -2469,7 +2469,7 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
bms_free(relation->rd_keyattr);
bms_free(relation->rd_pkattr);
bms_free(relation->rd_idattr);
- bms_free(relation->rd_hotblockingattr);
+ bms_free(relation->rd_indexedattr);
bms_free(relation->rd_summarizedattr);
if (relation->rd_pubdesc)
pfree(relation->rd_pubdesc);
@@ -5271,8 +5271,8 @@ RelationGetIndexPredicate(Relation relation)
* (beware: even if PK is deferrable!)
* INDEX_ATTR_BITMAP_IDENTITY_KEY Columns in the table's replica identity
* index (empty if FULL)
- * INDEX_ATTR_BITMAP_HOT_BLOCKING Columns that block updates from being HOT
- * INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
+ * INDEX_ATTR_BITMAP_INDEXED Columns referenced by indexes
+ * INDEX_ATTR_BITMAP_SUMMARIZED Columns only included in summarizing indexes
*
* Attribute numbers are offset by FirstLowInvalidHeapAttributeNumber so that
* we can include system attributes (e.g., OID) in the bitmap representation.
@@ -5295,8 +5295,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
Bitmapset *uindexattrs; /* columns in unique indexes */
Bitmapset *pkindexattrs; /* columns in the primary index */
Bitmapset *idindexattrs; /* columns in the replica identity */
- Bitmapset *hotblockingattrs; /* columns with HOT blocking indexes */
- Bitmapset *summarizedattrs; /* columns with summarizing indexes */
+ Bitmapset *indexedattrs; /* columns referenced by indexes */
+ Bitmapset *summarizedattrs; /* columns only in summarizing indexes */
List *indexoidlist;
List *newindexoidlist;
Oid relpkindex;
@@ -5315,8 +5315,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
return bms_copy(relation->rd_pkattr);
case INDEX_ATTR_BITMAP_IDENTITY_KEY:
return bms_copy(relation->rd_idattr);
- case INDEX_ATTR_BITMAP_HOT_BLOCKING:
- return bms_copy(relation->rd_hotblockingattr);
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return bms_copy(relation->rd_indexedattr);
case INDEX_ATTR_BITMAP_SUMMARIZED:
return bms_copy(relation->rd_summarizedattr);
default:
@@ -5361,7 +5361,7 @@ restart:
uindexattrs = NULL;
pkindexattrs = NULL;
idindexattrs = NULL;
- hotblockingattrs = NULL;
+ indexedattrs = NULL;
summarizedattrs = NULL;
foreach(l, indexoidlist)
{
@@ -5421,7 +5421,7 @@ restart:
if (indexDesc->rd_indam->amsummarizing)
attrs = &summarizedattrs;
else
- attrs = &hotblockingattrs;
+ attrs = &indexedattrs;
/* Collect simple attribute references */
for (i = 0; i < indexDesc->rd_index->indnatts; i++)
@@ -5430,9 +5430,9 @@ restart:
/*
* Since we have covering indexes with non-key columns, we must
- * handle them accurately here. non-key columns must be added into
- * hotblockingattrs or summarizedattrs, since they are in index,
- * and update shouldn't miss them.
+ * handle them accurately here. Non-key columns must be added into
+ * indexedattrs or summarizedattrs, since they are in index, and
+ * update shouldn't miss them.
*
* Summarizing indexes do not block HOT, but do need to be updated
* when the column value changes, thus require a separate
@@ -5493,12 +5493,20 @@ restart:
bms_free(uindexattrs);
bms_free(pkindexattrs);
bms_free(idindexattrs);
- bms_free(hotblockingattrs);
+ bms_free(indexedattrs);
bms_free(summarizedattrs);
goto restart;
}
+ /*
+ * Record what attributes are only referenced by summarizing indexes. Then
+ * add that into the other indexed attributes to track all referenced
+ * attributes.
+ */
+ summarizedattrs = bms_del_members(summarizedattrs, indexedattrs);
+ indexedattrs = bms_add_members(indexedattrs, summarizedattrs);
+
/* Don't leak the old values of these bitmaps, if any */
relation->rd_attrsvalid = false;
bms_free(relation->rd_keyattr);
@@ -5507,8 +5515,8 @@ restart:
relation->rd_pkattr = NULL;
bms_free(relation->rd_idattr);
relation->rd_idattr = NULL;
- bms_free(relation->rd_hotblockingattr);
- relation->rd_hotblockingattr = NULL;
+ bms_free(relation->rd_indexedattr);
+ relation->rd_indexedattr = NULL;
bms_free(relation->rd_summarizedattr);
relation->rd_summarizedattr = NULL;
@@ -5523,7 +5531,7 @@ restart:
relation->rd_keyattr = bms_copy(uindexattrs);
relation->rd_pkattr = bms_copy(pkindexattrs);
relation->rd_idattr = bms_copy(idindexattrs);
- relation->rd_hotblockingattr = bms_copy(hotblockingattrs);
+ relation->rd_indexedattr = bms_copy(indexedattrs);
relation->rd_summarizedattr = bms_copy(summarizedattrs);
relation->rd_attrsvalid = true;
MemoryContextSwitchTo(oldcxt);
@@ -5537,8 +5545,8 @@ restart:
return pkindexattrs;
case INDEX_ATTR_BITMAP_IDENTITY_KEY:
return idindexattrs;
- case INDEX_ATTR_BITMAP_HOT_BLOCKING:
- return hotblockingattrs;
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return indexedattrs;
case INDEX_ATTR_BITMAP_SUMMARIZED:
return summarizedattrs;
default:
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 305ecc31a9e..31a688cf05b 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -403,10 +403,9 @@ extern TM_Result heap_delete(Relation relation, const ItemPointerData *tid,
extern void heap_finish_speculative(Relation relation, const ItemPointerData *tid);
extern void heap_abort_speculative(Relation relation, const ItemPointerData *tid);
extern TM_Result heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, const LockTupleMode lockmode,
+ const Bitmapset *modified_idx_attrs, const bool hot_allowed);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -469,6 +468,12 @@ extern void log_heap_prune_and_freeze(Relation relation, Buffer buffer,
OffsetNumber *dead, int ndead,
OffsetNumber *unused, int nunused);
+/* in heap/heapam.c */
+extern bool HeapUpdateHotAllowable(Relation relation, const Bitmapset *modified_idx_attrs,
+ bool *summarized_only);
+extern LockTupleMode HeapUpdateDetermineLockmode(Relation relation,
+ const Bitmapset *modified_idx_attrs);
+
/* in heap/vacuumlazy.c */
extern void heap_vacuum_rel(Relation rel,
const VacuumParams params, BufferAccessStrategy bstrategy);
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 06084752245..8ec20dcfc11 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -549,6 +549,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
+ const Bitmapset *modified_idx_attrs,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1523,12 +1524,12 @@ static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ const Bitmapset *modified_idx_attrs, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
- wait, tmfd,
- lockmode, update_indexes);
+ wait, tmfd, lockmode,
+ modified_idx_attrs, update_indexes);
}
/*
@@ -2009,6 +2010,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
+ const Bitmapset *modified_idx_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 07f4b1f7490..d294789c441 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -18,6 +18,7 @@
#include "datatype/timestamp.h"
#include "executor/execdesc.h"
#include "fmgr.h"
+#include "nodes/execnodes.h"
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
@@ -610,6 +611,10 @@ extern TupleDesc ExecCleanTypeFromTL(List *targetList);
extern TupleDesc ExecTypeFromExprList(List *exprList);
extern void ExecTypeSetColNames(TupleDesc typeInfo, List *namesList);
extern void UpdateChangedParamSet(PlanState *node, Bitmapset *newchg);
+extern Bitmapset *ExecCompareSlotAttrs(Bitmapset *attrs,
+ TupleDesc tupdesc,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
typedef struct TupOutputState
{
@@ -807,5 +812,8 @@ extern ResultRelInfo *ExecLookupResultRelByOid(ModifyTableState *node,
Oid resultoid,
bool missing_ok,
bool update_cache);
+extern Bitmapset *ExecUpdateModifiedIdxAttrs(ResultRelInfo *relinfo,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
#endif /* EXECUTOR_H */
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 236830f6b93..11460e134f0 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -162,7 +162,7 @@ typedef struct RelationData
Bitmapset *rd_keyattr; /* cols that can be ref'd by foreign keys */
Bitmapset *rd_pkattr; /* cols included in primary key */
Bitmapset *rd_idattr; /* included in replica identity index */
- Bitmapset *rd_hotblockingattr; /* cols blocking HOT update */
+ Bitmapset *rd_indexedattr; /* all cols referenced by indexes */
Bitmapset *rd_summarizedattr; /* cols indexed by summarizing indexes */
PublicationDesc *rd_pubdesc; /* publication descriptor, or NULL */
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2700224939a..d4db82496b4 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -69,7 +69,7 @@ typedef enum IndexAttrBitmapKind
INDEX_ATTR_BITMAP_KEY,
INDEX_ATTR_BITMAP_PRIMARY_KEY,
INDEX_ATTR_BITMAP_IDENTITY_KEY,
- INDEX_ATTR_BITMAP_HOT_BLOCKING,
+ INDEX_ATTR_BITMAP_INDEXED,
INDEX_ATTR_BITMAP_SUMMARIZED,
} IndexAttrBitmapKind;
diff --git a/src/test/modules/injection_points/expected/syscache-update-pruned.out b/src/test/modules/injection_points/expected/syscache-update-pruned.out
index a6a4e8db996..07ef67a1eb4 100644
--- a/src/test/modules/injection_points/expected/syscache-update-pruned.out
+++ b/src/test/modules/injection_points/expected/syscache-update-pruned.out
@@ -16,8 +16,8 @@ step wakeinval4:
step at2: <... completed>
step wakeinval4: <... completed>
step wakegrant4:
- SELECT FROM injection_points_detach('heap_update-before-pin');
- SELECT FROM injection_points_wakeup('heap_update-before-pin');
+ SELECT FROM injection_points_detach('simple_heap_update-before-pin');
+ SELECT FROM injection_points_wakeup('simple_heap_update-before-pin');
<waiting ...>
step grant1: <... completed>
ERROR: tuple concurrently deleted
@@ -42,8 +42,8 @@ step mkrels4:
SELECT FROM vactest.mkrels('intruder', 1, 100); -- repopulate LP_UNUSED
step wakegrant4:
- SELECT FROM injection_points_detach('heap_update-before-pin');
- SELECT FROM injection_points_wakeup('heap_update-before-pin');
+ SELECT FROM injection_points_detach('simple_heap_update-before-pin');
+ SELECT FROM injection_points_wakeup('simple_heap_update-before-pin');
<waiting ...>
step grant1: <... completed>
ERROR: duplicate key value violates unique constraint "pg_class_oid_index"
@@ -71,8 +71,8 @@ step at2: <... completed>
step wakeinval4: <... completed>
step at4: ALTER TABLE vactest.child50 INHERIT vactest.orig50;
step wakegrant4:
- SELECT FROM injection_points_detach('heap_update-before-pin');
- SELECT FROM injection_points_wakeup('heap_update-before-pin');
+ SELECT FROM injection_points_detach('simple_heap_update-before-pin');
+ SELECT FROM injection_points_wakeup('simple_heap_update-before-pin');
<waiting ...>
step grant1: <... completed>
step wakegrant4: <... completed>
diff --git a/src/test/modules/injection_points/specs/syscache-update-pruned.spec b/src/test/modules/injection_points/specs/syscache-update-pruned.spec
index e3a4295bd12..fef9ac895a1 100644
--- a/src/test/modules/injection_points/specs/syscache-update-pruned.spec
+++ b/src/test/modules/injection_points/specs/syscache-update-pruned.spec
@@ -103,7 +103,7 @@ session s1
setup {
SET debug_discard_caches = 0;
SELECT FROM injection_points_set_local();
- SELECT FROM injection_points_attach('heap_update-before-pin', 'wait');
+ SELECT FROM injection_points_attach('simple_heap_update-before-pin', 'wait');
}
step cachefill1 { SELECT FROM vactest.reloid_catcache_set('vactest.orig50'); }
step grant1 { GRANT SELECT ON vactest.orig50 TO PUBLIC; }
@@ -140,8 +140,8 @@ step mkrels4 {
SELECT FROM vactest.mkrels('intruder', 1, 100); -- repopulate LP_UNUSED
}
step wakegrant4 {
- SELECT FROM injection_points_detach('heap_update-before-pin');
- SELECT FROM injection_points_wakeup('heap_update-before-pin');
+ SELECT FROM injection_points_detach('simple_heap_update-before-pin');
+ SELECT FROM injection_points_wakeup('simple_heap_update-before-pin');
}
step at4 { ALTER TABLE vactest.child50 INHERIT vactest.orig50; }
step wakeinval4 {
diff --git a/src/test/regress/expected/generated_virtual.out b/src/test/regress/expected/generated_virtual.out
index 6dab60c937b..7ebb7890d96 100644
--- a/src/test/regress/expected/generated_virtual.out
+++ b/src/test/regress/expected/generated_virtual.out
@@ -287,7 +287,7 @@ DETAIL: Column "b" is a generated column.
INSERT INTO gtest1v VALUES (8, DEFAULT), (9, DEFAULT); -- error
ERROR: cannot insert a non-DEFAULT value into column "b"
DETAIL: Column "b" is a generated column.
-SELECT * FROM gtest1v;
+SELECT * FROM gtest1v ORDER BY a;
a | b
---+----
3 | 6
diff --git a/src/test/regress/expected/triggers.out b/src/test/regress/expected/triggers.out
index 98dee63b50a..ef98fd0cccf 100644
--- a/src/test/regress/expected/triggers.out
+++ b/src/test/regress/expected/triggers.out
@@ -959,16 +959,24 @@ NOTICE: main_view BEFORE UPDATE STATEMENT (before_view_upd_stmt)
NOTICE: main_view AFTER UPDATE STATEMENT (after_view_upd_stmt)
UPDATE 0
-- Delete from view using trigger
-DELETE FROM main_view WHERE a IN (20,21);
+DELETE FROM main_view WHERE a = 20 AND b = 31;
NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
-NOTICE: OLD: (21,10)
-NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
NOTICE: OLD: (20,31)
+NOTICE: main_view AFTER DELETE STATEMENT (after_view_del_stmt)
+DELETE 1
+DELETE FROM main_view WHERE a = 21 AND b = 10;
+NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
+NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
+NOTICE: OLD: (21,10)
+NOTICE: main_view AFTER DELETE STATEMENT (after_view_del_stmt)
+DELETE 1
+DELETE FROM main_view WHERE a = 21 AND b = 32;
+NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
NOTICE: OLD: (21,32)
NOTICE: main_view AFTER DELETE STATEMENT (after_view_del_stmt)
-DELETE 3
+DELETE 1
DELETE FROM main_view WHERE a = 31 RETURNING a, b;
NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
diff --git a/src/test/regress/expected/tsearch.out b/src/test/regress/expected/tsearch.out
index 9287c440709..c604ec35fa5 100644
--- a/src/test/regress/expected/tsearch.out
+++ b/src/test/regress/expected/tsearch.out
@@ -2483,7 +2483,8 @@ SELECT to_tsquery('SKIES & My | booKs');
'sky' | 'book'
(1 row)
---trigger
+-- tsvector_update_trigger() uses heap_modify_tuple() to set column 'a'
+-- without going through the executor's SET-clause tracking.
CREATE TRIGGER tsvectorupdate
BEFORE UPDATE OR INSERT ON test_tsvector
FOR EACH ROW EXECUTE PROCEDURE tsvector_update_trigger(a, 'pg_catalog.english', t);
diff --git a/src/test/regress/expected/updatable_views.out b/src/test/regress/expected/updatable_views.out
index 9cea538b8e8..4877a1ddce9 100644
--- a/src/test/regress/expected/updatable_views.out
+++ b/src/test/regress/expected/updatable_views.out
@@ -372,15 +372,15 @@ INSERT INTO rw_view16 (a, b) VALUES (3, 'Row 3'); -- should be OK
UPDATE rw_view16 SET a=3, aa=-3 WHERE a=3; -- should fail
ERROR: multiple assignments to same column "a"
UPDATE rw_view16 SET aa=-3 WHERE a=3; -- should be OK
-SELECT * FROM base_tbl;
+SELECT * FROM base_tbl ORDER BY a;
a | b
----+--------
+ -3 | Row 3
-2 | Row -2
-1 | Row -1
0 | Row 0
1 | Row 1
2 | Row 2
- -3 | Row 3
(6 rows)
DELETE FROM rw_view16 WHERE a=-3; -- should be OK
diff --git a/src/test/regress/sql/generated_virtual.sql b/src/test/regress/sql/generated_virtual.sql
index e750866d2d8..877152d6d69 100644
--- a/src/test/regress/sql/generated_virtual.sql
+++ b/src/test/regress/sql/generated_virtual.sql
@@ -127,7 +127,7 @@ ALTER VIEW gtest1v ALTER COLUMN b SET DEFAULT 100;
INSERT INTO gtest1v VALUES (8, DEFAULT); -- error
INSERT INTO gtest1v VALUES (8, DEFAULT), (9, DEFAULT); -- error
-SELECT * FROM gtest1v;
+SELECT * FROM gtest1v ORDER BY a;
DELETE FROM gtest1v WHERE a >= 5;
DROP VIEW gtest1v;
diff --git a/src/test/regress/sql/triggers.sql b/src/test/regress/sql/triggers.sql
index ea39817ee3d..6ceb61608ae 100644
--- a/src/test/regress/sql/triggers.sql
+++ b/src/test/regress/sql/triggers.sql
@@ -660,7 +660,9 @@ UPDATE main_view SET b = 32 WHERE a = 21 AND b = 31 RETURNING a, b;
UPDATE main_view SET b = 0 WHERE false;
-- Delete from view using trigger
-DELETE FROM main_view WHERE a IN (20,21);
+DELETE FROM main_view WHERE a = 20 AND b = 31;
+DELETE FROM main_view WHERE a = 21 AND b = 10;
+DELETE FROM main_view WHERE a = 21 AND b = 32;
DELETE FROM main_view WHERE a = 31 RETURNING a, b;
\set QUIET true
diff --git a/src/test/regress/sql/tsearch.sql b/src/test/regress/sql/tsearch.sql
index dc74aa0c889..77ac5fd3c5a 100644
--- a/src/test/regress/sql/tsearch.sql
+++ b/src/test/regress/sql/tsearch.sql
@@ -752,7 +752,8 @@ SELECT to_tsvector('SKIES My booKs');
SELECT plainto_tsquery('SKIES My booKs');
SELECT to_tsquery('SKIES & My | booKs');
---trigger
+-- tsvector_update_trigger() uses heap_modify_tuple() to set column 'a'
+-- without going through the executor's SET-clause tracking.
CREATE TRIGGER tsvectorupdate
BEFORE UPDATE OR INSERT ON test_tsvector
FOR EACH ROW EXECUTE PROCEDURE tsvector_update_trigger(a, 'pg_catalog.english', t);
diff --git a/src/test/regress/sql/updatable_views.sql b/src/test/regress/sql/updatable_views.sql
index 1635adde2d4..160e7799715 100644
--- a/src/test/regress/sql/updatable_views.sql
+++ b/src/test/regress/sql/updatable_views.sql
@@ -125,7 +125,7 @@ INSERT INTO rw_view16 VALUES (3, 'Row 3', 3); -- should fail
INSERT INTO rw_view16 (a, b) VALUES (3, 'Row 3'); -- should be OK
UPDATE rw_view16 SET a=3, aa=-3 WHERE a=3; -- should fail
UPDATE rw_view16 SET aa=-3 WHERE a=3; -- should be OK
-SELECT * FROM base_tbl;
+SELECT * FROM base_tbl ORDER BY a;
DELETE FROM rw_view16 WHERE a=-3; -- should be OK
-- Read-only views
INSERT INTO ro_view17 VALUES (3, 'ROW 3');
--
2.51.2
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2026-03-02 19:08 Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-11 15:51 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-12 20:33 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2026-03-12 21:31 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-15 21:11 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-16 16:23 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-16 17:55 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-17 16:38 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-17 18:04 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-23 18:39 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2026-03-24 18:02 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
@ 2026-03-24 19:44 ` Nathan Bossart <[email protected]>
2026-03-25 17:16 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
0 siblings, 1 reply; 37+ messages in thread
From: Nathan Bossart @ 2026-03-24 19:44 UTC (permalink / raw)
To: Greg Burd <[email protected]>; +Cc: Jeff Davis <[email protected]>; pgsql-hackers
On Tue, Mar 24, 2026 at 02:02:07PM -0400, Greg Burd wrote:
> On Mon, Mar 23, 2026, at 2:39 PM, Nathan Bossart wrote:
>> On Tue, Mar 17, 2026 at 02:04:11PM -0400, Greg Burd wrote:
>>> - * INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
>>> + * INDEX_ATTR_BITMAP_INDEXED Columns referenced by indexes
>>> + * INDEX_ATTR_BITMAP_SUMMARIZED Columns only included in summarizing indexes
>>
>>> - Bitmapset *summarizedattrs; /* columns with summarizing indexes */
>>> + Bitmapset *indexedattrs; /* columns referenced by indexes */
>>> + Bitmapset *summarizedattrs; /* columns only in summarizing indexes */
>>
>> As before, the comment changes for the summarized-attr-related stuff seem
>> unnecessary.
>
> I disagree, the "only" is required to highlight the logic change here.
> Before this patch summarized attrs could overlap with indexed attrs, now
> it should not. This makes the logic a bit easier later in
> HeapUpdateHotAllowable().
My bad, you are right.
> So, we go from 3 calls to RelationGetIndexAttrBitmap() to 1, or at most 2
> when there's a summarizing index (which is frequently the case).
>
> This feels more logical, cleaner, and has less overhead but supports the
> same HOT logic.
Nice.
--
nathan
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: Expanding HOT updates for expression and partial indexes
2026-03-02 19:08 Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-11 15:51 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-12 20:33 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2026-03-12 21:31 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-15 21:11 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-16 16:23 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-16 17:55 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-17 16:38 ` Re: Expanding HOT updates for expression and partial indexes Jeff Davis <[email protected]>
2026-03-17 18:04 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-23 18:39 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
2026-03-24 18:02 ` Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-24 19:44 ` Re: Expanding HOT updates for expression and partial indexes Nathan Bossart <[email protected]>
@ 2026-03-25 17:16 ` Nathan Bossart <[email protected]>
0 siblings, 0 replies; 37+ messages in thread
From: Nathan Bossart @ 2026-03-25 17:16 UTC (permalink / raw)
To: Greg Burd <[email protected]>; +Cc: Jeff Davis <[email protected]>; pgsql-hackers
I just spoke to Greg off-list and wanted to share my current thoughts on
the list as well. In short, while we feel that the patch is in decent
shape and seems to be performance neutral (or maybe even positive in some
cases), it obviously doesn't accomplish $subject, and only a couple of
folks have looked at it in depth. Furthermore, if this patch was committed
and someone did find a problem, it'd be hard to justify anything except a
revert. So, it's probably better to keep working on the full patch set and
try to get $subject committed much earlier in the development cycle.
If someone thinks that we should seriously consider committing this for
v19, please let us know.
--
nathan
^ permalink raw reply [nested|flat] 37+ messages in thread
end of thread, other threads:[~2026-03-25 17:16 UTC | newest]
Thread overview: 37+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2025-03-05 23:39 Re: Expanding HOT updates for expression and partial indexes Matthias van de Meent <[email protected]>
2025-03-06 12:40 ` Burd, Greg <[email protected]>
2025-03-07 22:47 ` Matthias van de Meent <[email protected]>
2025-03-25 11:47 ` Burd, Greg <[email protected]>
2025-03-25 14:21 ` Burd, Greg <[email protected]>
2025-07-02 18:10 ` Greg Burd <[email protected]>
2025-10-07 21:36 ` Greg Burd <[email protected]>
2025-10-08 20:48 ` Nathan Bossart <[email protected]>
2025-10-09 19:08 ` Jeff Davis <[email protected]>
2025-10-09 21:13 ` Greg Burd <[email protected]>
2025-10-09 20:57 ` Greg Burd <[email protected]>
2025-10-09 19:27 ` Jeff Davis <[email protected]>
2025-10-14 17:46 ` Greg Burd <[email protected]>
2025-10-14 18:43 ` Jeff Davis <[email protected]>
2025-11-16 18:53 ` Greg Burd <[email protected]>
2025-11-19 18:00 ` Greg Burd <[email protected]>
2025-11-19 18:21 ` Greg Burd <[email protected]>
2025-11-21 15:25 ` Matthias van de Meent <[email protected]>
2025-11-22 21:30 ` Greg Burd <[email protected]>
2025-11-24 18:59 ` Matthias van de Meent <[email protected]>
2025-12-03 22:06 ` Greg Burd <[email protected]>
2025-12-15 21:46 ` Greg Burd <[email protected]>
2026-03-02 19:08 Re: Expanding HOT updates for expression and partial indexes Greg Burd <[email protected]>
2026-03-11 15:51 ` Greg Burd <[email protected]>
2026-03-12 20:33 ` Nathan Bossart <[email protected]>
2026-03-12 21:31 ` Greg Burd <[email protected]>
2026-03-15 21:11 ` Jeff Davis <[email protected]>
2026-03-16 16:23 ` Greg Burd <[email protected]>
2026-03-16 17:29 ` Nathan Bossart <[email protected]>
2026-03-16 17:55 ` Jeff Davis <[email protected]>
2026-03-17 15:22 ` Nathan Bossart <[email protected]>
2026-03-17 16:38 ` Jeff Davis <[email protected]>
2026-03-17 18:04 ` Greg Burd <[email protected]>
2026-03-23 18:39 ` Nathan Bossart <[email protected]>
2026-03-24 18:02 ` Greg Burd <[email protected]>
2026-03-24 19:44 ` Nathan Bossart <[email protected]>
2026-03-25 17:16 ` Nathan Bossart <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox