public inbox for [email protected]
help / color / mirror / Atom feedFrom: Greg Burd <[email protected]>
To: =?utf-8?Q?pgsql-hackers=40postgresql.org?= <[email protected]>
Subject: Re: Expanding HOT updates for expression and partial indexes
Date: Sun, 16 Nov 2025 13:53:09 -0500
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
Hello again.
This idea started over a year ago for me while working on a project that
used JSONB and had horrible "bloat" with update times that were not
fantastic. The root cause, expression indexes prevent HOT updates and
all indexes on JSONB are by definition, expressions. That's the backstory.
The idea for the solution came from a patch [1] applied [2], then later
reverted [3], that basically evaluated the before/after tuple
expressions in heap_update() at the point where newbuff == buffer just
before deciding to use_hot_update or not. When the evaluated
expressions produced equal results using a binary comparison the index
didn't need to be updated. While this approach sorta worked, but it was
reverted for a few reasons, here's Tom's summary: "The problem here is
that [the code that checks for equality] thinks that the type of the
index column is identical to the type of the source datum for it, which
is not true for any opclass making use of the opckeytype property. [4]"
Still, expanding the domain of what might go HOT seems like a good goal
to me and it was at the heart of the issues I was facing on the project
using JSONB, so I kept at it.
The patches I've sent on this thread have evolved from that first idea.
First they addressed the specific issue raised by Tom. Then they
expanded to include partial indexes as well. This worked, and I helped
to ship a fork of Postgres used this approach and solved customer issues
with the JSONB use case. Customer experience was better, no unnecessary
index updates when indexed data wasn't modified meant no more heap/index
bloat and faster update times. Vacuum could finally keep up and
performance and storage overhead was much better.
For this patch set v1 through v17 were essentially work that
refined/reworked that original approach, but there was a serious flaw
that I didn't fully appreciate until around v16/17. I was evaluating
the expressions while holding a lock on the buffer page which a) expands
the time the lock is held, and b) opens the door to self-deadlock. No bueno.
Then there was v18, a quick work-around for that. I moved the call that
invokes the executor to the beginning of heap_update() before taking the
lock on the page. To do this I had to find the set of updated
attributes, which I discovered was available in the executor as the
updated tuple is created. This was a viable fix, but didn't really go
far enough and was a bit hackish IMO. Jeff Davis and others challenged
me to move the work to identify what's changed into the executor and
clean it up. I'm a sucker for a challenge.
More generally, this idea made sense. While Postgres has many places
where logic is tightly coupled to the way the heap works this didn't
have to be one. To make this work I needed the update path in the
executor to be interested in:
a) knowing what columns were specified in the UPDATE statement and those
impacted by before/after triggers,
b) reducing that set to those attributes known to be both indexed and to
have changed value,
c) finding which of those (and possibly other) attributes that force new
index updates.
Why? We'll, that code already exists in a few places and in some cases
is replicated; for (a) there is ExecGetAllUpdatedCols(), for (b)
HeapDetermineColumnsInfo() and index_unchanged_by_update().
An interesting thing to note is that HeapDetermineColumnsInfo() might
return a set that includes columns not returned by
ExecGetAllUpdatedCols() because HeapDetermineColumnsInfo() iterates over
all indexed attributes looking for changes and that might find an
indexed attribute that was changed by heap_modify_tuple() but not
knowable by ExecGetAllUpdatedCols(). This happens in tsvector code, see
tsvector_op.c tsvector_update_trigger() where if (update_needed)
heap_modify_tuple_by_cols(). That column isn't known to ExecGetAllUpdatedCols().
HeapDetermineColumnsInfo() is also critical when modifying catalog
tuples. Catalog tuples are modified using either Form/GETSTRUCT or
values/nulls/replaces then using heap_modify_tuple() and calling into
CatalogTupleUpdate() which calls simple_heap_update() that calls
heap_update() where we find HeapDetermineColumnsInfo(). The interesting
thing here is that when modifying catalog tuples there is knowledge of
what attributes are changed, but that knowledge isn't preserved and
passed into CatalogTupleUpdate(), rather it is re-discovered in
HeapDetermineColumnsInfo(). That's how catalog tuples are able to take
the HOT path, they re-use that same logic. There is a fix for that [5]
too (and I really hope that lands in master ASAP), but that's not the
subject of this thread.
HeapDetermineColumnsInfo() also helps inform a few other decisions in
heap_update(), but these have to happen after taking the buffer lock and
are very heap-specific, namely:
1. Do either the replica identity key attributes overlap with the
modified index attributes or do they need to be stored externally, this
is passed on to ExtractReplicaIdentity() to find out if we must augment
the WAL log or not.
2. Are there any modified indexed attributes that intersect with the
primary keys of the relation, if not lower the lock mode to enable
multixact to work.
HeapDetermineColumnsInfo() also takes a pragmatic approach to testing
for equality when looking for modified indexed attributes, it uses
datumIsEqual() which boils down to a simple memcmp() of the before/after
HeapTuple datum. This is fine in most cases, but limits the scope of
what can be HOT.
Interestingly, this requirement of binary equality has leaked into other
parts of the code, namely nbtree's deduplication of TIDs on page split.
That code uses binary equality as well. A nbtree index with collation
for case insensitive must store both "A" and "a" despite those being
type-equal because they are not binary equivalent. More on this later.
At this point, I had my sights set on HeapDetermineColumnsInfo(). I
felt that what it was doing should move into the executor, well as much
of that work as possible, and outside of the buffer lock. This would
also open the door for removal of redundant code. My thought was that
the table AM update API should have an additional argument, the
"modified indexed attributes" or "mix_attrs", passed in.
So, here we are at the door of v19... let's begin.
0001 - Reorganize heap update logic
This is preparatory work for the larger goal in that heap_update()
serves two masters: CatalogTupleUpdate()/simple_heap_update() and
heap_tuple_update() and in reality, they were different but needed most
of the same logic that happens at the start of heap_update(). This
patch splits that logic out and moves it into heap_tuple_update() and
simple_heap_update(). Functionally nothing changes. That's the meat of
this patch.Reorganize heap update logic
0002 - Track changed indexed columns in the executor during UPDATEs
This is the first core set of changes, it doesn't expand HOT updates but
it does restructure where HeapDetermineColumnsInfo()'s core work happens.
A new function ExecCheckIndexedAttrsForChanges() in nodeModifyTable.c is
now responsible for checking for changes between Datum in the old/new
TupleTableSlots. This is different from before in that we're not
checking the new HeapTuple Datum verses the HeapTuple we read from the
buffer page while holding the lock on that page.
An update starts off by reading the existing tuple using the table AM.
Then a new updated tuple is created as the set of changes to the old.
Then the new TupleTableSlot is the combination of the existing one we
just read and the changes we just recorded. So, in the executor before
calling into the table AM's update function we have a pin on the buffer
and the before/after TupleTableSlots for this update. So, I've put the
call to my new ExecCheckIndexedAttrsForChanges() function just before
calling table_tuple_update() and I've added the "mix_attrs" into that
call which get passed on to the heap in heap_tuple_update() and then
heap_update() and all is well.
Why is this safe? The way I read heap_update() is that it has always
historically had code to deal with cases where the tuple is concurrently
updated and react accordingly thanks to HeapTupleSatisfiesUpdate() which
remains where it was in heap_update(). Visibility checks happened when
we first read the tuple to form the updated tuple and later in
heap_update() when we call HeapTupleSatisfiesVisibility() to check for
transaction-snapshot mode RI updates.
So, this new update path from the executor into the table AM seems to me
to be okay and almost functionally equivalent. But there is one big
change to discuss before moving to the simple_heap_update() path.
In nodeModifyTable.c tts_attr_equal() replaces heap_attr_equal()
changing the test for equality when calling into heap_tuple_update().
In the past we used datumIsEqual(), essentially a binary comparison
using memcmp(), now the comparison code in tts_attr_equal uses
type-specific equality function when available and falls back to
datumIsEqual() when not.
The other parts of HeapDetermineColumnsInfo() remain in the code, but
they still happen within the simple_heap_update() and
heap_tuple_update() code. That's where you'll find that after the
buffer is locked we do (1) and (2) from above. This keeps the
heap-specific work in the heap, but we've moved some work up into the
executor and outside the buffer lock.
While this accomplishes the goal of removing HeapDetermineColumnsInfo()
from heap_update() on the path that uses the table AM API
heap_tuple_update() but it doesn't on the simple_heap_update() path.
That remains the same as it was in the previous patch. Ideally, my
patch to restructure how catalog tuples are updated [5] is committed and
we can fully remove HeapDetermineColumnsInfo() and likely speed up all
catalog updates in the process. That's what motivated [5], please take
a look, it required a huge number of changes so I thought it deserved a
life/thread of its own.
Finally, there is the ExecSimpleRelationUpdate() path and
slot_modify_data(). On this path we know what attributes are being
updated, so we just check to see if they changed and then intersect that
with the set of indexed attributes and we have our modified indexed
attributes set to pass into simple_heap_update().
0003 - Replace index_unchanged_by_update with ri_ChangedIndexedCols
This patch removes the function index_unchanged_by_update() in
execIndexing.c and simply re-uses the modified indexed attributes that
we've stashed away in ResultRelInfo as ri_ChangedIndexedCols. This
provides a hint when calling into the index AM's index_insert() function
indicating if the UPDATE was without logical change to the data or not.
We've done that check, we don't need to do it again.
0004 - Enable HOT updates for expression and partial indexes
This finally gets us back to where this project started, but on much
more firm ground than before because we're not going to self-deadlock.
The idea has grown from a small function into something larger, but only
out of necessity.
In this patch I add ExecWhichIndexesRequireUpdates() in execIndexing.c
which implements (c) finding the set of attributes that force new index
updates. This set can be very different from the modified indexed
attributes. We know that some attributes are not equal to their
previous versions, but does that mean that the index that references
that attribute needs a new index tuple? It may, or it may not. Here's
the comment on that function that explains:
/*
* ExecWhichIndexesRequireUpdates
*
* Determine which indexes need updating given modified indexed attributes.
* This function is a companion to ExecCheckIndexedAttrsForChanges().
On the
* surface, they appear similar but they are doing two very different things.
*
* For a standard index on a set of attributes this is the intersection of
* the mix_attrs and the index attrs (key, expression, but not predicate).
*
* For expression indexes and indexes which implement the amcomparedatums()
* index AM API we'll need to form index datum and compare each
attribute to
* see if any actually changed.
*
* For expression indexes the result of the expression might not change
at all,
* this is common with JSONB columns which require expression indexes
and where
* it is commonplace to index a field within a document and have updates that
* generally don't update that field.
*
* Partial indexes won't trigger index tuples when the old/new tuples
are both
* outside of the predicate range.
*
* For nbtree the amcomparedatums() API is critical as it requires that key
* attributes are equal when they memcmp(), which might not be the case when
* using type-specific comparison or factoring in collation which might make
* an index case insensitive.
*
* All of this is to say that the goal is for the executor to know,
ahead of
* calling into the table AM for the update and before calling into the index
* AM for inserting new index tuples, which attributes at a minimum will
* necessitate a new index tuple.
*
...
*/
Whereas before we were comparing Datum in the table relation, now we're
comparing Datum in the index relation. Index AMs are free to store what
they want, we need to know if what's changed and referenced by the index
means that the index needs a new tuple or not.
In the case of a JSONB expression index (or any expression index) the
expression is evaluated when calling FormIndexDatum(). The result of
the expression is the Datum in the values/isnull arrays. Then we need
to compare them to see if they changed. This can be done using the same
tts_attr_equal() function, but with the attribute desc of the index, not
table relation.
In some cases that's not enough, for instance nbtree and it's more
stringent equality requirements. For that reason (and another coming up
in a second) we need a new optional index AM API, amcomparedatums().
Indexes implementing this function have the ability to compare two
Datums for equality in what ever way they want. For nbtree, that's a
binary comparison.
The new case here that this also supports is for indexes like GIN/RUM
where there is an opclass that can extract zero or more pieces from the
attribute and form multiple index entries when needed. Those extracted
pieces might have odd equality rules as well. This opens the door for
index implementations to provide information that will help inform heap
when making HOT decisions that didn't exist before.
Take for example the Linux Foundation's DocumentDB [6] project [7] which
aims to be an open source alternative to MongoDB built on top of
PostgreSQL. One of the pieces of that project is its "extended RUM"
index AM implementation. This index extracts portions of the BSON
documents stored and forms index keys from that. Here is an example:
CREATE INDEX documents_rum_index_14002
ON documentdb_data.documents_14001
USING documentdb_rum
(document bson_rum_single_path_ops (path=a, iswildcard='true', tl='2699'))
An index on the "document" column which uses the
"bson_rum_single_path_ops" opclass to extract a portion of the BSON
document that matches "path=a".
For this to potentially be stored by heap as a HOT update we need to
know that what changed within that document didn't intersect with the
"path=a" and if it did that the new value(s) were all equal to the old
values. Equality in DocumentDB isn't what you think, it's quite odd and
specific to BSON and rules defined by MongoDB so it's important to allow
the index AM to execute what's necessary for it's use case.
For the more common JSONB use case we can now have heap make an informed
decision about HOT updates after evaluating the expression. For
GIN/RUM/etc. implementation there is a path to HOT. For nbtree there is
a way to maintain its requirements.
Is there a cost to all this? Yes, of course. There is net new work
being done on some paths. IndexFormDatum() will be called more
frequently and sometimes twice for the same thing. This could be
improved, there might be a way to cache that information. But to have
tests of old/new we'll have to do that work at least twice.
Is there a benefit? Yes, of course. Some redundant code paths are gone
and in the end we've increased the number of cases where HOT updates are
a possibility. This especially helps out users of JSONB, but not only them.
What's left undone?
* I need to check code coverage so that I might
* create tests covering all the new cases
* update the README.HOT documentation, wiki, etc.
* performance...
For performance I'd like to examine some worst cases as in lots of
indexes that have a lot of new code to exercise and all but the last
index would allow for a HOT update. That should represent the maximum
amount of new overhead for this code. Then, the other side of the
equation, how much does this help JSONB? I think that is something to
measure in terms of TPS as well as "bloat" avoided and time spent vacuuming.
I also don't like the TU_Updating enum, I think it's a leaky abstraction
and really pointless now. I'd like to remove it in favor of the bitmap
of attributes known to force index tuples to be inserted. Maybe I'll
layer that into the next set.
In the end, this is a lot of work and I believe that it moves the ball
forward. I'll have more metrics on that soon I hope, but I wanted to
get the conversation re-started ASAP as we're late in the v19 cycle.
Finally, the "elephant" in the room (ha!) is PHOT[9]/WARM[10][11]. Yes,
some of this work does help make a solution to allow HOT updates when
only updating a subset of indexes closer to reality, I'd be lying not to
mention that here, it is a big part of my overall plan for my next year
and (I hope) v20 and I need this work to get there. Specifically, PHOT
is in part based on HeapDetermineColumnsInfo() and I needed to make that
more truthful for it to work.
I hope you see the value in this work and will partner with me to
finalize it and get it into master.
best.
-greg
[1] https://www.postgresql.org/message-id/flat/4d9928ee-a9e6-15f9-9c82-5981f13ffca6%40postgrespro.ru
[2] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=c203d6cf8
[3] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=05f84605dbeb9cf8279a157234b24bbb706c5...
[4] https://www.postgresql.org/message-id/2877.1541538838%40sss.pgh.pa.us
[5] https://www.postgresql.org/message-id/flat/[email protected]
[6] https://www.linuxfoundation.org/press/linux-foundation-welcomes-documentdb-to-advance-open-developer...
[7] https://github.com/documentdb/documentdb
[8] https://github.com/documentdb/documentdb/tree/main/pg_documentdb_extended_rum
[9] https://www.postgresql.org/message-id/flat/2ECBBCA0-4D8D-4841-8872-4A5BBDC063D2%40amazon.com
[10] https://www.postgresql.org/message-id/flat/CABOikdMop5Rb_RnS2xFdAXMZGSqcJ-P-BY2ruMd%2BbuUkJ4iDPw%40m...
[11]
https://www.postgresql.org/message-id/flat/CABOikdMNy6yowA%2BwTGK9RVd8iw%2BCzqHeQSGpW7Yka_4RSZ_LOQ%4...
Attachments:
[application/octet-stream] v19-0001-Reorganize-heap-update-logic.patch (47.6K, 2-v19-0001-Reorganize-heap-update-logic.patch)
download | inline diff:
From 6d76d6ad5bc7077727f92bdb9c33b78ca3d3c14e Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 2 Nov 2025 11:36:20 -0500
Subject: [PATCH v19 1/4] Reorganize heap update logic
This commit refactors the interaction between heap_tuple_update(),
heap_update(), and simple_heap_update() to improve code organization
and flexibility. The changes are functionally equivalent to the
previous implementation and have no performance impact.
The primary motivation is to prepare for upcoming modifications to
how and where modified attributes are identified during the update
path, particularly for catalog updates.
As part of this reorganization, the handling of replica identity key
attributes has been adjusted. Instead of fetching a second copy of
the bitmap during an update operation, the caller is now required to
provide it. This change applies to both heap_update() and
heap_delete().
No user-visible changes.
---
src/backend/access/heap/heapam.c | 568 +++++++++++------------
src/backend/access/heap/heapam_handler.c | 117 ++++-
src/include/access/heapam.h | 24 +-
3 files changed, 410 insertions(+), 299 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 4b0c49f4bb0..aff47481345 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -39,18 +39,24 @@
#include "access/syncscan.h"
#include "access/valid.h"
#include "access/visibilitymap.h"
+#include "access/xact.h"
#include "access/xloginsert.h"
+#include "catalog/catalog.h"
#include "catalog/pg_database.h"
#include "catalog/pg_database_d.h"
#include "commands/vacuum.h"
+#include "nodes/bitmapset.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
+#include "storage/bufmgr.h"
+#include "storage/itemptr.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
#include "storage/procarray.h"
#include "utils/datum.h"
#include "utils/injection_point.h"
#include "utils/inval.h"
+#include "utils/relcache.h"
#include "utils/spccache.h"
#include "utils/syscache.h"
@@ -62,16 +68,8 @@ static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
HeapTuple newtup, HeapTuple old_key_tuple,
bool all_visible_cleared, bool new_all_visible_cleared);
#ifdef USE_ASSERT_CHECKING
-static void check_lock_if_inplace_updateable_rel(Relation relation,
- const ItemPointerData *otid,
- HeapTuple newtup);
static void check_inplace_rel_lock(HeapTuple oldtup);
#endif
-static Bitmapset *HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external);
static bool heap_acquire_tuplock(Relation relation, const ItemPointerData *tid,
LockTupleMode mode, LockWaitPolicy wait_policy,
bool *have_tuple_lock);
@@ -103,10 +101,10 @@ static bool ConditionalMultiXactIdWait(MultiXactId multi, MultiXactStatus status
static void index_delete_sort(TM_IndexDeleteOp *delstate);
static int bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate);
static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
-static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
+static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp,
+ Bitmapset *rid_attrs, bool key_required,
bool *copy);
-
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
* corresponding MultiXactStatuses (one to merely lock tuples, another one to
@@ -2799,6 +2797,7 @@ heap_delete(Relation relation, const ItemPointerData *tid,
Buffer buffer;
Buffer vmbuffer = InvalidBuffer;
TransactionId new_xmax;
+ Bitmapset *rid_attrs;
uint16 new_infomask,
new_infomask2;
bool have_tuple_lock = false;
@@ -2811,6 +2810,8 @@ heap_delete(Relation relation, const ItemPointerData *tid,
AssertHasSnapshotForToast(relation);
+ rid_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
/*
* Forbid this during a parallel operation, lest it allocate a combo CID.
* Other workers might need that combo CID for visibility checks, and we
@@ -3014,6 +3015,7 @@ l1:
UnlockTupleTuplock(relation, &(tp.t_self), LockTupleExclusive);
if (vmbuffer != InvalidBuffer)
ReleaseBuffer(vmbuffer);
+ bms_free(rid_attrs);
return result;
}
@@ -3035,7 +3037,10 @@ l1:
* Compute replica identity tuple before entering the critical section so
* we don't PANIC upon a memory allocation failure.
*/
- old_key_tuple = ExtractReplicaIdentity(relation, &tp, true, &old_key_copied);
+ old_key_tuple = ExtractReplicaIdentity(relation, &tp, rid_attrs,
+ true, &old_key_copied);
+ bms_free(rid_attrs);
+ rid_attrs = NULL;
/*
* If this is the first possibly-multixact-able operation in the current
@@ -3247,7 +3252,10 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* heap_update - replace a tuple
*
* See table_tuple_update() for an explanation of the parameters, except that
- * this routine directly takes a tuple rather than a slot.
+ * this routine directly takes a heap tuple rather than a slot.
+ *
+ * It's required that the caller has acquired the pin and lock on the buffer.
+ * That lock and pin will be managed here, not in the caller.
*
* In the failure cases, the routine fills *tmfd with the tuple's t_ctid,
* t_xmax (resolving a possible MultiXact, if necessary), and t_cmax (the last
@@ -3255,30 +3263,21 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* generated by another transaction).
*/
TM_Result
-heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+heap_update(Relation relation, HeapTupleData *oldtup,
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
+ Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
+ Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
+ Bitmapset *mix_attrs, Buffer *vmbuffer,
+ bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
- Bitmapset *hot_attrs;
- Bitmapset *sum_attrs;
- Bitmapset *key_attrs;
- Bitmapset *id_attrs;
- Bitmapset *interesting_attrs;
- Bitmapset *modified_attrs;
- ItemId lp;
- HeapTupleData oldtup;
HeapTuple heaptup;
HeapTuple old_key_tuple = NULL;
bool old_key_copied = false;
- Page page;
- BlockNumber block;
MultiXactStatus mxact_status;
- Buffer buffer,
- newbuf,
- vmbuffer = InvalidBuffer,
+ Buffer newbuf,
vmbuffer_new = InvalidBuffer;
bool need_toast;
Size newtupsize,
@@ -3292,7 +3291,6 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
bool all_visible_cleared_new = false;
bool checked_lockers;
bool locker_remains;
- bool id_has_external = false;
TransactionId xmax_new_tuple,
xmax_old_tuple;
uint16 infomask_old_tuple,
@@ -3300,144 +3298,13 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
infomask_new_tuple,
infomask2_new_tuple;
- Assert(ItemPointerIsValid(otid));
-
- /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
- Assert(HeapTupleHeaderGetNatts(newtup->t_data) <=
- RelationGetNumberOfAttributes(relation));
-
+ Assert(BufferIsLockedByMe(buffer));
+ Assert(ItemIdIsNormal(lp));
AssertHasSnapshotForToast(relation);
- /*
- * Forbid this during a parallel operation, lest it allocate a combo CID.
- * Other workers might need that combo CID for visibility checks, and we
- * have no provision for broadcasting it to them.
- */
- if (IsInParallelMode())
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
- errmsg("cannot update tuples during a parallel operation")));
-
-#ifdef USE_ASSERT_CHECKING
- check_lock_if_inplace_updateable_rel(relation, otid, newtup);
-#endif
-
- /*
- * Fetch the list of attributes to be checked for various operations.
- *
- * For HOT considerations, this is wasted effort if we fail to update or
- * have to put the new tuple on a different page. But we must compute the
- * list before obtaining buffer lock --- in the worst case, if we are
- * doing an update on one of the relevant system catalogs, we could
- * deadlock if we try to fetch the list later. In any case, the relcache
- * caches the data so this is usually pretty cheap.
- *
- * We also need columns used by the replica identity and columns that are
- * considered the "key" of rows in the table.
- *
- * Note that we get copies of each bitmap, so we need not worry about
- * relcache flush happening midway through.
- */
- hot_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_HOT_BLOCKING);
- sum_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_SUMMARIZED);
- key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
- id_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
- interesting_attrs = NULL;
- interesting_attrs = bms_add_members(interesting_attrs, hot_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, sum_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, key_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, id_attrs);
-
- block = ItemPointerGetBlockNumber(otid);
- INJECTION_POINT("heap_update-before-pin", NULL);
- buffer = ReadBuffer(relation, block);
- page = BufferGetPage(buffer);
-
- /*
- * Before locking the buffer, pin the visibility map page if it appears to
- * be necessary. Since we haven't got the lock yet, someone else might be
- * in the middle of changing this, so we'll need to recheck after we have
- * the lock.
- */
- if (PageIsAllVisible(page))
- visibilitymap_pin(relation, block, &vmbuffer);
-
- LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
-
- lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
-
- /*
- * Usually, a buffer pin and/or snapshot blocks pruning of otid, ensuring
- * we see LP_NORMAL here. When the otid origin is a syscache, we may have
- * neither a pin nor a snapshot. Hence, we may see other LP_ states, each
- * of which indicates concurrent pruning.
- *
- * Failing with TM_Updated would be most accurate. However, unlike other
- * TM_Updated scenarios, we don't know the successor ctid in LP_UNUSED and
- * LP_DEAD cases. While the distinction between TM_Updated and TM_Deleted
- * does matter to SQL statements UPDATE and MERGE, those SQL statements
- * hold a snapshot that ensures LP_NORMAL. Hence, the choice between
- * TM_Updated and TM_Deleted affects only the wording of error messages.
- * Settle on TM_Deleted, for two reasons. First, it avoids complicating
- * the specification of when tmfd->ctid is valid. Second, it creates
- * error log evidence that we took this branch.
- *
- * Since it's possible to see LP_UNUSED at otid, it's also possible to see
- * LP_NORMAL for a tuple that replaced LP_UNUSED. If it's a tuple for an
- * unrelated row, we'll fail with "duplicate key value violates unique".
- * XXX if otid is the live, newer version of the newtup row, we'll discard
- * changes originating in versions of this catalog row after the version
- * the caller got from syscache. See syscache-update-pruned.spec.
- */
- if (!ItemIdIsNormal(lp))
- {
- Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
-
- UnlockReleaseBuffer(buffer);
- Assert(!have_tuple_lock);
- if (vmbuffer != InvalidBuffer)
- ReleaseBuffer(vmbuffer);
- tmfd->ctid = *otid;
- tmfd->xmax = InvalidTransactionId;
- tmfd->cmax = InvalidCommandId;
- *update_indexes = TU_None;
-
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- /* modified_attrs not yet initialized */
- bms_free(interesting_attrs);
- return TM_Deleted;
- }
-
- /*
- * Fill in enough data in oldtup for HeapDetermineColumnsInfo to work
- * properly.
- */
- oldtup.t_tableOid = RelationGetRelid(relation);
- oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
- oldtup.t_len = ItemIdGetLength(lp);
- oldtup.t_self = *otid;
-
- /* the new tuple is ready, except for this: */
+ /* The new tuple is ready, except for this */
newtup->t_tableOid = RelationGetRelid(relation);
- /*
- * Determine columns modified by the update. Additionally, identify
- * whether any of the unmodified replica identity key attributes in the
- * old tuple is externally stored or not. This is required because for
- * such attributes the flattened value won't be WAL logged as part of the
- * new tuple so we must include it as part of the old_key_tuple. See
- * ExtractReplicaIdentity.
- */
- modified_attrs = HeapDetermineColumnsInfo(relation, interesting_attrs,
- id_attrs, &oldtup,
- newtup, &id_has_external);
-
/*
* If we're not updating any "key" column, we can grab a weaker lock type.
* This allows for more concurrency when we are running simultaneously
@@ -3449,7 +3316,7 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
* is updates that don't manipulate key columns, not those that
* serendipitously arrive at the same key values.
*/
- if (!bms_overlap(modified_attrs, key_attrs))
+ if (!bms_overlap(mix_attrs, pk_attrs))
{
*lockmode = LockTupleNoKeyExclusive;
mxact_status = MultiXactStatusNoKeyUpdate;
@@ -3473,17 +3340,10 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
key_intact = false;
}
- /*
- * Note: beyond this point, use oldtup not otid to refer to old tuple.
- * otid may very well point at newtup->t_self, which we will overwrite
- * with the new tuple's location, so there's great risk of confusion if we
- * use otid anymore.
- */
-
l2:
checked_lockers = false;
locker_remains = false;
- result = HeapTupleSatisfiesUpdate(&oldtup, cid, buffer);
+ result = HeapTupleSatisfiesUpdate(oldtup, cid, buffer);
/* see below about the "no wait" case */
Assert(result != TM_BeingModified || wait);
@@ -3515,8 +3375,8 @@ l2:
*/
/* must copy state data before unlocking buffer */
- xwait = HeapTupleHeaderGetRawXmax(oldtup.t_data);
- infomask = oldtup.t_data->t_infomask;
+ xwait = HeapTupleHeaderGetRawXmax(oldtup->t_data);
+ infomask = oldtup->t_data->t_infomask;
/*
* Now we have to do something about the existing locker. If it's a
@@ -3556,13 +3416,12 @@ l2:
* requesting a lock and already have one; avoids deadlock).
*/
if (!current_is_member)
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &oldtup->t_self, *lockmode,
LockWaitBlock, &have_tuple_lock);
/* wait for multixact */
MultiXactIdWait((MultiXactId) xwait, mxact_status, infomask,
- relation, &oldtup.t_self, XLTW_Update,
- &remain);
+ relation, &oldtup->t_self, XLTW_Update, &remain);
checked_lockers = true;
locker_remains = remain != 0;
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
@@ -3572,9 +3431,9 @@ l2:
* could update this tuple before we get to this point. Check
* for xmax change, and start over if so.
*/
- if (xmax_infomask_changed(oldtup.t_data->t_infomask,
+ if (xmax_infomask_changed(oldtup->t_data->t_infomask,
infomask) ||
- !TransactionIdEquals(HeapTupleHeaderGetRawXmax(oldtup.t_data),
+ !TransactionIdEquals(HeapTupleHeaderGetRawXmax(oldtup->t_data),
xwait))
goto l2;
}
@@ -3599,8 +3458,8 @@ l2:
* before this one, which are important to keep in case this
* subxact aborts.
*/
- if (!HEAP_XMAX_IS_LOCKED_ONLY(oldtup.t_data->t_infomask))
- update_xact = HeapTupleGetUpdateXid(oldtup.t_data);
+ if (!HEAP_XMAX_IS_LOCKED_ONLY(oldtup->t_data->t_infomask))
+ update_xact = HeapTupleGetUpdateXid(oldtup->t_data);
else
update_xact = InvalidTransactionId;
@@ -3641,9 +3500,9 @@ l2:
* lock.
*/
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &oldtup->t_self, *lockmode,
LockWaitBlock, &have_tuple_lock);
- XactLockTableWait(xwait, relation, &oldtup.t_self,
+ XactLockTableWait(xwait, relation, &oldtup->t_self,
XLTW_Update);
checked_lockers = true;
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
@@ -3653,20 +3512,20 @@ l2:
* other xact could update this tuple before we get to this point.
* Check for xmax change, and start over if so.
*/
- if (xmax_infomask_changed(oldtup.t_data->t_infomask, infomask) ||
+ if (xmax_infomask_changed(oldtup->t_data->t_infomask, infomask) ||
!TransactionIdEquals(xwait,
- HeapTupleHeaderGetRawXmax(oldtup.t_data)))
+ HeapTupleHeaderGetRawXmax(oldtup->t_data)))
goto l2;
/* Otherwise check if it committed or aborted */
- UpdateXmaxHintBits(oldtup.t_data, buffer, xwait);
- if (oldtup.t_data->t_infomask & HEAP_XMAX_INVALID)
+ UpdateXmaxHintBits(oldtup->t_data, buffer, xwait);
+ if (oldtup->t_data->t_infomask & HEAP_XMAX_INVALID)
can_continue = true;
}
if (can_continue)
result = TM_Ok;
- else if (!ItemPointerEquals(&oldtup.t_self, &oldtup.t_data->t_ctid))
+ else if (!ItemPointerEquals(&oldtup->t_self, &oldtup->t_data->t_ctid))
result = TM_Updated;
else
result = TM_Deleted;
@@ -3679,39 +3538,33 @@ l2:
result == TM_Updated ||
result == TM_Deleted ||
result == TM_BeingModified);
- Assert(!(oldtup.t_data->t_infomask & HEAP_XMAX_INVALID));
+ Assert(!(oldtup->t_data->t_infomask & HEAP_XMAX_INVALID));
Assert(result != TM_Updated ||
- !ItemPointerEquals(&oldtup.t_self, &oldtup.t_data->t_ctid));
+ !ItemPointerEquals(&oldtup->t_self, &oldtup->t_data->t_ctid));
}
if (crosscheck != InvalidSnapshot && result == TM_Ok)
{
/* Perform additional check for transaction-snapshot mode RI updates */
- if (!HeapTupleSatisfiesVisibility(&oldtup, crosscheck, buffer))
+ if (!HeapTupleSatisfiesVisibility(oldtup, crosscheck, buffer))
result = TM_Updated;
}
if (result != TM_Ok)
{
- tmfd->ctid = oldtup.t_data->t_ctid;
- tmfd->xmax = HeapTupleHeaderGetUpdateXid(oldtup.t_data);
+ tmfd->ctid = oldtup->t_data->t_ctid;
+ tmfd->xmax = HeapTupleHeaderGetUpdateXid(oldtup->t_data);
if (result == TM_SelfModified)
- tmfd->cmax = HeapTupleHeaderGetCmax(oldtup.t_data);
+ tmfd->cmax = HeapTupleHeaderGetCmax(oldtup->t_data);
else
tmfd->cmax = InvalidCommandId;
UnlockReleaseBuffer(buffer);
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
- if (vmbuffer != InvalidBuffer)
- ReleaseBuffer(vmbuffer);
+ UnlockTupleTuplock(relation, &oldtup->t_self, *lockmode);
+ if (*vmbuffer != InvalidBuffer)
+ ReleaseBuffer(*vmbuffer);
*update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
return result;
}
@@ -3724,10 +3577,10 @@ l2:
* tuple has been locked or updated under us, but hopefully it won't
* happen very often.
*/
- if (vmbuffer == InvalidBuffer && PageIsAllVisible(page))
+ if (*vmbuffer == InvalidBuffer && PageIsAllVisible(page))
{
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- visibilitymap_pin(relation, block, &vmbuffer);
+ visibilitymap_pin(relation, block, vmbuffer);
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
goto l2;
}
@@ -3738,9 +3591,9 @@ l2:
* If the tuple we're updating is locked, we need to preserve the locking
* info in the old tuple's Xmax. Prepare a new Xmax value for this.
*/
- compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
- oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2,
+ compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup->t_data),
+ oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2,
xid, *lockmode, true,
&xmax_old_tuple, &infomask_old_tuple,
&infomask2_old_tuple);
@@ -3752,12 +3605,12 @@ l2:
* tuple. (In rare cases that might also be InvalidTransactionId and yet
* not have the HEAP_XMAX_INVALID bit set; that's fine.)
*/
- if ((oldtup.t_data->t_infomask & HEAP_XMAX_INVALID) ||
- HEAP_LOCKED_UPGRADED(oldtup.t_data->t_infomask) ||
+ if ((oldtup->t_data->t_infomask & HEAP_XMAX_INVALID) ||
+ HEAP_LOCKED_UPGRADED(oldtup->t_data->t_infomask) ||
(checked_lockers && !locker_remains))
xmax_new_tuple = InvalidTransactionId;
else
- xmax_new_tuple = HeapTupleHeaderGetRawXmax(oldtup.t_data);
+ xmax_new_tuple = HeapTupleHeaderGetRawXmax(oldtup->t_data);
if (!TransactionIdIsValid(xmax_new_tuple))
{
@@ -3772,7 +3625,7 @@ l2:
* Note that since we're doing an update, the only possibility is that
* the lockers had FOR KEY SHARE lock.
*/
- if (oldtup.t_data->t_infomask & HEAP_XMAX_IS_MULTI)
+ if (oldtup->t_data->t_infomask & HEAP_XMAX_IS_MULTI)
{
GetMultiXactIdHintBits(xmax_new_tuple, &infomask_new_tuple,
&infomask2_new_tuple);
@@ -3800,7 +3653,7 @@ l2:
* Replace cid with a combo CID if necessary. Note that we already put
* the plain cid into the new tuple.
*/
- HeapTupleHeaderAdjustCmax(oldtup.t_data, &cid, &iscombo);
+ HeapTupleHeaderAdjustCmax(oldtup->t_data, &cid, &iscombo);
/*
* If the toaster needs to be activated, OR if the new tuple will not fit
@@ -3817,12 +3670,12 @@ l2:
relation->rd_rel->relkind != RELKIND_MATVIEW)
{
/* toast table entries should never be recursively toasted */
- Assert(!HeapTupleHasExternal(&oldtup));
+ Assert(!HeapTupleHasExternal(oldtup));
Assert(!HeapTupleHasExternal(newtup));
need_toast = false;
}
else
- need_toast = (HeapTupleHasExternal(&oldtup) ||
+ need_toast = (HeapTupleHasExternal(oldtup) ||
HeapTupleHasExternal(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
@@ -3855,9 +3708,9 @@ l2:
* updating, because the potentially created multixact would otherwise
* be wrong.
*/
- compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
- oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2,
+ compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup->t_data),
+ oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2,
xid, *lockmode, false,
&xmax_lock_old_tuple, &infomask_lock_old_tuple,
&infomask2_lock_old_tuple);
@@ -3867,18 +3720,18 @@ l2:
START_CRIT_SECTION();
/* Clear obsolete visibility flags ... */
- oldtup.t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
- oldtup.t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
- HeapTupleClearHotUpdated(&oldtup);
+ oldtup->t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
+ oldtup->t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
+ HeapTupleClearHotUpdated(oldtup);
/* ... and store info about transaction updating this tuple */
Assert(TransactionIdIsValid(xmax_lock_old_tuple));
- HeapTupleHeaderSetXmax(oldtup.t_data, xmax_lock_old_tuple);
- oldtup.t_data->t_infomask |= infomask_lock_old_tuple;
- oldtup.t_data->t_infomask2 |= infomask2_lock_old_tuple;
- HeapTupleHeaderSetCmax(oldtup.t_data, cid, iscombo);
+ HeapTupleHeaderSetXmax(oldtup->t_data, xmax_lock_old_tuple);
+ oldtup->t_data->t_infomask |= infomask_lock_old_tuple;
+ oldtup->t_data->t_infomask2 |= infomask2_lock_old_tuple;
+ HeapTupleHeaderSetCmax(oldtup->t_data, cid, iscombo);
/* temporarily make it look not-updated, but locked */
- oldtup.t_data->t_ctid = oldtup.t_self;
+ oldtup->t_data->t_ctid = oldtup->t_self;
/*
* Clear all-frozen bit on visibility map if needed. We could
@@ -3887,7 +3740,7 @@ l2:
* worthwhile.
*/
if (PageIsAllVisible(page) &&
- visibilitymap_clear(relation, block, vmbuffer,
+ visibilitymap_clear(relation, block, *vmbuffer,
VISIBILITYMAP_ALL_FROZEN))
cleared_all_frozen = true;
@@ -3901,10 +3754,10 @@ l2:
XLogBeginInsert();
XLogRegisterBuffer(0, buffer, REGBUF_STANDARD);
- xlrec.offnum = ItemPointerGetOffsetNumber(&oldtup.t_self);
+ xlrec.offnum = ItemPointerGetOffsetNumber(&oldtup->t_self);
xlrec.xmax = xmax_lock_old_tuple;
- xlrec.infobits_set = compute_infobits(oldtup.t_data->t_infomask,
- oldtup.t_data->t_infomask2);
+ xlrec.infobits_set = compute_infobits(oldtup->t_data->t_infomask,
+ oldtup->t_data->t_infomask2);
xlrec.flags =
cleared_all_frozen ? XLH_LOCK_ALL_FROZEN_CLEARED : 0;
XLogRegisterData(&xlrec, SizeOfHeapLock);
@@ -3926,7 +3779,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = heap_toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = heap_toast_insert_or_update(relation, newtup, oldtup, 0);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
@@ -3962,20 +3815,20 @@ l2:
/* It doesn't fit, must use RelationGetBufferForTuple. */
newbuf = RelationGetBufferForTuple(relation, heaptup->t_len,
buffer, 0, NULL,
- &vmbuffer_new, &vmbuffer,
+ &vmbuffer_new, vmbuffer,
0);
/* We're all done. */
break;
}
/* Acquire VM page pin if needed and we don't have it. */
- if (vmbuffer == InvalidBuffer && PageIsAllVisible(page))
- visibilitymap_pin(relation, block, &vmbuffer);
+ if (*vmbuffer == InvalidBuffer && PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, vmbuffer);
/* Re-acquire the lock on the old tuple's page. */
LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
/* Re-check using the up-to-date free space */
pagefree = PageGetHeapFreeSpace(page);
if (newtupsize > pagefree ||
- (vmbuffer == InvalidBuffer && PageIsAllVisible(page)))
+ (*vmbuffer == InvalidBuffer && PageIsAllVisible(page)))
{
/*
* Rats, it doesn't fit anymore, or somebody just now set the
@@ -4013,7 +3866,7 @@ l2:
* will include checking the relation level, there is no benefit to a
* separate check for the new tuple.
*/
- CheckForSerializableConflictIn(relation, &oldtup.t_self,
+ CheckForSerializableConflictIn(relation, &oldtup->t_self,
BufferGetBlockNumber(buffer));
/*
@@ -4021,7 +3874,6 @@ l2:
* has enough space for the new tuple. If they are the same buffer, only
* one pin is held.
*/
-
if (newbuf == buffer)
{
/*
@@ -4029,7 +3881,7 @@ l2:
* to do a HOT update. Check if any of the index columns have been
* changed.
*/
- if (!bms_overlap(modified_attrs, hot_attrs))
+ if (!bms_overlap(mix_attrs, hot_attrs))
{
use_hot_update = true;
@@ -4040,7 +3892,7 @@ l2:
* indexes if the columns were updated, or we may fail to detect
* e.g. value bound changes in BRIN minmax indexes.
*/
- if (bms_overlap(modified_attrs, sum_attrs))
+ if (bms_overlap(mix_attrs, sum_attrs))
summarized_update = true;
}
}
@@ -4057,10 +3909,8 @@ l2:
* logged. Pass old key required as true only if the replica identity key
* columns are modified or it has external data.
*/
- old_key_tuple = ExtractReplicaIdentity(relation, &oldtup,
- bms_overlap(modified_attrs, id_attrs) ||
- id_has_external,
- &old_key_copied);
+ old_key_tuple = ExtractReplicaIdentity(relation, oldtup, rid_attrs,
+ rep_id_key_required, &old_key_copied);
/* NO EREPORT(ERROR) from here till changes are logged */
START_CRIT_SECTION();
@@ -4082,7 +3932,7 @@ l2:
if (use_hot_update)
{
/* Mark the old tuple as HOT-updated */
- HeapTupleSetHotUpdated(&oldtup);
+ HeapTupleSetHotUpdated(oldtup);
/* And mark the new tuple as heap-only */
HeapTupleSetHeapOnly(heaptup);
/* Mark the caller's copy too, in case different from heaptup */
@@ -4091,7 +3941,7 @@ l2:
else
{
/* Make sure tuples are correctly marked as not-HOT */
- HeapTupleClearHotUpdated(&oldtup);
+ HeapTupleClearHotUpdated(oldtup);
HeapTupleClearHeapOnly(heaptup);
HeapTupleClearHeapOnly(newtup);
}
@@ -4100,17 +3950,17 @@ l2:
/* Clear obsolete visibility flags, possibly set by ourselves above... */
- oldtup.t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
- oldtup.t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
+ oldtup->t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
+ oldtup->t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
/* ... and store info about transaction updating this tuple */
Assert(TransactionIdIsValid(xmax_old_tuple));
- HeapTupleHeaderSetXmax(oldtup.t_data, xmax_old_tuple);
- oldtup.t_data->t_infomask |= infomask_old_tuple;
- oldtup.t_data->t_infomask2 |= infomask2_old_tuple;
- HeapTupleHeaderSetCmax(oldtup.t_data, cid, iscombo);
+ HeapTupleHeaderSetXmax(oldtup->t_data, xmax_old_tuple);
+ oldtup->t_data->t_infomask |= infomask_old_tuple;
+ oldtup->t_data->t_infomask2 |= infomask2_old_tuple;
+ HeapTupleHeaderSetCmax(oldtup->t_data, cid, iscombo);
/* record address of new tuple in t_ctid of old one */
- oldtup.t_data->t_ctid = heaptup->t_self;
+ oldtup->t_data->t_ctid = heaptup->t_self;
/* clear PD_ALL_VISIBLE flags, reset all visibilitymap bits */
if (PageIsAllVisible(BufferGetPage(buffer)))
@@ -4118,7 +3968,7 @@ l2:
all_visible_cleared = true;
PageClearAllVisible(BufferGetPage(buffer));
visibilitymap_clear(relation, BufferGetBlockNumber(buffer),
- vmbuffer, VISIBILITYMAP_VALID_BITS);
+ *vmbuffer, VISIBILITYMAP_VALID_BITS);
}
if (newbuf != buffer && PageIsAllVisible(BufferGetPage(newbuf)))
{
@@ -4143,12 +3993,12 @@ l2:
*/
if (RelationIsAccessibleInLogicalDecoding(relation))
{
- log_heap_new_cid(relation, &oldtup);
+ log_heap_new_cid(relation, oldtup);
log_heap_new_cid(relation, heaptup);
}
recptr = log_heap_update(relation, buffer,
- newbuf, &oldtup, heaptup,
+ newbuf, oldtup, heaptup,
old_key_tuple,
all_visible_cleared,
all_visible_cleared_new);
@@ -4173,7 +4023,7 @@ l2:
* both tuple versions in one call to inval.c so we can avoid redundant
* sinval messages.)
*/
- CacheInvalidateHeapTuple(relation, &oldtup, heaptup);
+ CacheInvalidateHeapTuple(relation, oldtup, heaptup);
/* Now we can release the buffer(s) */
if (newbuf != buffer)
@@ -4181,14 +4031,14 @@ l2:
ReleaseBuffer(buffer);
if (BufferIsValid(vmbuffer_new))
ReleaseBuffer(vmbuffer_new);
- if (BufferIsValid(vmbuffer))
- ReleaseBuffer(vmbuffer);
+ if (BufferIsValid(*vmbuffer))
+ ReleaseBuffer(*vmbuffer);
/*
* Release the lmgr tuple lock, if we had it.
*/
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &oldtup->t_self, *lockmode);
pgstat_count_heap_update(relation, use_hot_update, newbuf != buffer);
@@ -4221,13 +4071,6 @@ l2:
if (old_key_tuple != NULL && old_key_copied)
heap_freetuple(old_key_tuple);
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
- bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
-
return TM_Ok;
}
@@ -4236,7 +4079,7 @@ l2:
* Confirm adequate lock held during heap_update(), per rules from
* README.tuplock section "Locking to write inplace-updated tables".
*/
-static void
+void
check_lock_if_inplace_updateable_rel(Relation relation,
const ItemPointerData *otid,
HeapTuple newtup)
@@ -4408,7 +4251,7 @@ heap_attr_equals(TupleDesc tupdesc, int attrnum, Datum value1, Datum value2,
* listed as interesting) of the old tuple is a member of external_cols and is
* stored externally.
*/
-static Bitmapset *
+Bitmapset *
HeapDetermineColumnsInfo(Relation relation,
Bitmapset *interesting_cols,
Bitmapset *external_cols,
@@ -4491,25 +4334,175 @@ HeapDetermineColumnsInfo(Relation relation,
}
/*
- * simple_heap_update - replace a tuple
- *
- * This routine may be used to update a tuple when concurrent updates of
- * the target tuple are not expected (for example, because we have a lock
- * on the relation associated with the tuple). Any failure is reported
- * via ereport().
+ * This routine may be used to update a tuple when concurrent updates of the
+ * target tuple are not expected (for example, because we have a lock on the
+ * relation associated with the tuple). Any failure is reported via ereport().
*/
void
-simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup,
+simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
TM_FailureData tmfd;
LockTupleMode lockmode;
+ Buffer buffer;
+ Buffer vmbuffer = InvalidBuffer;
+ Page page;
+ BlockNumber block;
+ Bitmapset *hot_attrs,
+ *sum_attrs,
+ *pk_attrs,
+ *rid_attrs,
+ *mix_attrs,
+ *idx_attrs;
+ ItemId lp;
+ HeapTupleData oldtup;
+ bool rep_id_key_required = false;
+
+ Assert(ItemPointerIsValid(otid));
+
+ /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
+ Assert(HeapTupleHeaderGetNatts(tuple->t_data) <=
+ RelationGetNumberOfAttributes(relation));
+
+ /*
+ * Forbid this during a parallel operation, lest it allocate a combo CID.
+ * Other workers might need that combo CID for visibility checks, and we
+ * have no provision for broadcasting it to them.
+ */
+ if (IsInParallelMode())
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
+ errmsg("cannot update tuples during a parallel operation")));
+
+#ifdef USE_ASSERT_CHECKING
+ check_lock_if_inplace_updateable_rel(relation, otid, tuple);
+#endif
+
+ /*
+ * Fetch the list of attributes to be checked for various operations.
+ *
+ * For HOT considerations, this is wasted effort if we fail to update or
+ * have to put the new tuple on a different page. But we must compute the
+ * list before obtaining buffer lock --- in the worst case, if we are
+ * doing an update on one of the relevant system catalogs, we could
+ * deadlock if we try to fetch the list later. In any case, the relcache
+ * caches the data so this is usually pretty cheap.
+ *
+ * We also need columns used by the replica identity and columns that are
+ * considered the "key" of rows in the table.
+ *
+ * Note that we get copies of each bitmap, so we need not worry about
+ * relcache flush happening midway through.
+ */
+ hot_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_HOT_BLOCKING);
+ sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+ pk_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
+ rid_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
+ idx_attrs = bms_copy(hot_attrs);
+ idx_attrs = bms_add_members(idx_attrs, sum_attrs);
+ idx_attrs = bms_add_members(idx_attrs, pk_attrs);
+ idx_attrs = bms_add_members(idx_attrs, rid_attrs);
+
+ block = ItemPointerGetBlockNumber(otid);
+ INJECTION_POINT("heap_update-before-pin", NULL);
+ buffer = ReadBuffer(relation, block);
+ page = BufferGetPage(buffer);
+
+ /*
+ * Before locking the buffer, pin the visibility map page if it appears to
+ * be necessary. Since we haven't got the lock yet, someone else might be
+ * in the middle of changing this, so we'll need to recheck after we have
+ * the lock.
+ */
+ if (PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, &vmbuffer);
+
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+
+ lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
+
+ /*
+ * Usually, a buffer pin and/or snapshot blocks pruning of otid, ensuring
+ * we see LP_NORMAL here. When the otid origin is a syscache, we may have
+ * neither a pin nor a snapshot. Hence, we may see other LP_ states, each
+ * of which indicates concurrent pruning.
+ *
+ * Failing with TM_Updated would be most accurate. However, unlike other
+ * TM_Updated scenarios, we don't know the successor ctid in LP_UNUSED and
+ * LP_DEAD cases. While the distinction between TM_Updated and TM_Deleted
+ * does matter to SQL statements UPDATE and MERGE, those SQL statements
+ * hold a snapshot that ensures LP_NORMAL. Hence, the choice between
+ * TM_Updated and TM_Deleted affects only the wording of error messages.
+ * Settle on TM_Deleted, for two reasons. First, it avoids complicating
+ * the specification of when tmfd->ctid is valid. Second, it creates
+ * error log evidence that we took this branch.
+ *
+ * Since it's possible to see LP_UNUSED at otid, it's also possible to see
+ * LP_NORMAL for a tuple that replaced LP_UNUSED. If it's a tuple for an
+ * unrelated row, we'll fail with "duplicate key value violates unique".
+ * XXX if otid is the live, newer version of the newtup row, we'll discard
+ * changes originating in versions of this catalog row after the version
+ * the caller got from syscache. See syscache-update-pruned.spec.
+ */
+ if (!ItemIdIsNormal(lp))
+ {
+ Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
+
+ UnlockReleaseBuffer(buffer);
+ if (vmbuffer != InvalidBuffer)
+ ReleaseBuffer(vmbuffer);
+ *update_indexes = TU_None;
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(idx_attrs);
+ /* mix_attrs not yet initialized */
+
+ elog(ERROR, "tuple concurrently deleted");
+
+ return;
+ }
+
+ /*
+ * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
+ * then pass that on to heap_update.
+ */
+ oldtup.t_tableOid = RelationGetRelid(relation);
+ oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
+ oldtup.t_len = ItemIdGetLength(lp);
+ oldtup.t_self = *otid;
+
+ mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
+ &oldtup, tuple, &rep_id_key_required);
+
+ /*
+ * We'll need to WAL log the replica identity attributes if either they
+ * overlap with the modified indexed attributes or, as we've checked for
+ * just now in HeapDetermineColumnsInfo, they were unmodified external
+ * indexed attributes.
+ */
+ rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+
+ result = heap_update(relation, &oldtup, tuple, GetCurrentCommandId(true),
+ InvalidSnapshot, true /* wait for commit */ , &tmfd, &lockmode,
+ buffer, page, block, lp, hot_attrs, sum_attrs, pk_attrs,
+ rid_attrs, mix_attrs, &vmbuffer, rep_id_key_required,
+ update_indexes);
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(mix_attrs);
+ bms_free(idx_attrs);
- result = heap_update(relation, otid, tup,
- GetCurrentCommandId(true), InvalidSnapshot,
- true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
switch (result)
{
case TM_SelfModified:
@@ -9149,12 +9142,11 @@ log_heap_new_cid(Relation relation, HeapTuple tup)
* the same tuple that was passed in.
*/
static HeapTuple
-ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
- bool *copy)
+ExtractReplicaIdentity(Relation relation, HeapTuple tp, Bitmapset *rid_attrs,
+ bool key_required, bool *copy)
{
TupleDesc desc = RelationGetDescr(relation);
char replident = relation->rd_rel->relreplident;
- Bitmapset *idattrs;
HeapTuple key_tuple;
bool nulls[MaxHeapAttributeNumber];
Datum values[MaxHeapAttributeNumber];
@@ -9185,17 +9177,13 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
if (!key_required)
return NULL;
- /* find out the replica identity columns */
- idattrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
-
/*
* If there's no defined replica identity columns, treat as !key_required.
* (This case should not be reachable from heap_update, since that should
* calculate key_required accurately. But heap_delete just passes
* constant true for key_required, so we can hit this case in deletes.)
*/
- if (bms_is_empty(idattrs))
+ if (bms_is_empty(rid_attrs))
return NULL;
/*
@@ -9208,7 +9196,7 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
for (int i = 0; i < desc->natts; i++)
{
if (bms_is_member(i + 1 - FirstLowInvalidHeapAttributeNumber,
- idattrs))
+ rid_attrs))
Assert(!nulls[i]);
else
nulls[i] = true;
@@ -9217,8 +9205,6 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
key_tuple = heap_form_tuple(desc, values, nulls);
*copy = true;
- bms_free(idattrs);
-
/*
* If the tuple, which by here only contains indexed columns, still has
* toasted columns, force them to be inlined. This is somewhat unlikely
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bcbac844bb6..1cf9a18775d 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -44,6 +44,7 @@
#include "storage/procarray.h"
#include "storage/smgr.h"
#include "utils/builtins.h"
+#include "utils/injection_point.h"
#include "utils/rel.h"
static void reform_and_rewrite_tuple(HeapTuple tuple,
@@ -312,23 +313,133 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
return heap_delete(relation, tid, cid, crosscheck, wait, tmfd, changingPart);
}
-
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd,
LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
{
+ bool rep_id_key_required = false;
bool shouldFree = true;
HeapTuple tuple = ExecFetchSlotHeapTuple(slot, true, &shouldFree);
+ HeapTupleData oldtup;
+ Buffer buffer;
+ Buffer vmbuffer = InvalidBuffer;
+ Page page;
+ BlockNumber block;
+ ItemId lp;
+ Bitmapset *hot_attrs,
+ *sum_attrs,
+ *pk_attrs,
+ *rid_attrs,
+ *mix_attrs,
+ *idx_attrs;
TM_Result result;
+ Assert(ItemPointerIsValid(otid));
+
+ /* Cheap, simplistic check that the tuple matches the rel's rowtype. */
+ Assert(HeapTupleHeaderGetNatts(tuple->t_data) <=
+ RelationGetNumberOfAttributes(relation));
+
+ /*
+ * Forbid this during a parallel operation, lest it allocate a combo CID.
+ * Other workers might need that combo CID for visibility checks, and we
+ * have no provision for broadcasting it to them.
+ */
+ if (IsInParallelMode())
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TRANSACTION_STATE),
+ errmsg("cannot update tuples during a parallel operation")));
+
+#ifdef USE_ASSERT_CHECKING
+ check_lock_if_inplace_updateable_rel(relation, otid, tuple);
+#endif
+
+ /*
+ * Fetch the list of attributes to be checked for various operations.
+ *
+ * For HOT considerations, this is wasted effort if we fail to update or
+ * have to put the new tuple on a different page. But we must compute the
+ * list before obtaining buffer lock --- in the worst case, if we are
+ * doing an update on one of the relevant system catalogs, we could
+ * deadlock if we try to fetch the list later. In any case, the relcache
+ * caches the data so this is usually pretty cheap.
+ *
+ * We also need columns used by the replica identity and columns that are
+ * considered the "key" of rows in the table.
+ *
+ * Note that we get copies of each bitmap, so we need not worry about
+ * relcache flush happening midway through.
+ */
+ hot_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_HOT_BLOCKING);
+ sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+ pk_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
+ rid_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_IDENTITY_KEY);
+
+ idx_attrs = bms_copy(hot_attrs);
+ idx_attrs = bms_add_members(idx_attrs, sum_attrs);
+ idx_attrs = bms_add_members(idx_attrs, pk_attrs);
+ idx_attrs = bms_add_members(idx_attrs, rid_attrs);
+
+ block = ItemPointerGetBlockNumber(otid);
+ INJECTION_POINT("heap_update-before-pin", NULL);
+ buffer = ReadBuffer(relation, block);
+ page = BufferGetPage(buffer);
+
+ /*
+ * Before locking the buffer, pin the visibility map page if it appears to
+ * be necessary. Since we haven't got the lock yet, someone else might be
+ * in the middle of changing this, so we'll need to recheck after we have
+ * the lock.
+ */
+ if (PageIsAllVisible(page))
+ visibilitymap_pin(relation, block, &vmbuffer);
+
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+
+ lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));
+
+ Assert(ItemIdIsNormal(lp));
+
+ /*
+ * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
+ * then pass that on to heap_update.
+ */
+ oldtup.t_tableOid = RelationGetRelid(relation);
+ oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
+ oldtup.t_len = ItemIdGetLength(lp);
+ oldtup.t_self = *otid;
+
+ mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
+ &oldtup, tuple, &rep_id_key_required);
+
+ /*
+ * We'll need to WAL log the replica identity attributes if either they
+ * overlap with the modified indexed attributes or, as we've checked for
+ * just now in HeapDetermineColumnsInfo, they were unmodified external
+ * indexed attributes.
+ */
+ rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
- result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
- tmfd, lockmode, update_indexes);
+ result = heap_update(relation, &oldtup, tuple, cid, crosscheck, wait, tmfd, lockmode,
+ buffer, page, block, lp, hot_attrs, sum_attrs, pk_attrs,
+ rid_attrs, mix_attrs, &vmbuffer, rep_id_key_required, update_indexes);
+
+ bms_free(hot_attrs);
+ bms_free(sum_attrs);
+ bms_free(pk_attrs);
+ bms_free(rid_attrs);
+ bms_free(mix_attrs);
+ bms_free(idx_attrs);
+
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 909db73b7bb..41d541aa6b2 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -321,11 +321,13 @@ extern TM_Result heap_delete(Relation relation, const ItemPointerData *tid,
TM_FailureData *tmfd, bool changingPart);
extern void heap_finish_speculative(Relation relation, const ItemPointerData *tid);
extern void heap_abort_speculative(Relation relation, const ItemPointerData *tid);
-extern TM_Result heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+extern TM_Result heap_update(Relation relation, HeapTupleData *oldtup,
+ HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
+ Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
+ Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
+ Bitmapset *mix_attrs, Buffer *vmbuffer,
+ bool rep_id_key_required, TU_UpdateIndexes *update_indexes);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -391,6 +393,18 @@ extern void log_heap_prune_and_freeze(Relation relation, Buffer buffer,
OffsetNumber *dead, int ndead,
OffsetNumber *unused, int nunused);
+/* in heap/heapam.c */
+extern Bitmapset *HeapDetermineColumnsInfo(Relation relation,
+ Bitmapset *interesting_cols,
+ Bitmapset *external_cols,
+ HeapTuple oldtup, HeapTuple newtup,
+ bool *has_external);
+#ifdef USE_ASSERT_CHECKING
+extern void check_lock_if_inplace_updateable_rel(Relation relation,
+ const ItemPointerData *otid,
+ HeapTuple newtup);
+#endif
+
/* in heap/vacuumlazy.c */
extern void heap_vacuum_rel(Relation rel,
const VacuumParams params, BufferAccessStrategy bstrategy);
--
2.49.0
[application/octet-stream] v19-0002-Track-changed-indexed-columns-in-the-executor-du.patch (34.3K, 3-v19-0002-Track-changed-indexed-columns-in-the-executor-du.patch)
download | inline diff:
From 69d226a736a77ca83e8df2e617226552c431da13 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 26 Oct 2025 10:49:25 -0400
Subject: [PATCH v19 2/4] Track changed indexed columns in the executor during
UPDATEs
Refactor executor update logic to determine which indexed columns have
actually changed during an UPDATE operation rather than leaving this up
to HeapDetermineColumnsInfo in heap_update. This enables the comparison
to happen without taking a lock on the page and opens the door to reuse
in other code paths.
Because heap_update now requires the caller to provide the modified
indexed columns simple_heap_update has become a tad more complex. It is
frequently called from CatalogTupleUpdate which either updates heap
tuples via their form or using heap_modify_tuple. In both cases the
caller does know the modified set of attributes, but sadly those
attributes are lost before being provided to simple_heap_update. Due to
that the "simple" path has to retain the HeapDetermineColumnsInfo logic
of old (for now). In order for that to work it was necessary to split
the (overly large) heap_update call itself up. This moves up into
simple_heap_update and heap_tuple_update a bit of what existed in
heap_update itself. Ideally this will be cleaned up once
CatalogTupleUpdate paths are all recording modified attributes
correctly, when that happens the "simple" path can be simplified again.
ExecCheckIndexedAttrsForChanges replaces HeapDeterminesColumnsInfo and
tts_attr_equal replaces heap_attr_equal changing the test for equality
when calling into heap_tuple_update (but not simple_heap_update). In
the past we used datumIsEqual(), essentially a binary comparison using
memcmp(), now the comparison code in tts_attr_equal uses type-specific
equality function when available and falls back to datumIsEqual() when
not. This change in equality testing has some intended implications and
opens the door for more HOT updates (foreshadowing). For instance,
indexes with collation information allowing more HOT updates when the
index is specified to be case insensitive.
This change forced some logic changes in execReplication on the update
paths is now it is required to have knowledge of the set of attributes
that are both changed and referenced by indexes. Luckilly, the this is
available within calls to slot_modify_data() where LogicalRepTupleData
is processed and has a set of updated attributes. In this case rather
than using ExecCheckIndexedAttrsForChanges we can preseve what
slot_modify_data() identifies as the modified set and then intersect
that with the set of indexes on the relation and get the correct set of
modified indexed attributes required on heap_update().
---
src/backend/access/heap/heapam.c | 12 +-
src/backend/access/heap/heapam_handler.c | 72 +++++--
src/backend/access/table/tableam.c | 5 +-
src/backend/executor/execMain.c | 1 +
src/backend/executor/execReplication.c | 7 +
src/backend/executor/nodeModifyTable.c | 247 ++++++++++++++++++++++-
src/backend/nodes/bitmapset.c | 4 +
src/backend/replication/logical/worker.c | 72 ++++++-
src/backend/utils/cache/relcache.c | 15 ++
src/include/access/tableam.h | 8 +-
src/include/executor/executor.h | 5 +
src/include/nodes/execnodes.h | 1 +
src/include/utils/rel.h | 1 +
src/include/utils/relcache.h | 1 +
14 files changed, 415 insertions(+), 36 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index aff47481345..1cdb72b3a7a 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -3263,12 +3263,12 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* generated by another transaction).
*/
TM_Result
-heap_update(Relation relation, HeapTupleData *oldtup,
- HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
- Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
- Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
- Bitmapset *mix_attrs, Buffer *vmbuffer,
+heap_update(Relation relation, HeapTupleData *oldtup, HeapTuple newtup,
+ CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, LockTupleMode *lockmode,
+ Buffer buffer, Page page, BlockNumber block, ItemId lp,
+ Bitmapset *hot_attrs, Bitmapset *sum_attrs, Bitmapset *pk_attrs,
+ Bitmapset *rid_attrs, Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 1cf9a18775d..ef08e1d3e10 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -315,9 +315,12 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
- CommandId cid, Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd,
- LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
+ CommandId cid, Snapshot snapshot,
+ Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd,
+ LockTupleMode *lockmode,
+ Bitmapset *mix_attrs,
+ TU_UpdateIndexes *update_indexes)
{
bool rep_id_key_required = false;
bool shouldFree = true;
@@ -332,7 +335,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
*sum_attrs,
*pk_attrs,
*rid_attrs,
- *mix_attrs,
*idx_attrs;
TM_Result result;
@@ -414,16 +416,61 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
oldtup.t_len = ItemIdGetLength(lp);
oldtup.t_self = *otid;
- mix_attrs = HeapDetermineColumnsInfo(relation, idx_attrs, rid_attrs,
- &oldtup, tuple, &rep_id_key_required);
-
/*
- * We'll need to WAL log the replica identity attributes if either they
- * overlap with the modified indexed attributes or, as we've checked for
- * just now in HeapDetermineColumnsInfo, they were unmodified external
- * indexed attributes.
+ * We'll need to include the replica identity key when either the identity
+ * key attributes overlap with the modified index attributes or when the
+ * replica identity attributes are stored externally. This is required
+ * because for such attributes the flattened value won't be WAL logged as
+ * part of the new tuple so we must determine if we need to extract and
+ * include them as part of the old_key_tuple (see ExtractReplicaIdentity).
*/
- rep_id_key_required = rep_id_key_required || bms_overlap(mix_attrs, rid_attrs);
+ rep_id_key_required = bms_overlap(mix_attrs, rid_attrs);
+ if (!rep_id_key_required)
+ {
+ Bitmapset *attrs;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ int attidx = -1;
+
+ /*
+ * We don't own idx_attrs so we'll copy it and remove the modified set
+ * to reduce the attributes we need to test in the while loop and
+ * avoid a two branches in the loop.
+ */
+ attrs = bms_difference(idx_attrs, mix_attrs);
+ attrs = bms_int_members(attrs, rid_attrs);
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /*
+ * attidx is zero-based, attrnum is the normal attribute number
+ */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value;
+ bool isnull;
+
+ /*
+ * System attributes are not added into interesting_attrs in
+ * relcache
+ */
+ Assert(attrnum > 0);
+
+ value = heap_getattr(&oldtup, attrnum, tupdesc, &isnull);
+
+ /* No need to check attributes that can't be stored externally */
+ if (isnull ||
+ TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
+ continue;
+
+ /* Check if the old tuple's attribute is stored externally */
+ if (VARATT_IS_EXTERNAL((struct varlena *) DatumGetPointer(value)))
+ {
+ rep_id_key_required = true;
+ break;
+ }
+ }
+
+ bms_free(attrs);
+ }
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
@@ -437,7 +484,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
bms_free(sum_attrs);
bms_free(pk_attrs);
bms_free(rid_attrs);
- bms_free(mix_attrs);
bms_free(idx_attrs);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 5e41404937e..dadcf03ed24 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -336,6 +336,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
+ Bitmapset *modified_indexed_cols,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -346,7 +347,9 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
GetCurrentCommandId(true),
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, &lockmode,
+ modified_indexed_cols,
+ update_indexes);
switch (result)
{
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 27c9eec697b..6b7b6bc8019 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1282,6 +1282,7 @@ InitResultRelInfo(ResultRelInfo *resultRelInfo,
/* The following fields are set later if needed */
resultRelInfo->ri_RowIdAttNo = 0;
resultRelInfo->ri_extraUpdatedCols = NULL;
+ resultRelInfo->ri_ChangedIndexedCols = NULL;
resultRelInfo->ri_projectNew = NULL;
resultRelInfo->ri_newTupleSlot = NULL;
resultRelInfo->ri_oldTupleSlot = NULL;
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index def32774c90..2709e2db0f2 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -32,6 +32,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
#include "utils/typcache.h"
@@ -936,7 +937,13 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ /*
+ * We're not going to call ExecCheckIndexedAttrsForChanges here
+ * because we've already identified the changes earlier on thanks to
+ * slot_modify_data.
+ */
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
+ resultRelInfo->ri_ChangedIndexedCols,
&update_indexes);
conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 4c5647ac38a..4d1cf50e369 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -17,6 +17,7 @@
* ExecModifyTable - retrieve the next tuple from the node
* ExecEndModifyTable - shut down the ModifyTable node
* ExecReScanModifyTable - rescan the ModifyTable node
+ * ExecCheckIndexedAttrsForChanges - find set of updated indexed columns
*
* NOTES
* The ModifyTable node receives input from its outerPlan, which is
@@ -54,11 +55,14 @@
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/tupconvert.h"
+#include "access/tupdesc.h"
#include "access/xact.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
#include "executor/nodeModifyTable.h"
+#include "executor/tuptable.h"
#include "foreign/fdwapi.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
@@ -68,6 +72,8 @@
#include "storage/lmgr.h"
#include "utils/builtins.h"
#include "utils/datum.h"
+#include "utils/float.h"
+#include "utils/lsyscache.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
@@ -176,6 +182,219 @@ static TupleTableSlot *ExecMergeNotMatched(ModifyTableContext *context,
bool canSetTag);
+/*
+ * Compare two datums using the type's default equality operator.
+ *
+ * Returns true if the values are equal according to the type's equality
+ * operator, false otherwise. Falls back to binary comparison if no
+ * type-specific operator is available.
+ *
+ * This function uses the TypeCache infrastructure which caches operator
+ * lookups for efficiency.
+ */
+bool
+tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
+ Datum value1, Datum value2)
+{
+ TypeCacheEntry *typentry;
+
+ LOCAL_FCINFO(fcinfo, 2);
+ Datum result;
+
+ /*
+ * Fast path for common types to avoid even the type cache lookup. These
+ * types have simple equality semantics.
+ */
+ switch (typid)
+ {
+ case INT2OID:
+ return DatumGetInt16(value1) == DatumGetInt16(value2);
+ case INT4OID:
+ return DatumGetInt32(value1) == DatumGetInt32(value2);
+ case INT8OID:
+ return DatumGetInt64(value1) == DatumGetInt64(value2);
+ case FLOAT4OID:
+ return !float4_cmp_internal(DatumGetFloat4(value1), DatumGetFloat4(value2));
+ case FLOAT8OID:
+ return !float8_cmp_internal(DatumGetFloat8(value1), DatumGetFloat8(value2));
+ case BOOLOID:
+ return DatumGetBool(value1) == DatumGetBool(value2);
+ case OIDOID:
+ case REGPROCOID:
+ case REGPROCEDUREOID:
+ case REGOPEROID:
+ case REGOPERATOROID:
+ case REGCLASSOID:
+ case REGTYPEOID:
+ case REGROLEOID:
+ case REGNAMESPACEOID:
+ case REGCONFIGOID:
+ case REGDICTIONARYOID:
+ return DatumGetObjectId(value1) == DatumGetObjectId(value2);
+ case CHAROID:
+ return DatumGetChar(value1) == DatumGetChar(value2);
+ default:
+ /* Continue to type cache lookup */
+ break;
+ }
+
+ /*
+ * Look up the type's equality operator using the type cache. Request both
+ * the operator OID and the function info for efficiency.
+ */
+ typentry = lookup_type_cache(typid,
+ TYPECACHE_EQ_OPR | TYPECACHE_EQ_OPR_FINFO);
+
+ /*
+ * If no equality operator is available, fall back to binary comparison.
+ * This handles types that don't have proper equality operators defined.
+ */
+ if (!OidIsValid(typentry->eq_opr))
+ return datumIsEqual(value1, value2, typbyval, typlen);
+
+ /*
+ * Use the cached function info if available, otherwise look it up. The
+ * type cache keeps this around so subsequent calls are fast.
+ */
+ if (typentry->eq_opr_finfo.fn_addr == NULL)
+ {
+ Oid eq_proc = get_opcode(typentry->eq_opr);
+
+ if (!OidIsValid(eq_proc))
+ /* Shouldn't happen, but fall back to binary comparison */
+ return datumIsEqual(value1, value2, typbyval, typlen);
+
+ fmgr_info_cxt(eq_proc, &typentry->eq_opr_finfo,
+ CacheMemoryContext);
+ }
+
+ /* Set up function call */
+ InitFunctionCallInfoData(*fcinfo, &typentry->eq_opr_finfo, 2,
+ collation, NULL, NULL);
+
+ fcinfo->args[0].value = value1;
+ fcinfo->args[0].isnull = false;
+ fcinfo->args[1].value = value2;
+ fcinfo->args[1].isnull = false;
+
+ /* Invoke the equality operator */
+ result = FunctionCallInvoke(fcinfo);
+
+ /*
+ * If the function returned NULL (shouldn't happen for equality ops),
+ * treat as not equal for safety.
+ */
+ if (fcinfo->isnull)
+ return false;
+
+ return DatumGetBool(result);
+}
+
+/*
+ * Determine which updated attributes actually changed values between old and
+ * new tuples and are referenced by indexes on the relation.
+ *
+ * Returns a Bitmapset of attribute offsets (0-based, adjusted by
+ * FirstLowInvalidHeapAttributeNumber) or NULL if no attributes changed.
+ */
+Bitmapset *
+ExecCheckIndexedAttrsForChanges(ResultRelInfo *relinfo,
+ TupleTableSlot *tts_old,
+ TupleTableSlot *tts_new)
+{
+ Relation relation = relinfo->ri_RelationDesc;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ Bitmapset *indexed_attrs;
+ Bitmapset *modified = NULL;
+ int attidx;
+
+ /* If no indexes, we're done */
+ if (relinfo->ri_NumIndices == 0)
+ return NULL;
+
+ /*
+ * Get the set of index key attributes. This includes summarizing,
+ * expression indexes and attributes mentioned in the predicate of a
+ * partition but not those in INCLUDING.
+ */
+ indexed_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_INDEXED);
+ Assert(!bms_is_empty(indexed_attrs));
+
+ /*
+ * NOTE: It is important to scan all indexed attributes in the tuples
+ * because ExecGetAllUpdatedCols won't include columns that may have been
+ * modified via heap_modify_tuple_by_col which is the case in
+ * tsvector_update_trigger.
+ */
+ attidx = -1;
+ while ((attidx = bms_next_member(indexed_attrs, attidx)) >= 0)
+ {
+ /* attidx is zero-based, attrnum is the normal attribute number */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Form_pg_attribute attr;
+ bool oldnull,
+ newnull;
+ Datum oldval,
+ newval;
+
+ /*
+ * If it's a whole-tuple reference, record as modified. It's not
+ * really worth supporting this case, since it could only succeed
+ * after a no-op update, which is hardly a case worth optimizing for.
+ */
+ if (attrnum == 0)
+ {
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /*
+ * Likewise, include in the modified set any system attribute other
+ * than tableOID; we cannot expect these to be consistent in a HOT
+ * chain, or even to be set correctly yet in the new tuple.
+ */
+ if (attrnum < 0)
+ {
+ if (attrnum != TableOidAttributeNumber)
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /* Extract values from both slots */
+ oldval = slot_getattr(tts_old, attrnum, &oldnull);
+ newval = slot_getattr(tts_new, attrnum, &newnull);
+
+ /* If one value is NULL and the other is not, they are not equal */
+ if (oldnull != newnull)
+ {
+ modified = bms_add_member(modified, attidx);
+ continue;
+ }
+
+ /* If both are NULL, consider them equal */
+ if (oldnull)
+ continue;
+
+ /* Get attribute metadata */
+ Assert(attrnum > 0 && attrnum <= tupdesc->natts);
+ attr = TupleDescAttr(tupdesc, attrnum - 1);
+
+ /* Compare using type-specific equality operator */
+ if (!tts_attr_equal(attr->atttypid,
+ attr->attcollation,
+ attr->attbyval,
+ attr->attlen,
+ oldval,
+ newval))
+ modified = bms_add_member(modified, attidx);
+ }
+
+ bms_free(indexed_attrs);
+
+ return modified;
+}
+
/*
* Verify that the tuples to be produced by INSERT match the
* target relation's rowtype
@@ -2168,8 +2387,8 @@ ExecUpdatePrepareSlot(ResultRelInfo *resultRelInfo,
*/
static TM_Result
ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
- ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *slot,
- bool canSetTag, UpdateContext *updateCxt)
+ ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *oldSlot,
+ TupleTableSlot *slot, bool canSetTag, UpdateContext *updateCxt)
{
EState *estate = context->estate;
Relation resultRelationDesc = resultRelInfo->ri_RelationDesc;
@@ -2291,6 +2510,16 @@ lreplace:
if (resultRelationDesc->rd_att->constr)
ExecConstraints(resultRelInfo, slot, estate);
+ /*
+ * Identify which, if any, indexed attributes were modified here so that
+ * we might reuse it in a few places.
+ */
+ bms_free(resultRelInfo->ri_ChangedIndexedCols);
+ resultRelInfo->ri_ChangedIndexedCols = NULL;
+
+ resultRelInfo->ri_ChangedIndexedCols =
+ ExecCheckIndexedAttrsForChanges(resultRelInfo, oldSlot, slot);
+
/*
* replace the heap tuple
*
@@ -2306,6 +2535,7 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
+ resultRelInfo->ri_ChangedIndexedCols,
&updateCxt->updateIndexes);
return result;
@@ -2524,8 +2754,9 @@ ExecUpdate(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
*/
redo_act:
lockedtid = *tupleid;
- result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, slot,
- canSetTag, &updateCxt);
+
+ result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, oldSlot,
+ slot, canSetTag, &updateCxt);
/*
* If ExecUpdateAct reports that a cross-partition update was done,
@@ -3222,8 +3453,8 @@ lmerge_matched:
Assert(oldtuple == NULL);
result = ExecUpdateAct(context, resultRelInfo, tupleid,
- NULL, newslot, canSetTag,
- &updateCxt);
+ NULL, resultRelInfo->ri_oldTupleSlot,
+ newslot, canSetTag, &updateCxt);
/*
* As in ExecUpdate(), if ExecUpdateAct() reports that a
@@ -3248,6 +3479,7 @@ lmerge_matched:
tupleid, NULL, newslot);
mtstate->mt_merge_updated += 1;
}
+
break;
case CMD_DELETE:
@@ -4333,7 +4565,7 @@ ExecModifyTable(PlanState *pstate)
* For UPDATE/DELETE/MERGE, fetch the row identity info for the tuple
* to be updated/deleted/merged. For a heap relation, that's a TID;
* otherwise we may have a wholerow junk attr that carries the old
- * tuple in toto. Keep this in step with the part of
+ * tuple in total. Keep this in step with the part of
* ExecInitModifyTable that sets up ri_RowIdAttNo.
*/
if (operation == CMD_UPDATE || operation == CMD_DELETE ||
@@ -4509,6 +4741,7 @@ ExecModifyTable(PlanState *pstate)
/* Now apply the update. */
slot = ExecUpdate(&context, resultRelInfo, tupleid, oldtuple,
oldSlot, slot, node->canSetTag);
+
if (tuplock)
UnlockTuple(resultRelInfo->ri_RelationDesc, tupleid,
InplaceUpdateTupleLock);
diff --git a/src/backend/nodes/bitmapset.c b/src/backend/nodes/bitmapset.c
index b4ecf0b0390..9014990267a 100644
--- a/src/backend/nodes/bitmapset.c
+++ b/src/backend/nodes/bitmapset.c
@@ -238,6 +238,10 @@ bms_make_singleton(int x)
void
bms_free(Bitmapset *a)
{
+#if USE_ASSERT_CHECKING
+ Assert(bms_is_valid_set(a));
+#endif
+
if (a)
pfree(a);
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 93970c6af29..b363eaa49cc 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -243,6 +243,8 @@
*/
#include "postgres.h"
+#include "access/sysattr.h"
+#include "nodes/bitmapset.h"
#include <sys/stat.h>
#include <unistd.h>
@@ -275,7 +277,6 @@
#include "replication/logicalrelation.h"
#include "replication/logicalworker.h"
#include "replication/origin.h"
-#include "replication/slot.h"
#include "replication/walreceiver.h"
#include "replication/worker_internal.h"
#include "rewrite/rewriteHandler.h"
@@ -291,6 +292,7 @@
#include "utils/memutils.h"
#include "utils/pg_lsn.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/rls.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
@@ -1110,15 +1112,18 @@ slot_store_data(TupleTableSlot *slot, LogicalRepRelMapEntry *rel,
* "slot" is filled with a copy of the tuple in "srcslot", replacing
* columns provided in "tupleData" and leaving others as-is.
*
+ * Returns a bitmap of the modified columns.
+ *
* Caution: unreplaced pass-by-ref columns in "slot" will point into the
* storage for "srcslot". This is OK for current usage, but someday we may
* need to materialize "slot" at the end to make it independent of "srcslot".
*/
-static void
+static Bitmapset *
slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
LogicalRepRelMapEntry *rel,
LogicalRepTupleData *tupleData)
{
+ Bitmapset *modified = NULL;
int natts = slot->tts_tupleDescriptor->natts;
int i;
@@ -1195,6 +1200,28 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
slot->tts_isnull[i] = true;
}
+ /*
+ * Determine if the replicated value changed the local value by
+ * comparing slots. This is a subset of
+ * ExecCheckIndexedAttrsForChanges.
+ */
+ if (srcslot->tts_isnull[i] != slot->tts_isnull[i])
+ {
+ /* One is NULL, the other is not so the value changed */
+ modified = bms_add_member(modified, i + 1 - FirstLowInvalidHeapAttributeNumber);
+ }
+ else if (!srcslot->tts_isnull[i])
+ {
+ /* Both are not NULL, compare their values */
+ if (!tts_attr_equal(att->atttypid,
+ att->attcollation,
+ att->attbyval,
+ att->attlen,
+ srcslot->tts_values[i],
+ slot->tts_values[i]))
+ modified = bms_add_member(modified, i + 1 - FirstLowInvalidHeapAttributeNumber);
+ }
+
/* Reset attnum for error callback */
apply_error_callback_arg.remote_attnum = -1;
}
@@ -1202,6 +1229,8 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
/* And finally, declare that "slot" contains a valid virtual tuple */
ExecStoreVirtualTuple(slot);
+
+ return modified;
}
/*
@@ -2918,6 +2947,7 @@ apply_handle_update_internal(ApplyExecutionData *edata,
ConflictTupleInfo conflicttuple = {0};
bool found;
MemoryContext oldctx;
+ Bitmapset *indexed = NULL;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
ExecOpenIndices(relinfo, false);
@@ -2934,6 +2964,8 @@ apply_handle_update_internal(ApplyExecutionData *edata,
*/
if (found)
{
+ Bitmapset *modified = NULL;
+
/*
* Report the conflict if the tuple was modified by a different
* origin.
@@ -2957,15 +2989,29 @@ apply_handle_update_internal(ApplyExecutionData *edata,
/* Process and store remote tuple in the slot */
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot, localslot, relmapentry, newtup);
+ modified = slot_modify_data(remoteslot, localslot, relmapentry, newtup);
MemoryContextSwitchTo(oldctx);
+ /*
+ * Normally we'd call ExecCheckIndexedAttrForChanges but here we have
+ * the record of changed columns in the replication state, so let's
+ * use that instead.
+ */
+ indexed = RelationGetIndexAttrBitmap(relinfo->ri_RelationDesc,
+ INDEX_ATTR_BITMAP_INDEXED);
+
+ bms_free(relinfo->ri_ChangedIndexedCols);
+ relinfo->ri_ChangedIndexedCols = bms_int_members(modified, indexed);
+ bms_free(indexed);
+
EvalPlanQualSetSlot(&epqstate, remoteslot);
InitConflictIndexes(relinfo);
- /* Do the actual update. */
+ /* First check privileges */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
+
+ /* Then do the actual update. */
ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
remoteslot);
}
@@ -3455,6 +3501,8 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
bool found;
EPQState epqstate;
ConflictTupleInfo conflicttuple = {0};
+ Bitmapset *modified = NULL;
+ Bitmapset *indexed;
/* Get the matching local tuple from the partition. */
found = FindReplTupleInLocalRel(edata, partrel,
@@ -3523,8 +3571,8 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
* remoteslot_part.
*/
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot_part, localslot, part_entry,
- newtup);
+ modified = slot_modify_data(remoteslot_part, localslot, part_entry,
+ newtup);
MemoryContextSwitchTo(oldctx);
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
@@ -3549,6 +3597,18 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
EvalPlanQualSetSlot(&epqstate, remoteslot_part);
TargetPrivilegesCheck(partrelinfo->ri_RelationDesc,
ACL_UPDATE);
+
+ /*
+ * Normally we'd call ExecCheckIndexedAttrForChanges but
+ * here we have the record of changed columns in the
+ * replication state, so let's use that instead.
+ */
+ indexed = RelationGetIndexAttrBitmap(partrelinfo->ri_RelationDesc,
+ INDEX_ATTR_BITMAP_INDEXED);
+ bms_free(partrelinfo->ri_ChangedIndexedCols);
+ partrelinfo->ri_ChangedIndexedCols = bms_int_members(modified, indexed);
+ bms_free(indexed);
+
ExecSimpleRelationUpdate(partrelinfo, estate, &epqstate,
localslot, remoteslot_part);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 915d0bc9084..32825596be1 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -2482,6 +2482,7 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
bms_free(relation->rd_idattr);
bms_free(relation->rd_hotblockingattr);
bms_free(relation->rd_summarizedattr);
+ bms_free(relation->rd_indexedattr);
if (relation->rd_pubdesc)
pfree(relation->rd_pubdesc);
if (relation->rd_options)
@@ -5283,6 +5284,7 @@ RelationGetIndexPredicate(Relation relation)
* index (empty if FULL)
* INDEX_ATTR_BITMAP_HOT_BLOCKING Columns that block updates from being HOT
* INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
+ * INDEX_ATTR_BITMAP_INDEXED Columns referenced by indexes
*
* Attribute numbers are offset by FirstLowInvalidHeapAttributeNumber so that
* we can include system attributes (e.g., OID) in the bitmap representation.
@@ -5307,6 +5309,7 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
Bitmapset *idindexattrs; /* columns in the replica identity */
Bitmapset *hotblockingattrs; /* columns with HOT blocking indexes */
Bitmapset *summarizedattrs; /* columns with summarizing indexes */
+ Bitmapset *indexedattrs; /* columns referenced by indexes */
List *indexoidlist;
List *newindexoidlist;
Oid relpkindex;
@@ -5329,6 +5332,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
return bms_copy(relation->rd_hotblockingattr);
case INDEX_ATTR_BITMAP_SUMMARIZED:
return bms_copy(relation->rd_summarizedattr);
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return bms_copy(relation->rd_indexedattr);
default:
elog(ERROR, "unknown attrKind %u", attrKind);
}
@@ -5373,6 +5378,7 @@ restart:
idindexattrs = NULL;
hotblockingattrs = NULL;
summarizedattrs = NULL;
+ indexedattrs = NULL;
foreach(l, indexoidlist)
{
Oid indexOid = lfirst_oid(l);
@@ -5505,10 +5511,14 @@ restart:
bms_free(idindexattrs);
bms_free(hotblockingattrs);
bms_free(summarizedattrs);
+ bms_free(indexedattrs);
goto restart;
}
+ /* Combine all index attributes */
+ indexedattrs = bms_union(hotblockingattrs, summarizedattrs);
+
/* Don't leak the old values of these bitmaps, if any */
relation->rd_attrsvalid = false;
bms_free(relation->rd_keyattr);
@@ -5521,6 +5531,8 @@ restart:
relation->rd_hotblockingattr = NULL;
bms_free(relation->rd_summarizedattr);
relation->rd_summarizedattr = NULL;
+ bms_free(relation->rd_indexedattr);
+ relation->rd_indexedattr = NULL;
/*
* Now save copies of the bitmaps in the relcache entry. We intentionally
@@ -5535,6 +5547,7 @@ restart:
relation->rd_idattr = bms_copy(idindexattrs);
relation->rd_hotblockingattr = bms_copy(hotblockingattrs);
relation->rd_summarizedattr = bms_copy(summarizedattrs);
+ relation->rd_indexedattr = bms_copy(indexedattrs);
relation->rd_attrsvalid = true;
MemoryContextSwitchTo(oldcxt);
@@ -5551,6 +5564,8 @@ restart:
return hotblockingattrs;
case INDEX_ATTR_BITMAP_SUMMARIZED:
return summarizedattrs;
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return indexedattrs;
default:
elog(ERROR, "unknown attrKind %u", attrKind);
return NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index e16bf025692..8a5931a3118 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -549,6 +549,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
+ Bitmapset *updated_cols,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1502,12 +1503,12 @@ static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ Bitmapset *updated_cols, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
- wait, tmfd,
- lockmode, update_indexes);
+ wait, tmfd, lockmode,
+ updated_cols, update_indexes);
}
/*
@@ -2010,6 +2011,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
+ Bitmapset *modified_indexe_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index fa2b657fb2f..993dc0e6ced 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -800,5 +800,10 @@ extern ResultRelInfo *ExecLookupResultRelByOid(ModifyTableState *node,
Oid resultoid,
bool missing_ok,
bool update_cache);
+extern Bitmapset *ExecCheckIndexedAttrsForChanges(ResultRelInfo *resultRelInfo,
+ TupleTableSlot *tts_old,
+ TupleTableSlot *tts_new);
+extern bool tts_attr_equal(Oid typid, Oid collation, bool typbyval, int16 typlen,
+ Datum value1, Datum value2);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 18ae8f0d4bb..8b08e0045ba 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -498,6 +498,7 @@ typedef struct ResultRelInfo
Bitmapset *ri_extraUpdatedCols;
/* true if the above has been computed */
bool ri_extraUpdatedCols_valid;
+ Bitmapset *ri_ChangedIndexedCols;
/* Projection to generate new tuple in an INSERT/UPDATE */
ProjectionInfo *ri_projectNew;
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 80286076a11..b23a7306e69 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -164,6 +164,7 @@ typedef struct RelationData
Bitmapset *rd_idattr; /* included in replica identity index */
Bitmapset *rd_hotblockingattr; /* cols blocking HOT update */
Bitmapset *rd_summarizedattr; /* cols indexed by summarizing indexes */
+ Bitmapset *rd_indexedattr; /* all cols referenced by indexes */
PublicationDesc *rd_pubdesc; /* publication descriptor, or NULL */
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 3561c6bef0b..d3fbb8b093a 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -71,6 +71,7 @@ typedef enum IndexAttrBitmapKind
INDEX_ATTR_BITMAP_IDENTITY_KEY,
INDEX_ATTR_BITMAP_HOT_BLOCKING,
INDEX_ATTR_BITMAP_SUMMARIZED,
+ INDEX_ATTR_BITMAP_INDEXED,
} IndexAttrBitmapKind;
extern Bitmapset *RelationGetIndexAttrBitmap(Relation relation,
--
2.49.0
[application/octet-stream] v19-0003-Replace-index_unchanged_by_update-with-ri_Change.patch (8.3K, 4-v19-0003-Replace-index_unchanged_by_update-with-ri_Change.patch)
download | inline diff:
From 8687e77751d0ee16d4d9495828099b918426423b Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Fri, 31 Oct 2025 14:55:25 -0400
Subject: [PATCH v19 3/4] Replace index_unchanged_by_update with
ri_ChangedIndexedCols
In execIndexing on updates we'd like to pass a hint to the indexing code
when the indexed attributes are unchanged. This commit replaces the now
redundant code in index_unchanged_by_update with the same information
found earlier in the update path.
---
src/backend/catalog/toasting.c | 2 -
src/backend/executor/execIndexing.c | 156 +---------------------------
src/backend/nodes/makefuncs.c | 2 -
src/include/nodes/execnodes.h | 4 -
4 files changed, 1 insertion(+), 163 deletions(-)
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 874a8fc89ad..5d819bda54a 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -300,8 +300,6 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
indexInfo->ii_Unique = true;
indexInfo->ii_NullsNotDistinct = false;
indexInfo->ii_ReadyForInserts = true;
- indexInfo->ii_CheckedUnchanged = false;
- indexInfo->ii_IndexUnchanged = false;
indexInfo->ii_Concurrent = false;
indexInfo->ii_BrokenHotChain = false;
indexInfo->ii_ParallelWorkers = 0;
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 401606f840a..fb1bc3a480d 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -138,11 +138,6 @@ static bool check_exclusion_or_unique_constraint(Relation heap, Relation index,
static bool index_recheck_constraint(Relation index, const Oid *constr_procs,
const Datum *existing_values, const bool *existing_isnull,
const Datum *new_values);
-static bool index_unchanged_by_update(ResultRelInfo *resultRelInfo,
- EState *estate, IndexInfo *indexInfo,
- Relation indexRelation);
-static bool index_expression_changed_walker(Node *node,
- Bitmapset *allUpdatedCols);
static void ExecWithoutOverlapsNotEmpty(Relation rel, NameData attname, Datum attval,
char typtype, Oid atttypid);
@@ -440,10 +435,7 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* index. If we're being called as part of an UPDATE statement,
* consider if the 'indexUnchanged' = true hint should be passed.
*/
- indexUnchanged = update && index_unchanged_by_update(resultRelInfo,
- estate,
- indexInfo,
- indexRelation);
+ indexUnchanged = update && bms_is_empty(resultRelInfo->ri_ChangedIndexedCols);
satisfiesConstraint =
index_insert(indexRelation, /* index relation */
@@ -993,152 +985,6 @@ index_recheck_constraint(Relation index, const Oid *constr_procs,
return true;
}
-/*
- * Check if ExecInsertIndexTuples() should pass indexUnchanged hint.
- *
- * When the executor performs an UPDATE that requires a new round of index
- * tuples, determine if we should pass 'indexUnchanged' = true hint for one
- * single index.
- */
-static bool
-index_unchanged_by_update(ResultRelInfo *resultRelInfo, EState *estate,
- IndexInfo *indexInfo, Relation indexRelation)
-{
- Bitmapset *updatedCols;
- Bitmapset *extraUpdatedCols;
- Bitmapset *allUpdatedCols;
- bool hasexpression = false;
- List *idxExprs;
-
- /*
- * Check cache first
- */
- if (indexInfo->ii_CheckedUnchanged)
- return indexInfo->ii_IndexUnchanged;
- indexInfo->ii_CheckedUnchanged = true;
-
- /*
- * Check for indexed attribute overlap with updated columns.
- *
- * Only do this for key columns. A change to a non-key column within an
- * INCLUDE index should not be counted here. Non-key column values are
- * opaque payload state to the index AM, a little like an extra table TID.
- *
- * Note that row-level BEFORE triggers won't affect our behavior, since
- * they don't affect the updatedCols bitmaps generally. It doesn't seem
- * worth the trouble of checking which attributes were changed directly.
- */
- updatedCols = ExecGetUpdatedCols(resultRelInfo, estate);
- extraUpdatedCols = ExecGetExtraUpdatedCols(resultRelInfo, estate);
- for (int attr = 0; attr < indexInfo->ii_NumIndexKeyAttrs; attr++)
- {
- int keycol = indexInfo->ii_IndexAttrNumbers[attr];
-
- if (keycol <= 0)
- {
- /*
- * Skip expressions for now, but remember to deal with them later
- * on
- */
- hasexpression = true;
- continue;
- }
-
- if (bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
- updatedCols) ||
- bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
- extraUpdatedCols))
- {
- /* Changed key column -- don't hint for this index */
- indexInfo->ii_IndexUnchanged = false;
- return false;
- }
- }
-
- /*
- * When we get this far and index has no expressions, return true so that
- * index_insert() call will go on to pass 'indexUnchanged' = true hint.
- *
- * The _absence_ of an indexed key attribute that overlaps with updated
- * attributes (in addition to the total absence of indexed expressions)
- * shows that the index as a whole is logically unchanged by UPDATE.
- */
- if (!hasexpression)
- {
- indexInfo->ii_IndexUnchanged = true;
- return true;
- }
-
- /*
- * Need to pass only one bms to expression_tree_walker helper function.
- * Avoid allocating memory in common case where there are no extra cols.
- */
- if (!extraUpdatedCols)
- allUpdatedCols = updatedCols;
- else
- allUpdatedCols = bms_union(updatedCols, extraUpdatedCols);
-
- /*
- * We have to work slightly harder in the event of indexed expressions,
- * but the principle is the same as before: try to find columns (Vars,
- * actually) that overlap with known-updated columns.
- *
- * If we find any matching Vars, don't pass hint for index. Otherwise
- * pass hint.
- */
- idxExprs = RelationGetIndexExpressions(indexRelation);
- hasexpression = index_expression_changed_walker((Node *) idxExprs,
- allUpdatedCols);
- list_free(idxExprs);
- if (extraUpdatedCols)
- bms_free(allUpdatedCols);
-
- if (hasexpression)
- {
- indexInfo->ii_IndexUnchanged = false;
- return false;
- }
-
- /*
- * Deliberately don't consider index predicates. We should even give the
- * hint when result rel's "updated tuple" has no corresponding index
- * tuple, which is possible with a partial index (provided the usual
- * conditions are met).
- */
- indexInfo->ii_IndexUnchanged = true;
- return true;
-}
-
-/*
- * Indexed expression helper for index_unchanged_by_update().
- *
- * Returns true when Var that appears within allUpdatedCols located.
- */
-static bool
-index_expression_changed_walker(Node *node, Bitmapset *allUpdatedCols)
-{
- if (node == NULL)
- return false;
-
- if (IsA(node, Var))
- {
- Var *var = (Var *) node;
-
- if (bms_is_member(var->varattno - FirstLowInvalidHeapAttributeNumber,
- allUpdatedCols))
- {
- /* Var was updated -- indicates that we should not hint */
- return true;
- }
-
- /* Still haven't found a reason to not pass the hint */
- return false;
- }
-
- return expression_tree_walker(node, index_expression_changed_walker,
- allUpdatedCols);
-}
-
/*
* ExecWithoutOverlapsNotEmpty - raise an error if the tuple has an empty
* range or multirange in the given attribute.
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index e2d9e9be41a..d69dc090aa4 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -845,8 +845,6 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
n->ii_Unique = unique;
n->ii_NullsNotDistinct = nulls_not_distinct;
n->ii_ReadyForInserts = isready;
- n->ii_CheckedUnchanged = false;
- n->ii_IndexUnchanged = false;
n->ii_Concurrent = concurrent;
n->ii_Summarizing = summarizing;
n->ii_WithoutOverlaps = withoutoverlaps;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 8b08e0045ba..898368fb8cb 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -202,10 +202,6 @@ typedef struct IndexInfo
bool ii_NullsNotDistinct;
/* is it valid for inserts? */
bool ii_ReadyForInserts;
- /* IndexUnchanged status determined yet? */
- bool ii_CheckedUnchanged;
- /* aminsert hint, cached for retail inserts */
- bool ii_IndexUnchanged;
/* are we doing a concurrent index build? */
bool ii_Concurrent;
/* did we detect any broken HOT chains? */
--
2.49.0
[application/octet-stream] v19-0004-Enable-HOT-updates-for-expression-and-partial-in.patch (95.5K, 5-v19-0004-Enable-HOT-updates-for-expression-and-partial-in.patch)
download | inline diff:
From 05c4e601b85be2fd79b642de2e1c194b5ad7ea80 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Sun, 26 Oct 2025 10:49:25 -0400
Subject: [PATCH v19 4/4] Enable HOT updates for expression and partial indexes
Currently, PostgreSQL conservatively prevents HOT (Heap-Only Tuple)
updates whenever any indexed column changes, even if the indexed
portion of that column remains identical. This is overly restrictive
for expression indexes (where f(column) might not change even when
column changes) and partial indexes (where both old and new tuples
might fall outside the predicate).
This patch introduces several improvements to enable HOT updates in
these cases:
Add amcomparedatums callback to IndexAmRoutine. This allows index
access methods like GIN to provide custom logic for comparing datums
by extracting and comparing index keys rather than comparing the raw
datums. GIN indexes now implement gincomparedatums() which extracts
keys from both datums and compares the resulting key sets.
Add ExecWhichIndexesRequireUpdates() to refine the set of modified
attributes and determine precisely which indexes need updating. For
partial indexes, this checks whether both old and new tuples satisfy
or fail the predicate. For expression indexes, this uses type-specific
equality operators to compare computed values. For extraction-based
indexes (GIN/RUM), this delegates to amcomparedatums.
Modify heap update paths to use the refined modified indexed attrs
bitmapset returned by ExecWhichIndexesRequireUpdates(). This allows
HOT updates when indexes don't actually require updating, while still
preventing HOT updates when they do.
Importantly, table access methods can still signal using TU_Update if
all, none, or only summarizing indexes should be updated. While the
executor layer now owns determining what has changed due to an update
and is interested in only updating the minimum number of indexes
possible, the table AM can override that while performing
table_tuple_update(), which is what heap does.
This optimization significantly improves update performance for tables
with expression indexes, partial indexes, and GIN/GiST indexes on
complex data types like JSONB and tsvector, while maintaining correct
index semantics. Minimal additional overhead due to type-specific
equality checking should be washed out by the benefits of updating
indexes fewer times.
---
src/backend/access/gin/ginutil.c | 94 ++-
src/backend/access/heap/heapam.c | 10 +-
src/backend/access/heap/heapam_handler.c | 6 +-
src/backend/access/nbtree/nbtree.c | 38 ++
src/backend/access/table/tableam.c | 4 +-
src/backend/bootstrap/bootstrap.c | 8 +
src/backend/catalog/index.c | 57 ++
src/backend/catalog/indexing.c | 16 +-
src/backend/catalog/toasting.c | 4 +
src/backend/executor/execIndexing.c | 326 ++++++++-
src/backend/executor/nodeModifyTable.c | 61 +-
src/backend/nodes/makefuncs.c | 4 +
src/include/access/amapi.h | 28 +
src/include/access/gin.h | 3 +
src/include/access/heapam.h | 6 +-
src/include/access/nbtree.h | 4 +
src/include/access/tableam.h | 8 +-
src/include/catalog/index.h | 1 +
src/include/executor/executor.h | 5 +
src/include/nodes/execnodes.h | 19 +
.../expected/hot_expression_indexes.out | 644 ++++++++++++++++++
src/test/regress/parallel_schedule | 6 +
.../regress/sql/hot_expression_indexes.sql | 491 +++++++++++++
src/tools/pgindent/typedefs.list | 1 +
24 files changed, 1794 insertions(+), 50 deletions(-)
create mode 100644 src/test/regress/expected/hot_expression_indexes.out
create mode 100644 src/test/regress/sql/hot_expression_indexes.sql
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index 78f7b7a2495..85e25ed73e8 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -26,6 +26,7 @@
#include "storage/indexfsm.h"
#include "utils/builtins.h"
#include "utils/index_selfuncs.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/typcache.h"
@@ -78,6 +79,7 @@ ginhandler(PG_FUNCTION_ARGS)
amroutine->amproperty = NULL;
amroutine->ambuildphasename = ginbuildphasename;
amroutine->amvalidate = ginvalidate;
+ amroutine->amcomparedatums = gincomparedatums;
amroutine->amadjustmembers = ginadjustmembers;
amroutine->ambeginscan = ginbeginscan;
amroutine->amrescan = ginrescan;
@@ -477,13 +479,6 @@ cmpEntries(const void *a, const void *b, void *arg)
return res;
}
-
-/*
- * Extract the index key values from an indexable item
- *
- * The resulting key values are sorted, and any duplicates are removed.
- * This avoids generating redundant index entries.
- */
Datum *
ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
@@ -729,3 +724,88 @@ ginbuildphasename(int64 phasenum)
return NULL;
}
}
+
+/*
+ * gincomparedatums - Compare two datums to determine if they produce identical keys
+ *
+ * This function extracts keys from both old_datum and new_datum using the
+ * opclass's extractValue function, then compares the extracted key arrays.
+ * Returns true if the key sets are identical (same keys, same counts).
+ *
+ * This enables HOT updates for GIN indexes when the indexed portions of a
+ * value haven't changed, even if the value itself has changed.
+ *
+ * Example: JSONB column with GIN index. If an update changes a non-indexed
+ * key in the JSONB document, the extracted keys are identical and we can
+ * do a HOT update.
+ */
+bool
+gincomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull)
+{
+ GinState ginstate;
+ Datum *old_keys;
+ Datum *new_keys;
+ GinNullCategory *old_categories;
+ GinNullCategory *new_categories;
+ int32 old_nkeys;
+ int32 new_nkeys;
+ MemoryContext tmpcontext;
+ MemoryContext oldcontext;
+ bool result = true;
+
+ /* Handle NULL cases */
+ if (old_isnull != new_isnull)
+ return false;
+ if (old_isnull)
+ return true;
+
+ /* Create temporary context for extraction work */
+ tmpcontext = AllocSetContextCreate(CurrentMemoryContext,
+ "GIN datum comparison",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(tmpcontext);
+
+ initGinState(&ginstate, index);
+
+ /*
+ * Extract keys from both datums using existing GIN infrastructure.
+ */
+ old_keys = ginExtractEntries(&ginstate, attnum, old_datum, old_isnull,
+ &old_nkeys, &old_categories);
+ new_keys = ginExtractEntries(&ginstate, attnum, new_datum, new_isnull,
+ &new_nkeys, &new_categories);
+
+ /* Different number of keys → definitely different */
+ if (old_nkeys != new_nkeys)
+ {
+ result = false;
+ goto cleanup;
+ }
+
+ /*
+ * Compare the sorted key arrays element-by-element. Since both arrays are
+ * already sorted by ginExtractEntries, we can do a simple O(n)
+ * comparison.
+ */
+ for (int i = 0; i < old_nkeys; i++)
+ {
+ int cmp = ginCompareEntries(&ginstate, attnum,
+ old_keys[i], old_categories[i],
+ new_keys[i], new_categories[i]);
+
+ if (cmp != 0)
+ {
+ result = false;
+ break;
+ }
+ }
+
+cleanup:
+ /* Clean up */
+ MemoryContextSwitchTo(oldcontext);
+ MemoryContextDelete(tmpcontext);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 1cdb72b3a7a..5b0ff13b13d 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -3268,7 +3268,7 @@ heap_update(Relation relation, HeapTupleData *oldtup, HeapTuple newtup,
TM_FailureData *tmfd, LockTupleMode *lockmode,
Buffer buffer, Page page, BlockNumber block, ItemId lp,
Bitmapset *hot_attrs, Bitmapset *sum_attrs, Bitmapset *pk_attrs,
- Bitmapset *rid_attrs, Bitmapset *mix_attrs, Buffer *vmbuffer,
+ Bitmapset *rid_attrs, const Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -4337,8 +4337,9 @@ HeapDetermineColumnsInfo(Relation relation,
* This routine may be used to update a tuple when concurrent updates of the
* target tuple are not expected (for example, because we have a lock on the
* relation associated with the tuple). Any failure is reported via ereport().
+ * Returns the set of modified indexed attributes.
*/
-void
+Bitmapset *
simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tuple,
TU_UpdateIndexes *update_indexes)
{
@@ -4467,7 +4468,7 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
elog(ERROR, "tuple concurrently deleted");
- return;
+ return NULL;
}
/*
@@ -4500,7 +4501,6 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
bms_free(sum_attrs);
bms_free(pk_attrs);
bms_free(rid_attrs);
- bms_free(mix_attrs);
bms_free(idx_attrs);
switch (result)
@@ -4526,6 +4526,8 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
elog(ERROR, "unrecognized heap_update status: %u", result);
break;
}
+
+ return mix_attrs;
}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index ef08e1d3e10..7527809ec08 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -319,7 +319,7 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
Snapshot crosscheck, bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
- Bitmapset *mix_attrs,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes)
{
bool rep_id_key_required = false;
@@ -407,10 +407,6 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
Assert(ItemIdIsNormal(lp));
- /*
- * Partially construct the oldtup for HeapDetermineColumnsInfo to work and
- * then pass that on to heap_update.
- */
oldtup.t_tableOid = RelationGetRelid(relation);
oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
oldtup.t_len = ItemIdGetLength(lp);
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index fdff960c130..73cc3208757 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -155,6 +155,7 @@ bthandler(PG_FUNCTION_ARGS)
amroutine->amproperty = btproperty;
amroutine->ambuildphasename = btbuildphasename;
amroutine->amvalidate = btvalidate;
+ amroutine->amcomparedatums = btcomparedatums;
amroutine->amadjustmembers = btadjustmembers;
amroutine->ambeginscan = btbeginscan;
amroutine->amrescan = btrescan;
@@ -1795,3 +1796,40 @@ bttranslatecmptype(CompareType cmptype, Oid opfamily)
return InvalidStrategy;
}
}
+
+/*
+ * btcomparedatums - Compare two datums for equality
+ *
+ * This function is necessary because nbtree requires that keys that are not
+ * binary identical not be "equal". Other indexes might allow "A" and "a" to
+ * be "equal" when collation is case insensative, but not nbtree. Why? Well,
+ * nbtree deduplicates TIDs on page split and the way it accomplish that is by
+ * doing a binary comparison of the keys.
+ */
+
+bool
+btcomparedatums(Relation index, int attrnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull)
+{
+ TupleDesc desc = RelationGetDescr(index);
+ CompactAttribute *att;
+
+ /*
+ * If one value is NULL and other is not, then they are certainly not
+ * equal
+ */
+ if (old_isnull != new_isnull)
+ return false;
+
+ /*
+ * If both are NULL, they can be considered equal.
+ */
+ if (old_isnull)
+ return true;
+
+ /* We do simple binary comparison of the two datums */
+ Assert(attrnum <= desc->natts);
+ att = TupleDescCompactAttr(desc, attrnum - 1);
+ return datumIsEqual(old_datum, new_datum, att->attbyval, att->attlen);
+}
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index dadcf03ed24..ef7736bfa76 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -336,7 +336,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
- Bitmapset *modified_indexed_cols,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -348,7 +348,7 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
snapshot, InvalidSnapshot,
true /* wait for commit */ ,
&tmfd, &lockmode,
- modified_indexed_cols,
+ mix_attrs,
update_indexes);
switch (result)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index fc8638c1b61..329c110d0bf 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -961,10 +961,18 @@ index_register(Oid heap,
newind->il_info->ii_Expressions =
copyObject(indexInfo->ii_Expressions);
newind->il_info->ii_ExpressionsState = NIL;
+ /* expression attrs will likely be null, but may as well copy it */
+ newind->il_info->ii_ExpressionsAttrs =
+ copyObject(indexInfo->ii_ExpressionsAttrs);
/* predicate will likely be null, but may as well copy it */
newind->il_info->ii_Predicate =
copyObject(indexInfo->ii_Predicate);
newind->il_info->ii_PredicateState = NULL;
+ /* predicate attrs will likely be null, but may as well copy it */
+ newind->il_info->ii_PredicateAttrs =
+ copyObject(indexInfo->ii_PredicateAttrs);
+ newind->il_info->ii_CheckedPredicate = false;
+ newind->il_info->ii_PredicateSatisfied = false;
/* no exclusion constraints at bootstrap time, so no need to copy */
Assert(indexInfo->ii_ExclusionOps == NULL);
Assert(indexInfo->ii_ExclusionProcs == NULL);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 5d9db167e59..29b8cc4badd 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -27,6 +27,7 @@
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/relscan.h"
+#include "access/sysattr.h"
#include "access/tableam.h"
#include "access/toast_compression.h"
#include "access/transam.h"
@@ -58,6 +59,7 @@
#include "commands/trigger.h"
#include "executor/executor.h"
#include "miscadmin.h"
+#include "nodes/execnodes.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/optimizer.h"
@@ -2414,6 +2416,61 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* ----------------------------------------------------------------
*/
+/* ----------------
+ * BuildUpdateIndexInfo
+ *
+ * For expression indexes updates may not change the indexed value allowing
+ * for a HOT update. Add information to the IndexInfo to allow for checking
+ * if the indexed value has changed.
+ *
+ * Do this processing here rather than in BuildIndexInfo() to not incur the
+ * overhead in the common non-expression cases.
+ * ----------------
+ */
+void
+BuildUpdateIndexInfo(ResultRelInfo *resultRelInfo)
+{
+ for (int j = 0; j < resultRelInfo->ri_NumIndices; j++)
+ {
+ int i;
+ int indnkeyatts;
+ Bitmapset *attrs = NULL;
+ IndexInfo *ii = resultRelInfo->ri_IndexRelationInfo[j];
+
+ /*
+ * Expressions are not allowed on non-key attributes, so we can skip
+ * them as they should show up in the index HOT-blocking attributes.
+ */
+ indnkeyatts = ii->ii_NumIndexKeyAttrs;
+
+ /* Collect key attributes used by the index */
+ for (i = 0; i < indnkeyatts; i++)
+ {
+ AttrNumber attnum = ii->ii_IndexAttrNumbers[i];
+
+ if (attnum != 0)
+ attrs = bms_add_member(attrs, attnum - FirstLowInvalidHeapAttributeNumber);
+ }
+
+ /* Collect attributes used in the expression */
+ if (ii->ii_Expressions)
+ pull_varattnos((Node *) ii->ii_Expressions,
+ resultRelInfo->ri_RangeTableIndex,
+ &ii->ii_ExpressionsAttrs);
+
+ /* Collect attributes used in the predicate */
+ if (ii->ii_Predicate)
+ pull_varattnos((Node *) ii->ii_Predicate,
+ resultRelInfo->ri_RangeTableIndex,
+ &ii->ii_PredicateAttrs);
+
+ ii->ii_IndexedAttrs = bms_union(attrs, ii->ii_ExpressionsAttrs);
+
+ /* All indexes should index *something*! */
+ Assert(!bms_is_empty(ii->ii_IndexedAttrs));
+ }
+}
+
/* ----------------
* BuildIndexInfo
* Construct an IndexInfo record for an open index
diff --git a/src/backend/catalog/indexing.c b/src/backend/catalog/indexing.c
index 004c5121000..a361c215490 100644
--- a/src/backend/catalog/indexing.c
+++ b/src/backend/catalog/indexing.c
@@ -102,7 +102,7 @@ CatalogIndexInsert(CatalogIndexState indstate, HeapTuple heapTuple,
* Get information from the state structure. Fall out if nothing to do.
*/
numIndexes = indstate->ri_NumIndices;
- if (numIndexes == 0)
+ if (numIndexes == 0 || updateIndexes == TU_None)
return;
relationDescs = indstate->ri_IndexRelationDescs;
indexInfoArray = indstate->ri_IndexRelationInfo;
@@ -314,15 +314,18 @@ CatalogTupleUpdate(Relation heapRel, const ItemPointerData *otid, HeapTuple tup)
{
CatalogIndexState indstate;
TU_UpdateIndexes updateIndexes = TU_All;
+ Bitmapset *updatedAttrs;
CatalogTupleCheckConstraints(heapRel, tup);
indstate = CatalogOpenIndexes(heapRel);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
-
+ updatedAttrs = simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = updatedAttrs;
CatalogIndexInsert(indstate, tup, updateIndexes);
+
CatalogCloseIndexes(indstate);
+ bms_free(updatedAttrs);
}
/*
@@ -338,12 +341,15 @@ CatalogTupleUpdateWithInfo(Relation heapRel, const ItemPointerData *otid, HeapTu
CatalogIndexState indstate)
{
TU_UpdateIndexes updateIndexes = TU_All;
+ Bitmapset *updatedAttrs;
CatalogTupleCheckConstraints(heapRel, tup);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
-
+ updatedAttrs = simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = updatedAttrs;
CatalogIndexInsert(indstate, tup, updateIndexes);
+ ((ResultRelInfo *) indstate)->ri_ChangedIndexedCols = NULL;
+ bms_free(updatedAttrs);
}
/*
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 5d819bda54a..c665aa744b3 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -292,8 +292,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
indexInfo->ii_IndexAttrNumbers[1] = 2;
indexInfo->ii_Expressions = NIL;
indexInfo->ii_ExpressionsState = NIL;
+ indexInfo->ii_ExpressionsAttrs = NULL;
indexInfo->ii_Predicate = NIL;
indexInfo->ii_PredicateState = NULL;
+ indexInfo->ii_PredicateAttrs = NULL;
+ indexInfo->ii_CheckedPredicate = false;
+ indexInfo->ii_PredicateSatisfied = false;
indexInfo->ii_ExclusionOps = NULL;
indexInfo->ii_ExclusionProcs = NULL;
indexInfo->ii_ExclusionStrats = NULL;
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index fb1bc3a480d..736543147e7 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -109,11 +109,15 @@
#include "access/genam.h"
#include "access/relscan.h"
#include "access/tableam.h"
+#include "access/sysattr.h"
#include "access/xact.h"
#include "catalog/index.h"
#include "executor/executor.h"
+#include "nodes/bitmapset.h"
+#include "nodes/execnodes.h"
#include "nodes/nodeFuncs.h"
#include "storage/lmgr.h"
+#include "utils/datum.h"
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
@@ -318,8 +322,8 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
Relation heapRelation;
IndexInfo **indexInfoArray;
ExprContext *econtext;
- Datum values[INDEX_MAX_KEYS];
- bool isnull[INDEX_MAX_KEYS];
+ Datum loc_values[INDEX_MAX_KEYS];
+ bool loc_isnull[INDEX_MAX_KEYS];
Assert(ItemPointerIsValid(tupleid));
@@ -343,13 +347,13 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
/* Arrange for econtext's scan tuple to be the tuple under test */
econtext->ecxt_scantuple = slot;
- /*
- * for each index, form and insert the index tuple
- */
+ /* Insert into each index that needs updating */
for (i = 0; i < numIndices; i++)
{
Relation indexRelation = relationDescs[i];
IndexInfo *indexInfo;
+ Datum *values;
+ bool *isnull;
bool applyNoDupErr;
IndexUniqueCheck checkUnique;
bool indexUnchanged;
@@ -366,7 +370,7 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
/*
* Skip processing of non-summarizing indexes if we only update
- * summarizing indexes
+ * summarizing indexes or if this index is unchanged.
*/
if (onlySummarizing && !indexInfo->ii_Summarizing)
continue;
@@ -387,8 +391,15 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
indexInfo->ii_PredicateState = predicate;
}
+ /* Check the index predicate if we haven't done so earlier on */
+ if (!indexInfo->ii_CheckedPredicate)
+ {
+ indexInfo->ii_PredicateSatisfied = ExecQual(predicate, econtext);
+ indexInfo->ii_CheckedPredicate = true;
+ }
+
/* Skip this index-update if the predicate isn't satisfied */
- if (!ExecQual(predicate, econtext))
+ if (!indexInfo->ii_PredicateSatisfied)
continue;
}
@@ -396,11 +407,10 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* FormIndexDatum fills in its values and isnull parameters with the
* appropriate values for the column(s) of the index.
*/
- FormIndexDatum(indexInfo,
- slot,
- estate,
- values,
- isnull);
+ FormIndexDatum(indexInfo, slot, estate, loc_values, loc_isnull);
+
+ values = loc_values;
+ isnull = loc_isnull;
/* Check whether to apply noDupErr to this index */
applyNoDupErr = noDupErr &&
@@ -435,7 +445,9 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* index. If we're being called as part of an UPDATE statement,
* consider if the 'indexUnchanged' = true hint should be passed.
*/
- indexUnchanged = update && bms_is_empty(resultRelInfo->ri_ChangedIndexedCols);
+ indexUnchanged = update &&
+ !bms_overlap(indexInfo->ii_IndexedAttrs,
+ resultRelInfo->ri_ChangedIndexedCols);
satisfiesConstraint =
index_insert(indexRelation, /* index relation */
@@ -604,7 +616,12 @@ ExecCheckIndexConstraints(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
checkedIndex = true;
/* Check for partial index */
- if (indexInfo->ii_Predicate != NIL)
+ if (indexInfo->ii_CheckedPredicate && !indexInfo->ii_PredicateSatisfied)
+ {
+ /* We've already checked and the predicate wasn't satisfied. */
+ continue;
+ }
+ else if (indexInfo->ii_Predicate != NIL)
{
ExprState *predicate;
@@ -1018,3 +1035,284 @@ ExecWithoutOverlapsNotEmpty(Relation rel, NameData attname, Datum attval, char t
errmsg("empty WITHOUT OVERLAPS value found in column \"%s\" in relation \"%s\"",
NameStr(attname), RelationGetRelationName(rel))));
}
+
+/*
+ * ExecWhichIndexesRequireUpdates
+ *
+ * Determine which indexes need updating given modified indexed attributes.
+ * This function is a companion to ExecCheckIndexedAttrsForChanges(). On the
+ * surface, they appear similar but they are doing two very different things.
+ *
+ * For a standard index on a set of attributes this is the intersection of
+ * the mix_attrs and the index attrs (key, expression, but not predicate).
+ *
+ * For expression indexes and indexes which implement the amcomparedatums()
+ * index AM API we'll need to form index datum and compare each attribute to
+ * see if any actually changed.
+ *
+ * For expression indexes the result of the expression might not change at all,
+ * this is common with JSONB columns which require expression indexes and where
+ * it is commonplace to index a field within a document and have updates that
+ * generally don't update that field.
+ *
+ * Partial indexes won't trigger index tuples when the old/new tuples are both
+ * outside of the predicate range.
+ *
+ * For nbtree the amcomparedatums() API is critical as it requires that key
+ * attributes are equal when they memcmp(), which might not be the case when
+ * using type-specific comparison or factoring in collation which might make
+ * an index case insensitive.
+ *
+ * All of this is to say that the goal is for the executor to know, ahead of
+ * calling into the table AM for the update and before calling into the index
+ * AM for inserting new index tuples, which attributes at a minimum will
+ * necessitate a new index tuple.
+ *
+ * The mix_attrs parameter contains attributes that:
+ * 1. Were refereced by the UPDATE statement and are known to have been modified
+ * 2. Are referenced by at least one index (expression, predicate, or otherwise)
+ *
+ * This function refines that set by determining which indexes actually
+ * need updates and only retaining the attributes that indicate that:
+ * - Partial index predicates: If both tuples fall outside, no update needed
+ * - Expression indexes: If expression results are identical, no update needed
+ * - Extraction indexes (GIN, RUM, etc): If extracted values are identical, no update needed
+ *
+ * Returns a refined Bitmapset of attributes that force index updates.
+ */
+Bitmapset *
+ExecWhichIndexesRequireUpdates(ResultRelInfo *relinfo,
+ Bitmapset *mix_attrs,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts)
+{
+ Bitmapset *result_attrs = NULL;
+ ExprContext *econtext = GetPerTupleExprContext(estate);
+ TupleTableSlot *save_scantuple;
+ int i;
+
+ if (relinfo->ri_NumIndices == 0 || bms_is_empty(mix_attrs))
+ return NULL;
+
+ /* Check each index */
+ for (i = 0; i < relinfo->ri_NumIndices; i++)
+ {
+ Relation indexRel = relinfo->ri_IndexRelationDescs[i];
+ IndexInfo *indexInfo = relinfo->ri_IndexRelationInfo[i];
+ IndexAmRoutine *amroutine = indexRel->rd_indam;
+ bool has_expressions = (indexInfo->ii_Expressions != NIL);
+ bool has_am_compare = (amroutine->amcomparedatums != NULL);
+ bool is_partial = (indexInfo->ii_Predicate != NIL);
+ Bitmapset *idx_attrs = indexInfo->ii_IndexedAttrs;
+ Bitmapset *pre_attrs = indexInfo->ii_PredicateAttrs;
+
+ /* Check partial index predicate iff attrs overlap with modified */
+ if (is_partial &&
+ !bms_is_empty((pre_attrs = bms_intersect(pre_attrs, mix_attrs))))
+ {
+ ExprState *pstate;
+ bool old_qualifies,
+ new_qualifies;
+
+ if (!indexInfo->ii_CheckedPredicate)
+ pstate = ExecPrepareQual(indexInfo->ii_Predicate, estate);
+ else
+ pstate = indexInfo->ii_PredicateState;
+
+ save_scantuple = econtext->ecxt_scantuple;
+
+ econtext->ecxt_scantuple = old_tts;
+ old_qualifies = ExecQual(pstate, econtext);
+
+ econtext->ecxt_scantuple = new_tts;
+ new_qualifies = ExecQual(pstate, econtext);
+
+ econtext->ecxt_scantuple = save_scantuple;
+
+ indexInfo->ii_CheckedPredicate = true;
+ indexInfo->ii_PredicateState = pstate;
+ indexInfo->ii_PredicateSatisfied = new_qualifies;
+
+ /* Both outside predicate, index doesn't need update */
+ if (!old_qualifies && !new_qualifies)
+ {
+ bms_free(pre_attrs);
+ continue;
+ }
+
+ /* Predicate transition, must update index, add predicate attrs */
+ if (old_qualifies != new_qualifies)
+ {
+ /*
+ * We can say for sure that the index needs update, so we can
+ * add in the attributes from the predicate that are also in
+ * the mix_attrs set, but we don't yet know if there are other
+ * attributes this index references that are modified (in the
+ * mix_attrs) and force index updates. The only way to know
+ * if to test them one by one.
+ */
+ result_attrs = bms_add_members(result_attrs, pre_attrs);
+ bms_free(pre_attrs);
+ }
+ }
+
+ /*
+ * If we've got a result set equal to our modified set then we've
+ * identified that all the attributes on the index need to trigger new
+ * index tuples. No need to keep checking, we're done not just with
+ * this index but in general, break out of the loop here.
+ */
+ if (bms_equal(result_attrs, mix_attrs))
+ break;
+
+ /*
+ * NOTE: While it feels like we could avoid checking any further when
+ * the indexed attributes (key and expression) do not overlap with the
+ * modified indexed attributes (mix_attrs) we can't. Why? Well, it
+ * turns out that it is allowable for index AMs to have a different
+ * notion of equality. For instance, nbtree requires that datum
+ * equality be based on binary comparison, not anything type-specific.
+ * So, it is possible that the set of mix_attrs from
+ * ExecCheckIndexedAttrsForChanges() found that the new value of an
+ * attribute was equal to the old value despite it having a different
+ * binary representation.
+ *
+ * XXX: maybe that's something we should enforce and change nbtree?
+ */
+
+ /*
+ * Expression index or extraction-based index require us to form index
+ * datums/tuples and compare. We've done all we can to avoid this
+ * overhead now it's time to bite the bullet and get it done.
+ */
+ if (has_expressions || has_am_compare)
+ {
+ Datum old_values[INDEX_MAX_KEYS];
+ bool old_isnull[INDEX_MAX_KEYS];
+ Datum new_values[INDEX_MAX_KEYS];
+ bool new_isnull[INDEX_MAX_KEYS];
+ TupleDesc indexdesc = RelationGetDescr(indexRel);
+
+ save_scantuple = econtext->ecxt_scantuple;
+
+ /* Evaluate expressions (if any) to get base datums */
+ econtext->ecxt_scantuple = old_tts;
+ FormIndexDatum(indexInfo, old_tts, estate,
+ old_values, old_isnull);
+
+ econtext->ecxt_scantuple = new_tts;
+ FormIndexDatum(indexInfo, new_tts, estate,
+ new_values, new_isnull);
+
+ econtext->ecxt_scantuple = save_scantuple;
+
+ /* Compare the index key datums */
+ for (int j = 0; j < indexInfo->ii_NumIndexKeyAttrs; j++)
+ {
+ AttrNumber attrnum = indexInfo->ii_IndexAttrNumbers[j];
+ bool values_equal;
+
+ /*
+ * Skip attributes that we've already identified as triggering
+ * an index update.
+ */
+ if (attrnum > 0 &&
+ bms_is_member(attrnum - FirstLowInvalidHeapAttributeNumber, result_attrs))
+ continue;
+
+ /* A change to/from NULL, record this attribute */
+ if (old_isnull[j] != new_isnull[j])
+ {
+ if (attrnum == 0)
+ result_attrs = bms_add_members(result_attrs, indexInfo->ii_ExpressionsAttrs);
+ else
+ result_attrs = bms_add_member(result_attrs, attrnum - FirstLowInvalidHeapAttributeNumber);
+ continue;
+ }
+ /* Both NULL, no change */
+ if (old_isnull[j])
+ continue;
+
+ /*
+ * Use index AM's comparison function if present when
+ * comparing the datum formed when creating an index key. This
+ * is different from the comparison of datum in the tuple
+ * destined for a storage AM, which is why we only need to use
+ * this here.
+ */
+ if (has_am_compare)
+ {
+ /*
+ * For nbtree to properly deduplicate TIDs on page split
+ * it must treat equality as binary comparison. So it is
+ * vital that we call it's comparedatums() function.
+ *
+ * In the case of GIN/RUM indexes they too behave
+ * differently and can even extract one or more portions
+ * of the datum when forming index tuples. We'd like to
+ * know if this update needs to trigger one or more index
+ * tuples, so we let the index AM perform their extraction
+ * and compare the results.
+ *
+ * There may be other similar index AM implementation with
+ * extraction where indexes are built using only part(s)
+ * of the Datum and might even need to invoke
+ * type-specific equality operators.
+ *
+ * NOTE: For AM comparison, pass the 1-based index
+ * attribute number. The AM's compare function expects the
+ * same numbering as used internally by the AM.
+ */
+ values_equal = amroutine->amcomparedatums(indexRel, j + 1,
+ old_values[j], old_isnull[j],
+ new_values[j], new_isnull[j]);
+ }
+ else
+ {
+ /*
+ * Expression index without custom AM comparison. Compare
+ * the expression results using type-specific equality via
+ * the TypeCache.
+ */
+ Form_pg_attribute attr = TupleDescAttr(indexdesc, j);
+
+ values_equal = tts_attr_equal(attr->atttypid,
+ attr->attcollation,
+ attr->attbyval,
+ attr->attlen,
+ old_values[j],
+ new_values[j]);
+ }
+
+ if (!values_equal)
+ {
+ if (attrnum == 0)
+ result_attrs = bms_add_members(result_attrs, indexInfo->ii_ExpressionsAttrs);
+ else
+ result_attrs = bms_add_member(result_attrs, attrnum - FirstLowInvalidHeapAttributeNumber);
+ }
+ }
+ }
+ else
+ {
+ /*
+ * Here we know that we're reviewing an index that doesn't have a
+ * partial predicate, isn't an expression index, and doesn't have
+ * a amcomparedatums() implementation. Attributes that overlap
+ * with those known to have changed are the ones we need to
+ * record.
+ */
+
+ /*
+ * NOTE: we intentionally copy via intersect and then free rather
+ * than modify on the set we're pointing at in IndexInfo.
+ */
+ idx_attrs = bms_intersect(idx_attrs, mix_attrs);
+ result_attrs = bms_add_members(result_attrs, idx_attrs);
+ bms_free(idx_attrs);
+ }
+ }
+
+ return result_attrs;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 4d1cf50e369..191748cdce8 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -54,10 +54,12 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/sysattr.h"
#include "access/tableam.h"
#include "access/tupconvert.h"
#include "access/tupdesc.h"
#include "access/xact.h"
+#include "catalog/index.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -75,6 +77,7 @@
#include "utils/float.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
@@ -2395,6 +2398,12 @@ ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
bool partition_constraint_failed;
TM_Result result;
+ /* the set of modified indexed attributes */
+ Bitmapset *mix_attrs = NULL;
+
+ /* the set of index attributes that trigger new index datum */
+ Bitmapset *iru_attrs = NULL;
+
updateCxt->crossPartUpdate = false;
/*
@@ -2517,13 +2526,48 @@ lreplace:
bms_free(resultRelInfo->ri_ChangedIndexedCols);
resultRelInfo->ri_ChangedIndexedCols = NULL;
- resultRelInfo->ri_ChangedIndexedCols =
- ExecCheckIndexedAttrsForChanges(resultRelInfo, oldSlot, slot);
+ /*
+ * At this point we know the set of attributes specified in the UPDATE
+ * statement and those referenced by triggers, so we have a complete view
+ * of the UPDATE attributes on the table. We could get this set via the
+ * ExecGetUpdatedCols() function, but we'll need to review all indexed
+ * attributes because extensions could just directly heap_modify_tuple()
+ * an attribute not known to ExecGetUpdatedCols().
+ *
+ * We want to know which, if any, attributes that are referenced by an
+ * index have changed value. This set of attributes will dictate the
+ * minimum number of indexes we need to update.
+ */
+ mix_attrs = ExecCheckIndexedAttrsForChanges(resultRelInfo, oldSlot, slot);
/*
- * replace the heap tuple
+ * During updates we'll need a bit more information in IndexInfo but we've
+ * delayed adding it until here. We check to ensure that there are
+ * indexes, that something has changed that is indexed, and that the first
+ * index doesn't yet have ii_IndexedAttrs set as a way to ensure we only
+ * build this when needed and only once. We don't build this in
+ * ExecOpenIndicies() as it is unnecessary overhead when not performing an
+ * update.
+ */
+ if (resultRelInfo->ri_NumIndices > 0 &&
+ bms_is_empty(resultRelInfo->ri_IndexRelationInfo[0]->ii_IndexedAttrs))
+ BuildUpdateIndexInfo(resultRelInfo);
+
+ /*
+ * The next step is to identify which indexes, at a minimum, require new
+ * index tuples. You might think that you could simply intersect the
+ * index key attributes with the modified attributes and be done, but then
+ * you'd have missed a few cases (expressions, partial, indexes with
+ * operators that index only a portion of a datum or many different
+ * portions of it).
+ */
+ iru_attrs = ExecWhichIndexesRequireUpdates(resultRelInfo, mix_attrs, estate,
+ oldSlot, slot);
+
+ /*
+ * Call into the table AM to update the heap tuple.
*
- * Note: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
+ * NOTE: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
* the row to be updated is visible to that snapshot, and throw a
* can't-serialize error if not. This is a special-case behavior needed
* for referential integrity updates in transaction-snapshot mode
@@ -2535,9 +2579,14 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
- resultRelInfo->ri_ChangedIndexedCols,
+ iru_attrs,
&updateCxt->updateIndexes);
+ Assert(bms_is_empty(resultRelInfo->ri_ChangedIndexedCols));
+ resultRelInfo->ri_ChangedIndexedCols = iru_attrs;
+
+ bms_free(mix_attrs);
+
return result;
}
@@ -2555,7 +2604,7 @@ ExecUpdateEpilogue(ModifyTableContext *context, UpdateContext *updateCxt,
ModifyTableState *mtstate = context->mtstate;
List *recheckIndexes = NIL;
- /* insert index entries for tuple if necessary */
+ /* Insert index entries for tuple if necessary */
if (resultRelInfo->ri_NumIndices > 0 && (updateCxt->updateIndexes != TU_None))
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
slot, context->estate,
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index d69dc090aa4..e9a53b95caf 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -855,10 +855,14 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
/* expressions */
n->ii_Expressions = expressions;
n->ii_ExpressionsState = NIL;
+ n->ii_ExpressionsAttrs = NULL;
/* predicates */
n->ii_Predicate = predicates;
n->ii_PredicateState = NULL;
+ n->ii_PredicateAttrs = NULL;
+ n->ii_CheckedPredicate = false;
+ n->ii_PredicateSatisfied = false;
/* exclusion constraints */
n->ii_ExclusionOps = NULL;
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index 63dd41c1f21..9bdf73eda59 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -211,6 +211,33 @@ typedef void (*ammarkpos_function) (IndexScanDesc scan);
/* restore marked scan position */
typedef void (*amrestrpos_function) (IndexScanDesc scan);
+/*
+ * amcomparedatums - Compare datums to determine if index update is needed
+ *
+ * This function compares old_datum and new_datum to determine if they would
+ * produce different index entries. For extraction-based indexes (GIN, RUM),
+ * this should:
+ * 1. Extract keys from old_datum using the opclass's extractValue function
+ * 2. Extract keys from new_datum using the opclass's extractValue function
+ * 3. Compare the two sets of keys using appropriate equality operators
+ * 4. Return true if the sets are equal (no index update needed)
+ *
+ * The comparison should account for:
+ * - Different numbers of extracted keys
+ * - NULL values
+ * - Type-specific equality (not just binary equality)
+ * - Opclass parameters (e.g., path in bson_rum_single_path_ops)
+ *
+ * For the DocumentDB example with path='a', this would extract values at
+ * path 'a' from both old and new BSON documents and compare them using
+ * BSON's equality operator.
+ */
+/* identify if updated datums would produce one or more index entries */
+typedef bool (*amcomparedatums_function) (Relation indexRelation,
+ int attno,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* Callback function signatures - for parallel index scans.
*/
@@ -313,6 +340,7 @@ typedef struct IndexAmRoutine
amendscan_function amendscan;
ammarkpos_function ammarkpos; /* can be NULL */
amrestrpos_function amrestrpos; /* can be NULL */
+ amcomparedatums_function amcomparedatums; /* can be NULL */
/* interface functions to support parallel index scans */
amestimateparallelscan_function amestimateparallelscan; /* can be NULL */
diff --git a/src/include/access/gin.h b/src/include/access/gin.h
index 13ea91922ef..2f265f4816c 100644
--- a/src/include/access/gin.h
+++ b/src/include/access/gin.h
@@ -100,6 +100,9 @@ extern PGDLLIMPORT int gin_pending_list_limit;
extern void ginGetStats(Relation index, GinStatsData *stats);
extern void ginUpdateStats(Relation index, const GinStatsData *stats,
bool is_build);
+extern bool gincomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
extern void _gin_parallel_build_main(dsm_segment *seg, shm_toc *toc);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 41d541aa6b2..59db389a546 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -326,7 +326,7 @@ extern TM_Result heap_update(Relation relation, HeapTupleData *oldtup,
TM_FailureData *tmfd, LockTupleMode *lockmode, Buffer buffer,
Page page, BlockNumber block, ItemId lp, Bitmapset *hot_attrs,
Bitmapset *sum_attrs, Bitmapset *pk_attrs, Bitmapset *rid_attrs,
- Bitmapset *mix_attrs, Buffer *vmbuffer,
+ const Bitmapset *mix_attrs, Buffer *vmbuffer,
bool rep_id_key_required, TU_UpdateIndexes *update_indexes);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
@@ -361,8 +361,8 @@ extern bool heap_tuple_needs_eventual_freeze(HeapTupleHeader tuple);
extern void simple_heap_insert(Relation relation, HeapTuple tup);
extern void simple_heap_delete(Relation relation, const ItemPointerData *tid);
-extern void simple_heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple tup, TU_UpdateIndexes *update_indexes);
+extern Bitmapset *simple_heap_update(Relation relation, const ItemPointerData *otid,
+ HeapTuple tup, TU_UpdateIndexes *update_indexes);
extern TransactionId heap_index_delete_tuples(Relation rel,
TM_IndexDeleteOp *delstate);
diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h
index 16be5c7a9c1..42bd329eaad 100644
--- a/src/include/access/nbtree.h
+++ b/src/include/access/nbtree.h
@@ -1210,6 +1210,10 @@ extern int btgettreeheight(Relation rel);
extern CompareType bttranslatestrategy(StrategyNumber strategy, Oid opfamily);
extern StrategyNumber bttranslatecmptype(CompareType cmptype, Oid opfamily);
+extern bool btcomparedatums(Relation index, int attnum,
+ Datum old_datum, bool old_isnull,
+ Datum new_datum, bool new_isnull);
+
/*
* prototypes for internal functions in nbtree.c
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 8a5931a3118..2b9206ff24a 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -549,7 +549,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
- Bitmapset *updated_cols,
+ const Bitmapset *updated_cols,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1503,12 +1503,12 @@ static inline TM_Result
table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- Bitmapset *updated_cols, TU_UpdateIndexes *update_indexes)
+ const Bitmapset *mix_cols, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, snapshot, crosscheck,
wait, tmfd, lockmode,
- updated_cols, update_indexes);
+ mix_cols, update_indexes);
}
/*
@@ -2011,7 +2011,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
- Bitmapset *modified_indexe_attrs,
+ const Bitmapset *mix_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index dda95e54903..8d364f8b30f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -132,6 +132,7 @@ extern bool CompareIndexInfo(const IndexInfo *info1, const IndexInfo *info2,
const AttrMap *attmap);
extern void BuildSpeculativeIndexInfo(Relation index, IndexInfo *ii);
+extern void BuildUpdateIndexInfo(ResultRelInfo *resultRelInfo);
extern void FormIndexDatum(IndexInfo *indexInfo,
TupleTableSlot *slot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 993dc0e6ced..dda48f17605 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -739,6 +739,11 @@ extern Bitmapset *ExecGetAllUpdatedCols(ResultRelInfo *relinfo, EState *estate);
*/
extern void ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative);
extern void ExecCloseIndices(ResultRelInfo *resultRelInfo);
+extern Bitmapset *ExecWhichIndexesRequireUpdates(ResultRelInfo *relinfo,
+ Bitmapset *mix_attrs,
+ EState *estate,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
extern List *ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
TupleTableSlot *slot, EState *estate,
bool update,
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 898368fb8cb..d8e88817206 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -174,15 +174,29 @@ typedef struct IndexInfo
*/
AttrNumber ii_IndexAttrNumbers[INDEX_MAX_KEYS];
+ /*
+ * All key, expression, sumarizing, and partition attributes referenced by
+ * this index
+ */
+ Bitmapset *ii_IndexedAttrs;
+
/* expr trees for expression entries, or NIL if none */
List *ii_Expressions; /* list of Expr */
/* exec state for expressions, or NIL if none */
List *ii_ExpressionsState; /* list of ExprState */
+ /* attributes exclusively referenced by expression indexes */
+ Bitmapset *ii_ExpressionsAttrs;
/* partial-index predicate, or NIL if none */
List *ii_Predicate; /* list of Expr */
/* exec state for expressions, or NIL if none */
ExprState *ii_PredicateState;
+ /* attributes referenced by the predicate */
+ Bitmapset *ii_PredicateAttrs;
+ /* partial index predicate determined yet? */
+ bool ii_CheckedPredicate;
+ /* amupdate hint used to avoid rechecking predicate */
+ bool ii_PredicateSatisfied;
/* Per-column exclusion operators, or NULL if none */
Oid *ii_ExclusionOps; /* array with one entry per column */
@@ -494,6 +508,11 @@ typedef struct ResultRelInfo
Bitmapset *ri_extraUpdatedCols;
/* true if the above has been computed */
bool ri_extraUpdatedCols_valid;
+
+ /*
+ * For UPDATE a Bitmapset of the attributes that are both indexed and have
+ * changed in value.
+ */
Bitmapset *ri_ChangedIndexedCols;
/* Projection to generate new tuple in an INSERT/UPDATE */
diff --git a/src/test/regress/expected/hot_expression_indexes.out b/src/test/regress/expected/hot_expression_indexes.out
new file mode 100644
index 00000000000..d3eb07742c6
--- /dev/null
+++ b/src/test/regress/expected/hot_expression_indexes.out
@@ -0,0 +1,644 @@
+-- ================================================================
+-- Test Suite for HOT Updates with Expression and Partial Indexes
+-- ================================================================
+-- Setup: Create function to measure HOT updates
+CREATE OR REPLACE FUNCTION check_hot_updates(
+ expected INT,
+ p_table_name TEXT DEFAULT 't',
+ p_schema_name TEXT DEFAULT current_schema()
+)
+RETURNS TABLE (
+ table_name TEXT,
+ total_updates BIGINT,
+ hot_updates BIGINT,
+ hot_update_percentage NUMERIC,
+ matches_expected BOOLEAN
+)
+LANGUAGE plpgsql
+AS $$
+DECLARE
+ v_relid oid;
+ v_qualified_name TEXT;
+ v_hot_updates BIGINT;
+ v_updates BIGINT;
+ v_xact_hot_updates BIGINT;
+ v_xact_updates BIGINT;
+BEGIN
+ -- Force statistics update
+ PERFORM pg_stat_force_next_flush();
+
+ -- Get table OID
+ v_qualified_name := quote_ident(p_schema_name) || '.' || quote_ident(p_table_name);
+ v_relid := v_qualified_name::regclass;
+
+ IF v_relid IS NULL THEN
+ RAISE EXCEPTION 'Table %.% not found', p_schema_name, p_table_name;
+ END IF;
+
+ -- Get cumulative + transaction stats
+ v_hot_updates := COALESCE(pg_stat_get_tuples_hot_updated(v_relid), 0);
+ v_updates := COALESCE(pg_stat_get_tuples_updated(v_relid), 0);
+ v_xact_hot_updates := COALESCE(pg_stat_get_xact_tuples_hot_updated(v_relid), 0);
+ v_xact_updates := COALESCE(pg_stat_get_xact_tuples_updated(v_relid), 0);
+
+ v_hot_updates := v_hot_updates + v_xact_hot_updates;
+ v_updates := v_updates + v_xact_updates;
+
+ RETURN QUERY
+ SELECT
+ p_table_name::TEXT,
+ v_updates::BIGINT,
+ v_hot_updates::BIGINT,
+ CASE WHEN v_updates > 0
+ THEN ROUND((v_hot_updates::numeric / v_updates::numeric * 100)::numeric, 2)
+ ELSE 0
+ END,
+ (v_hot_updates = expected)::BOOLEAN;
+END;
+$$;
+-- ================================================================
+-- Basic JSONB Expression Index
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_name_idx ON t((docs->>'name'));
+INSERT INTO t VALUES (1, '{"name": "alice", "age": 30}');
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET docs = '{"name": "alice", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update indexed JSONB field - should NOT be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update non-indexed field again - should be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 32}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 2 | 66.67 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Partial Index with Predicate Transitions
+-- ================================================================
+CREATE TABLE t(id INT, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_value_idx ON t(value) WHERE value > 10;
+INSERT INTO t VALUES (1, 5);
+-- Both outside predicate - should be HOT
+UPDATE t SET value = 8 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET value = 15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Both inside predicate, value changes - should NOT be HOT
+UPDATE t SET value = 20 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Transition out of predicate - should NOT be HOT
+UPDATE t SET value = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 1 | 25.00 | t
+(1 row)
+
+-- Both outside predicate again - should be HOT
+UPDATE t SET value = 3 WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 5 | 2 | 40.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Expression Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((docs->>'status'))
+ WHERE (docs->>'priority')::int > 5;
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3}');
+-- Both outside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 4}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 10}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Inside predicate, status changes - should NOT be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 10}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Inside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 8}';
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 4 | 2 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Index on JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data);
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "database"]}');
+-- Change tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Change tags again - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Add field without changing existing keys - GIN keys changed (added "note"), NOT HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "note": "test"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN Index with Unchanged Keys
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create GIN index on specific path
+CREATE INDEX t_gin_idx ON t USING gin((data->'tags'));
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "sql"], "status": "active"}');
+-- Change non-indexed field - GIN keys on 'tags' unchanged, should be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change indexed tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- GIN with jsonb_path_ops
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data jsonb_path_ops);
+INSERT INTO t VALUES (1, '{"user": {"name": "alice"}, "tags": ["a", "b"]}');
+-- Change value at different path - keys changed, NOT HOT
+UPDATE t SET data = '{"user": {"name": "bob"}, "tags": ["a", "b"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- BRIN Index
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- BRIN tracks min/max per block range
+CREATE INDEX t_brin_idx ON t USING brin(value);
+INSERT INTO t VALUES (1, 100, 'initial');
+-- Update non-indexed column - BRIN doesn't care, should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update indexed column but stay in same range - should still be HOT
+-- (Note: BRIN tracks ranges, not exact values)
+UPDATE t SET value = 105 WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 2 | 100.00 | t
+(1 row)
+
+-- Change to significantly different value - still HOT for single row
+-- (BRIN summary won't change for single-row updates in same block)
+UPDATE t SET value = 1000 WHERE id = 1;
+SELECT * FROM check_hot_updates(3, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 3 | 100.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Multi-Column Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, a INT, b INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t(id, abs(a), abs(b));
+INSERT INTO t VALUES (1, -5, -10);
+-- Change sign but not abs value - should be HOT
+UPDATE t SET a = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change abs value - should NOT be HOT
+UPDATE t SET b = -15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Change id - should NOT be HOT
+UPDATE t SET id = 2 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Mixed Index Types (BRIN + Expression)
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_idx ON t USING brin(value);
+CREATE INDEX t_expr_idx ON t((data->>'status'));
+INSERT INTO t VALUES (1, 100, '{"status": "active"}');
+-- Update only BRIN column - should be HOT
+UPDATE t SET value = 200 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Update only expression column - should NOT be HOT
+UPDATE t SET data = '{"status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Update both - should NOT be HOT
+UPDATE t SET value = 300, data = '{"status": "pending"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Expression with COLLATION
+-- ================================================================
+CREATE TABLE t(id INT, name TEXT COLLATE "C")
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_lower_idx ON t(lower(name));
+INSERT INTO t VALUES (1, 'ALICE');
+-- Change case but not lowercase value - should be HOT
+UPDATE t SET name = 'Alice' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change lowercase value - should NOT be HOT
+UPDATE t SET name = 'BOB' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Array Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_array_len_idx ON t(array_length(tags, 1));
+INSERT INTO t VALUES (1, ARRAY['a', 'b', 'c']);
+-- Same length, different elements - should be HOT
+UPDATE t SET tags = ARRAY['d', 'e', 'f'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Different length - should NOT be HOT
+UPDATE t SET tags = ARRAY['d', 'e'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Nested JSONB Expression
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_nested_idx ON t((data->'user'->>'name'));
+INSERT INTO t VALUES ('{"user": {"name": "alice", "age": 30}}');
+-- Change nested non-indexed field - should be HOT
+UPDATE t SET data = '{"user": {"name": "alice", "age": 31}}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change nested indexed field - should NOT be HOT
+UPDATE t SET data = '{"user": {"name": "bob", "age": 31}}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- Complex Predicate on Multiple JSONB Fields
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((data->>'status'))
+ WHERE (data->>'priority')::int > 5
+ AND (data->>'active')::boolean = true;
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3, "active": true}');
+-- Outside predicate (priority too low) - should be HOT
+UPDATE t SET data = '{"status": "done", "priority": 3, "active": true}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": true}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Inside predicate, change to outside (active = false) - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": false}';
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- TOASTed Values in Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, large_text TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_substr_idx ON t(substr(large_text, 1, 10));
+INSERT INTO t VALUES (1, repeat('x', 5000) || 'identifier');
+-- Change end of string, prefix unchanged - should be HOT
+UPDATE t SET large_text = repeat('x', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Change prefix - should NOT be HOT
+UPDATE t SET large_text = repeat('y', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+DROP TABLE t;
+-- ================================================================
+-- TEST: GIN with TOASTed TEXT (tsvector)
+-- ================================================================
+CREATE TABLE t(id INT, content TEXT, search_vec tsvector)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create trigger to maintain tsvector
+CREATE TRIGGER tsvectorupdate_toast
+ BEFORE INSERT OR UPDATE ON t
+ FOR EACH ROW EXECUTE FUNCTION
+ tsvector_update_trigger(search_vec, 'pg_catalog.english', content);
+CREATE INDEX t_gin ON t USING gin(search_vec);
+-- Insert with large content (will be TOASTed)
+INSERT INTO t (id, content) VALUES
+ (1, repeat('important keyword ', 1000) || repeat('filler text ', 10000));
+-- Verify initial state
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('important');
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+-- IMPORTANT: The BEFORE UPDATE trigger modifies search_vec, so by the time
+-- ExecWhichIndexesRequireUpdates() runs, search_vec has already changed.
+-- This means the comparison sees old tsvector vs. trigger-modified tsvector,
+-- not the natural progression. HOT won't happen because the trigger changed
+-- the indexed column.
+-- Update: Even though content keywords unchanged, trigger still fires
+UPDATE t
+SET content = repeat('important keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (trigger modifies search_vec, blocking HOT)
+-- This is actually correct behavior - the trigger updated an indexed column
+-- Update: Change indexed keywords
+UPDATE t
+SET content = repeat('critical keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (index keys changed)
+-- Verify query correctness
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('critical');
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+DROP TABLE t CASCADE;
+-- ================================================================
+-- TEST: GIN with TOASTed JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin((data->'tags'));
+-- Insert with TOASTed JSONB
+INSERT INTO t (id, data) VALUES
+ (1, jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('x', 10000)
+ ));
+-- Update: Change large_field, tags unchanged - should be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 1 | 100.00 | t
+(1 row)
+
+-- Expected: 1 HOT update
+-- Update: Change tags - should NOT be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "sql"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Expected: Still 1 HOT
+-- Verify correctness
+SELECT count(*) FROM t WHERE data->'tags' @> '["database"]'::jsonb;
+ count
+-------
+ 0
+(1 row)
+
+-- Expected: 0 rows
+SELECT count(*) FROM t WHERE data->'tags' @> '["sql"]'::jsonb;
+ count
+-------
+ 1
+(1 row)
+
+-- Expected: 1 row
+DROP TABLE t CASCADE;
+-- ================================================================
+-- TEST: GIN with Array of Large Strings
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin(tags);
+-- Insert with large array elements (might be TOASTed)
+INSERT INTO t (id, tags) VALUES
+ (1, ARRAY[repeat('tag1', 1000), repeat('tag2', 1000)]);
+-- Update: Change to different large values - NOT HOT
+UPDATE t
+SET tags = ARRAY[repeat('tag3', 1000), repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 1 | 0 | 0.00 | t
+(1 row)
+
+-- Expected: 0 HOT (keys actually changed)
+-- Update: Keep same tag values, just reorder - SHOULD BE HOT
+-- (GIN is order-insensitive: both [tag3,tag4] and [tag4,tag3]
+-- extract to the same sorted key set ['tag3','tag4'])
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000), repeat('tag3', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 2 | 1 | 50.00 | t
+(1 row)
+
+-- Expected: 1 HOT (GIN keys semantically identical)
+-- Update: Remove an element - NOT HOT (keys changed)
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+ table_name | total_updates | hot_updates | hot_update_percentage | matches_expected
+------------+---------------+-------------+-----------------------+------------------
+ t | 3 | 1 | 33.33 | t
+(1 row)
+
+-- Expected: Still 1 HOT (not this one)
+DROP TABLE t CASCADE;
+-- Cleanup
+DROP FUNCTION check_hot_updates(int, text, text);
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index f56482fb9f1..4459625a59b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -125,6 +125,12 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression compression_lz4 memoize stats predicate numa eager_aggregate
+
+# ----------
+# Another group of parallel tests, these focused on heap HOT updates
+# ----------
+test: hot_expression_indexes
+
# event_trigger depends on create_am and cannot run concurrently with
# any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/sql/hot_expression_indexes.sql b/src/test/regress/sql/hot_expression_indexes.sql
new file mode 100644
index 00000000000..5dcadbde465
--- /dev/null
+++ b/src/test/regress/sql/hot_expression_indexes.sql
@@ -0,0 +1,491 @@
+-- ================================================================
+-- Test Suite for HOT Updates with Expression and Partial Indexes
+-- ================================================================
+
+-- Setup: Create function to measure HOT updates
+CREATE OR REPLACE FUNCTION check_hot_updates(
+ expected INT,
+ p_table_name TEXT DEFAULT 't',
+ p_schema_name TEXT DEFAULT current_schema()
+)
+RETURNS TABLE (
+ table_name TEXT,
+ total_updates BIGINT,
+ hot_updates BIGINT,
+ hot_update_percentage NUMERIC,
+ matches_expected BOOLEAN
+)
+LANGUAGE plpgsql
+AS $$
+DECLARE
+ v_relid oid;
+ v_qualified_name TEXT;
+ v_hot_updates BIGINT;
+ v_updates BIGINT;
+ v_xact_hot_updates BIGINT;
+ v_xact_updates BIGINT;
+BEGIN
+ -- Force statistics update
+ PERFORM pg_stat_force_next_flush();
+
+ -- Get table OID
+ v_qualified_name := quote_ident(p_schema_name) || '.' || quote_ident(p_table_name);
+ v_relid := v_qualified_name::regclass;
+
+ IF v_relid IS NULL THEN
+ RAISE EXCEPTION 'Table %.% not found', p_schema_name, p_table_name;
+ END IF;
+
+ -- Get cumulative + transaction stats
+ v_hot_updates := COALESCE(pg_stat_get_tuples_hot_updated(v_relid), 0);
+ v_updates := COALESCE(pg_stat_get_tuples_updated(v_relid), 0);
+ v_xact_hot_updates := COALESCE(pg_stat_get_xact_tuples_hot_updated(v_relid), 0);
+ v_xact_updates := COALESCE(pg_stat_get_xact_tuples_updated(v_relid), 0);
+
+ v_hot_updates := v_hot_updates + v_xact_hot_updates;
+ v_updates := v_updates + v_xact_updates;
+
+ RETURN QUERY
+ SELECT
+ p_table_name::TEXT,
+ v_updates::BIGINT,
+ v_hot_updates::BIGINT,
+ CASE WHEN v_updates > 0
+ THEN ROUND((v_hot_updates::numeric / v_updates::numeric * 100)::numeric, 2)
+ ELSE 0
+ END,
+ (v_hot_updates = expected)::BOOLEAN;
+END;
+$$;
+
+-- ================================================================
+-- Basic JSONB Expression Index
+-- ================================================================
+CREATE TABLE t(id INT PRIMARY KEY, docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_docs_name_idx ON t((docs->>'name'));
+INSERT INTO t VALUES (1, '{"name": "alice", "age": 30}');
+
+-- Update non-indexed JSONB field - should be HOT
+UPDATE t SET docs = '{"name": "alice", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update indexed JSONB field - should NOT be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 31}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update non-indexed field again - should be HOT
+UPDATE t SET docs = '{"name": "bob", "age": 32}' WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Partial Index with Predicate Transitions
+-- ================================================================
+CREATE TABLE t(id INT, value INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_value_idx ON t(value) WHERE value > 10;
+INSERT INTO t VALUES (1, 5);
+
+-- Both outside predicate - should be HOT
+UPDATE t SET value = 8 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET value = 15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Both inside predicate, value changes - should NOT be HOT
+UPDATE t SET value = 20 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Transition out of predicate - should NOT be HOT
+UPDATE t SET value = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Both outside predicate again - should be HOT
+UPDATE t SET value = 3 WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Expression Index with Partial Predicate
+-- ================================================================
+CREATE TABLE t(docs JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((docs->>'status'))
+ WHERE (docs->>'priority')::int > 5;
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3}');
+
+-- Both outside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 4}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET docs = '{"status": "pending", "priority": 10}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Inside predicate, status changes - should NOT be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 10}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Inside predicate, status unchanged - should be HOT
+UPDATE t SET docs = '{"status": "active", "priority": 8}';
+SELECT * FROM check_hot_updates(2, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN Index on JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data);
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "database"]}');
+
+-- Change tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+-- Change tags again - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+-- Add field without changing existing keys - GIN keys changed (added "note"), NOT HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "note": "test"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN Index with Unchanged Keys
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- Create GIN index on specific path
+CREATE INDEX t_gin_idx ON t USING gin((data->'tags'));
+INSERT INTO t VALUES (1, '{"tags": ["postgres", "sql"], "status": "active"}');
+
+-- Change non-indexed field - GIN keys on 'tags' unchanged, should be HOT
+UPDATE t SET data = '{"tags": ["postgres", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change indexed tags - GIN keys changed, should NOT be HOT
+UPDATE t SET data = '{"tags": ["mysql", "sql"], "status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- GIN with jsonb_path_ops
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin_idx ON t USING gin(data jsonb_path_ops);
+INSERT INTO t VALUES (1, '{"user": {"name": "alice"}, "tags": ["a", "b"]}');
+
+-- Change value at different path - keys changed, NOT HOT
+UPDATE t SET data = '{"user": {"name": "bob"}, "tags": ["a", "b"]}' WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- BRIN Index
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+-- BRIN tracks min/max per block range
+CREATE INDEX t_brin_idx ON t USING brin(value);
+INSERT INTO t VALUES (1, 100, 'initial');
+
+-- Update non-indexed column - BRIN doesn't care, should be HOT
+UPDATE t SET data = 'updated' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update indexed column but stay in same range - should still be HOT
+-- (Note: BRIN tracks ranges, not exact values)
+UPDATE t SET value = 105 WHERE id = 1;
+SELECT * FROM check_hot_updates(2, 't');
+
+-- Change to significantly different value - still HOT for single row
+-- (BRIN summary won't change for single-row updates in same block)
+UPDATE t SET value = 1000 WHERE id = 1;
+SELECT * FROM check_hot_updates(3, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Multi-Column Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, a INT, b INT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t(id, abs(a), abs(b));
+INSERT INTO t VALUES (1, -5, -10);
+
+-- Change sign but not abs value - should be HOT
+UPDATE t SET a = 5 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change abs value - should NOT be HOT
+UPDATE t SET b = -15 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change id - should NOT be HOT
+UPDATE t SET id = 2 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Mixed Index Types (BRIN + Expression)
+-- ================================================================
+CREATE TABLE t(id INT, value INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_brin_idx ON t USING brin(value);
+CREATE INDEX t_expr_idx ON t((data->>'status'));
+INSERT INTO t VALUES (1, 100, '{"status": "active"}');
+
+-- Update only BRIN column - should be HOT
+UPDATE t SET value = 200 WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update only expression column - should NOT be HOT
+UPDATE t SET data = '{"status": "inactive"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Update both - should NOT be HOT
+UPDATE t SET value = 300, data = '{"status": "pending"}' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Expression with COLLATION
+-- ================================================================
+CREATE TABLE t(id INT, name TEXT COLLATE "C")
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_lower_idx ON t(lower(name));
+INSERT INTO t VALUES (1, 'ALICE');
+
+-- Change case but not lowercase value - should be HOT
+UPDATE t SET name = 'Alice' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change lowercase value - should NOT be HOT
+UPDATE t SET name = 'BOB' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Array Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_array_len_idx ON t(array_length(tags, 1));
+INSERT INTO t VALUES (1, ARRAY['a', 'b', 'c']);
+
+-- Same length, different elements - should be HOT
+UPDATE t SET tags = ARRAY['d', 'e', 'f'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Different length - should NOT be HOT
+UPDATE t SET tags = ARRAY['d', 'e'] WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Nested JSONB Expression
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_nested_idx ON t((data->'user'->>'name'));
+INSERT INTO t VALUES ('{"user": {"name": "alice", "age": 30}}');
+
+-- Change nested non-indexed field - should be HOT
+UPDATE t SET data = '{"user": {"name": "alice", "age": 31}}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change nested indexed field - should NOT be HOT
+UPDATE t SET data = '{"user": {"name": "bob", "age": 31}}';
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- Complex Predicate on Multiple JSONB Fields
+-- ================================================================
+CREATE TABLE t(data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_idx ON t((data->>'status'))
+ WHERE (data->>'priority')::int > 5
+ AND (data->>'active')::boolean = true;
+
+INSERT INTO t VALUES ('{"status": "pending", "priority": 3, "active": true}');
+
+-- Outside predicate (priority too low) - should be HOT
+UPDATE t SET data = '{"status": "done", "priority": 3, "active": true}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Transition into predicate - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": true}';
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Inside predicate, change to outside (active = false) - should NOT be HOT
+UPDATE t SET data = '{"status": "done", "priority": 10, "active": false}';
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- TOASTed Values in Expression Index
+-- ================================================================
+CREATE TABLE t(id INT, large_text TEXT)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_substr_idx ON t(substr(large_text, 1, 10));
+
+INSERT INTO t VALUES (1, repeat('x', 5000) || 'identifier');
+
+-- Change end of string, prefix unchanged - should be HOT
+UPDATE t SET large_text = repeat('x', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+-- Change prefix - should NOT be HOT
+UPDATE t SET large_text = repeat('y', 5000) || 'different' WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+
+DROP TABLE t;
+
+-- ================================================================
+-- TEST: GIN with TOASTed TEXT (tsvector)
+-- ================================================================
+CREATE TABLE t(id INT, content TEXT, search_vec tsvector)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+-- Create trigger to maintain tsvector
+CREATE TRIGGER tsvectorupdate_toast
+ BEFORE INSERT OR UPDATE ON t
+ FOR EACH ROW EXECUTE FUNCTION
+ tsvector_update_trigger(search_vec, 'pg_catalog.english', content);
+
+CREATE INDEX t_gin ON t USING gin(search_vec);
+
+-- Insert with large content (will be TOASTed)
+INSERT INTO t (id, content) VALUES
+ (1, repeat('important keyword ', 1000) || repeat('filler text ', 10000));
+
+-- Verify initial state
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('important');
+-- Expected: 1 row
+
+-- IMPORTANT: The BEFORE UPDATE trigger modifies search_vec, so by the time
+-- ExecWhichIndexesRequireUpdates() runs, search_vec has already changed.
+-- This means the comparison sees old tsvector vs. trigger-modified tsvector,
+-- not the natural progression. HOT won't happen because the trigger changed
+-- the indexed column.
+
+-- Update: Even though content keywords unchanged, trigger still fires
+UPDATE t
+SET content = repeat('important keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+-- Expected: 0 HOT (trigger modifies search_vec, blocking HOT)
+-- This is actually correct behavior - the trigger updated an indexed column
+
+-- Update: Change indexed keywords
+UPDATE t
+SET content = repeat('critical keyword ', 1000) || repeat('different filler ', 10000)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+-- Expected: 0 HOT (index keys changed)
+
+-- Verify query correctness
+SELECT count(*) FROM t WHERE search_vec @@ to_tsquery('critical');
+-- Expected: 1 row
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- TEST: GIN with TOASTed JSONB
+-- ================================================================
+CREATE TABLE t(id INT, data JSONB)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin((data->'tags'));
+
+-- Insert with TOASTed JSONB
+INSERT INTO t (id, data) VALUES
+ (1, jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('x', 10000)
+ ));
+
+-- Update: Change large_field, tags unchanged - should be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "database"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: 1 HOT update
+
+-- Update: Change tags - should NOT be HOT
+UPDATE t
+SET data = jsonb_build_object(
+ 'tags', '["postgres", "sql"]'::jsonb,
+ 'large_field', repeat('y', 10000)
+)
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: Still 1 HOT
+
+-- Verify correctness
+SELECT count(*) FROM t WHERE data->'tags' @> '["database"]'::jsonb;
+-- Expected: 0 rows
+SELECT count(*) FROM t WHERE data->'tags' @> '["sql"]'::jsonb;
+-- Expected: 1 row
+
+DROP TABLE t CASCADE;
+
+-- ================================================================
+-- TEST: GIN with Array of Large Strings
+-- ================================================================
+CREATE TABLE t(id INT, tags TEXT[])
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+CREATE INDEX t_gin ON t USING gin(tags);
+
+-- Insert with large array elements (might be TOASTed)
+INSERT INTO t (id, tags) VALUES
+ (1, ARRAY[repeat('tag1', 1000), repeat('tag2', 1000)]);
+
+-- Update: Change to different large values - NOT HOT
+UPDATE t
+SET tags = ARRAY[repeat('tag3', 1000), repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(0, 't');
+-- Expected: 0 HOT (keys actually changed)
+
+-- Update: Keep same tag values, just reorder - SHOULD BE HOT
+-- (GIN is order-insensitive: both [tag3,tag4] and [tag4,tag3]
+-- extract to the same sorted key set ['tag3','tag4'])
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000), repeat('tag3', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: 1 HOT (GIN keys semantically identical)
+
+-- Update: Remove an element - NOT HOT (keys changed)
+UPDATE t
+SET tags = ARRAY[repeat('tag4', 1000)]
+WHERE id = 1;
+SELECT * FROM check_hot_updates(1, 't');
+-- Expected: Still 1 HOT (not this one)
+
+DROP TABLE t CASCADE;
+
+-- Cleanup
+DROP FUNCTION check_hot_updates(int, text, text);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 23bce72ae64..52ef8f10b35 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -390,6 +390,7 @@ CachedFunctionCompileCallback
CachedFunctionDeleteCallback
CachedFunctionHashEntry
CachedFunctionHashKey
+CachedIndexDatum
CachedPlan
CachedPlanSource
CallContext
--
2.49.0
view thread (35+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected]
Subject: Re: Expanding HOT updates for expression and partial indexes
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox