Heap-Only Tuples (<acronym>HOT</acronym>)

Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oEhzj-0007Wy-KC for pgsql-docs@arkaria.postgresql.org; Fri, 22 Jul 2022 02:02:23 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1oEhzi-0005uu-FF for pgsql-docs@arkaria.postgresql.org; Fri, 22 Jul 2022 02:02:22 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oEhzi-0005ul-3I for pgsql-docs@lists.postgresql.org; Fri, 22 Jul 2022 02:02:22 +0000 Received: from momjian.us ([72.94.173.45]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oEhzf-0006tv-6a; Fri, 22 Jul 2022 02:02:21 +0000 Received: from bruce by momjian.us with local (Exim 4.94.2) (envelope-from ) id 1oEhze-006hNt-1a; Thu, 21 Jul 2022 22:02:18 -0400 Date: Thu, 21 Jul 2022 22:02:18 -0400 From: Bruce Momjian To: "Jonathan S. Katz" Cc: "David G. Johnston" , Pg Docs Subject: Re: documentation on HOT Message-ID: References:

MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="EBbfoCfuvWag0gsw" Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --EBbfoCfuvWag0gsw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Sun, Feb 6, 2022 at 09:29:56PM -0500, Jonathan Katz wrote: > I agree with Bruce's point that we should have a new section (or > subsection). As I mentioned in my previous post, given HOT involves > indexing, I would suggest putting it there. > > I think that something that follows the general outline of Laurenz's post > would satisfy the user requirements. It explains at a high level what HOT > is, it's advantages, and how it works. Attached is a patch that adds a new HOT section to the storage chapter. You can see the results here: https://momjian.us/tmp/pgsql/storage-hot.html -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson --EBbfoCfuvWag0gsw Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="hot.diff" diff --git a/doc/src/sgml/acronyms.sgml b/doc/src/sgml/acronyms.sgml index 9ed148ab84..2df6559acc 100644 --- a/doc/src/sgml/acronyms.sgml +++ b/doc/src/sgml/acronyms.sgml @@ -299,9 +299,7 @@ HOT - Heap-Only - Tuples + Heap-Only Tuples diff --git a/doc/src/sgml/btree.sgml b/doc/src/sgml/btree.sgml index a9200ee52e..6f608a14bf 100644 --- a/doc/src/sgml/btree.sgml +++ b/doc/src/sgml/btree.sgml @@ -639,7 +639,8 @@ options(relopts local_relopts *) returns accumulate and adversely affect query latency and throughput. This typically occurs with UPDATE-heavy workloads where most individual updates cannot apply the - HOT optimization. Changing the value of only + HOT optimization. + Changing the value of only one column covered by one index during an UPDATE always necessitates a new set of index tuples — one for each and every index on the diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index a186e35f00..248dbc0e26 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -4381,7 +4381,7 @@ SCRAM-SHA-256$<iteration count>:&l If true, queries must not use the index until the xmin of this pg_index row is below their TransactionXmin - event horizon, because the table may contain broken HOT chains with + event horizon, because the table may contain broken HOT chains with incompatible rows that they can see diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index e2d728e0c4..e5a84ed76d 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -4482,7 +4482,8 @@ ANY num_sync ( HOT updates + will defer cleanup of dead row versions. The default is zero transactions, meaning that dead row versions can be removed as soon as possible, that is, as soon as they are no longer visible to any open transaction. You may wish to set this to a diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml index cf359fa9ff..4f83970c85 100644 --- a/doc/src/sgml/indexam.sgml +++ b/doc/src/sgml/indexam.sgml @@ -45,7 +45,8 @@ extant versions of the same logical row; to an index, each tuple is an independent object that needs its own index entry. Thus, an update of a row always creates all-new index entries for the row, even if - the key values did not change. (HOT tuples are an exception to this + the key values did not change. (HOT + tuples are an exception to this statement; but indexes do not deal with those, either.) Index entries for dead tuples are reclaimed (by vacuuming) when the dead tuples themselves are reclaimed. diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 023157d888..42e1e86c8a 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -749,7 +749,7 @@ CREATE INDEX people_names ON people ((first_name || ' ' || last_name)); Index expressions are relatively expensive to maintain, because the derived expression(s) must be computed for each row insertion - and non-HOT update. However, the index expressions are + and non-HOT update. However, the index expressions are not recomputed during an indexed search, since they are already stored in the index. In both examples above, the system sees the query as just WHERE indexedcolumn = 'constant' diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 7dbbab6f5c..6408d28c5d 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4426,7 +4426,7 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i n_tup_upd bigint - Number of rows updated (includes HOT updated rows) + Number of rows updated (includes HOT updated rows) diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml index f4b9f66589..fec9e17985 100644 --- a/doc/src/sgml/storage.sgml +++ b/doc/src/sgml/storage.sgml @@ -1075,4 +1075,65 @@ data. Empty in ordinary tables. + + + Heap-Only Tuples (<acronym>HOT</acronym>) + + + To allow for high concurrency, PostgreSQL + uses multiversion concurrency + control (MVCC) to store rows. However, + MVCC has some downsides for update queries. + Specifically, updates cause additional rows to be added to tables. + This can also require new index entries for each updated row, and + removal of old versions of rows can be expensive. + + + + Fortunately, there is an automatic system called heap-only tuples + (HOT) which minimizes the downsides of + PostgreSQL updates. This optimization is + possible when: + + + + + The table's indexed columns are not modified by the update. + + + + + There is sufficient free space on the page containing the old row + for the updated row. + + + + + In such cases, heap-only tuples provide two optimizations: + + + + + New index entries are not needed to represent the updated row. + + + + + Old versions of the updated row can be removed during normal + operation, including SELECTs, instead of requiring + periodic vacuum operations. + + + + + + + In summary, to make heap-only tuple updates more + likely, indexing of frequently-updated columns + should be avoided, and the use of non-default table fillfactor settings + is recommended. + + + --EBbfoCfuvWag0gsw--