Subject: FrozenTransactionId

From: Egor Rogov <[email-masked]>
Date: Sun, 12 Jun 2016 18:01:41 +0300

Hi,
Since 9.4 FrozenTransactionId is no longer used to freeze row versions. 
The change was done by commit 37484ad:

 >     Instead of changing the tuple xmin to FrozenTransactionId, the 
combination
 >     of HEAP_XMIN_COMMITTED and HEAP_XMIN_INVALID, which were 
previously never
 >     set together, is now defined as HEAP_XMIN_FROZEN.

But we still have FrozenTransactionId mentioned in section 23.1.5.
Please consider the attached patch to fix it.

Thanks,
Egor Rogov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 5204b34..8b966cc 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -403,28 +403,22 @@
 
    <para>
     The reason that periodic vacuuming solves the problem is that
-    <command>VACUUM</> will mark rows as <emphasis>frozen</>, indicating that
-    they were inserted by a transaction which committed sufficiently far in
-    the past that the effects of the inserting transaction is certain to be
-    visible, from an MVCC perspective, to all current and future transactions.
-    <productname>PostgreSQL</> reserves a special XID,
-    <literal>FrozenTransactionId</>, which does not follow the normal XID
-    comparison rules and is always considered older
-    than every normal XID. Normal XIDs are
-    compared using modulo-2<superscript>32</> arithmetic. This means
-    that for every normal XID, there are two billion XIDs that are
-    <quote>older</> and two billion that are <quote>newer</>; another
-    way to say it is that the normal XID space is circular with no
-    endpoint. Therefore, once a row version has been created with a particular
-    normal XID, the row version will appear to be <quote>in the past</> for
-    the next two billion transactions, no matter which normal XID we are
-    talking about. If the row version still exists after more than two billion
-    transactions, it will suddenly appear to be in the future. To
-    prevent this, frozen row versions are treated as if the inserting XID were
-    <literal>FrozenTransactionId</>, so that they will appear to be
-    <quote>in the past</> to all normal transactions regardless of wraparound
-    issues, and so such row versions will be valid until deleted, no matter
-    how long that is.
+    <command>VACUUM</> will mark rows as <emphasis>frozen</> (by setting
+    appropriate hint bits), indicating that they were inserted by a transaction
+    which committed sufficiently far in the past that the effects of the
+    inserting transaction is certain to be visible, from an MVCC perspective,
+    to all current and future transactions.  Usually XIDs are compared using
+    modulo-2<superscript>32</> arithmetic. This means that for every XID, there
+    are two billion XIDs that are <quote>older</> and two billion that are
+    <quote>newer</>; another way to say it is that the XID space is circular
+    with no endpoint. Therefore, once a row version has been created with a
+    particular XID, the row version will appear to be <quote>in the past</> for
+    the next two billion transactions. If the row version still exists after
+    more than two billion transactions, it will suddenly appear to be in the
+    future. To prevent this, usual visibility rules are not applied to frozen
+    row versions. Instead they are considered to be <quote>in the past</> to
+    all transactions regardless of wraparound issues, and so such row versions
+    will be valid until deleted, no matter how long that is.
    </para>
 
    <para>

-- 
Sent via pgsql-docs mailing list ([email-masked])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-docs

---

From: Egor Rogov <[email-masked]>
Date: Mon, 5 Sep 2016 10:27:25 -0700 (MST)

Hi,
This documentation bug is still in place for upcoming 9.6. Aren't anyone
interested in fixing it?

Regards,
Egor Rogov

--
View this message in context: http://postgresql.nabble.com/FrozenTransactionId-tp5907478p5919545.html
Sent from the PostgreSQL - docs mailing list archive at Nabble.com.

---

From: Tom Lane <[email-masked]>
Date: Mon, 05 Sep 2016 13:46:24 -0400

> This documentation bug is still in place for upcoming 9.6. Aren't anyone

It'd be more likely to get pushed if you'd submitted it in an easily
reviewable form, ie without reflowing the entire (rather long) paragraph.
As-is, it's much too hard to see what you changed or didn't change.

			regards, tom lane

---

From: Egor Rogov <[email-masked]>
Date: Mon, 5 Sep 2016 15:09:53 -0700 (MST)

> As-is, it's much too hard to see what you changed or didn't change.
Please find another patch attached. I hope it is a bit easier to review now.

Regards,
Egor Rogov

maintenance-2.patch (2K) <http://postgresql.nabble.com/attachment/5919578/0/maintenance-2.patch>

--
View this message in context: http://postgresql.nabble.com/FrozenTransactionId-tp5907478p5919578.html
Sent from the PostgreSQL - docs mailing list archive at Nabble.com.

---

From: Egor Rogov <[email-masked]>
Date: Tue, 6 Sep 2016 10:24:05 +0300

Sorry, my last email went without the attachment.

On 06.09.2016 01:09, Egor Rogov wrote:
> Tom Lane <[hidden email] 
> </user/SendEmail.jtp?type=node&node=5919578&i=0>> writes:
> *maintenance-2.patch* (2K) Download Attachment 
> <http://postgresql.nabble.com/attachment/5919578/0/maintenance-2.patch>
>
Regards,
Egor Rogov

diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 2713883..7d1ee4e 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -406,26 +406,23 @@
 
    <para>
     The reason that periodic vacuuming solves the problem is that
-    <command>VACUUM</> will mark rows as <emphasis>frozen</>, indicating that
+    <command>VACUUM</> will mark rows as <emphasis>frozen</> (by setting appropriate hint bits), indicating that
     they were inserted by a transaction which committed sufficiently far in
     the past that the effects of the inserting transaction is certain to be
     visible, from an MVCC perspective, to all current and future transactions.
-    <productname>PostgreSQL</> reserves a special XID,
-    <literal>FrozenTransactionId</>, which does not follow the normal XID
-    comparison rules and is always considered older
-    than every normal XID. Normal XIDs are
+    Usually XIDs are
     compared using modulo-2<superscript>32</> arithmetic. This means
-    that for every normal XID, there are two billion XIDs that are
+    that for every XID, there are two billion XIDs that are
     <quote>older</> and two billion that are <quote>newer</>; another
-    way to say it is that the normal XID space is circular with no
+    way to say it is that the XID space is circular with no
     endpoint. Therefore, once a row version has been created with a particular
-    normal XID, the row version will appear to be <quote>in the past</> for
-    the next two billion transactions, no matter which normal XID we are
-    talking about. If the row version still exists after more than two billion
+    XID, the row version will appear to be <quote>in the past</> for
+    the next two billion transactions.
+    If the row version still exists after more than two billion
     transactions, it will suddenly appear to be in the future. To
-    prevent this, frozen row versions are treated as if the inserting XID were
-    <literal>FrozenTransactionId</>, so that they will appear to be
-    <quote>in the past</> to all normal transactions regardless of wraparound
+    prevent this, usual visibility rules are not applied to frozen row versions.
+    Instead they are considered to be
+    <quote>in the past</> to all transactions regardless of wraparound
     issues, and so such row versions will be valid until deleted, no matter
     how long that is.
    </para>

---

From: Tom Lane <[email-masked]>
Date: Tue, 06 Sep 2016 08:41:34 -0400

>> Please find another patch attached. I hope it is a bit easier to 

Hmm.  I do not actually see anything wrong with the existing text:
it does not say that FrozenTransactionId is what appears on disk,
it says that frozen tuples are treated as if their xmin were
FrozenTransactionId.  That's an accurate description of the current
implementation.  I'm not sure that removing all mention of special
XIDs is a good thing, especially since they will still exist on-disk
in pg_upgrade'd installations.

It does seem like this paragraph could do with some copy-editing,
but I doubt that removing information is the way to go about that.
What I'd change:
1. Grammar fix: s/effects ... is/effects ... are/
2. Remove the useless phrase ", from an MVCC perspective,"
3. Reorder the sentences just after that, so that we have
"Normal XIDs are compared ... suddenly appear to be in the future.
To prevent this, PostgreSQL reserves a special XID, FrozenTransactionId,
... considered older than every normal XID.
Frozen row versions are treated as if ..."

It might also be worth mentioning explicitly that FrozenTransactionId
can appear on-disk in old installations.  And I'd be slightly tempted
to put in an explanation of BootstrapTransactionId, because users
will see that as well if they look into any system catalogs.

Comments?

			regards, tom lane

---

From: Egor Rogov <[email-masked]>
Date: Tue, 6 Sep 2016 23:15:53 +0300

> in pg_upgrade'd installations.

Okay, I see my mistake now: this old FrozenTransactionId mechanics is 
still valid to support upgraded installations.
Anyway, while the description is technically accurate, it is misleading. 
Right, it does not say that FrozenTransactionId is what appears on disk, 
but what is? There is no such information anywhere in the doc. Since 9.4 
frozen transactions have their normal XIDs preserved, so how a user can 
tell normal transaction from frozen one? This is what needs to be 
explained, I believe.

---

From: Tom Lane <[email-masked]>
Date: Tue, 06 Sep 2016 16:41:39 -0400

> tell normal transaction from frozen one? This is what needs to be 

I'm afraid the answer is "you can't tell".  The infomask bits in tuple
headers aren't exposed via SQL.  If you're really desperate,
contrib/pageinspect would help, but I don't propose mentioning that here.

In general, I'm not really sure why users would care very much at
a tuple-by-tuple level.  Aggregate statistics would be interesting,
which raises the question why contrib/pgstattuple doesn't provide
frozen-tuples counts.

			regards, tom lane

---

From: Tom Lane <[email-masked]>
Date: Tue, 06 Sep 2016 17:52:37 -0400

I pushed some edits based on this discussion at
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=975768f8eae2581b89ceafe8b16a77ff375207fe

			regards, tom lane

---

From: Egor Rogov <[email-masked]>
Date: Wed, 7 Sep 2016 01:31:49 +0300

On 06.09.2016 23:41, Tom Lane wrote:
> a tuple-by-tuple level.

"Normal" users shouldn't care of course, but DBAs sometimes have to. The 
old description could be easily misunderstood and one might expect to 
see FrozenTransactionId in xmin field (I know several people who wasted 
hours trying to figure out what's wrong with vacuum freeze). Now the 
description is much cleaner, thanks!

> Aggregate statistics would be interesting,
Regards,
Egor Rogov