[PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS

public inbox for [email protected]  
help / color / mirror / Atom feed

[PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
68+ messages / 3 participants
[nested] [flat]

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/hio.c          |  8 +++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--CE+1k2dSO48ffgeK--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  5 +++--
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6d357565a8 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -147,8 +147,9 @@ The following infomask bits are applicable:
 
 - HEAP_KEYS_UPDATED
   This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  this tuple and changed the key values, or it deleted the tuple.  It can also
+  be set in combination of HEAP_XMAX_LOCK_ONLY.  It's set regardless of whether
+  the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--+HP7ph2BbKc20aGI--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination.
@ 2021-01-21 16:06 Julien Rouhaud <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Julien Rouhaud @ 2021-01-21 16:06 UTC (permalink / raw)

This combination of hint bits was previously detected as a form of corruption,
but it can be obtained with some mix of SELECT ... FOR UPDATE and UPDATE
queries.

Discussion: https://postgr.es/m/20210124061758.GA11756@nol
---
 contrib/amcheck/t/001_verify_heapam.pl | 14 +++++++++++++-
 contrib/amcheck/verify_heapam.c        |  7 -------
 src/backend/access/heap/README.tuplock |  7 ++++---
 src/backend/access/heap/hio.c          |  8 +++-----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index a2f65b826d..6050feb712 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -4,7 +4,7 @@ use warnings;
 use PostgresNode;
 use TestLib;
 
-use Test::More tests => 79;
+use Test::More tests => 80;
 
 my ($node, $result);
 
@@ -47,6 +47,9 @@ detects_heap_corruption(
 #
 fresh_test_table('test');
 $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
+detects_no_corruption(
+	"verify_heapam('test')",
+	"all-frozen not corrupted table");
 corrupt_first_page('test');
 detects_heap_corruption("verify_heapam('test')",
 	"all-frozen corrupted table");
@@ -92,6 +95,15 @@ sub fresh_test_table
 		ALTER TABLE $relname ALTER b SET STORAGE external;
 		INSERT INTO $relname (a, b)
 			(SELECT gs, repeat('b',gs*10) FROM generate_series(1,1000) gs);
+		BEGIN;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		RELEASE s1;
+		SAVEPOINT s1;
+		SELECT 1 FROM $relname WHERE a = 42 FOR UPDATE;
+		UPDATE $relname SET b = b WHERE a = 42;
+		COMMIT;
 	));
 }
 
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 88ab32490c..49f5ca0ef2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -608,13 +608,6 @@ check_tuple_header_and_visibilty(HeapTupleHeader tuphdr, HeapCheckContext *ctx)
 								   ctx->tuphdr->t_hoff, ctx->lp_len));
 		header_garbled = true;
 	}
-	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-		(ctx->tuphdr->t_infomask2 & HEAP_KEYS_UPDATED))
-	{
-		report_corruption(ctx,
-						  pstrdup("tuple is marked as only locked, but also claims key columns were updated"));
-		header_garbled = true;
-	}
 
 	if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) &&
 		(ctx->tuphdr->t_infomask & HEAP_XMAX_IS_MULTI))
diff --git a/src/backend/access/heap/README.tuplock b/src/backend/access/heap/README.tuplock
index d03ddf6cdc..6441e8baf0 100644
--- a/src/backend/access/heap/README.tuplock
+++ b/src/backend/access/heap/README.tuplock
@@ -146,9 +146,10 @@ The following infomask bits are applicable:
   FOR UPDATE; this is implemented by the HEAP_KEYS_UPDATED bit.
 
 - HEAP_KEYS_UPDATED
-  This bit lives in t_infomask2.  If set, indicates that the XMAX updated
-  this tuple and changed the key values, or it deleted the tuple.
-  It's set regardless of whether the XMAX is a TransactionId or a MultiXactId.
+  This bit lives in t_infomask2.  If set, indicates that the operation(s) done
+  by the XMAX compromise the tuple key, such as a SELECT FOR UPDATE, an UPDATE
+  that modifies the columns of the key, or a DELETE.  It's set regardless of
+  whether the XMAX is a TransactionId or a MultiXactId.
 
 We currently never set the HEAP_XMAX_COMMITTED when the HEAP_XMAX_IS_MULTI bit
 is set.
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c
index fb7ad0bab4..75223c9bea 100644
--- a/src/backend/access/heap/hio.c
+++ b/src/backend/access/heap/hio.c
@@ -49,12 +49,10 @@ RelationPutHeapTuple(Relation relation,
 
 	/*
 	 * Do not allow tuples with invalid combinations of hint bits to be placed
-	 * on a page.  These combinations are detected as corruption by the
-	 * contrib/amcheck logic, so if you disable one or both of these
-	 * assertions, make corresponding changes there.
+	 * on a page.  This combination is detected as corruption by the
+	 * contrib/amcheck logic, so if you disable this assertion, make
+	 * corresponding changes there.
 	 */
-	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_LOCK_ONLY) &&
-			 (tuple->t_data->t_infomask2 & HEAP_KEYS_UPDATED)));
 	Assert(!((tuple->t_data->t_infomask & HEAP_XMAX_COMMITTED) &&
 			 (tuple->t_data->t_infomask & HEAP_XMAX_IS_MULTI)));
 
-- 
2.20.1


--0ntfKIWw70PvrIHh--





^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* Re: Named Operators
@ 2023-01-20 17:32 Ted Yu <[email protected]>
  2023-01-20 18:56 ` Re: Named Operators Gurjeet Singh <[email protected]>
  0 siblings, 1 reply; 68+ messages in thread

From: Ted Yu @ 2023-01-20 17:32 UTC (permalink / raw)
  To: Gurjeet Singh <[email protected]>; +Cc: Vik Fearing <[email protected]>; Tom Lane <[email protected]>; Isaac Morland <[email protected]>; Matthias van de Meent <[email protected]>; pgsql-hackers

On Fri, Jan 20, 2023 at 9:17 AM Gurjeet Singh <[email protected]> wrote:

> On Sat, Jan 14, 2023 at 6:14 AM Gurjeet Singh <[email protected]> wrote:
> >
> > I agree that an identifier _surrounded_ by the same token (e.g. #foo#)
> > or the pairing token (e.g. {foo}) looks better aesthetically, so I am
> > okay with any of the following variations of the scheme, as well:
> >
> > \#foo\#  (tested; works)
> > \#foo#   (not tested; reduces ident length by 1)
> >
> > We can choose a different character, instead of #. Perhaps \{foo} !
>
> Please find attached the patch that uses \{foo} styled Named
> Operators. This is in line with Tom's reluctant hint at possibly using
> curly braces as delimiter characters. Since the curly braces are used
> by the SQL Specification for row pattern recognition, this patch
> proposes escaping the first of the curly braces.
>
> We can get rid of the leading backslash, if (a) we're confident that
> SQL committee will not use curly braces anywhere else, and (b) if
> we're confident that if/when Postgres supports Row Pattern Recognition
> feature, we'll be able to treat curly braces inside the PATTERN clause
> specially. Since both of those conditions are unlikely, I think we
> must settle for the escaped-first-curly-brace style for the naming our
> operators.
>
> Keeping with the previous posts, here's a sample SQL script showing
> what the proposed syntax will look like in action. Personally, I
> prefer the \#foo style, since the \# prefix stands out among the text,
> better than \{..} does, and because # character is a better signal of
> an operator than {.
>
> create operator \{add_point}
>     (function = box_add, leftarg = box, rightarg = point);
> create table test(a box);
> insert into test values('((0,0),(1,1))'), ('((0,0),(2,1))');
> select a as original, a \{add_point} '(1,1)' as modified from test;
> drop operator \{add_point}(box, point);
>
> Best regards,
> Gurjeet
> http://Gurje.et


Hi,
Since `validIdentifier` doesn't modify the contents of `name` string, it
seems that there is no need to create `tmp` string in `validNamedOperator`.
You can pass the start and end offsets into the string (name) as second and
third parameters to `validIdentifier`.

Cheers


^ permalink  raw  reply  [nested|flat] 68+ messages in thread

* Re: Named Operators
  2023-01-20 17:32 Re: Named Operators Ted Yu <[email protected]>
@ 2023-01-20 18:56 ` Gurjeet Singh <[email protected]>
  0 siblings, 0 replies; 68+ messages in thread

From: Gurjeet Singh @ 2023-01-20 18:56 UTC (permalink / raw)
  To: Ted Yu <[email protected]>; +Cc: Vik Fearing <[email protected]>; Tom Lane <[email protected]>; Isaac Morland <[email protected]>; Matthias van de Meent <[email protected]>; pgsql-hackers

On Fri, Jan 20, 2023 at 9:32 AM Ted Yu <[email protected]> wrote:
>
> Since `validIdentifier` doesn't modify the contents of `name` string, it seems that there is no need to create `tmp` string in `validNamedOperator`.
> You can pass the start and end offsets into the string (name) as second and third parameters to `validIdentifier`.

Thanks for reviewing the patch!

I was making a temporary copy of the string, since I had to modify it
before the validation, whereas the callee expects a `const char*`. I
agree that the same check can be done with more elegance, while
eliminating the temporary allocation. Please find the updated patch
attached.

Instead of passing the start and end of region I want to check, as
suggested, I'm now passing just the length of the string I want
validated. But I think that's for the better, since it now aligns with
the comment that validIdentifier() does not check if the passed string
is shorter than NAMEDATALEN.

Best regards,
Gurjeet
http://Gurje.et


Attachments:

  [application/octet-stream] named_operators_v4.patch (5.6K, ../../CABwTF4UnQf1L414+8Ls_9Qu4j1H-rzHyg6TzGmx2Omd9HS21pA@mail.gmail.com/2-named_operators_v4.patch)
  download | inline diff:
diff --git a/src/backend/catalog/pg_operator.c b/src/backend/catalog/pg_operator.c
index 1017f2eed1..c5b8562cb5 100644
--- a/src/backend/catalog/pg_operator.c
+++ b/src/backend/catalog/pg_operator.c
@@ -31,6 +31,7 @@
 #include "catalog/pg_type.h"
 #include "miscadmin.h"
 #include "parser/parse_oper.h"
+#include "parser/scansup.h"
 #include "utils/acl.h"
 #include "utils/builtins.h"
 #include "utils/lsyscache.h"
@@ -79,6 +80,10 @@ validOperatorName(const char *name)
 	if (len == 0 || len >= NAMEDATALEN)
 		return false;
 
+	/* Is this a Named Operator? */
+	if (validNamedOperator(name))
+		return true;
+
 	/* Can't contain any invalid characters */
 	/* Test string here should match op_chars in scan.l */
 	if (strspn(name, "~!@#^&|`?+-*/%<>=") != len)
diff --git a/src/backend/parser/scan.l b/src/backend/parser/scan.l
index db8b0fe8eb..8587b82c8d 100644
--- a/src/backend/parser/scan.l
+++ b/src/backend/parser/scan.l
@@ -379,6 +379,16 @@ self			[,()\[\].;\:\+\-\*\/\%\^\<\>\=]
 op_chars		[\~\!\@\#\^\&\|\`\?\+\-\*\/\%\<\>\=]
 operator		{op_chars}+
 
+/*
+ * Named Operators, e.g. \{foo}
+ *
+ * {namedopfailed*} are error rules to avoid scanner backup when
+ * {namedop} fails to match its trailing tokens.
+ */
+namedop			\\\{{identifier}\}
+namedopfailed1	\\\{{identifier}
+namedopfailed2	\\\{
+
 /*
  * Numbers
  *
@@ -768,6 +778,23 @@ other			.
 				}
 <xdolq><<EOF>>	{ yyerror("unterminated dollar-quoted string"); }
 
+{namedop}		{
+					SET_YYLLOC();
+					if (yyleng >= NAMEDATALEN)
+						yyerror("operator name too long");
+					/* XXX Should we support double-quoted, case sensitive names? */
+					yylval->str = downcase_identifier(yytext, yyleng, false, false);
+					return Op;
+				}
+
+{namedopfailed1}	{
+					yyerror("unexpected token");
+				}
+
+{namedopfailed2}	{
+					yyerror("unexpected token");
+				}
+
 {xdstart}		{
 					SET_YYLLOC();
 					BEGIN(xd);
diff --git a/src/backend/parser/scansup.c b/src/backend/parser/scansup.c
index 602108a40f..f209ac3752 100644
--- a/src/backend/parser/scansup.c
+++ b/src/backend/parser/scansup.c
@@ -125,3 +125,65 @@ scanner_isspace(char ch)
 		return true;
 	return false;
 }
+
+/*
+ * validNamedOperator() -- return true if name adheres to the scanner rule
+ * {namedop}
+ */
+bool
+validNamedOperator(const char *name)
+{
+	size_t	len = strlen(name);
+	bool	valid_identifier;
+
+	if (len < 4 || len >= NAMEDATALEN)
+	   return false;
+
+	if (name[0] != '\\' || name[1] != '{' || name[len-1] != '}')
+		return false;
+
+	// Disregard the delimiters
+	valid_identifier = validIdentifier(name + 2, len - 3);
+
+	return valid_identifier;
+}
+
+/*
+ * validIdentifier() -- return true if name adheres to the scanner rule
+ * {identifier}
+ *
+ * Note: this function does not check if the identifier length
+ * is less than NAMEDATALEN.
+ */
+bool
+validIdentifier(const char *name, size_t len)
+{
+	uint8	c;
+	size_t	i;
+
+	// Reject if first character is not part of ident_start
+	c = name[0];
+	if ( !(c == '_'
+		|| (c >='A' && c <= 'Z')
+		|| (c >='a' && c <= 'z')
+		|| (c >= 0200 && c <= 0377)))
+	{
+		return false;
+	}
+
+	// Reject if other characters are not part of ident_cont
+	for (i = 1; i < len; ++i)
+	{
+		c = name[i];
+		if ( !(c == '_' || c == '$'
+			|| (c >='A' && c <= 'Z')
+			|| (c >='a' && c <= 'z')
+			|| (c >='0' && c <= '9')
+			|| (c >= 0200 && c <= 0377)))
+		{
+			return false;
+		}
+	}
+
+	return true;
+}
diff --git a/src/fe_utils/psqlscan.l b/src/fe_utils/psqlscan.l
index ae531ec240..98a2561886 100644
--- a/src/fe_utils/psqlscan.l
+++ b/src/fe_utils/psqlscan.l
@@ -317,6 +317,16 @@ self			[,()\[\].;\:\+\-\*\/\%\^\<\>\=]
 op_chars		[\~\!\@\#\^\&\|\`\?\+\-\*\/\%\<\>\=]
 operator		{op_chars}+
 
+/*
+ * Named Operators, e.g. :foo:
+ *
+ * {namedopfailed*} are error rules to avoid scanner backup when
+ * {namedop} fails to match its trailing tokens.
+ */
+namedop			\\\{{identifier}\}
+namedopfailed1	\\\{{identifier}
+namedopfailed2	\\\{
+
 /*
  * Numbers
  *
@@ -570,6 +580,18 @@ other			.
 					ECHO;
 				}
 
+{namedop}		{
+					ECHO;
+				}
+
+{namedopfailed1}	{
+					ECHO;
+				}
+
+{namedopfailed2}	{
+					ECHO;
+				}
+
 {xdstart}		{
 					BEGIN(xd);
 					ECHO;
diff --git a/src/include/parser/scansup.h b/src/include/parser/scansup.h
index ff65224bf6..b9eb0a96a4 100644
--- a/src/include/parser/scansup.h
+++ b/src/include/parser/scansup.h
@@ -24,4 +24,7 @@ extern void truncate_identifier(char *ident, int len, bool warn);
 
 extern bool scanner_isspace(char ch);
 
+extern bool validNamedOperator(const char *name);
+extern bool validIdentifier(const char *name, size_t len);
+
 #endif							/* SCANSUP_H */
diff --git a/test.sql b/test.sql
new file mode 100644
index 0000000000..4d0c44c629
--- /dev/null
+++ b/test.sql
@@ -0,0 +1,38 @@
+
+create operator \{hello_named_operators} (function = box_add, leftarg = box, rightarg = point);
+
+select '((0,0), (1,1))'::box \{hello_named_operators} '(1,1)'::point;
+
+drop operator \{hello_named_operators}(box, point);
+
+create operator \{add_point} (function = box_add, leftarg = box, rightarg = point);
+
+create table test(a box);
+
+insert into test values('((0,0),(1,1))'), ('((0,0),(2,1))');
+
+select a as original, a \{add_point} '(1,1)' as modified from test;
+
+CREATE OPERATOR \{equal_1} (
+    LEFTARG = int,
+    RIGHTARG = int,
+    PROCEDURE = int4eq,
+    COMMUTATOR = \{equal_1}
+);
+
+CREATE OPERATOR \{equal_2} (
+    LEFTARG = int,
+    RIGHTARG = int,
+    PROCEDURE = int4eq,
+    COMMUTATOR = equal_2 -- The absence of delimiters causes an expected failure
+);
+
+
+\! pg_dump -U postgres --schema public
+
+drop operator \{add_point}(box, point);
+drop operator \{equal_1}(int, int);
+drop operator \{equal_2}(int, int);
+
+drop table test;
+


^ permalink  raw  reply  [nested|flat] 68+ messages in thread

end of thread, other threads:[~2023-01-20 18:56 UTC | newest]

Thread overview: 68+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v1] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v2] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2021-01-21 16:06 [PATCH v3] Allow HEAP_XMAX_LOCK_ONLY and HEAP_KEYS_UPDATED combination. Julien Rouhaud <[email protected]>
2023-01-20 17:32 Re: Named Operators Ted Yu <[email protected]>
2023-01-20 18:56 ` Re: Named Operators Gurjeet Singh <[email protected]>

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox