pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: pageinspect: Fix handling of all-zero pages
6+ messages / 1 participants
[nested] [flat]

* pgsql: pageinspect: Fix handling of all-zero pages
@ 2022-04-14 06:11 Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2022-04-14 06:11 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

pageinspect: Fix handling of all-zero pages

Getting from get_raw_page() an all-zero page is considered as a valid
case by the buffer manager and it can happen for example when finding a
corrupted page with zero_damaged_pages enabled (using zero_damaged_pages
to look at corrupted pages happens), or after a crash when a relation
file is extended before any WAL for its new data is generated (before a
vacuum or autovacuum job comes in to do some cleanup).

However, all the functions of pageinspect, as of the index AMs (except
hash that has its own idea of new pages), heap, the FSM or the page
header have never worked with all-zero pages, causing various crashes
when going through the page internals.

This commit changes all the pageinspect functions to be compliant with
all-zero pages, where the choice is made to return NULL or no rows for
SRFs when finding a new page.  get_raw_page() still works the same way,
returning a batch of zeros in the bytea of the page retrieved.  A hard
error could be used but NULL, while more invasive, is useful when
scanning relation files in full to get a batch of results for a single
relation in one query.  Tests are added for all the code paths
impacted.

Reported-by: Daria Lepikhova
Author: Michael Paquier
Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru
Backpatch-through: 10

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/cd4868a5700fadf5a840d44686658517433b338c

Modified Files
--------------
contrib/pageinspect/brinfuncs.c        | 22 ++++++++++++++++++++++
contrib/pageinspect/btreefuncs.c       |  6 ++++++
contrib/pageinspect/expected/brin.out  | 25 +++++++++++++++++++++++++
contrib/pageinspect/expected/btree.out |  6 ++++++
contrib/pageinspect/expected/gin.out   | 14 ++++++++++++++
contrib/pageinspect/expected/gist.out  | 18 ++++++++++++++++++
contrib/pageinspect/expected/hash.out  | 12 ++++++++++++
contrib/pageinspect/expected/page.out  | 20 ++++++++++++++++++++
contrib/pageinspect/fsmfuncs.c         |  4 ++++
contrib/pageinspect/ginfuncs.c         | 13 +++++++++++++
contrib/pageinspect/gistfuncs.c        | 12 ++++++++++++
contrib/pageinspect/rawpage.c          |  3 +++
contrib/pageinspect/sql/brin.sql       |  7 +++++++
contrib/pageinspect/sql/btree.sql      |  4 ++++
contrib/pageinspect/sql/gin.sql        |  6 ++++++
contrib/pageinspect/sql/gist.sql       |  6 ++++++
contrib/pageinspect/sql/hash.sql       |  7 +++++++
contrib/pageinspect/sql/page.sql       |  6 ++++++
18 files changed, 191 insertions(+)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: pageinspect: Fix handling of all-zero pages
@ 2022-04-14 06:11 Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2022-04-14 06:11 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

pageinspect: Fix handling of all-zero pages

Getting from get_raw_page() an all-zero page is considered as a valid
case by the buffer manager and it can happen for example when finding a
corrupted page with zero_damaged_pages enabled (using zero_damaged_pages
to look at corrupted pages happens), or after a crash when a relation
file is extended before any WAL for its new data is generated (before a
vacuum or autovacuum job comes in to do some cleanup).

However, all the functions of pageinspect, as of the index AMs (except
hash that has its own idea of new pages), heap, the FSM or the page
header have never worked with all-zero pages, causing various crashes
when going through the page internals.

This commit changes all the pageinspect functions to be compliant with
all-zero pages, where the choice is made to return NULL or no rows for
SRFs when finding a new page.  get_raw_page() still works the same way,
returning a batch of zeros in the bytea of the page retrieved.  A hard
error could be used but NULL, while more invasive, is useful when
scanning relation files in full to get a batch of results for a single
relation in one query.  Tests are added for all the code paths
impacted.

Reported-by: Daria Lepikhova
Author: Michael Paquier
Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru
Backpatch-through: 10

Branch
------
REL_14_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/df6bbe73b86cda28472b9ecc61ac2deff5b2837b

Modified Files
--------------
contrib/pageinspect/brinfuncs.c        | 22 ++++++++++++++++++++++
contrib/pageinspect/btreefuncs.c       |  6 ++++++
contrib/pageinspect/expected/brin.out  | 25 +++++++++++++++++++++++++
contrib/pageinspect/expected/btree.out |  6 ++++++
contrib/pageinspect/expected/gin.out   | 14 ++++++++++++++
contrib/pageinspect/expected/gist.out  | 18 ++++++++++++++++++
contrib/pageinspect/expected/hash.out  | 12 ++++++++++++
contrib/pageinspect/expected/page.out  | 20 ++++++++++++++++++++
contrib/pageinspect/fsmfuncs.c         |  4 ++++
contrib/pageinspect/ginfuncs.c         | 12 ++++++++++++
contrib/pageinspect/gistfuncs.c        | 12 ++++++++++++
contrib/pageinspect/rawpage.c          |  3 +++
contrib/pageinspect/sql/brin.sql       |  7 +++++++
contrib/pageinspect/sql/btree.sql      |  4 ++++
contrib/pageinspect/sql/gin.sql        |  6 ++++++
contrib/pageinspect/sql/gist.sql       |  6 ++++++
contrib/pageinspect/sql/hash.sql       |  7 +++++++
contrib/pageinspect/sql/page.sql       |  6 ++++++
18 files changed, 190 insertions(+)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: pageinspect: Fix handling of all-zero pages
@ 2022-04-14 06:11 Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2022-04-14 06:11 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

pageinspect: Fix handling of all-zero pages

Getting from get_raw_page() an all-zero page is considered as a valid
case by the buffer manager and it can happen for example when finding a
corrupted page with zero_damaged_pages enabled (using zero_damaged_pages
to look at corrupted pages happens), or after a crash when a relation
file is extended before any WAL for its new data is generated (before a
vacuum or autovacuum job comes in to do some cleanup).

However, all the functions of pageinspect, as of the index AMs (except
hash that has its own idea of new pages), heap, the FSM or the page
header have never worked with all-zero pages, causing various crashes
when going through the page internals.

This commit changes all the pageinspect functions to be compliant with
all-zero pages, where the choice is made to return NULL or no rows for
SRFs when finding a new page.  get_raw_page() still works the same way,
returning a batch of zeros in the bytea of the page retrieved.  A hard
error could be used but NULL, while more invasive, is useful when
scanning relation files in full to get a batch of results for a single
relation in one query.  Tests are added for all the code paths
impacted.

Reported-by: Daria Lepikhova
Author: Michael Paquier
Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru
Backpatch-through: 10

Branch
------
REL_13_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/2275d044d0849251c5e0fa10ee16c73124731f5c

Modified Files
--------------
contrib/pageinspect/brinfuncs.c        | 22 ++++++++++++++++++++++
contrib/pageinspect/btreefuncs.c       |  6 ++++++
contrib/pageinspect/expected/brin.out  | 25 +++++++++++++++++++++++++
contrib/pageinspect/expected/btree.out |  6 ++++++
contrib/pageinspect/expected/gin.out   | 14 ++++++++++++++
contrib/pageinspect/expected/hash.out  | 12 ++++++++++++
contrib/pageinspect/expected/page.out  | 20 ++++++++++++++++++++
contrib/pageinspect/fsmfuncs.c         |  4 ++++
contrib/pageinspect/ginfuncs.c         | 12 ++++++++++++
contrib/pageinspect/rawpage.c          |  3 +++
contrib/pageinspect/sql/brin.sql       |  7 +++++++
contrib/pageinspect/sql/btree.sql      |  4 ++++
contrib/pageinspect/sql/gin.sql        |  6 ++++++
contrib/pageinspect/sql/hash.sql       |  7 +++++++
contrib/pageinspect/sql/page.sql       |  6 ++++++
15 files changed, 154 insertions(+)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: pageinspect: Fix handling of all-zero pages
@ 2022-04-14 06:11 Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2022-04-14 06:11 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

pageinspect: Fix handling of all-zero pages

Getting from get_raw_page() an all-zero page is considered as a valid
case by the buffer manager and it can happen for example when finding a
corrupted page with zero_damaged_pages enabled (using zero_damaged_pages
to look at corrupted pages happens), or after a crash when a relation
file is extended before any WAL for its new data is generated (before a
vacuum or autovacuum job comes in to do some cleanup).

However, all the functions of pageinspect, as of the index AMs (except
hash that has its own idea of new pages), heap, the FSM or the page
header have never worked with all-zero pages, causing various crashes
when going through the page internals.

This commit changes all the pageinspect functions to be compliant with
all-zero pages, where the choice is made to return NULL or no rows for
SRFs when finding a new page.  get_raw_page() still works the same way,
returning a batch of zeros in the bytea of the page retrieved.  A hard
error could be used but NULL, while more invasive, is useful when
scanning relation files in full to get a batch of results for a single
relation in one query.  Tests are added for all the code paths
impacted.

Reported-by: Daria Lepikhova
Author: Michael Paquier
Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru
Backpatch-through: 10

Branch
------
REL_12_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/5378d55cb2f38eed4a7423cd3aa16d9de80b5313

Modified Files
--------------
contrib/pageinspect/brinfuncs.c        | 22 ++++++++++++++++++++++
contrib/pageinspect/btreefuncs.c       |  6 ++++++
contrib/pageinspect/expected/brin.out  | 25 +++++++++++++++++++++++++
contrib/pageinspect/expected/btree.out |  6 ++++++
contrib/pageinspect/expected/gin.out   | 14 ++++++++++++++
contrib/pageinspect/expected/hash.out  | 12 ++++++++++++
contrib/pageinspect/expected/page.out  | 20 ++++++++++++++++++++
contrib/pageinspect/fsmfuncs.c         |  4 ++++
contrib/pageinspect/ginfuncs.c         | 12 ++++++++++++
contrib/pageinspect/rawpage.c          |  3 +++
contrib/pageinspect/sql/brin.sql       |  7 +++++++
contrib/pageinspect/sql/btree.sql      |  4 ++++
contrib/pageinspect/sql/gin.sql        |  6 ++++++
contrib/pageinspect/sql/hash.sql       |  7 +++++++
contrib/pageinspect/sql/page.sql       |  6 ++++++
15 files changed, 154 insertions(+)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: pageinspect: Fix handling of all-zero pages
@ 2022-04-14 06:11 Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2022-04-14 06:11 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

pageinspect: Fix handling of all-zero pages

Getting from get_raw_page() an all-zero page is considered as a valid
case by the buffer manager and it can happen for example when finding a
corrupted page with zero_damaged_pages enabled (using zero_damaged_pages
to look at corrupted pages happens), or after a crash when a relation
file is extended before any WAL for its new data is generated (before a
vacuum or autovacuum job comes in to do some cleanup).

However, all the functions of pageinspect, as of the index AMs (except
hash that has its own idea of new pages), heap, the FSM or the page
header have never worked with all-zero pages, causing various crashes
when going through the page internals.

This commit changes all the pageinspect functions to be compliant with
all-zero pages, where the choice is made to return NULL or no rows for
SRFs when finding a new page.  get_raw_page() still works the same way,
returning a batch of zeros in the bytea of the page retrieved.  A hard
error could be used but NULL, while more invasive, is useful when
scanning relation files in full to get a batch of results for a single
relation in one query.  Tests are added for all the code paths
impacted.

Reported-by: Daria Lepikhova
Author: Michael Paquier
Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru
Backpatch-through: 10

Branch
------
REL_11_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/79fed072ba5f96a8136053d20c8d74e0d59fe0e3

Modified Files
--------------
contrib/pageinspect/brinfuncs.c        | 22 ++++++++++++++++++++++
contrib/pageinspect/btreefuncs.c       |  6 ++++++
contrib/pageinspect/expected/brin.out  | 25 +++++++++++++++++++++++++
contrib/pageinspect/expected/btree.out |  6 ++++++
contrib/pageinspect/expected/gin.out   | 14 ++++++++++++++
contrib/pageinspect/expected/hash.out  | 12 ++++++++++++
contrib/pageinspect/expected/page.out  | 20 ++++++++++++++++++++
contrib/pageinspect/fsmfuncs.c         |  4 ++++
contrib/pageinspect/ginfuncs.c         | 12 ++++++++++++
contrib/pageinspect/rawpage.c          |  3 +++
contrib/pageinspect/sql/brin.sql       |  7 +++++++
contrib/pageinspect/sql/btree.sql      |  4 ++++
contrib/pageinspect/sql/gin.sql        |  6 ++++++
contrib/pageinspect/sql/hash.sql       |  7 +++++++
contrib/pageinspect/sql/page.sql       |  6 ++++++
15 files changed, 154 insertions(+)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: pageinspect: Fix handling of all-zero pages
@ 2022-04-14 06:11 Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2022-04-14 06:11 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

pageinspect: Fix handling of all-zero pages

Getting from get_raw_page() an all-zero page is considered as a valid
case by the buffer manager and it can happen for example when finding a
corrupted page with zero_damaged_pages enabled (using zero_damaged_pages
to look at corrupted pages happens), or after a crash when a relation
file is extended before any WAL for its new data is generated (before a
vacuum or autovacuum job comes in to do some cleanup).

However, all the functions of pageinspect, as of the index AMs (except
hash that has its own idea of new pages), heap, the FSM or the page
header have never worked with all-zero pages, causing various crashes
when going through the page internals.

This commit changes all the pageinspect functions to be compliant with
all-zero pages, where the choice is made to return NULL or no rows for
SRFs when finding a new page.  get_raw_page() still works the same way,
returning a batch of zeros in the bytea of the page retrieved.  A hard
error could be used but NULL, while more invasive, is useful when
scanning relation files in full to get a batch of results for a single
relation in one query.  Tests are added for all the code paths
impacted.

Reported-by: Daria Lepikhova
Author: Michael Paquier
Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru
Backpatch-through: 10

Branch
------
REL_10_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/12d32b7bc15183be063516732d7458c58e2d2156

Modified Files
--------------
contrib/pageinspect/brinfuncs.c        | 22 ++++++++++++++++++++++
contrib/pageinspect/btreefuncs.c       |  6 ++++++
contrib/pageinspect/expected/brin.out  | 25 +++++++++++++++++++++++++
contrib/pageinspect/expected/btree.out |  6 ++++++
contrib/pageinspect/expected/gin.out   | 14 ++++++++++++++
contrib/pageinspect/expected/hash.out  | 12 ++++++++++++
contrib/pageinspect/expected/page.out  | 20 ++++++++++++++++++++
contrib/pageinspect/fsmfuncs.c         |  4 ++++
contrib/pageinspect/ginfuncs.c         | 12 ++++++++++++
contrib/pageinspect/rawpage.c          |  3 +++
contrib/pageinspect/sql/brin.sql       |  7 +++++++
contrib/pageinspect/sql/btree.sql      |  4 ++++
contrib/pageinspect/sql/gin.sql        |  6 ++++++
contrib/pageinspect/sql/hash.sql       |  7 +++++++
contrib/pageinspect/sql/page.sql       |  6 ++++++
15 files changed, 154 insertions(+)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread


end of thread, other threads:[~2022-04-14 06:11 UTC | newest]

Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-04-14 06:11 pgsql: pageinspect: Fix handling of all-zero pages Michael Paquier <michael@paquier.xyz>
2022-04-14 06:11 pgsql: pageinspect: Fix handling of all-zero pages Michael Paquier <michael@paquier.xyz>
2022-04-14 06:11 pgsql: pageinspect: Fix handling of all-zero pages Michael Paquier <michael@paquier.xyz>
2022-04-14 06:11 pgsql: pageinspect: Fix handling of all-zero pages Michael Paquier <michael@paquier.xyz>
2022-04-14 06:11 pgsql: pageinspect: Fix handling of all-zero pages Michael Paquier <michael@paquier.xyz>
2022-04-14 06:11 pgsql: pageinspect: Fix handling of all-zero pages Michael Paquier <michael@paquier.xyz>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox