pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: pageinspect: Fix handling of all-zero pages 6+ messages / 1 participants [nested] [flat]
* pgsql: pageinspect: Fix handling of all-zero pages @ 2022-04-14 06:11 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2022-04-14 06:11 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org pageinspect: Fix handling of all-zero pages Getting from get_raw_page() an all-zero page is considered as a valid case by the buffer manager and it can happen for example when finding a corrupted page with zero_damaged_pages enabled (using zero_damaged_pages to look at corrupted pages happens), or after a crash when a relation file is extended before any WAL for its new data is generated (before a vacuum or autovacuum job comes in to do some cleanup). However, all the functions of pageinspect, as of the index AMs (except hash that has its own idea of new pages), heap, the FSM or the page header have never worked with all-zero pages, causing various crashes when going through the page internals. This commit changes all the pageinspect functions to be compliant with all-zero pages, where the choice is made to return NULL or no rows for SRFs when finding a new page. get_raw_page() still works the same way, returning a batch of zeros in the bytea of the page retrieved. A hard error could be used but NULL, while more invasive, is useful when scanning relation files in full to get a batch of results for a single relation in one query. Tests are added for all the code paths impacted. Reported-by: Daria Lepikhova Author: Michael Paquier Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru Backpatch-through: 10 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/cd4868a5700fadf5a840d44686658517433b338c Modified Files -------------- contrib/pageinspect/brinfuncs.c | 22 ++++++++++++++++++++++ contrib/pageinspect/btreefuncs.c | 6 ++++++ contrib/pageinspect/expected/brin.out | 25 +++++++++++++++++++++++++ contrib/pageinspect/expected/btree.out | 6 ++++++ contrib/pageinspect/expected/gin.out | 14 ++++++++++++++ contrib/pageinspect/expected/gist.out | 18 ++++++++++++++++++ contrib/pageinspect/expected/hash.out | 12 ++++++++++++ contrib/pageinspect/expected/page.out | 20 ++++++++++++++++++++ contrib/pageinspect/fsmfuncs.c | 4 ++++ contrib/pageinspect/ginfuncs.c | 13 +++++++++++++ contrib/pageinspect/gistfuncs.c | 12 ++++++++++++ contrib/pageinspect/rawpage.c | 3 +++ contrib/pageinspect/sql/brin.sql | 7 +++++++ contrib/pageinspect/sql/btree.sql | 4 ++++ contrib/pageinspect/sql/gin.sql | 6 ++++++ contrib/pageinspect/sql/gist.sql | 6 ++++++ contrib/pageinspect/sql/hash.sql | 7 +++++++ contrib/pageinspect/sql/page.sql | 6 ++++++ 18 files changed, 191 insertions(+) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: pageinspect: Fix handling of all-zero pages @ 2022-04-14 06:11 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2022-04-14 06:11 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org pageinspect: Fix handling of all-zero pages Getting from get_raw_page() an all-zero page is considered as a valid case by the buffer manager and it can happen for example when finding a corrupted page with zero_damaged_pages enabled (using zero_damaged_pages to look at corrupted pages happens), or after a crash when a relation file is extended before any WAL for its new data is generated (before a vacuum or autovacuum job comes in to do some cleanup). However, all the functions of pageinspect, as of the index AMs (except hash that has its own idea of new pages), heap, the FSM or the page header have never worked with all-zero pages, causing various crashes when going through the page internals. This commit changes all the pageinspect functions to be compliant with all-zero pages, where the choice is made to return NULL or no rows for SRFs when finding a new page. get_raw_page() still works the same way, returning a batch of zeros in the bytea of the page retrieved. A hard error could be used but NULL, while more invasive, is useful when scanning relation files in full to get a batch of results for a single relation in one query. Tests are added for all the code paths impacted. Reported-by: Daria Lepikhova Author: Michael Paquier Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru Backpatch-through: 10 Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/df6bbe73b86cda28472b9ecc61ac2deff5b2837b Modified Files -------------- contrib/pageinspect/brinfuncs.c | 22 ++++++++++++++++++++++ contrib/pageinspect/btreefuncs.c | 6 ++++++ contrib/pageinspect/expected/brin.out | 25 +++++++++++++++++++++++++ contrib/pageinspect/expected/btree.out | 6 ++++++ contrib/pageinspect/expected/gin.out | 14 ++++++++++++++ contrib/pageinspect/expected/gist.out | 18 ++++++++++++++++++ contrib/pageinspect/expected/hash.out | 12 ++++++++++++ contrib/pageinspect/expected/page.out | 20 ++++++++++++++++++++ contrib/pageinspect/fsmfuncs.c | 4 ++++ contrib/pageinspect/ginfuncs.c | 12 ++++++++++++ contrib/pageinspect/gistfuncs.c | 12 ++++++++++++ contrib/pageinspect/rawpage.c | 3 +++ contrib/pageinspect/sql/brin.sql | 7 +++++++ contrib/pageinspect/sql/btree.sql | 4 ++++ contrib/pageinspect/sql/gin.sql | 6 ++++++ contrib/pageinspect/sql/gist.sql | 6 ++++++ contrib/pageinspect/sql/hash.sql | 7 +++++++ contrib/pageinspect/sql/page.sql | 6 ++++++ 18 files changed, 190 insertions(+) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: pageinspect: Fix handling of all-zero pages @ 2022-04-14 06:11 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2022-04-14 06:11 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org pageinspect: Fix handling of all-zero pages Getting from get_raw_page() an all-zero page is considered as a valid case by the buffer manager and it can happen for example when finding a corrupted page with zero_damaged_pages enabled (using zero_damaged_pages to look at corrupted pages happens), or after a crash when a relation file is extended before any WAL for its new data is generated (before a vacuum or autovacuum job comes in to do some cleanup). However, all the functions of pageinspect, as of the index AMs (except hash that has its own idea of new pages), heap, the FSM or the page header have never worked with all-zero pages, causing various crashes when going through the page internals. This commit changes all the pageinspect functions to be compliant with all-zero pages, where the choice is made to return NULL or no rows for SRFs when finding a new page. get_raw_page() still works the same way, returning a batch of zeros in the bytea of the page retrieved. A hard error could be used but NULL, while more invasive, is useful when scanning relation files in full to get a batch of results for a single relation in one query. Tests are added for all the code paths impacted. Reported-by: Daria Lepikhova Author: Michael Paquier Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru Backpatch-through: 10 Branch ------ REL_13_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/2275d044d0849251c5e0fa10ee16c73124731f5c Modified Files -------------- contrib/pageinspect/brinfuncs.c | 22 ++++++++++++++++++++++ contrib/pageinspect/btreefuncs.c | 6 ++++++ contrib/pageinspect/expected/brin.out | 25 +++++++++++++++++++++++++ contrib/pageinspect/expected/btree.out | 6 ++++++ contrib/pageinspect/expected/gin.out | 14 ++++++++++++++ contrib/pageinspect/expected/hash.out | 12 ++++++++++++ contrib/pageinspect/expected/page.out | 20 ++++++++++++++++++++ contrib/pageinspect/fsmfuncs.c | 4 ++++ contrib/pageinspect/ginfuncs.c | 12 ++++++++++++ contrib/pageinspect/rawpage.c | 3 +++ contrib/pageinspect/sql/brin.sql | 7 +++++++ contrib/pageinspect/sql/btree.sql | 4 ++++ contrib/pageinspect/sql/gin.sql | 6 ++++++ contrib/pageinspect/sql/hash.sql | 7 +++++++ contrib/pageinspect/sql/page.sql | 6 ++++++ 15 files changed, 154 insertions(+) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: pageinspect: Fix handling of all-zero pages @ 2022-04-14 06:11 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2022-04-14 06:11 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org pageinspect: Fix handling of all-zero pages Getting from get_raw_page() an all-zero page is considered as a valid case by the buffer manager and it can happen for example when finding a corrupted page with zero_damaged_pages enabled (using zero_damaged_pages to look at corrupted pages happens), or after a crash when a relation file is extended before any WAL for its new data is generated (before a vacuum or autovacuum job comes in to do some cleanup). However, all the functions of pageinspect, as of the index AMs (except hash that has its own idea of new pages), heap, the FSM or the page header have never worked with all-zero pages, causing various crashes when going through the page internals. This commit changes all the pageinspect functions to be compliant with all-zero pages, where the choice is made to return NULL or no rows for SRFs when finding a new page. get_raw_page() still works the same way, returning a batch of zeros in the bytea of the page retrieved. A hard error could be used but NULL, while more invasive, is useful when scanning relation files in full to get a batch of results for a single relation in one query. Tests are added for all the code paths impacted. Reported-by: Daria Lepikhova Author: Michael Paquier Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru Backpatch-through: 10 Branch ------ REL_12_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/5378d55cb2f38eed4a7423cd3aa16d9de80b5313 Modified Files -------------- contrib/pageinspect/brinfuncs.c | 22 ++++++++++++++++++++++ contrib/pageinspect/btreefuncs.c | 6 ++++++ contrib/pageinspect/expected/brin.out | 25 +++++++++++++++++++++++++ contrib/pageinspect/expected/btree.out | 6 ++++++ contrib/pageinspect/expected/gin.out | 14 ++++++++++++++ contrib/pageinspect/expected/hash.out | 12 ++++++++++++ contrib/pageinspect/expected/page.out | 20 ++++++++++++++++++++ contrib/pageinspect/fsmfuncs.c | 4 ++++ contrib/pageinspect/ginfuncs.c | 12 ++++++++++++ contrib/pageinspect/rawpage.c | 3 +++ contrib/pageinspect/sql/brin.sql | 7 +++++++ contrib/pageinspect/sql/btree.sql | 4 ++++ contrib/pageinspect/sql/gin.sql | 6 ++++++ contrib/pageinspect/sql/hash.sql | 7 +++++++ contrib/pageinspect/sql/page.sql | 6 ++++++ 15 files changed, 154 insertions(+) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: pageinspect: Fix handling of all-zero pages @ 2022-04-14 06:11 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2022-04-14 06:11 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org pageinspect: Fix handling of all-zero pages Getting from get_raw_page() an all-zero page is considered as a valid case by the buffer manager and it can happen for example when finding a corrupted page with zero_damaged_pages enabled (using zero_damaged_pages to look at corrupted pages happens), or after a crash when a relation file is extended before any WAL for its new data is generated (before a vacuum or autovacuum job comes in to do some cleanup). However, all the functions of pageinspect, as of the index AMs (except hash that has its own idea of new pages), heap, the FSM or the page header have never worked with all-zero pages, causing various crashes when going through the page internals. This commit changes all the pageinspect functions to be compliant with all-zero pages, where the choice is made to return NULL or no rows for SRFs when finding a new page. get_raw_page() still works the same way, returning a batch of zeros in the bytea of the page retrieved. A hard error could be used but NULL, while more invasive, is useful when scanning relation files in full to get a batch of results for a single relation in one query. Tests are added for all the code paths impacted. Reported-by: Daria Lepikhova Author: Michael Paquier Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru Backpatch-through: 10 Branch ------ REL_11_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/79fed072ba5f96a8136053d20c8d74e0d59fe0e3 Modified Files -------------- contrib/pageinspect/brinfuncs.c | 22 ++++++++++++++++++++++ contrib/pageinspect/btreefuncs.c | 6 ++++++ contrib/pageinspect/expected/brin.out | 25 +++++++++++++++++++++++++ contrib/pageinspect/expected/btree.out | 6 ++++++ contrib/pageinspect/expected/gin.out | 14 ++++++++++++++ contrib/pageinspect/expected/hash.out | 12 ++++++++++++ contrib/pageinspect/expected/page.out | 20 ++++++++++++++++++++ contrib/pageinspect/fsmfuncs.c | 4 ++++ contrib/pageinspect/ginfuncs.c | 12 ++++++++++++ contrib/pageinspect/rawpage.c | 3 +++ contrib/pageinspect/sql/brin.sql | 7 +++++++ contrib/pageinspect/sql/btree.sql | 4 ++++ contrib/pageinspect/sql/gin.sql | 6 ++++++ contrib/pageinspect/sql/hash.sql | 7 +++++++ contrib/pageinspect/sql/page.sql | 6 ++++++ 15 files changed, 154 insertions(+) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: pageinspect: Fix handling of all-zero pages @ 2022-04-14 06:11 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2022-04-14 06:11 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org pageinspect: Fix handling of all-zero pages Getting from get_raw_page() an all-zero page is considered as a valid case by the buffer manager and it can happen for example when finding a corrupted page with zero_damaged_pages enabled (using zero_damaged_pages to look at corrupted pages happens), or after a crash when a relation file is extended before any WAL for its new data is generated (before a vacuum or autovacuum job comes in to do some cleanup). However, all the functions of pageinspect, as of the index AMs (except hash that has its own idea of new pages), heap, the FSM or the page header have never worked with all-zero pages, causing various crashes when going through the page internals. This commit changes all the pageinspect functions to be compliant with all-zero pages, where the choice is made to return NULL or no rows for SRFs when finding a new page. get_raw_page() still works the same way, returning a batch of zeros in the bytea of the page retrieved. A hard error could be used but NULL, while more invasive, is useful when scanning relation files in full to get a batch of results for a single relation in one query. Tests are added for all the code paths impacted. Reported-by: Daria Lepikhova Author: Michael Paquier Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru Backpatch-through: 10 Branch ------ REL_10_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/12d32b7bc15183be063516732d7458c58e2d2156 Modified Files -------------- contrib/pageinspect/brinfuncs.c | 22 ++++++++++++++++++++++ contrib/pageinspect/btreefuncs.c | 6 ++++++ contrib/pageinspect/expected/brin.out | 25 +++++++++++++++++++++++++ contrib/pageinspect/expected/btree.out | 6 ++++++ contrib/pageinspect/expected/gin.out | 14 ++++++++++++++ contrib/pageinspect/expected/hash.out | 12 ++++++++++++ contrib/pageinspect/expected/page.out | 20 ++++++++++++++++++++ contrib/pageinspect/fsmfuncs.c | 4 ++++ contrib/pageinspect/ginfuncs.c | 12 ++++++++++++ contrib/pageinspect/rawpage.c | 3 +++ contrib/pageinspect/sql/brin.sql | 7 +++++++ contrib/pageinspect/sql/btree.sql | 4 ++++ contrib/pageinspect/sql/gin.sql | 6 ++++++ contrib/pageinspect/sql/hash.sql | 7 +++++++ contrib/pageinspect/sql/page.sql | 6 ++++++ 15 files changed, 154 insertions(+) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2022-04-14 06:11 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2022-04-14 06:11 pgsql: pageinspect: Fix handling of all-zero pages Michael Paquier <michael@paquier.xyz> 2022-04-14 06:11 pgsql: pageinspect: Fix handling of all-zero pages Michael Paquier <michael@paquier.xyz> 2022-04-14 06:11 pgsql: pageinspect: Fix handling of all-zero pages Michael Paquier <michael@paquier.xyz> 2022-04-14 06:11 pgsql: pageinspect: Fix handling of all-zero pages Michael Paquier <michael@paquier.xyz> 2022-04-14 06:11 pgsql: pageinspect: Fix handling of all-zero pages Michael Paquier <michael@paquier.xyz> 2022-04-14 06:11 pgsql: pageinspect: Fix handling of all-zero pages Michael Paquier <michael@paquier.xyz>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox