pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: pageinspect: Fix handling of page sizes and AM types
8+ messages / 1 participants
[nested] [flat]
Commits from this thread:
2022-03-16 4477dcb207 dc5b3bdae3 028a3c6b1c 5cb7702101 09c97746d0 07e343513b pageinspect: Fix memory context allocation of page in brin_revmap_data()
* pgsql: pageinspect: Fix handling of page sizes and AM types
@ 2022-03-16 02:21 Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 8+ messages in thread
From: Michael Paquier @ 2022-03-16 02:21 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
pageinspect: Fix handling of page sizes and AM types
This commit fixes a set of issues related to the use of the SQL
functions in this module when the caller is able to pass down raw page
data as input argument:
- The page size check was fuzzy in a couple of places, sometimes
looking after only a sub-range, but what we are looking for is an exact
match on BLCKSZ. After considering a few options here, I have settled
down to do a generalization of get_page_from_raw(). Most of the SQL
functions already used that, and this is not strictly required if not
accessing an 8-byte-wide value from a raw page, but this feels safer in
the long run for alignment-picky environment, particularly if a code
path begins to access such values. This also reduces the number of
strings that need to be translated.
- The BRIN function brin_page_items() uses a Relation but it did not
check the access method of the opened index, potentially leading to
crashes. All the other functions in need of a Relation already did
that.
- Some code paths could fail on elog(), but we should to use ereport()
for failures that can be triggered by the user.
Tests are added to stress all the cases that are fixed as of this
commit, with some junk raw pages (\set VERBOSITY ensures that this works
across all page sizes) and unexpected index types when functions open
relations.
Author: Michael Paquier, Justin Prysby
Discussion: https://postgr.es/m/20220218030020.GA1137@telsasoft.com
Backpatch-through: 10
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/076f4d9539e9687d68ada32353c0c16d9bfa3cfb
Modified Files
--------------
contrib/pageinspect/brinfuncs.c | 36 ++++++++++++----------------------
contrib/pageinspect/btreefuncs.c | 28 ++++++++++++--------------
contrib/pageinspect/expected/brin.out | 4 ++++
contrib/pageinspect/expected/btree.out | 15 ++++++++++++++
contrib/pageinspect/expected/gin.out | 11 +++++++++++
contrib/pageinspect/expected/gist.out | 15 ++++++++++++++
contrib/pageinspect/expected/hash.out | 17 ++++++++++++++++
contrib/pageinspect/expected/page.out | 11 +++++++++++
contrib/pageinspect/fsmfuncs.c | 4 +++-
contrib/pageinspect/gistfuncs.c | 9 +++++++++
contrib/pageinspect/hashfuncs.c | 6 ++++--
contrib/pageinspect/rawpage.c | 29 +++------------------------
contrib/pageinspect/sql/brin.sql | 4 ++++
contrib/pageinspect/sql/btree.sql | 13 ++++++++++++
contrib/pageinspect/sql/gin.sql | 9 +++++++++
contrib/pageinspect/sql/gist.sql | 13 ++++++++++++
contrib/pageinspect/sql/hash.sql | 13 ++++++++++++
contrib/pageinspect/sql/page.sql | 9 +++++++++
18 files changed, 179 insertions(+), 67 deletions(-)
^ permalink raw reply [nested|flat] 8+ messages in thread
* pgsql: pageinspect: Fix handling of page sizes and AM types
@ 2022-03-16 02:21 Michael Paquier <michael@paquier.xyz>
0 siblings, 0 replies; 8+ messages in thread
From: Michael Paquier @ 2022-03-16 02:21 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
pageinspect: Fix handling of page sizes and AM types
This commit fixes a set of issues related to the use of the SQL
functions in this module when the caller is able to pass down raw page
data as input argument:
- The page size check was fuzzy in a couple of places, sometimes
looking after only a sub-range, but what we are looking for is an exact
match on BLCKSZ. After considering a few options here, I have settled
down to do a generalization of get_page_from_raw(). Most of the SQL
functions already used that, and this is not strictly required if not
accessing an 8-byte-wide value from a raw page, but this feels safer in
the long run for alignment-picky environment, particularly if a code
path begins to access such values. This also reduces the number of
strings that need to be translated.
- The BRIN function brin_page_items() uses a Relation but it did not
check the access method of the opened index, potentially leading to
crashes. All the other functions in need of a Relation already did
that.
- Some code paths could fail on elog(), but we should to use ereport()
for failures that can be triggered by the user.
Tests are added to stress all the cases that are fixed as of this
commit, with some junk raw pages (\set VERBOSITY ensures that this works
across all page sizes) and unexpected index types when functions open
relations.
Author: Michael Paquier, Justin Prysby
Discussion: https://postgr.es/m/20220218030020.GA1137@telsasoft.com
Backpatch-through: 10
Branch
------
REL_13_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/d3a9b83c30b62898f6823d127c6b6ddef98f7cba
Modified Files
--------------
contrib/pageinspect/brinfuncs.c | 36 ++++++++++++----------------------
contrib/pageinspect/btreefuncs.c | 28 ++++++++++++--------------
contrib/pageinspect/expected/brin.out | 4 ++++
contrib/pageinspect/expected/btree.out | 15 ++++++++++++++
contrib/pageinspect/expected/gin.out | 11 +++++++++++
contrib/pageinspect/expected/hash.out | 17 ++++++++++++++++
contrib/pageinspect/expected/page.out | 11 +++++++++++
contrib/pageinspect/fsmfuncs.c | 4 +++-
contrib/pageinspect/hashfuncs.c | 6 ++++--
contrib/pageinspect/rawpage.c | 29 +++------------------------
contrib/pageinspect/sql/brin.sql | 4 ++++
contrib/pageinspect/sql/btree.sql | 13 ++++++++++++
contrib/pageinspect/sql/gin.sql | 9 +++++++++
contrib/pageinspect/sql/hash.sql | 13 ++++++++++++
contrib/pageinspect/sql/page.sql | 9 +++++++++
15 files changed, 142 insertions(+), 67 deletions(-)
^ permalink raw reply [nested|flat] 8+ messages in thread
* pgsql: pageinspect: Fix handling of page sizes and AM types
@ 2022-03-16 02:21 Michael Paquier <michael@paquier.xyz>
0 siblings, 0 replies; 8+ messages in thread
From: Michael Paquier @ 2022-03-16 02:21 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
pageinspect: Fix handling of page sizes and AM types
This commit fixes a set of issues related to the use of the SQL
functions in this module when the caller is able to pass down raw page
data as input argument:
- The page size check was fuzzy in a couple of places, sometimes
looking after only a sub-range, but what we are looking for is an exact
match on BLCKSZ. After considering a few options here, I have settled
down to do a generalization of get_page_from_raw(). Most of the SQL
functions already used that, and this is not strictly required if not
accessing an 8-byte-wide value from a raw page, but this feels safer in
the long run for alignment-picky environment, particularly if a code
path begins to access such values. This also reduces the number of
strings that need to be translated.
- The BRIN function brin_page_items() uses a Relation but it did not
check the access method of the opened index, potentially leading to
crashes. All the other functions in need of a Relation already did
that.
- Some code paths could fail on elog(), but we should to use ereport()
for failures that can be triggered by the user.
Tests are added to stress all the cases that are fixed as of this
commit, with some junk raw pages (\set VERBOSITY ensures that this works
across all page sizes) and unexpected index types when functions open
relations.
Author: Michael Paquier, Justin Prysby
Discussion: https://postgr.es/m/20220218030020.GA1137@telsasoft.com
Backpatch-through: 10
Branch
------
REL_12_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/af8a8eb54245b2f24bbe1b93811fffbb71cca235
Modified Files
--------------
contrib/pageinspect/brinfuncs.c | 36 ++++++++++++----------------------
contrib/pageinspect/btreefuncs.c | 28 ++++++++++++--------------
contrib/pageinspect/expected/brin.out | 4 ++++
contrib/pageinspect/expected/btree.out | 15 ++++++++++++++
contrib/pageinspect/expected/gin.out | 11 +++++++++++
contrib/pageinspect/expected/hash.out | 17 ++++++++++++++++
contrib/pageinspect/expected/page.out | 11 +++++++++++
contrib/pageinspect/fsmfuncs.c | 4 +++-
contrib/pageinspect/hashfuncs.c | 6 ++++--
contrib/pageinspect/rawpage.c | 29 +++------------------------
contrib/pageinspect/sql/brin.sql | 4 ++++
contrib/pageinspect/sql/btree.sql | 13 ++++++++++++
contrib/pageinspect/sql/gin.sql | 9 +++++++++
contrib/pageinspect/sql/hash.sql | 13 ++++++++++++
contrib/pageinspect/sql/page.sql | 9 +++++++++
15 files changed, 142 insertions(+), 67 deletions(-)
^ permalink raw reply [nested|flat] 8+ messages in thread
* pgsql: pageinspect: Fix handling of page sizes and AM types
@ 2022-03-16 02:21 Michael Paquier <michael@paquier.xyz>
0 siblings, 0 replies; 8+ messages in thread
From: Michael Paquier @ 2022-03-16 02:21 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
pageinspect: Fix handling of page sizes and AM types
This commit fixes a set of issues related to the use of the SQL
functions in this module when the caller is able to pass down raw page
data as input argument:
- The page size check was fuzzy in a couple of places, sometimes
looking after only a sub-range, but what we are looking for is an exact
match on BLCKSZ. After considering a few options here, I have settled
down to do a generalization of get_page_from_raw(). Most of the SQL
functions already used that, and this is not strictly required if not
accessing an 8-byte-wide value from a raw page, but this feels safer in
the long run for alignment-picky environment, particularly if a code
path begins to access such values. This also reduces the number of
strings that need to be translated.
- The BRIN function brin_page_items() uses a Relation but it did not
check the access method of the opened index, potentially leading to
crashes. All the other functions in need of a Relation already did
that.
- Some code paths could fail on elog(), but we should to use ereport()
for failures that can be triggered by the user.
Tests are added to stress all the cases that are fixed as of this
commit, with some junk raw pages (\set VERBOSITY ensures that this works
across all page sizes) and unexpected index types when functions open
relations.
Author: Michael Paquier, Justin Prysby
Discussion: https://postgr.es/m/20220218030020.GA1137@telsasoft.com
Backpatch-through: 10
Branch
------
REL_14_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/b61e6214cface3def6241295edadf4eeaf4c884d
Modified Files
--------------
contrib/pageinspect/brinfuncs.c | 36 ++++++++++++----------------------
contrib/pageinspect/btreefuncs.c | 28 ++++++++++++--------------
contrib/pageinspect/expected/brin.out | 4 ++++
contrib/pageinspect/expected/btree.out | 15 ++++++++++++++
contrib/pageinspect/expected/gin.out | 11 +++++++++++
contrib/pageinspect/expected/gist.out | 15 ++++++++++++++
contrib/pageinspect/expected/hash.out | 17 ++++++++++++++++
contrib/pageinspect/expected/page.out | 11 +++++++++++
contrib/pageinspect/fsmfuncs.c | 4 +++-
contrib/pageinspect/gistfuncs.c | 9 +++++++++
contrib/pageinspect/hashfuncs.c | 6 ++++--
contrib/pageinspect/rawpage.c | 29 +++------------------------
contrib/pageinspect/sql/brin.sql | 4 ++++
contrib/pageinspect/sql/btree.sql | 13 ++++++++++++
contrib/pageinspect/sql/gin.sql | 9 +++++++++
contrib/pageinspect/sql/gist.sql | 13 ++++++++++++
contrib/pageinspect/sql/hash.sql | 13 ++++++++++++
contrib/pageinspect/sql/page.sql | 9 +++++++++
18 files changed, 179 insertions(+), 67 deletions(-)
^ permalink raw reply [nested|flat] 8+ messages in thread
* pgsql: pageinspect: Fix handling of page sizes and AM types
@ 2022-03-16 02:21 Michael Paquier <michael@paquier.xyz>
0 siblings, 0 replies; 8+ messages in thread
From: Michael Paquier @ 2022-03-16 02:21 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
pageinspect: Fix handling of page sizes and AM types
This commit fixes a set of issues related to the use of the SQL
functions in this module when the caller is able to pass down raw page
data as input argument:
- The page size check was fuzzy in a couple of places, sometimes
looking after only a sub-range, but what we are looking for is an exact
match on BLCKSZ. After considering a few options here, I have settled
down to do a generalization of get_page_from_raw(). Most of the SQL
functions already used that, and this is not strictly required if not
accessing an 8-byte-wide value from a raw page, but this feels safer in
the long run for alignment-picky environment, particularly if a code
path begins to access such values. This also reduces the number of
strings that need to be translated.
- The BRIN function brin_page_items() uses a Relation but it did not
check the access method of the opened index, potentially leading to
crashes. All the other functions in need of a Relation already did
that.
- Some code paths could fail on elog(), but we should to use ereport()
for failures that can be triggered by the user.
Tests are added to stress all the cases that are fixed as of this
commit, with some junk raw pages (\set VERBOSITY ensures that this works
across all page sizes) and unexpected index types when functions open
relations.
Author: Michael Paquier, Justin Prysby
Discussion: https://postgr.es/m/20220218030020.GA1137@telsasoft.com
Backpatch-through: 10
Branch
------
REL_10_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/c42b88932b9ddf6d4f6f509e62aec6d1f2e73111
Modified Files
--------------
contrib/pageinspect/brinfuncs.c | 36 ++++++++++++----------------------
contrib/pageinspect/btreefuncs.c | 28 ++++++++++++--------------
contrib/pageinspect/expected/brin.out | 4 ++++
contrib/pageinspect/expected/btree.out | 15 ++++++++++++++
contrib/pageinspect/expected/gin.out | 11 +++++++++++
contrib/pageinspect/expected/hash.out | 17 ++++++++++++++++
contrib/pageinspect/expected/page.out | 11 +++++++++++
contrib/pageinspect/fsmfuncs.c | 4 +++-
contrib/pageinspect/hashfuncs.c | 6 ++++--
contrib/pageinspect/rawpage.c | 29 +++------------------------
contrib/pageinspect/sql/brin.sql | 4 ++++
contrib/pageinspect/sql/btree.sql | 13 ++++++++++++
contrib/pageinspect/sql/gin.sql | 9 +++++++++
contrib/pageinspect/sql/hash.sql | 13 ++++++++++++
contrib/pageinspect/sql/page.sql | 9 +++++++++
15 files changed, 142 insertions(+), 67 deletions(-)
^ permalink raw reply [nested|flat] 8+ messages in thread
* pgsql: pageinspect: Fix handling of page sizes and AM types
@ 2022-03-16 02:21 Michael Paquier <michael@paquier.xyz>
0 siblings, 0 replies; 8+ messages in thread
From: Michael Paquier @ 2022-03-16 02:21 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
pageinspect: Fix handling of page sizes and AM types
This commit fixes a set of issues related to the use of the SQL
functions in this module when the caller is able to pass down raw page
data as input argument:
- The page size check was fuzzy in a couple of places, sometimes
looking after only a sub-range, but what we are looking for is an exact
match on BLCKSZ. After considering a few options here, I have settled
down to do a generalization of get_page_from_raw(). Most of the SQL
functions already used that, and this is not strictly required if not
accessing an 8-byte-wide value from a raw page, but this feels safer in
the long run for alignment-picky environment, particularly if a code
path begins to access such values. This also reduces the number of
strings that need to be translated.
- The BRIN function brin_page_items() uses a Relation but it did not
check the access method of the opened index, potentially leading to
crashes. All the other functions in need of a Relation already did
that.
- Some code paths could fail on elog(), but we should to use ereport()
for failures that can be triggered by the user.
Tests are added to stress all the cases that are fixed as of this
commit, with some junk raw pages (\set VERBOSITY ensures that this works
across all page sizes) and unexpected index types when functions open
relations.
Author: Michael Paquier, Justin Prysby
Discussion: https://postgr.es/m/20220218030020.GA1137@telsasoft.com
Backpatch-through: 10
Branch
------
REL_11_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/2389ee8dd800b7e95eb6ee8238e67db88c60d996
Modified Files
--------------
contrib/pageinspect/brinfuncs.c | 36 ++++++++++++----------------------
contrib/pageinspect/btreefuncs.c | 28 ++++++++++++--------------
contrib/pageinspect/expected/brin.out | 4 ++++
contrib/pageinspect/expected/btree.out | 15 ++++++++++++++
contrib/pageinspect/expected/gin.out | 11 +++++++++++
contrib/pageinspect/expected/hash.out | 17 ++++++++++++++++
contrib/pageinspect/expected/page.out | 11 +++++++++++
contrib/pageinspect/fsmfuncs.c | 4 +++-
contrib/pageinspect/hashfuncs.c | 6 ++++--
contrib/pageinspect/rawpage.c | 29 +++------------------------
contrib/pageinspect/sql/brin.sql | 4 ++++
contrib/pageinspect/sql/btree.sql | 13 ++++++++++++
contrib/pageinspect/sql/gin.sql | 9 +++++++++
contrib/pageinspect/sql/hash.sql | 13 ++++++++++++
contrib/pageinspect/sql/page.sql | 9 +++++++++
15 files changed, 142 insertions(+), 67 deletions(-)
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: pgsql: pageinspect: Fix handling of page sizes and AM types
@ 2022-03-16 02:45 Michael Paquier <michael@paquier.xyz>
parent: Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 8+ messages in thread
From: Michael Paquier @ 2022-03-16 02:45 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
On Wed, Mar 16, 2022 at 02:21:42AM +0000, Michael Paquier wrote:
> pageinspect: Fix handling of page sizes and AM types
sifaka and some other animals have been complaining here:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=sifaka&dt=2022-03-16%2002%3A24%3A28
I have reproduced the problem locally, just looking into why the page
alignment and/or the allocation change seem to matter here..
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: pgsql: pageinspect: Fix handling of page sizes and AM types
@ 2022-03-16 02:50 Michael Paquier <michael@paquier.xyz>
parent: Michael Paquier <michael@paquier.xyz>
0 siblings, 0 replies; 8+ messages in thread
From: Michael Paquier @ 2022-03-16 02:50 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
On Wed, Mar 16, 2022 at 11:45:14AM +0900, Michael Paquier wrote:
> I have reproduced the problem locally, just looking into why the page
> alignment and/or the allocation change seem to matter here..
Bleh. That's a simple oversight with the multi-call memory context
where the aligned copy of the page is not saved into the correct
context. Will fix shortly.
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2022-03-16 02:50 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-03-16 02:21 pgsql: pageinspect: Fix handling of page sizes and AM types Michael Paquier <michael@paquier.xyz>
2022-03-16 02:45 ` Michael Paquier <michael@paquier.xyz>
2022-03-16 02:50 ` Michael Paquier <michael@paquier.xyz>
2022-03-16 02:21 pgsql: pageinspect: Fix handling of page sizes and AM types Michael Paquier <michael@paquier.xyz>
2022-03-16 02:21 pgsql: pageinspect: Fix handling of page sizes and AM types Michael Paquier <michael@paquier.xyz>
2022-03-16 02:21 pgsql: pageinspect: Fix handling of page sizes and AM types Michael Paquier <michael@paquier.xyz>
2022-03-16 02:21 pgsql: pageinspect: Fix handling of page sizes and AM types Michael Paquier <michael@paquier.xyz>
2022-03-16 02:21 pgsql: pageinspect: Fix handling of page sizes and AM types Michael Paquier <michael@paquier.xyz>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox