pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: intarray: Prevent out-of-bound memory reads with gist__int_ops
6+ messages / 1 participants
[nested] [flat]

* pgsql: intarray: Prevent out-of-bound memory reads with gist__int_ops
@ 2023-06-15 04:46  Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2023-06-15 04:46 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

intarray: Prevent out-of-bound memory reads with gist__int_ops

As gist__int_ops stands in intarray, it is possible to store GiST
entries for leaf pages that can cause corruptions when decompressed.
Leaf nodes are stored as decompressed all the time by the compression
method, and the decompression method should map with that, retrieving
the contents of the page without doing any decompression.  However, the
code authorized the insertion of leaf page data with a higher number of
array items than what can be supported, generating a NOTICE message to
inform about this matter (199 for a 8k page, for reference).  When
calling the decompression method, a decompression would be attempted on
this leaf node item but the contents should be retrieved as they are.

The NOTICE message generated when dealing with the compression of a leaf
page and too many elements in the input array for gist__int_ops has been
introduced by 08ee64e, removing the marker stored in the array to track
if this is actually a leaf node.  However, it also missed the fact that
the decompression path should do nothing for a leaf page.  Hence, as the
code stand, a too-large array would be stored as uncompressed but the
decompression path would attempt a decompression rather that retrieving
the contents as they are.

This leads to various problems.  First, even if 08ee64e tried to address
that, it is possible to do out-of-bound chunk writes with a large input
array, with the backend informing about that with WARNINGs.  On
decompression, retrieving the stored leaf data would lead to incorrect
memory reads, leading to crashes or even worse.

Perhaps somebody would be interested in expanding the number of array
items that can be handled in a leaf page for this operator in the
future, which would require revisiting the choice done in 08ee64e, but
based on the lack of reports about this problem since 2005 it does not
look so.  For now, this commit prevents the insertion of data for leaf
pages when using more array items that the code can handle on
decompression, switching the NOTICE message to an ERROR.  If one wishes
to use more array items, gist__intbig_ops is an optional choice.

While on it, use ERRCODE_PROGRAM_LIMIT_EXCEEDED as error code when a
limit is reached, because that's what the module is facing in such
cases.

Author: Ankit Kumar Pandey, Alexander Lakhin
Reviewed-by: Richard Guo, Michael Paquier
Discussion: https://postgr.es/m/796b65c3-57b7-bddf-b0d5-a8afafb8b627@gmail.com
Discussion: https://postgr.es/m/17888-f72930e6b5ce8c14@postgresql.org
Backpatch-through: 11

Branch
------
REL_15_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/4be308edee4495fe72075397efa0f6e0e0f9693b

Modified Files
--------------
contrib/intarray/_int_gist.c       | 12 ++++++++----
contrib/intarray/expected/_int.out |  2 ++
contrib/intarray/sql/_int.sql      |  2 ++
3 files changed, 12 insertions(+), 4 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: intarray: Prevent out-of-bound memory reads with gist__int_ops
@ 2023-06-15 04:46  Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2023-06-15 04:46 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

intarray: Prevent out-of-bound memory reads with gist__int_ops

As gist__int_ops stands in intarray, it is possible to store GiST
entries for leaf pages that can cause corruptions when decompressed.
Leaf nodes are stored as decompressed all the time by the compression
method, and the decompression method should map with that, retrieving
the contents of the page without doing any decompression.  However, the
code authorized the insertion of leaf page data with a higher number of
array items than what can be supported, generating a NOTICE message to
inform about this matter (199 for a 8k page, for reference).  When
calling the decompression method, a decompression would be attempted on
this leaf node item but the contents should be retrieved as they are.

The NOTICE message generated when dealing with the compression of a leaf
page and too many elements in the input array for gist__int_ops has been
introduced by 08ee64e, removing the marker stored in the array to track
if this is actually a leaf node.  However, it also missed the fact that
the decompression path should do nothing for a leaf page.  Hence, as the
code stand, a too-large array would be stored as uncompressed but the
decompression path would attempt a decompression rather that retrieving
the contents as they are.

This leads to various problems.  First, even if 08ee64e tried to address
that, it is possible to do out-of-bound chunk writes with a large input
array, with the backend informing about that with WARNINGs.  On
decompression, retrieving the stored leaf data would lead to incorrect
memory reads, leading to crashes or even worse.

Perhaps somebody would be interested in expanding the number of array
items that can be handled in a leaf page for this operator in the
future, which would require revisiting the choice done in 08ee64e, but
based on the lack of reports about this problem since 2005 it does not
look so.  For now, this commit prevents the insertion of data for leaf
pages when using more array items that the code can handle on
decompression, switching the NOTICE message to an ERROR.  If one wishes
to use more array items, gist__intbig_ops is an optional choice.

While on it, use ERRCODE_PROGRAM_LIMIT_EXCEEDED as error code when a
limit is reached, because that's what the module is facing in such
cases.

Author: Ankit Kumar Pandey, Alexander Lakhin
Reviewed-by: Richard Guo, Michael Paquier
Discussion: https://postgr.es/m/796b65c3-57b7-bddf-b0d5-a8afafb8b627@gmail.com
Discussion: https://postgr.es/m/17888-f72930e6b5ce8c14@postgresql.org
Backpatch-through: 11

Branch
------
REL_11_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/ab40b0395a75e3faf43e5a7d4b521b3bc1130d3a

Modified Files
--------------
contrib/intarray/_int_gist.c       | 12 ++++++++----
contrib/intarray/expected/_int.out |  2 ++
contrib/intarray/sql/_int.sql      |  2 ++
3 files changed, 12 insertions(+), 4 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: intarray: Prevent out-of-bound memory reads with gist__int_ops
@ 2023-06-15 04:46  Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2023-06-15 04:46 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

intarray: Prevent out-of-bound memory reads with gist__int_ops

As gist__int_ops stands in intarray, it is possible to store GiST
entries for leaf pages that can cause corruptions when decompressed.
Leaf nodes are stored as decompressed all the time by the compression
method, and the decompression method should map with that, retrieving
the contents of the page without doing any decompression.  However, the
code authorized the insertion of leaf page data with a higher number of
array items than what can be supported, generating a NOTICE message to
inform about this matter (199 for a 8k page, for reference).  When
calling the decompression method, a decompression would be attempted on
this leaf node item but the contents should be retrieved as they are.

The NOTICE message generated when dealing with the compression of a leaf
page and too many elements in the input array for gist__int_ops has been
introduced by 08ee64e, removing the marker stored in the array to track
if this is actually a leaf node.  However, it also missed the fact that
the decompression path should do nothing for a leaf page.  Hence, as the
code stand, a too-large array would be stored as uncompressed but the
decompression path would attempt a decompression rather that retrieving
the contents as they are.

This leads to various problems.  First, even if 08ee64e tried to address
that, it is possible to do out-of-bound chunk writes with a large input
array, with the backend informing about that with WARNINGs.  On
decompression, retrieving the stored leaf data would lead to incorrect
memory reads, leading to crashes or even worse.

Perhaps somebody would be interested in expanding the number of array
items that can be handled in a leaf page for this operator in the
future, which would require revisiting the choice done in 08ee64e, but
based on the lack of reports about this problem since 2005 it does not
look so.  For now, this commit prevents the insertion of data for leaf
pages when using more array items that the code can handle on
decompression, switching the NOTICE message to an ERROR.  If one wishes
to use more array items, gist__intbig_ops is an optional choice.

While on it, use ERRCODE_PROGRAM_LIMIT_EXCEEDED as error code when a
limit is reached, because that's what the module is facing in such
cases.

Author: Ankit Kumar Pandey, Alexander Lakhin
Reviewed-by: Richard Guo, Michael Paquier
Discussion: https://postgr.es/m/796b65c3-57b7-bddf-b0d5-a8afafb8b627@gmail.com
Discussion: https://postgr.es/m/17888-f72930e6b5ce8c14@postgresql.org
Backpatch-through: 11

Branch
------
REL_13_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/ae9aac64a38faee824a920efd2c5631175eb30af

Modified Files
--------------
contrib/intarray/_int_gist.c       | 12 ++++++++----
contrib/intarray/expected/_int.out |  2 ++
contrib/intarray/sql/_int.sql      |  2 ++
3 files changed, 12 insertions(+), 4 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: intarray: Prevent out-of-bound memory reads with gist__int_ops
@ 2023-06-15 04:46  Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2023-06-15 04:46 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

intarray: Prevent out-of-bound memory reads with gist__int_ops

As gist__int_ops stands in intarray, it is possible to store GiST
entries for leaf pages that can cause corruptions when decompressed.
Leaf nodes are stored as decompressed all the time by the compression
method, and the decompression method should map with that, retrieving
the contents of the page without doing any decompression.  However, the
code authorized the insertion of leaf page data with a higher number of
array items than what can be supported, generating a NOTICE message to
inform about this matter (199 for a 8k page, for reference).  When
calling the decompression method, a decompression would be attempted on
this leaf node item but the contents should be retrieved as they are.

The NOTICE message generated when dealing with the compression of a leaf
page and too many elements in the input array for gist__int_ops has been
introduced by 08ee64e, removing the marker stored in the array to track
if this is actually a leaf node.  However, it also missed the fact that
the decompression path should do nothing for a leaf page.  Hence, as the
code stand, a too-large array would be stored as uncompressed but the
decompression path would attempt a decompression rather that retrieving
the contents as they are.

This leads to various problems.  First, even if 08ee64e tried to address
that, it is possible to do out-of-bound chunk writes with a large input
array, with the backend informing about that with WARNINGs.  On
decompression, retrieving the stored leaf data would lead to incorrect
memory reads, leading to crashes or even worse.

Perhaps somebody would be interested in expanding the number of array
items that can be handled in a leaf page for this operator in the
future, which would require revisiting the choice done in 08ee64e, but
based on the lack of reports about this problem since 2005 it does not
look so.  For now, this commit prevents the insertion of data for leaf
pages when using more array items that the code can handle on
decompression, switching the NOTICE message to an ERROR.  If one wishes
to use more array items, gist__intbig_ops is an optional choice.

While on it, use ERRCODE_PROGRAM_LIMIT_EXCEEDED as error code when a
limit is reached, because that's what the module is facing in such
cases.

Author: Ankit Kumar Pandey, Alexander Lakhin
Reviewed-by: Richard Guo, Michael Paquier
Discussion: https://postgr.es/m/796b65c3-57b7-bddf-b0d5-a8afafb8b627@gmail.com
Discussion: https://postgr.es/m/17888-f72930e6b5ce8c14@postgresql.org
Backpatch-through: 11

Branch
------
REL_12_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/6ecc1c02ade5388b5bd5aa957787d405ed489cf2

Modified Files
--------------
contrib/intarray/_int_gist.c       | 12 ++++++++----
contrib/intarray/expected/_int.out |  2 ++
contrib/intarray/sql/_int.sql      |  2 ++
3 files changed, 12 insertions(+), 4 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: intarray: Prevent out-of-bound memory reads with gist__int_ops
@ 2023-06-15 04:46  Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2023-06-15 04:46 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

intarray: Prevent out-of-bound memory reads with gist__int_ops

As gist__int_ops stands in intarray, it is possible to store GiST
entries for leaf pages that can cause corruptions when decompressed.
Leaf nodes are stored as decompressed all the time by the compression
method, and the decompression method should map with that, retrieving
the contents of the page without doing any decompression.  However, the
code authorized the insertion of leaf page data with a higher number of
array items than what can be supported, generating a NOTICE message to
inform about this matter (199 for a 8k page, for reference).  When
calling the decompression method, a decompression would be attempted on
this leaf node item but the contents should be retrieved as they are.

The NOTICE message generated when dealing with the compression of a leaf
page and too many elements in the input array for gist__int_ops has been
introduced by 08ee64e, removing the marker stored in the array to track
if this is actually a leaf node.  However, it also missed the fact that
the decompression path should do nothing for a leaf page.  Hence, as the
code stand, a too-large array would be stored as uncompressed but the
decompression path would attempt a decompression rather that retrieving
the contents as they are.

This leads to various problems.  First, even if 08ee64e tried to address
that, it is possible to do out-of-bound chunk writes with a large input
array, with the backend informing about that with WARNINGs.  On
decompression, retrieving the stored leaf data would lead to incorrect
memory reads, leading to crashes or even worse.

Perhaps somebody would be interested in expanding the number of array
items that can be handled in a leaf page for this operator in the
future, which would require revisiting the choice done in 08ee64e, but
based on the lack of reports about this problem since 2005 it does not
look so.  For now, this commit prevents the insertion of data for leaf
pages when using more array items that the code can handle on
decompression, switching the NOTICE message to an ERROR.  If one wishes
to use more array items, gist__intbig_ops is an optional choice.

While on it, use ERRCODE_PROGRAM_LIMIT_EXCEEDED as error code when a
limit is reached, because that's what the module is facing in such
cases.

Author: Ankit Kumar Pandey, Alexander Lakhin
Reviewed-by: Richard Guo, Michael Paquier
Discussion: https://postgr.es/m/796b65c3-57b7-bddf-b0d5-a8afafb8b627@gmail.com
Discussion: https://postgr.es/m/17888-f72930e6b5ce8c14@postgresql.org
Backpatch-through: 11

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/c00fbe89dc51c13eae1ca71460bcddfc3dde5081

Modified Files
--------------
contrib/intarray/_int_gist.c       | 12 ++++++++----
contrib/intarray/expected/_int.out |  2 ++
contrib/intarray/sql/_int.sql      |  2 ++
3 files changed, 12 insertions(+), 4 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: intarray: Prevent out-of-bound memory reads with gist__int_ops
@ 2023-06-15 04:46  Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2023-06-15 04:46 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

intarray: Prevent out-of-bound memory reads with gist__int_ops

As gist__int_ops stands in intarray, it is possible to store GiST
entries for leaf pages that can cause corruptions when decompressed.
Leaf nodes are stored as decompressed all the time by the compression
method, and the decompression method should map with that, retrieving
the contents of the page without doing any decompression.  However, the
code authorized the insertion of leaf page data with a higher number of
array items than what can be supported, generating a NOTICE message to
inform about this matter (199 for a 8k page, for reference).  When
calling the decompression method, a decompression would be attempted on
this leaf node item but the contents should be retrieved as they are.

The NOTICE message generated when dealing with the compression of a leaf
page and too many elements in the input array for gist__int_ops has been
introduced by 08ee64e, removing the marker stored in the array to track
if this is actually a leaf node.  However, it also missed the fact that
the decompression path should do nothing for a leaf page.  Hence, as the
code stand, a too-large array would be stored as uncompressed but the
decompression path would attempt a decompression rather that retrieving
the contents as they are.

This leads to various problems.  First, even if 08ee64e tried to address
that, it is possible to do out-of-bound chunk writes with a large input
array, with the backend informing about that with WARNINGs.  On
decompression, retrieving the stored leaf data would lead to incorrect
memory reads, leading to crashes or even worse.

Perhaps somebody would be interested in expanding the number of array
items that can be handled in a leaf page for this operator in the
future, which would require revisiting the choice done in 08ee64e, but
based on the lack of reports about this problem since 2005 it does not
look so.  For now, this commit prevents the insertion of data for leaf
pages when using more array items that the code can handle on
decompression, switching the NOTICE message to an ERROR.  If one wishes
to use more array items, gist__intbig_ops is an optional choice.

While on it, use ERRCODE_PROGRAM_LIMIT_EXCEEDED as error code when a
limit is reached, because that's what the module is facing in such
cases.

Author: Ankit Kumar Pandey, Alexander Lakhin
Reviewed-by: Richard Guo, Michael Paquier
Discussion: https://postgr.es/m/796b65c3-57b7-bddf-b0d5-a8afafb8b627@gmail.com
Discussion: https://postgr.es/m/17888-f72930e6b5ce8c14@postgresql.org
Backpatch-through: 11

Branch
------
REL_14_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/019a40d61959cf9f35795bcd1bc1f3d8a15476c1

Modified Files
--------------
contrib/intarray/_int_gist.c       | 12 ++++++++----
contrib/intarray/expected/_int.out |  2 ++
contrib/intarray/sql/_int.sql      |  2 ++
3 files changed, 12 insertions(+), 4 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread


end of thread, other threads:[~2023-06-15 04:46 UTC | newest]

Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-06-15 04:46 pgsql: intarray: Prevent out-of-bound memory reads with gist__int_ops Michael Paquier <michael@paquier.xyz>
2023-06-15 04:46 pgsql: intarray: Prevent out-of-bound memory reads with gist__int_ops Michael Paquier <michael@paquier.xyz>
2023-06-15 04:46 pgsql: intarray: Prevent out-of-bound memory reads with gist__int_ops Michael Paquier <michael@paquier.xyz>
2023-06-15 04:46 pgsql: intarray: Prevent out-of-bound memory reads with gist__int_ops Michael Paquier <michael@paquier.xyz>
2023-06-15 04:46 pgsql: intarray: Prevent out-of-bound memory reads with gist__int_ops Michael Paquier <michael@paquier.xyz>
2023-06-15 04:46 pgsql: intarray: Prevent out-of-bound memory reads with gist__int_ops Michael Paquier <michael@paquier.xyz>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox