pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: Fix buffer overrun in unicode string normalization with empty in
6+ messages / 1 participants
[nested] [flat]

* pgsql: Fix buffer overrun in unicode string normalization with empty in
@ 2021-11-11 06:03 Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2021-11-11 06:03 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix buffer overrun in unicode string normalization with empty input

PostgreSQL 13 and newer versions are directly impacted by that through
the SQL function normalize(), which would cause a call of this function
to write one byte past its allocation if using in input an empty
string after recomposing the string with NFC and NFKC.  Older versions
(v10~v12) are not directly affected by this problem as the only code
path using normalization is SASLprep in SCRAM authentication that
forbids the case of an empty string, but let's make the code more robust
anyway there so as any out-of-core callers of this function are covered.

The solution chosen to fix this issue is simple, with the addition of a
fast-exit path if the decomposed string is found as empty.  This would
only happen for an empty string as at its lowest level a codepoint would
be decomposed as itself if it has no entry in the decomposition table or
if it has a decomposition size of 0.

Some tests are added to cover this issue in v13~.  Note that an empty
string has always been considered as normalized (grammar "IS NF[K]{C,D}
NORMALIZED", through the SQL function is_normalized()) for all the
operations allowed (NFC, NFD, NFKC and NFKD) since this feature has been
introduced as of 2991ac5.  This behavior is unchanged but some tests are
added in v13~ to check after that.

I have also checked "make normalization-check" in src/common/unicode/,
while on it (works in 13~, and breaks in older stable branches
independently of this commit).

The release notes should just mention this commit for v13~.

Reported-by: Matthijs van der Vleuten
Discussion: https://postgr.es/m/17277-0c527a373794e802@postgresql.org
Backpatch-through: 10

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/098c134556664d37b78ae87853a82f4a9d23a2c8

Modified Files
--------------
src/common/unicode_norm.c             |  4 ++++
src/test/regress/expected/unicode.out | 12 ++++++++++--
src/test/regress/sql/unicode.sql      |  4 +++-
3 files changed, 17 insertions(+), 3 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Fix buffer overrun in unicode string normalization with empty in
@ 2021-11-11 06:03 Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2021-11-11 06:03 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix buffer overrun in unicode string normalization with empty input

PostgreSQL 13 and newer versions are directly impacted by that through
the SQL function normalize(), which would cause a call of this function
to write one byte past its allocation if using in input an empty
string after recomposing the string with NFC and NFKC.  Older versions
(v10~v12) are not directly affected by this problem as the only code
path using normalization is SASLprep in SCRAM authentication that
forbids the case of an empty string, but let's make the code more robust
anyway there so as any out-of-core callers of this function are covered.

The solution chosen to fix this issue is simple, with the addition of a
fast-exit path if the decomposed string is found as empty.  This would
only happen for an empty string as at its lowest level a codepoint would
be decomposed as itself if it has no entry in the decomposition table or
if it has a decomposition size of 0.

Some tests are added to cover this issue in v13~.  Note that an empty
string has always been considered as normalized (grammar "IS NF[K]{C,D}
NORMALIZED", through the SQL function is_normalized()) for all the
operations allowed (NFC, NFD, NFKC and NFKD) since this feature has been
introduced as of 2991ac5.  This behavior is unchanged but some tests are
added in v13~ to check after that.

I have also checked "make normalization-check" in src/common/unicode/,
while on it (works in 13~, and breaks in older stable branches
independently of this commit).

The release notes should just mention this commit for v13~.

Reported-by: Matthijs van der Vleuten
Discussion: https://postgr.es/m/17277-0c527a373794e802@postgresql.org
Backpatch-through: 10

Branch
------
REL_14_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/b609db7155d9c94011a6e16d0e6a60d62e322694

Modified Files
--------------
src/common/unicode_norm.c             |  4 ++++
src/test/regress/expected/unicode.out | 12 ++++++++++--
src/test/regress/sql/unicode.sql      |  4 +++-
3 files changed, 17 insertions(+), 3 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Fix buffer overrun in unicode string normalization with empty in
@ 2021-11-11 06:03 Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2021-11-11 06:03 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix buffer overrun in unicode string normalization with empty input

PostgreSQL 13 and newer versions are directly impacted by that through
the SQL function normalize(), which would cause a call of this function
to write one byte past its allocation if using in input an empty
string after recomposing the string with NFC and NFKC.  Older versions
(v10~v12) are not directly affected by this problem as the only code
path using normalization is SASLprep in SCRAM authentication that
forbids the case of an empty string, but let's make the code more robust
anyway there so as any out-of-core callers of this function are covered.

The solution chosen to fix this issue is simple, with the addition of a
fast-exit path if the decomposed string is found as empty.  This would
only happen for an empty string as at its lowest level a codepoint would
be decomposed as itself if it has no entry in the decomposition table or
if it has a decomposition size of 0.

Some tests are added to cover this issue in v13~.  Note that an empty
string has always been considered as normalized (grammar "IS NF[K]{C,D}
NORMALIZED", through the SQL function is_normalized()) for all the
operations allowed (NFC, NFD, NFKC and NFKD) since this feature has been
introduced as of 2991ac5.  This behavior is unchanged but some tests are
added in v13~ to check after that.

I have also checked "make normalization-check" in src/common/unicode/,
while on it (works in 13~, and breaks in older stable branches
independently of this commit).

The release notes should just mention this commit for v13~.

Reported-by: Matthijs van der Vleuten
Discussion: https://postgr.es/m/17277-0c527a373794e802@postgresql.org
Backpatch-through: 10

Branch
------
REL_13_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/13c8adf90e9d9bc58209ab820775949336d901f7

Modified Files
--------------
src/common/unicode_norm.c             |  4 ++++
src/test/regress/expected/unicode.out | 12 ++++++++++--
src/test/regress/sql/unicode.sql      |  4 +++-
3 files changed, 17 insertions(+), 3 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Fix buffer overrun in unicode string normalization with empty in
@ 2021-11-11 06:03 Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2021-11-11 06:03 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix buffer overrun in unicode string normalization with empty input

PostgreSQL 13 and newer versions are directly impacted by that through
the SQL function normalize(), which would cause a call of this function
to write one byte past its allocation if using in input an empty
string after recomposing the string with NFC and NFKC.  Older versions
(v10~v12) are not directly affected by this problem as the only code
path using normalization is SASLprep in SCRAM authentication that
forbids the case of an empty string, but let's make the code more robust
anyway there so as any out-of-core callers of this function are covered.

The solution chosen to fix this issue is simple, with the addition of a
fast-exit path if the decomposed string is found as empty.  This would
only happen for an empty string as at its lowest level a codepoint would
be decomposed as itself if it has no entry in the decomposition table or
if it has a decomposition size of 0.

Some tests are added to cover this issue in v13~.  Note that an empty
string has always been considered as normalized (grammar "IS NF[K]{C,D}
NORMALIZED", through the SQL function is_normalized()) for all the
operations allowed (NFC, NFD, NFKC and NFKD) since this feature has been
introduced as of 2991ac5.  This behavior is unchanged but some tests are
added in v13~ to check after that.

I have also checked "make normalization-check" in src/common/unicode/,
while on it (works in 13~, and breaks in older stable branches
independently of this commit).

The release notes should just mention this commit for v13~.

Reported-by: Matthijs van der Vleuten
Discussion: https://postgr.es/m/17277-0c527a373794e802@postgresql.org
Backpatch-through: 10

Branch
------
REL_12_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/de343812b97a0e46388517f80bdd9c403e81b3bd

Modified Files
--------------
src/common/unicode_norm.c | 4 ++++
1 file changed, 4 insertions(+)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Fix buffer overrun in unicode string normalization with empty in
@ 2021-11-11 06:03 Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2021-11-11 06:03 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix buffer overrun in unicode string normalization with empty input

PostgreSQL 13 and newer versions are directly impacted by that through
the SQL function normalize(), which would cause a call of this function
to write one byte past its allocation if using in input an empty
string after recomposing the string with NFC and NFKC.  Older versions
(v10~v12) are not directly affected by this problem as the only code
path using normalization is SASLprep in SCRAM authentication that
forbids the case of an empty string, but let's make the code more robust
anyway there so as any out-of-core callers of this function are covered.

The solution chosen to fix this issue is simple, with the addition of a
fast-exit path if the decomposed string is found as empty.  This would
only happen for an empty string as at its lowest level a codepoint would
be decomposed as itself if it has no entry in the decomposition table or
if it has a decomposition size of 0.

Some tests are added to cover this issue in v13~.  Note that an empty
string has always been considered as normalized (grammar "IS NF[K]{C,D}
NORMALIZED", through the SQL function is_normalized()) for all the
operations allowed (NFC, NFD, NFKC and NFKD) since this feature has been
introduced as of 2991ac5.  This behavior is unchanged but some tests are
added in v13~ to check after that.

I have also checked "make normalization-check" in src/common/unicode/,
while on it (works in 13~, and breaks in older stable branches
independently of this commit).

The release notes should just mention this commit for v13~.

Reported-by: Matthijs van der Vleuten
Discussion: https://postgr.es/m/17277-0c527a373794e802@postgresql.org
Backpatch-through: 10

Branch
------
REL_11_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/56c5a069e04b99835b6c179c37109ad4ff7ab236

Modified Files
--------------
src/common/unicode_norm.c | 4 ++++
1 file changed, 4 insertions(+)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Fix buffer overrun in unicode string normalization with empty in
@ 2021-11-11 06:03 Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2021-11-11 06:03 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix buffer overrun in unicode string normalization with empty input

PostgreSQL 13 and newer versions are directly impacted by that through
the SQL function normalize(), which would cause a call of this function
to write one byte past its allocation if using in input an empty
string after recomposing the string with NFC and NFKC.  Older versions
(v10~v12) are not directly affected by this problem as the only code
path using normalization is SASLprep in SCRAM authentication that
forbids the case of an empty string, but let's make the code more robust
anyway there so as any out-of-core callers of this function are covered.

The solution chosen to fix this issue is simple, with the addition of a
fast-exit path if the decomposed string is found as empty.  This would
only happen for an empty string as at its lowest level a codepoint would
be decomposed as itself if it has no entry in the decomposition table or
if it has a decomposition size of 0.

Some tests are added to cover this issue in v13~.  Note that an empty
string has always been considered as normalized (grammar "IS NF[K]{C,D}
NORMALIZED", through the SQL function is_normalized()) for all the
operations allowed (NFC, NFD, NFKC and NFKD) since this feature has been
introduced as of 2991ac5.  This behavior is unchanged but some tests are
added in v13~ to check after that.

I have also checked "make normalization-check" in src/common/unicode/,
while on it (works in 13~, and breaks in older stable branches
independently of this commit).

The release notes should just mention this commit for v13~.

Reported-by: Matthijs van der Vleuten
Discussion: https://postgr.es/m/17277-0c527a373794e802@postgresql.org
Backpatch-through: 10

Branch
------
REL_10_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/7653f6f5f53f146b140b507b8651d107ff6957d0

Modified Files
--------------
src/common/unicode_norm.c | 4 ++++
1 file changed, 4 insertions(+)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread


end of thread, other threads:[~2021-11-11 06:03 UTC | newest]

Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-11-11 06:03 pgsql: Fix buffer overrun in unicode string normalization with empty in Michael Paquier <michael@paquier.xyz>
2021-11-11 06:03 pgsql: Fix buffer overrun in unicode string normalization with empty in Michael Paquier <michael@paquier.xyz>
2021-11-11 06:03 pgsql: Fix buffer overrun in unicode string normalization with empty in Michael Paquier <michael@paquier.xyz>
2021-11-11 06:03 pgsql: Fix buffer overrun in unicode string normalization with empty in Michael Paquier <michael@paquier.xyz>
2021-11-11 06:03 pgsql: Fix buffer overrun in unicode string normalization with empty in Michael Paquier <michael@paquier.xyz>
2021-11-11 06:03 pgsql: Fix buffer overrun in unicode string normalization with empty in Michael Paquier <michael@paquier.xyz>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox