pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Fix scale clamping in numeric round() and trunc(). 7+ messages / 1 participants [nested] [flat]
* pgsql: Fix scale clamping in numeric round() and trunc(). @ 2024-07-08 17:00 Dean Rasheed <dean.a.rasheed@gmail.com> 0 siblings, 0 replies; 7+ messages in thread From: Dean Rasheed @ 2024-07-08 17:00 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix scale clamping in numeric round() and trunc(). The numeric round() and trunc() functions clamp the scale argument to the range between +/- NUMERIC_MAX_RESULT_SCALE (2000), which is much smaller than the actual allowed range of type numeric. As a result, they return incorrect results when asked to round/truncate more than 2000 digits before or after the decimal point. Fix by using the correct upper and lower scale limits based on the actual allowed (and documented) range of type numeric. While at it, use the new NUMERIC_WEIGHT_MAX constant instead of SHRT_MAX in all other overflow checks, and fix a comment thinko in power_var() introduced by e54a758d24 -- the minimum value of ln_dweight is -NUMERIC_DSCALE_MAX (-16383), not -SHRT_MAX, though this doesn't affect the point being made in the comment, that the resulting local_rscale value may exceed NUMERIC_MAX_DISPLAY_SCALE (1000). Back-patch to all supported branches. Dean Rasheed, reviewed by Joel Jacobson. Discussion: https://postgr.es/m/CAEZATCXB%2BrDTuMjhK5ZxcouufigSc-X4tGJCBTMpZ3n%3DxxQuhg%40mail.gmail.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/1ff39f4ff2b68625cdc37076e5c8281a4eab7f4f Modified Files -------------- src/backend/utils/adt/numeric.c | 43 +++++++++----- src/test/regress/expected/numeric.out | 102 ++++++++++++++++++++++++++++++++++ src/test/regress/sql/numeric.sql | 25 +++++++++ 3 files changed, 157 insertions(+), 13 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix scale clamping in numeric round() and trunc(). @ 2024-07-08 17:00 Dean Rasheed <dean.a.rasheed@gmail.com> 0 siblings, 0 replies; 7+ messages in thread From: Dean Rasheed @ 2024-07-08 17:00 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix scale clamping in numeric round() and trunc(). The numeric round() and trunc() functions clamp the scale argument to the range between +/- NUMERIC_MAX_RESULT_SCALE (2000), which is much smaller than the actual allowed range of type numeric. As a result, they return incorrect results when asked to round/truncate more than 2000 digits before or after the decimal point. Fix by using the correct upper and lower scale limits based on the actual allowed (and documented) range of type numeric. While at it, use the new NUMERIC_WEIGHT_MAX constant instead of SHRT_MAX in all other overflow checks, and fix a comment thinko in power_var() introduced by e54a758d24 -- the minimum value of ln_dweight is -NUMERIC_DSCALE_MAX (-16383), not -SHRT_MAX, though this doesn't affect the point being made in the comment, that the resulting local_rscale value may exceed NUMERIC_MAX_DISPLAY_SCALE (1000). Back-patch to all supported branches. Dean Rasheed, reviewed by Joel Jacobson. Discussion: https://postgr.es/m/CAEZATCXB%2BrDTuMjhK5ZxcouufigSc-X4tGJCBTMpZ3n%3DxxQuhg%40mail.gmail.com Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/7a8977d2587f48c545acd9dab9c76ad24eb09eea Modified Files -------------- src/backend/utils/adt/numeric.c | 43 +++++++++----- src/test/regress/expected/numeric.out | 102 ++++++++++++++++++++++++++++++++++ src/test/regress/sql/numeric.sql | 25 +++++++++ 3 files changed, 157 insertions(+), 13 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix scale clamping in numeric round() and trunc(). @ 2024-07-08 17:00 Dean Rasheed <dean.a.rasheed@gmail.com> 0 siblings, 0 replies; 7+ messages in thread From: Dean Rasheed @ 2024-07-08 17:00 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix scale clamping in numeric round() and trunc(). The numeric round() and trunc() functions clamp the scale argument to the range between +/- NUMERIC_MAX_RESULT_SCALE (2000), which is much smaller than the actual allowed range of type numeric. As a result, they return incorrect results when asked to round/truncate more than 2000 digits before or after the decimal point. Fix by using the correct upper and lower scale limits based on the actual allowed (and documented) range of type numeric. While at it, use the new NUMERIC_WEIGHT_MAX constant instead of SHRT_MAX in all other overflow checks, and fix a comment thinko in power_var() introduced by e54a758d24 -- the minimum value of ln_dweight is -NUMERIC_DSCALE_MAX (-16383), not -SHRT_MAX, though this doesn't affect the point being made in the comment, that the resulting local_rscale value may exceed NUMERIC_MAX_DISPLAY_SCALE (1000). Back-patch to all supported branches. Dean Rasheed, reviewed by Joel Jacobson. Discussion: https://postgr.es/m/CAEZATCXB%2BrDTuMjhK5ZxcouufigSc-X4tGJCBTMpZ3n%3DxxQuhg%40mail.gmail.com Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/f7aec8c1df904888e6fc0e817e460c73aad4cbd2 Modified Files -------------- src/backend/utils/adt/numeric.c | 43 +++++++++----- src/test/regress/expected/numeric.out | 102 ++++++++++++++++++++++++++++++++++ src/test/regress/sql/numeric.sql | 25 +++++++++ 3 files changed, 157 insertions(+), 13 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix scale clamping in numeric round() and trunc(). @ 2024-07-08 17:00 Dean Rasheed <dean.a.rasheed@gmail.com> 0 siblings, 0 replies; 7+ messages in thread From: Dean Rasheed @ 2024-07-08 17:00 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix scale clamping in numeric round() and trunc(). The numeric round() and trunc() functions clamp the scale argument to the range between +/- NUMERIC_MAX_RESULT_SCALE (2000), which is much smaller than the actual allowed range of type numeric. As a result, they return incorrect results when asked to round/truncate more than 2000 digits before or after the decimal point. Fix by using the correct upper and lower scale limits based on the actual allowed (and documented) range of type numeric. While at it, use the new NUMERIC_WEIGHT_MAX constant instead of SHRT_MAX in all other overflow checks, and fix a comment thinko in power_var() introduced by e54a758d24 -- the minimum value of ln_dweight is -NUMERIC_DSCALE_MAX (-16383), not -SHRT_MAX, though this doesn't affect the point being made in the comment, that the resulting local_rscale value may exceed NUMERIC_MAX_DISPLAY_SCALE (1000). Back-patch to all supported branches. Dean Rasheed, reviewed by Joel Jacobson. Discussion: https://postgr.es/m/CAEZATCXB%2BrDTuMjhK5ZxcouufigSc-X4tGJCBTMpZ3n%3DxxQuhg%40mail.gmail.com Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/47ca912de02975458fca657435d558725dbd6b64 Modified Files -------------- src/backend/utils/adt/numeric.c | 35 +++++++++--- src/test/regress/expected/numeric.out | 102 ++++++++++++++++++++++++++++++++++ src/test/regress/sql/numeric.sql | 25 +++++++++ 3 files changed, 153 insertions(+), 9 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix scale clamping in numeric round() and trunc(). @ 2024-07-08 17:00 Dean Rasheed <dean.a.rasheed@gmail.com> 0 siblings, 0 replies; 7+ messages in thread From: Dean Rasheed @ 2024-07-08 17:00 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix scale clamping in numeric round() and trunc(). The numeric round() and trunc() functions clamp the scale argument to the range between +/- NUMERIC_MAX_RESULT_SCALE (2000), which is much smaller than the actual allowed range of type numeric. As a result, they return incorrect results when asked to round/truncate more than 2000 digits before or after the decimal point. Fix by using the correct upper and lower scale limits based on the actual allowed (and documented) range of type numeric. While at it, use the new NUMERIC_WEIGHT_MAX constant instead of SHRT_MAX in all other overflow checks, and fix a comment thinko in power_var() introduced by e54a758d24 -- the minimum value of ln_dweight is -NUMERIC_DSCALE_MAX (-16383), not -SHRT_MAX, though this doesn't affect the point being made in the comment, that the resulting local_rscale value may exceed NUMERIC_MAX_DISPLAY_SCALE (1000). Back-patch to all supported branches. Dean Rasheed, reviewed by Joel Jacobson. Discussion: https://postgr.es/m/CAEZATCXB%2BrDTuMjhK5ZxcouufigSc-X4tGJCBTMpZ3n%3DxxQuhg%40mail.gmail.com Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/a3c0124f69ffd2a299f099d4037ba8c6d2293aba Modified Files -------------- src/backend/utils/adt/numeric.c | 35 +++++++++--- src/test/regress/expected/numeric.out | 102 ++++++++++++++++++++++++++++++++++ src/test/regress/sql/numeric.sql | 25 +++++++++ 3 files changed, 153 insertions(+), 9 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix scale clamping in numeric round() and trunc(). @ 2024-07-08 17:00 Dean Rasheed <dean.a.rasheed@gmail.com> 0 siblings, 0 replies; 7+ messages in thread From: Dean Rasheed @ 2024-07-08 17:00 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix scale clamping in numeric round() and trunc(). The numeric round() and trunc() functions clamp the scale argument to the range between +/- NUMERIC_MAX_RESULT_SCALE (2000), which is much smaller than the actual allowed range of type numeric. As a result, they return incorrect results when asked to round/truncate more than 2000 digits before or after the decimal point. Fix by using the correct upper and lower scale limits based on the actual allowed (and documented) range of type numeric. While at it, use the new NUMERIC_WEIGHT_MAX constant instead of SHRT_MAX in all other overflow checks, and fix a comment thinko in power_var() introduced by e54a758d24 -- the minimum value of ln_dweight is -NUMERIC_DSCALE_MAX (-16383), not -SHRT_MAX, though this doesn't affect the point being made in the comment, that the resulting local_rscale value may exceed NUMERIC_MAX_DISPLAY_SCALE (1000). Back-patch to all supported branches. Dean Rasheed, reviewed by Joel Jacobson. Discussion: https://postgr.es/m/CAEZATCXB%2BrDTuMjhK5ZxcouufigSc-X4tGJCBTMpZ3n%3DxxQuhg%40mail.gmail.com Branch ------ REL_13_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/ece2969266d1f99f50e921ecd8d8754a6876a0dd Modified Files -------------- src/backend/utils/adt/numeric.c | 35 +++++++++--- src/test/regress/expected/numeric.out | 102 ++++++++++++++++++++++++++++++++++ src/test/regress/sql/numeric.sql | 25 +++++++++ 3 files changed, 153 insertions(+), 9 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix scale clamping in numeric round() and trunc(). @ 2024-07-08 17:00 Dean Rasheed <dean.a.rasheed@gmail.com> 0 siblings, 0 replies; 7+ messages in thread From: Dean Rasheed @ 2024-07-08 17:00 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix scale clamping in numeric round() and trunc(). The numeric round() and trunc() functions clamp the scale argument to the range between +/- NUMERIC_MAX_RESULT_SCALE (2000), which is much smaller than the actual allowed range of type numeric. As a result, they return incorrect results when asked to round/truncate more than 2000 digits before or after the decimal point. Fix by using the correct upper and lower scale limits based on the actual allowed (and documented) range of type numeric. While at it, use the new NUMERIC_WEIGHT_MAX constant instead of SHRT_MAX in all other overflow checks, and fix a comment thinko in power_var() introduced by e54a758d24 -- the minimum value of ln_dweight is -NUMERIC_DSCALE_MAX (-16383), not -SHRT_MAX, though this doesn't affect the point being made in the comment, that the resulting local_rscale value may exceed NUMERIC_MAX_DISPLAY_SCALE (1000). Back-patch to all supported branches. Dean Rasheed, reviewed by Joel Jacobson. Discussion: https://postgr.es/m/CAEZATCXB%2BrDTuMjhK5ZxcouufigSc-X4tGJCBTMpZ3n%3DxxQuhg%40mail.gmail.com Branch ------ REL_12_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/8badee7875c0eac9e2c4fd61f23eceaa64276b3f Modified Files -------------- src/backend/utils/adt/numeric.c | 35 +++++++++--- src/test/regress/expected/numeric.out | 102 ++++++++++++++++++++++++++++++++++ src/test/regress/sql/numeric.sql | 25 +++++++++ 3 files changed, 153 insertions(+), 9 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2024-07-08 17:00 UTC | newest] Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-07-08 17:00 pgsql: Fix scale clamping in numeric round() and trunc(). Dean Rasheed <dean.a.rasheed@gmail.com> 2024-07-08 17:00 pgsql: Fix scale clamping in numeric round() and trunc(). Dean Rasheed <dean.a.rasheed@gmail.com> 2024-07-08 17:00 pgsql: Fix scale clamping in numeric round() and trunc(). Dean Rasheed <dean.a.rasheed@gmail.com> 2024-07-08 17:00 pgsql: Fix scale clamping in numeric round() and trunc(). Dean Rasheed <dean.a.rasheed@gmail.com> 2024-07-08 17:00 pgsql: Fix scale clamping in numeric round() and trunc(). Dean Rasheed <dean.a.rasheed@gmail.com> 2024-07-08 17:00 pgsql: Fix scale clamping in numeric round() and trunc(). Dean Rasheed <dean.a.rasheed@gmail.com> 2024-07-08 17:00 pgsql: Fix scale clamping in numeric round() and trunc(). Dean Rasheed <dean.a.rasheed@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox