From: Dean Rasheed <dean.a.rasheed@gmail.com>
To: pgsql-committers@postgresql.org
Subject: pgsql: Fix corner-case loss of precision in numeric pow() calculation
Date: Thu, 05 May 2016 10:19:42 +0000
Message-ID: <E1ayGNi-0002bk-Lv@gemulon.postgresql.org> (raw)
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-committers>
Fix corner-case loss of precision in numeric pow() calculation
Commit 7d9a4737c268f61fb8800957631f12d3f13be218 greatly improved the
accuracy of the numeric transcendental functions, however it failed to
consider the case where the result from pow() is close to the overflow
threshold, for example 0.12 ^ -2345.6. For such inputs, where the
result has more than 2000 digits before the decimal point, the decimal
result weight estimate was being clamped to 2000, leading to a loss of
precision in the final calculation.
Fix this by replacing the clamping code with an overflow test that
aborts the calculation early if the final result is sure to overflow,
based on the overflow limit in exp_var(). This provides the same
protection against integer overflow in the subsequent result scale
computation as the original clamping code, but it also ensures that
precision is never lost and saves compute cycles in cases that are
sure to overflow.
The new early overflow test works with the initial low-precision
result (expected to be accurate to around 8 significant digits) and
includes a small fuzz factor to ensure that it doesn't kick in for
values that would not overflow exp_var(), so the overall overflow
threshold of pow() is unchanged and consistent for all inputs with
non-integer exponents.
Author: Dean Rasheed
Reviewed-by: Tom Lane
Discussion: http://www.postgresql.org/message-id/CAEZATCUj3U-cQj0jjoia=qgs0SjE3auroxh8swvNKvZWUqegrg@mail.gmail....
See-also: http://www.postgresql.org/message-id/CAEZATCV7w+8iB=07dJ8Q0zihXQT1semcQuTeK+4_rogC_zq5Hw@mail.gmail....
Branch
------
master
Details
-------
http://git.postgresql.org/pg/commitdiff/18a02ad2a506e4425c6dd2ea235039cd5659467d
Modified Files
--------------
src/backend/utils/adt/numeric.c | 20 ++++++++++++++++----
src/test/regress/expected/numeric_big.out | 14 ++++++++++++++
src/test/regress/sql/numeric_big.sql | 10 ++++++++++
3 files changed, 40 insertions(+), 4 deletions(-)
--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-committers@postgresql.org
Cc: dean.a.rasheed@gmail.com
Subject: Re: pgsql: Fix corner-case loss of precision in numeric pow() calculation
In-Reply-To: <E1ayGNi-0002bk-Lv@gemulon.postgresql.org>
References: <E1ayGNi-0002bk-Lv@gemulon.postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox