pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Dean Rasheed <dean.a.rasheed@gmail.com>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Fix corner-case loss of precision in numeric_power().
Date: Wed, 06 Oct 2021 12:29:47 +0000
Message-ID: <E1mY63P-0004Xt-2g@gemulon.postgresql.org> (raw)

Fix corner-case loss of precision in numeric_power().

This fixes a loss of precision that occurs when the first input is
very close to 1, so that its logarithm is very small.

Formerly, during the initial low-precision calculation to estimate the
result weight, the logarithm was computed to a local rscale that was
capped to NUMERIC_MAX_DISPLAY_SCALE (1000). However, the base may be
as close as 1e-16383 to 1, hence its logarithm may be as small as
1e-16383, and so the local rscale needs to be allowed to exceed 16383,
otherwise all precision is lost, leading to a poor choice of rscale
for the full-precision calculation.

Fix this by removing the cap on the local rscale during the initial
low-precision calculation, as we already do in the full-precision
calculation. This doesn't change the fact that the initial calculation
is a low-precision approximation, computing the logarithm to around 8
significant digits, which is very fast, especially when the base is
very close to 1.

Patch by me, reviewed by Alvaro Herrera.

Discussion: https://postgr.es/m/CAEZATCV-Ceu%2BHpRMf416yUe4KKFv%3DtdgXQAe5-7S9tD%3D5E-T1g%40mail.gmail.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/e54a758d24dab056bb7f50d26c57a3c8761cc44a

Modified Files
--------------
src/backend/utils/adt/numeric.c       | 6 +++++-
src/test/regress/expected/numeric.out | 6 ++++++
src/test/regress/sql/numeric.sql      | 1 +
3 files changed, 12 insertions(+), 1 deletion(-)



view thread (9+ messages)  latest in thread

Message-ID: <E1mY63P-0004Xt-2g@gemulon.postgresql.org>
Permalink:  ../E1mY63P-0004Xt-2g@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1mY63P-0004Xt-2g@gemulon.postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: dean.a.rasheed@gmail.com
  Subject: Re: pgsql: Fix corner-case loss of precision in numeric_power().
  In-Reply-To: <E1mY63P-0004Xt-2g@gemulon.postgresql.org>
  References: <E1mY63P-0004Xt-2g@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox