pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Dean Rasheed <dean.a.rasheed@gmail.com>
Cc: anyhowstep@hotmail.com, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Subject: Re: BUG #16281: LN() function inaccurate at 1000th fractional digit
Date: Sat, 29 Feb 2020 14:12:39 -0500
Message-ID: <13048.1583003559@sss.pgh.pa.us> (raw)
In-Reply-To: <CAEZATCUyWV57u_W1kC1v2OsHYezG_b0ZFf8R4HOa1hRecszW5Q@mail.gmail.com>
References: <16281-279f299d9c06e56f@postgresql.org>
	<CAEZATCVi=KpqHid+DBkn27Es9+SOkFyo2_wSX892aUP_8P6_-w@mail.gmail.com>
	<962.1582904341@sss.pgh.pa.us>
	<CAEZATCUyWV57u_W1kC1v2OsHYezG_b0ZFf8R4HOa1hRecszW5Q@mail.gmail.com>

Dean Rasheed <dean.a.rasheed@gmail.com> writes:
> Looking more closely at ln_var(), it seems that there was an oversight
> related to the way that it computes the local_rscale for the Taylor
> series expansion --- it fails to account for the fact that the result
> is multiplied by fact (2^(nsqrt+1), where nsqrt is the number of
> square roots performed in the range reduction phase, which in practice
> is at most 22).
> Since 2^22 has 7 decimal digits, multiplying by 2^22 almost entirely
> wipes out the 8-digit safety margin used in the Taylor series
> expansion. The attached patch corrects that.

Looks sane to me.

> With this patch, all the examples originally posted return the correct
> results (calculated with bc). I'd be interested to know how the OP
> constructed these examples, and whether there were any that were off
> by more than 1 ULP.

Yeah, that would be interesting.

			regards, tom lane





view thread (7+ messages)  latest in thread

Message-ID: <13048.1583003559@sss.pgh.pa.us>
Permalink:  ../13048.1583003559@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/13048.1583003559@sss.pgh.pa.us

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-bugs@postgresql.org
  Cc: tgl@sss.pgh.pa.us, dean.a.rasheed@gmail.com
  Subject: Re: BUG #16281: LN() function inaccurate at 1000th fractional digit
  In-Reply-To: <13048.1583003559@sss.pgh.pa.us>
  References: <16281-279f299d9c06e56f@postgresql.org> <CAEZATCVi=KpqHid+DBkn27Es9+SOkFyo2_wSX892aUP_8P6_-w@mail.gmail.com> <962.1582904341@sss.pgh.pa.us> <CAEZATCUyWV57u_W1kC1v2OsHYezG_b0ZFf8R4HOa1hRecszW5Q@mail.gmail.com> <13048.1583003559@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox