From: Tom Lane <tgl@sss.pgh.pa.us>
To: Will Pugh <willpugh@gmail.com>
Cc: pgsql-sql@postgresql.org
Subject: Re: How does Numeric division determine precision?
Date: Thu, 12 Jul 2012 13:30:19 -0400
Message-ID: <29730.1342114219@sss.pgh.pa.us> (raw)
In-Reply-To: <CAM39vH_0snUv_YRCU11L2iB4zj=Dz7F9B_RhUPtYDqdSKymaDw@mail.gmail.com>
References: <CAM39vH_0snUv_YRCU11L2iB4zj=Dz7F9B_RhUPtYDqdSKymaDw@mail.gmail.com>
Will Pugh <willpugh@gmail.com> writes:
> It seems that is 9.1, numerics that don't have a specified precision
> and scale are arbitrary scale/precision.
> For many operations this is straightforward. However, when doing a
> division operation that does not terminate, I'm curious about how the
> number of digits is determined.
According to select_div_scale() in src/backend/utils/adt/numeric.c,
/*
* The result scale of a division isn't specified in any SQL standard. For
* PostgreSQL we select a result scale that will give at least
* NUMERIC_MIN_SIG_DIGITS significant digits, so that numeric gives a
* result no less accurate than float8; but use a scale not less than
* either input's display scale.
*/
I wouldn't necessarily claim that that couldn't be improved on,
but that's what it does now.
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-sql@postgresql.org
Cc: tgl@sss.pgh.pa.us, willpugh@gmail.com
Subject: Re: How does Numeric division determine precision?
In-Reply-To: <29730.1342114219@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox