Message-ID: <005b01c48a43$630db0a0$7201a8c0@mst1x5r347kymb>
From: "Iain" <iain@mst.co.jp>
To: "Adam Lancaster" <alancaster@preventsys.com>,
	<pgsql-sql@postgresql.org>
References: <8758F8D58219684FAB0239EE8967048A010807@calculon.preventsys.com>
	<200408241252.30182.josh@agliodbs.com>
	<1093378875.15248.39.camel@linda>
Subject: Re: Possible rounding error of large float values?
Date: Wed, 25 Aug 2004 10:32:29 +0900
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-2022-jp"
Content-Transfer-Encoding: 7bit


# select 9223372036854775807 = 9223372036854775807::float;
 ?column?
----------
 t
(1 row)

# select 9223372036854775807 = 9223372036854775807.0::float;
 ?column?
----------
 t
(1 row)

# select 9223372036854775807 = 9223372036854775807.0000::numeric;
 ?column?
----------
 t
(1 row)

This and the fact that it's still possible to find the row using the
original value would seem to indicate that the rounding is just a display
artifact..

----- Original Message ----- 
From: "Oliver Elphick" <olly@lfix.co.uk>
To: "Josh Berkus" <josh@agliodbs.com>
Cc: "Adam Lancaster" <alancaster@preventsys.com>; <pgsql-sql@postgresql.org>
Sent: Wednesday, August 25, 2004 5:21 AM
Subject: Re: [SQL] Possible rounding error of large float values?


> On Tue, 2004-08-24 at 20:52, Josh Berkus wrote:
> > Adam,
> >
> > > 9223372036854775807
> > >
> > > It gets selected out as:
> > >
> > > 9.22337203685478E18
> >
> > This is a property of FLOAT data types.  They round.
> >
> > > Which appears to be rounded. When we cast it to numeric type we get:
> > >
> > > 9223372036854780000
> > >
> > > Which also is rounded. It is still possible to find the row using the
> > > original value
> >
> > Hmmm ... is 15 digits the limit of NUMERIC?   It may be.
>
> It must be the limit of float.  Numeric can't produce any more than it
> was given and it was cast from float.
>
> bray=# select 653596708775675750507850507570708696432 ::numeric;
>                  numeric
> -----------------------------------------
>  653596708775675750507850507570708696432
> (1 row)
>
> -- 
> Oliver Elphick                                          olly@lfix.co.uk
> Isle of Wight                              http://www.lfix.co.uk/oliver
> GPG: 1024D/A54310EA  92C8 39E7 280E 3631 3F0E  1EC0 5664 7A2F A543 10EA
>                  ========================================
>      "I saw in the night visions, and, behold, one like the
>       Son of man came with the clouds of heaven, and came to
>       the Ancient of days, and they brought him near before
>       him. And there was given him dominion, and glory, and
>       a kingdom, that all people, nations, and languages,
>       should serve him; his dominion is an everlasting
>       dominion, which shall not pass away, and his kingdom
>       that which shall not be destroyed."
>                                     Daniel 7:13,14
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster