From: Stephan Szabo <sszabo@megazone.bigpanda.com>
To: Iain <iain@mst.co.jp>
Cc: Adam Lancaster <alancaster@preventsys.com>
Cc: pgsql-sql@postgresql.org
Subject: Re: Possible rounding error of large float values?
Date: Tue, 24 Aug 2004 18:41:37 -0700 (PDT)
Message-ID: <20040824183612.M67501@megazone.bigpanda.com> (raw)
In-Reply-To: <005b01c48a43$630db0a0$7201a8c0@mst1x5r347kymb>
References: <8758F8D58219684FAB0239EE8967048A010807@calculon.preventsys.com>
<200408241252.30182.josh@agliodbs.com>
<1093378875.15248.39.camel@linda>
<005b01c48a43$630db0a0$7201a8c0@mst1x5r347kymb>
On Wed, 25 Aug 2004, Iain wrote:
>
> # select 9223372036854775807 = 9223372036854775807::float;
> ?column?
> ----------
> t
> (1 row)
> This and the fact that it's still possible to find the row using the
> original value would seem to indicate that the rounding is just a display
> artifact..
I don't think that's true since the number ending in 807 is equal to a
version ending in 808.
sszabo=# select 9223372036854775807 = 9223372036854775808::float;
?column?
----------
t
(1 row)
(In this next one I'm casting both because I want to choose the types of
both literals)
sszabo=# select 9223372036854775807::float =
9223372036854775808::numeric;
?column?
----------
t
(1 row)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-sql@postgresql.org
Cc: sszabo@megazone.bigpanda.com, iain@mst.co.jp, alancaster@preventsys.com
Subject: Re: Possible rounding error of large float values?
In-Reply-To: <20040824183612.M67501@megazone.bigpanda.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox