X-Original-To: pgsql-sql-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 9E4E15E46F5 for ; Tue, 24 Aug 2004 22:41:41 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 26101-05 for ; Wed, 25 Aug 2004 01:41:34 +0000 (GMT) Received: from megazone.bigpanda.com (megazone.bigpanda.com [64.147.171.210]) by svr1.postgresql.org (Postfix) with ESMTP id 035845E46EC for ; Tue, 24 Aug 2004 22:41:32 -0300 (ADT) Received: by megazone.bigpanda.com (Postfix, from userid 1001) id 77710354DC; Tue, 24 Aug 2004 18:41:37 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by megazone.bigpanda.com (Postfix) with ESMTP id 75B8735422; Tue, 24 Aug 2004 18:41:37 -0700 (PDT) Date: Tue, 24 Aug 2004 18:41:37 -0700 (PDT) From: Stephan Szabo To: Iain Cc: Adam Lancaster , pgsql-sql@postgresql.org Subject: Re: Possible rounding error of large float values? In-Reply-To: <005b01c48a43$630db0a0$7201a8c0@mst1x5r347kymb> Message-ID: <20040824183612.M67501@megazone.bigpanda.com> References: <8758F8D58219684FAB0239EE8967048A010807@calculon.preventsys.com> <200408241252.30182.josh@agliodbs.com> <1093378875.15248.39.camel@linda> <005b01c48a43$630db0a0$7201a8c0@mst1x5r347kymb> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200408/260 X-Sequence-Number: 18757 On Wed, 25 Aug 2004, Iain wrote: > > # select 9223372036854775807 = 9223372036854775807::float; > ?column? > ---------- > t > (1 row) > This and the fact that it's still possible to find the row using the > original value would seem to indicate that the rounding is just a display > artifact.. I don't think that's true since the number ending in 807 is equal to a version ending in 808. sszabo=# select 9223372036854775807 = 9223372036854775808::float; ?column? ---------- t (1 row) (In this next one I'm casting both because I want to choose the types of both literals) sszabo=# select 9223372036854775807::float = 9223372036854775808::numeric; ?column? ---------- t (1 row)