X-Original-To: pgsql-sql-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 49F645E46C2 for ; Tue, 24 Aug 2004 21:39:38 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 48334-06 for ; Wed, 25 Aug 2004 00:39:37 +0000 (GMT) Received: from smtp111.mail.sc5.yahoo.com (smtp111.mail.sc5.yahoo.com [66.163.170.9]) by svr1.postgresql.org (Postfix) with SMTP id 1C6E75E46E1 for ; Tue, 24 Aug 2004 21:39:35 -0300 (ADT) Received: from unknown (HELO jupiter.black-lion.info) (janwieck@68.80.245.191 with login) by smtp111.mail.sc5.yahoo.com with SMTP; 25 Aug 2004 00:39:39 -0000 Received: from [172.21.8.18] ([192.168.192.101]) (authenticated bits=0) by jupiter.black-lion.info (8.12.10/8.12.9) with ESMTP id i7P0dST7086813; Tue, 24 Aug 2004 20:39:29 -0400 (EDT) (envelope-from JanWieck@Yahoo.com) Message-ID: <412BDFBA.7080405@Yahoo.com> Date: Tue, 24 Aug 2004 20:39:22 -0400 From: Jan Wieck User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.1) Gecko/20040707 X-Accept-Language: en-us, en MIME-Version: 1.0 To: olly@lfix.co.uk Cc: Josh Berkus , Adam Lancaster , pgsql-sql@postgresql.org Subject: Re: Possible rounding error of large float values? References: <8758F8D58219684FAB0239EE8967048A010807@calculon.preventsys.com> <200408241252.30182.josh@agliodbs.com> <1093378875.15248.39.camel@linda> In-Reply-To: <1093378875.15248.39.camel@linda> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200408/258 X-Sequence-Number: 18755 On 8/24/2004 4:21 PM, Oliver Elphick wrote: > On Tue, 2004-08-24 at 20:52, Josh Berkus wrote: >> Adam, >> >> > 9223372036854775807 >> > >> > It gets selected out as: >> > >> > 9.22337203685478E18 >> >> This is a property of FLOAT data types. They round. >> >> > Which appears to be rounded. When we cast it to numeric type we get: >> > >> > 9223372036854780000 >> > >> > Which also is rounded. It is still possible to find the row using the >> > original value >> >> Hmmm ... is 15 digits the limit of NUMERIC? It may be. The original numeric code I presented contained even log(), ln() and trigonometric functions that produced results of up to 1,000 digits precision. They wheren't considered usefull, so they didn't survive, but numeric is that good after all. Jan > > It must be the limit of float. Numeric can't produce any more than it > was given and it was cast from float. > > bray=# select 653596708775675750507850507570708696432 ::numeric; > numeric > ----------------------------------------- > 653596708775675750507850507570708696432 > (1 row) > -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #