Received: from localhost (unknown [200.46.204.182]) by postgresql.org (Postfix) with ESMTP id 4FA752E32A0 for ; Mon, 14 Jan 2008 15:01:42 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.182]) (amavisd-maia, port 10024) with ESMTP id 55551-06 for ; Mon, 14 Jan 2008 15:01:36 -0400 (AST) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.180]) by postgresql.org (Postfix) with ESMTP id 9B7BF2E2ECF for ; Mon, 14 Jan 2008 15:01:37 -0400 (AST) Received: by py-out-1112.google.com with SMTP id d37so3462267pye.29 for ; Mon, 14 Jan 2008 11:01:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; bh=ZcRnWw6Uj3W74qMt7oq2dooWSFK/0jAz5GsBinUOirU=; b=PMeukRy5k0Nm2rWZf9PoQkBaUMmKVQ0HjmxI6SPTwBKBzaZm9OtNf+0ya2rGoOw9zOdxcPH+7TS8R7BmgU6SIRDRR8xsY62JSySN1o1KhM0akyYH0sZYbwlW0ZryvwWHlKk+cQSomBG74ZNzESQav/Tht6fOuRgT0Y/uWDSBtYI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=GGAZNXagD98vT2SA/24PMhLEE33ck4HTO6VAoh8vOyOH/ctSalx97XpCpVVl15cTMWqBfT0qy7W0GsKLPIXFpUDN6p1IUCjA8dvxF3G52JmWYyPk0GluD+RhCaHWooujxeqdERtbv8WPOySJfGNkeSnWj113W8w0W//g2GYmmCk= Received: by 10.64.156.2 with SMTP id d2mr15233424qbe.54.1200337296081; Mon, 14 Jan 2008 11:01:36 -0800 (PST) Received: by 10.64.242.15 with HTTP; Mon, 14 Jan 2008 11:01:36 -0800 (PST) Message-ID: <8078a1730801141101j4bc0dc4chdffee5eaebdb99da@mail.gmail.com> Date: Mon, 14 Jan 2008 11:01:36 -0800 From: "Medi Montaseri" To: "Steve Midgley" Subject: Re: UTF8 encoding and non-text data types Cc: pgsql-sql@postgresql.org In-Reply-To: <478aea79.03098c0a.5f32.ffffa6e4SMTPIN_ADDED@mx.google.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_12508_27420390.1200337296057" References: <20080113222706.73BE416DA2AB@mx1.hub.org> <478aea79.03098c0a.5f32.ffffa6e4SMTPIN_ADDED@mx.google.com> X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200801/113 X-Sequence-Number: 29966 ------=_Part_12508_27420390.1200337296057 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline Thanks Steve, Actually I do not insert text data into my numeric field. As I mentioned given create table t1 { name text, cost decimal } then I would like to insert numeric data into column "cost" because then I can later benefit from numerical operators like SUM, AVG, etc More specifically, I am using HTML, Perl and PG. So from the HTML point of view a textfield is just some strings. So my user would enter 12345 but expressed in UTF8. Perl would get this and use DBI to insert it into PG What I am experiencing now is that DB errors that I am trying to insert an incorrect data into column "cost" which is numeric and the data is coming in from HTML in UTF8 Mybe I have to convert it to ASCII numbers in Perl before inserting them into PG Thanks Medi On Jan 13, 2008 8:51 PM, Steve Midgley wrote: > At 02:22 PM 1/13/2008, pgsql-sql-owner@postgresql.org wrote: > >Date: Sat, 12 Jan 2008 14:21:00 -0800 > >From: "Medi Montaseri" > >To: pgsql-sql@postgresql.org > >Subject: UTF8 encoding and non-text data types > >Message-ID: > ><8078a1730801121421l1c9b90c1lc65d36cf6e752a6f@mail.gmail.com> > > > >I understand PG supports UTF-8 encoding and I have sucessfully > >inserted > >Unicode text into columns. I was wondering about other data types such > >as > >numbers, decimal, dates > > > >That is, say I have a table t1 with > >create table t1 { name text, cost decimal } > >I can insert UTF8 text datatype into this table with no problem > >But if my application attempts to insert numbers encloded in UTF8, > >then I > >get wrong datatype error > > > >Is the solution for the application layer (not database) to convert > >the > >non-text UTF8 numbers to ASCII and then insert it into database ? > > > >Thanks > >Medi > > Hi Medi, > > I have only limited experience in this area, but it sounds like you > sending your numbers as strings? In your example: > > >create table t1 { name text, cost decimal }; > > insert into t1 (name, cost) values ('name1', '1'); > > I can't think of how else you're sending numeric values as UTF8? I know > that Pg will accept numbers as strings and convert internally (that has > worked for me in some object relational environments where I don't > choose to cope with data types), but I think it would be better if you > simply didn't send your numeric data in quotations, whether as UTF8 or > ASCII. If you don't have control over this layer (that quotes your > values), then I'd say converting to ASCII would solve the problem. But > better to convert to numeric and not ship quoted strings at all. > > I may be totally off-base and missing something fundamental and I'm > very open to correction (by anyone), but that's what I can see here. > > Best regards, > > Steve > > ------=_Part_12508_27420390.1200337296057 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline Thanks Steve,

Actually I do not insert text data into my numeric field.
As I mentioned given
create table t1 { name text, cost decimal }
then I would like to insert numeric data into column "cost" because then I can later benefit from numerical operators like SUM, AVG, etc

More specifically, I am using HTML, Perl and PG. So from the HTML point of view a textfield is just some strings. So my user would enter 12345 but expressed in UTF8. Perl would get this and use DBI to insert it into PG

What I am experiencing now is that DB errors that I am trying to insert an incorrect data into column "cost" which is numeric and the data is coming in from HTML in UTF8

Mybe I have to convert it to ASCII numbers in Perl before inserting  them into PG

Thanks
Medi

On Jan 13, 2008 8:51 PM, Steve Midgley <public@misuse.org> wrote:
At 02:22 PM 1/13/2008, pgsql-sql-owner@postgresql.org wrote:
>Date: Sat, 12 Jan 2008 14:21:00 -0800
>From: "Medi Montaseri" < montaseri@gmail.com>
>To: pgsql-sql@postgresql.org
>Subject: UTF8 encoding and non-text data types
>Message-ID:
>< 8078a1730801121421l1c9b90c1lc65d36cf6e752a6f@mail.gmail.com>
>
>I understand PG supports UTF-8 encoding and I have sucessfully
>inserted
>Unicode text into columns. I was wondering about other data types such
>as
>numbers, decimal, dates
>
>That is, say I have a table t1 with
>create table t1 { name text, cost decimal }
>I can insert UTF8 text datatype into this table with no problem
>But if my application attempts to insert numbers encloded in UTF8,
>then I
>get wrong datatype error
>
>Is the solution for the application layer (not database) to convert
>the
>non-text UTF8 numbers to ASCII and then insert it into database ?
>
>Thanks
>Medi

Hi Medi,

I have only limited experience in this area, but it sounds like you
sending your numbers as strings? In your example:

>create table t1 { name text, cost decimal };

insert into t1 (name, cost) values ('name1', '1');

I can't think of how else you're sending numeric values as UTF8? I know
that Pg will accept numbers as strings and convert internally (that has
worked for me in some object relational environments where I don't
choose to cope with data types), but I think it would be better if you
simply didn't send your numeric data in quotations, whether as UTF8 or
ASCII. If you don't have control over this layer (that quotes your
values), then I'd say converting to ASCII would solve the problem. But
better to convert to numeric and not ship quoted strings at all.

I may be totally off-base and missing something fundamental and I'm
very open to correction (by anyone), but that's what I can see here.

Best regards,

Steve

------=_Part_12508_27420390.1200337296057--