X-Original-To: pgsql-php-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 511545E46C3 for ; Sat, 28 Aug 2004 11:39:51 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 93489-03 for ; Sat, 28 Aug 2004 14:39:43 +0000 (GMT) Received: from web11.manitu.net (web11.manitu.net [217.11.48.111]) by svr1.postgresql.org (Postfix) with ESMTP id C65545E46F8 for ; Sat, 28 Aug 2004 11:39:40 -0300 (ADT) Received: from [192.168.0.1] (dsl-082-082-193-241.arcor-ip.net [82.82.193.241]) (authenticated) by web11.manitu.net (8.10.2-SOL3/8.10.2) with ESMTP id i7SEdd313860; Sat, 28 Aug 2004 16:39:39 +0200 Subject: Re: Space requirements (with respect to foriegn languages) From: Markus Bertheau To: Gerard Samuel Cc: pgsql-php@postgresql.org In-Reply-To: <412E49C9.8050609@trini0.org> References: <412E49C9.8050609@trini0.org> Content-Type: text/plain; charset=UTF-8 Message-Id: <1093703976.2732.5.camel@teetnang> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 (1.4.6-2) Date: Sat, 28 Aug 2004 16:39:37 +0200 Content-Transfer-Encoding: quoted-printable X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200408/44 X-Sequence-Number: 2932 =D0=92 =D0=A7=D1=82=D0=B2, 26.08.2004, =D0=B2 22:36, Gerard Samuel =D0=BF= =D0=B8=D1=88=D0=B5=D1=82: > My site/code/database is developed primarily for the english language. > I've had people from "The Far East" add content to my site using their > native language, and it is displaying properly in the site. > But Im a bit concerned about the number of characters these languages use. > For example, I've had someone enter -> > chinese testing=E3=80=80=E4=B8=AD=E6=96=87 >=20 > It is saved in the database as -> > chinese testing 中文 Your web page uses a character set that does not contain chinese characters. So the browser decided to send their respective HTML entities instead. These entities, as you correctly observed, amount to more than one (latin, ASCII) character. > Now, forgive my ignorance, but I have no idea what the additional > chinese characters mean, but from the values in the database, Im > assuming that it amounts to 3 characters. > But if Im correct that those are 3 characters, it is > using up 24 characters in a column. >=20 > My concern is that what if I were to limit a column to say 25 "english" > characters, and a chinese fellow, comes by and hypothetically says > "Hello World" in chinese and goes over the limit of the column, the data > will be truncated. PostgreSQL will not truncate the data, but reject it; but the general point is correct. > Is there anything that can be done to overcome this shortcoming? >=20 > Im currently using PostgreSQL 7.4.2, using SQL_ASCII as the database > characterset, FreeBSD 4.10, php 4.3.6. Change your site to use a character set that includes chinese characters, for example Unicode. The most common encoding of Unicode on the web is UTF-8. It's also the encoding PostgreSQL uses when you use UNICODE as the database encoding. If you decide to switch your site to UTF-8 and want varchar(25) to mean 25 characters, and not 25 bytes, you have to change the database encoding to UNICODE accordingly. --=20 Markus Bertheau