Subject: multi-language web application: is it possible?

From: "Elim Qiu" <[email-masked]>
Date: Fri, 4 May 2007 08:20:20 -0600

I have many tables like the table Person:below, in mysql database.

person_id, first_name,last_name, mi, gb_first_name, gb_last_name, b5_first_name, b5_last_name, gender, dob

where different columns storing strings in different encodings. At anytime, a web user can switch the language and the application will get the values in the right columns to generate web pages. The purpose of Multi-language tables is to make multilanguage dynamic content management easier for web applications.  For example, to add a person record, the user enter the English name, then switch the session language to gb2312, enter the Chinese name in gb2312, and then switch the session language to big5, enter the name in big5. And then commit the data into the database. The whole thing sounds complicated but can be treated as a pattern and let a framework to take care of those and the code can be as clean as a single language app. I actually have the framework that works well for me with mysql database.

I'm trying to port the app to pgsql database but got trouble doing so. I can read and set a row with multiple languages (in some paricular cases), but cannot do queries like  (gb_first_name = 'A' and b5_first_name = 'B') with A a gb2312 string, B a big5 string. The tables are of unicode encoded, and the dbclient encoding is set to GBK. The application's char set are selectable by user among iso-8859-1,gb2312 and big5.

I didn't do anything about language encoding in mysql database, it just worked for me. At least with english, gb2312 and big5 altogether in a table like table Person above. I noticed that (english, gb2312, big5, Jp) cannot work together even in mysql database.  My approach seems fine with most western languages

So after all such experimental work, I still don't know how to make a real multi language web app such that the languages are switchable within the same session. 

Any suggestions?  Any web application known to be able to solve the problem? Thanks

---

From: Tino Wildenhain <[email-masked]>
Date: Fri, 04 May 2007 16:59:10 +0200

Elim Qiu schrieb:
> where different columns storing strings in different encodings. At 
> anytime, a web user can switch the language and the application will get 
> the values in the right columns to generate web pages. The purpose of 
> Multi-language tables is to make multilanguage dynamic content 
> management easier for web applications.  For example, to add a person 
> record, the user enter the English name, then switch the session 
> language to gb2312, enter the Chinese name in gb2312, and then switch 
> the session language to big5, enter the name in big5. And then commit 
> the data into the database. The whole thing sounds complicated but can 
> be treated as a pattern and let a framework to take care of those and 
> the code can be as clean as a single language app. I actually have the 
> framework that works well for me with mysql database.

It would be easier to normalize the tables and have a referral.
Also person names arent usually localized - at least this would
be very uncommon. If you want you can transcribe them when you
output it (e.g. replace umlauts with their dual char ascii
representation)

> I'm trying to port the app to pgsql database but got trouble doing so. I 
> can read and set a row with multiple languages (in some paricular 
> cases), but cannot do queries like  (gb_first_name = 'A' and 
> b5_first_name = 'B') with A a gb2312 string, B a big5 string. The tables 
> are of unicode encoded, and the dbclient encoding is set to GBK. The 
> application's char set are selectable by user among iso-8859-1,gb2312 
> and big5.

the language and their encoding are more or less orthogonal. So if
you settle for one common encoding which covers all the language
you want to use, then its easy to have all the language words
side by side. General unicode (in its utf-8 representation) is
used in postgres for such a general case.

> I didn't do anything about language encoding in mysql database, it just 
> worked for me. At least with english, gb2312 and big5 altogether in a 
> table like table Person above. I noticed that (english, gb2312, big5, 
> Jp) cannot work together even in mysql database.  My approach seems fine 
> with most western languages

Well it might work but it might not work as expected. Beside the
encoding, you also have a collating order to obey. This is very
depending on the language (and the portion of the charset it
uses) so this would not even work in mySQL or other databases
w/o any specification.

In PG, assuming you would have a table with the texts, it would
carry the language_id too and you could provide a functional index
which would take care of the sorting of the individual language
based on the language id.

> So after all such experimental work, I still don't know how to make a 
> real multi language web app such that the languages are switchable 
> within the same session.

I'd highly recommend restructuring the tables to be more flexible
and to have a faster and cleaner approach. Joins do not hurt so
much in postgres so use them to your advantage.

Regards
Tino

---

From: "Adrian Maier" <[email-masked]>
Date: Fri, 4 May 2007 20:12:09 +0300

On 5/4/07, Elim Qiu <[email-masked]> wrote:
> where different columns storing strings in different encodings. At anytime,
> a web user can switch the language and the application will get the values
> in the right columns to generate web pages. The purpose of Multi-language
> tables is to make multilanguage dynamic content management easier for web
> applications.  For example, to add a person record, the user enter the
> English name, then switch the session language to gb2312, enter the Chinese
> name in gb2312, and then switch the session language to big5, enter the name
> in big5. And then commit the data into the database. The whole thing sounds
> complicated but can be treated as a pattern and let a framework to take care
> of those and the code can be as clean as a single language app. I actually
> have the framework that works well for me with mysql database.
>
> I'm trying to port the app to pgsql database but got trouble doing so. I can
> read and set a row with multiple languages (in some paricular cases), but
> cannot do queries like  (gb_first_name = 'A' and b5_first_name = 'B') with A
> a gb2312 string, B a big5 string. The tables are of unicode encoded, and the
> dbclient encoding is set to GBK. The application's char set are selectable
> by user among iso-8859-1,gb2312 and big5.
>
> worked for me. At least with english, gb2312 and big5 altogether in a table
> like table Person above. I noticed that (english, gb2312, big5, Jp) cannot
> work together even in mysql database.  My approach seems fine with most
> western languages
>
> So after all such experimental work, I still don't know how to make a real
> multi language web app such that the languages are switchable within the
> same session.
>

Elim,

the pgsql-www mailing list is dedicated to discussions regarding the PostgreSQL
website , and not to discussions about developing web applications.
I've noticed
that you've sent the email to pgsql-general as well .   pgsql-general
is probably a
better place for asking your question  .

Best wishes,
Adrian Maier