Subject: PostgreSQL website translations

From: "Adrian Maier" <[email-masked]>
Date: Mon, 3 Mar 2008 14:53:34 +0200

Hello,

Back in November 2007  I used to have a complete Romanian
translation in sync with the English content. At that time noone
has replied to my message regarding adding the translation to
the CVS repository.    At that time I have assumed that the
people involved in server adminstration were too busy with things
like gborg migration ,   so I thought that it's best not too insist.

But now, after the 8.3  release  I guess that it's time to raise
the issue again.

Of course,  the first thing that needs to be done (by me ..;-)  is to
bring the translation up  to date (from november to present) .  But
before doing that it would be nice to know whether there is enough
interest in my translation.

So:  should I spend time with bringing my Romanian translation
of the website up-to-date ?    or there is too little interest  ?

Cheers,
Adrian Maier

---

From: Guillaume Lelarge <[email-masked]>
Date: Sun, 09 Mar 2008 08:48:43 +0100

Adrian Maier a écrit :
> [...]

Despite the lack of answer, I think there is interest in translating the 
website. But there's a few things to do before adding the translation. I 
need to remember what Magnus told me about this last november :) ( ... 
if I remember well, I had to find some way to figure out if pages are 
out of date and if possible show the english page if the translated page 
is out of date ... something like this but Magnus will correct me if I'm 
wrong). Oh, and the pictures need to be "translatable" too.

-- 
Guillaume.
  http://www.postgresqlfr.org
  http://dalibo.com

---

From: Magnus Hagander <[email-masked]>
Date: Sun, 09 Mar 2008 12:00:21 +0100

> wrong). Oh, and the pictures need to be "translatable" too.

My apologies for not responding to this one earlier. It's certainly bee 
on my list to do.

I think what we really need before we can proceed properly with this is 
a "www code hacker" who can work the translation code. Not just a 
translator (don't get me wrong, we need a bunch of translators as well 
of course). Someone who can basically take responsibility for fixing up 
the code to work with the translations. There are a lot of small things 
broken in it AFAIK, and they need to be fixed. Guillaume mentions a 
couple of them, I think there are a few more. And some sections just 
don't have translatability implemented (yet).

There's no rocket science level code in the website, but said someone 
certainly needs to have a fairly good grasp of PHP and of coding in 
general. And most important - needs to have time to work on it.

As usual, this is best done in an incremental fashion rather than 
dumping a mega-patch in there - it'll be much more likely to be reviewed 
and applied quickly then.

But in the end, I think there is certainly interest in doing it. The 
reason the "current web team" is a bit slow on this is that we don't 
have what it takes to maintain such things once they are in right now - 
which is why we need someone to step up for the long term there. (IMO, 
of course, other members of the team may disagree on this, but I don't 
think they do)

//Magnus

---

From: "Adrian Maier" <[email-masked]>
Date: Mon, 10 Mar 2008 10:39:19 +0200

>  wrong). Oh, and the pictures need to be "translatable" too.

Sure: currently it's extremely painful to keep track of the
modifications.  And the last time this was discussed i've
understood that  for the time being the translators can
only rely on cvs and the pgwww-commits mailing list
because there were no resources for creating something
more elaborate ( especially because there were no
translations available ).

Another thing that will need serious thinking is, IMHO, the
mechanism used for selecting the language.  I personally
hate when a website decides the language based on
something unintuitive and transparent like the default
language in the browser or the IP location.    I mean:  it's
much better to have a set of small flags or a combobox
so that the user can easily select the desired language
instead of digging into web browser 's configuration.

-- 
Adrian Maier

---

From: Magnus Hagander <[email-masked]>
Date: Mon, 10 Mar 2008 10:05:31 +0100

> translations available ).

We now have svn, if that makes a difference :-)

> instead of digging into web browser 's configuration.

I think it's reasonable to have it *default* to the language from your
browser. Doint it based on IP like google does just completely *sucks*, but
the browser langauge makes sense. However, on top of that there should be a
way to switch your language on the site as well, to override what you have
as the default in the browser.

//Magnus

---

From: Stefan Kaltenbrunner <[email-masked]>
Date: Mon, 10 Mar 2008 10:13:28 +0100

> On Mon, Mar 10, 2008 at 10:39:19AM +0200, Adrian Maier wrote:

[..]

> as the default in the browser.

yes that is what I would like to see as well - default to browser 
language and have a manual override on the site.

Stefan

---

From: "Adrian Maier" <[email-masked]>
Date: Mon, 10 Mar 2008 11:22:12 +0200

>  as the default in the browser.

Sure, the language can be picked from the browser default language.
But apart from that the ability to select the language easily and
manually inside the website should be considered a must.

Not to mention that having many flags/languages somewhere in
the page header is good for advocacy because it shows that
PostgreSQL is spread all over the world.

Cheers,
Adrian Maier

---

From: "Adrian Maier" <[email-masked]>
Date: Mon, 10 Mar 2008 11:32:28 +0200

>  of course, other members of the team may disagree on this, but I don't

Shame I didn't know 6 months ago that there are little chances to
have a translation accepted before certain things are adjusted in
the web infrastructure first :  i had plenty of time last autumn and
winter. And instead of translating the already outdated pages and
receiving no feedback I could have digged into the PHP parts of
the site ...

Starting from 24 march i'm changing the job,  so it's hard to predict
how much spare time shall I have.  How many hours per week do you
think someone would need for taking care of the translation-related
issues of the website ?

Perhaps it would be useful to have the things that need to be fixed
added on Trac ?

-- 
Adrian Maier

---

From: Magnus Hagander <[email-masked]>
Date: Mon, 10 Mar 2008 10:49:28 +0100

> receiving no feedback I could have digged into the PHP parts of

Yeah, apoligies for that. I think it was kind of implied, but we never did
spell it out. Which we should've. 

> think someone would need for taking care of the translation-related

I don't think it'll be all that much, really. What's needed is a "big push"
to get things ready in the first place, and that's going to take quite a
bit of work. Once that's there, the main ongoing work will be updating
translations. There's almost no time spent "maintaining" the PHP code for
the main site when there aren't new features added, and I expect the same
thing for the translation stuff. However, everytime we add new features and
such, it has to be handled.

> Perhaps it would be useful to have the things that need to be fixed

That's a very good idea. First we just have to agree on what they are :-)

I think one big question that we never did solve the last time is what to
do with pages that are out-of-date. E.g. when the english version of a page
is updated, and the translated one isn't, what do we do with it... Show it
anyway, remove it, somehow indicate it... Once we've decided what we want
to do about it, then we can start debating how to do it (which is where the
point of cvs/svn etc comes back in), and start adding tickets for them.

//Magnus

---

From: "Adrian Maier" <[email-masked]>
Date: Mon, 10 Mar 2008 12:53:38 +0200

>  point of cvs/svn etc comes back in), and start adding tickets for them.

When an English page is modified we could :
- immediately send a notice to the translators.  Automatically added Trac
tickets seem to be a cool idea if it's possible.
- keep displaying the old translated page for 4-5 days
- if the translation is not yet ready we could invalidate the
page : move it somewhere else and start displaying the
English page until the translation is ready.
Or, instead of moving away the page we could add an
"outdated page since yyyy-mm-dd" warning .

Either way , I think that it would be better to give the translators
a few days to do the update before hiding or marking the page.

But first of all,  we'll need a good way to store the last_modification_date
for each page : each page (English and translations alike) need to
be "stamped"  with a date or date+time .   The date of the svn
revision is not good enough because relying on it is the perfect
recipe for missing modifications if they are occuring before the
translators finishes the translation of a previous version.

Here is a scenario :
- 8-apr: the English page is modified+committed
- 8-apr: the translator starts translating
- 9-apr: the English page is modified+committed again
- 10-apr: the translation gets committed (but it has the contents for 8-apr)

So,  I think that we'll need to build some mechanisms for dealing
with the page versions/trabslations outside of what does svn offer.

Cheers,
Adrian Maier

---

From: Guillaume Lelarge <[email-masked]>
Date: Mon, 10 Mar 2008 12:13:08 +0100

> - keep displaying the old translated page for 4-5 days

Nope. If a page is modified, it should just display the english one.

> "outdated page since yyyy-mm-dd" warning .

I don't think we need to move the obsolete translated webpage. The PHP 
code should simply display the english one.

> a few days to do the update before hiding or marking the page.

Nope. There's no reason to do this. If the english page fixes an error 
(content, not grammar or things like this), I prefer to have the fixed 
english one rather than the translated but wrong one.

> - 10-apr: the translation gets committed (but it has the contents for 8-apr)

Oh OK. I understand your example. But if we can't rely on a svn hook to 
stamp each file, I also don't think we can rely on a manual 
modification, do we ?

> with the page versions/trabslations outside of what does svn offer.

Do you have any ideas of what this could be ?

There's another issue that we'll need to answer : localised search. If I 
remember well Magnus's talk at FOSDEM, the search on www.postgresql.org 
relies on an english only dictionnary. How will french, romanian or 
spanish pages affect the current search feature ?

Regards.

---

From: "Adrian Maier" <[email-masked]>
Date: Mon, 10 Mar 2008 14:41:15 +0200

>  code should simply display the english one.

Ok , that's fine with me : the English page replaces the translated page
until the translator updates it.

>  stamp each file, I also don't think we can rely on a manual

The core of the problem is finding a good place to store the
correspondance between a given page and its translations.

My point is that we need a way to store information like
"ro/about/advantages.html rev5001 (10-apr-2008) is the translation of
en/about/advantages.html rev4995 (8-apr-2008) " .

It's excellent if this can be achieved with svn hooks , but I doubt that
it's possible.

>  Do you have any ideas of what this could be ?

I'm dreaming of having "something" (some scripts or maybe
or even a web application for translators)  that gives:
- the list of pages that need to be modified or that are new
- for each listed file :
      *  the diff between the "English text that was translated last time
into my language" and "the current English text"
      *  the ability to download the latest translated text
      *  the ability to upload the updated translated text

>  spanish pages affect the current search feature ?

I can't comment on this (not qualified ... ) .

Cheers,
Adrian Maier

---

From: Guillaume Lelarge <[email-masked]>
Date: Sun, 23 Mar 2008 23:26:43 +0100

> until the translator updates it.

Well, I'm not so sure now. For someone who doesn't speak english, an old 
translated page could be better than the current one in english. I mean, 
there's no need to have an uptodate translation of the books webpage for 
example.

Probably we can do the same as the Debian guys : add some text on the 
translated webpage which can specify that a more up to date page is 
available in english and can be found "here".

> It's excellent if this can be achieved with svn hooks , but I doubt that

I doubt too. And, really, I don't think we need this much of a system. 
Your example is quite interesting but how many times will we have this 
issue ? I don't think many times. Perhaps we should just trust the 
translator. Before commiting his stuff, he needs to check if someone 
else changes the webpage. Trusting the translator will simplify the 
issue of the obsolete page. We'll simply need to check the modification 
date to decide if the website should display the translated one or the 
english one.

> - the list of pages that need to be modified or that are new

Easy to write a script that does just this if we agree on the 
"modification date check".

> into my language" and "the current English text"

svn diff ? svn browser ?

---

From: "Adrian Maier" <[email-masked]>
Date: Thu, 27 Mar 2008 13:58:51 +0200

On Mon, Mar 24, 2008 at 12:26 AM, Guillaume Lelarge
<[email-masked]> wrote:
>  issue ?  I don't think many times.

It would happen frequently enough to give headaches to any translator
who is trying to add a new language .  At least until the new translation
is accepted and committed.

>  Perhaps we should just trust the

I don't think that the existing tools (svn) is enough to make the life
easy for the potential website translators.

I like and use svn daily, but this doesn't mean that it's the right tool
for handling translations ...

>  > into my language" and "the current English text"

In order to do "svn diff"  one needs to know the revision or the
timestamp of the "previously translated English text".  Sure : the
translator can write down the revision number and use that
revision when doing 'svn diff'   -  but such a system would be
too manual and painful  to seriously consider a solution.

Ok:  I understand that there are no work resources for building some
web application for translating the website. Some time ago i've been
told that i'd have to rely on just CVS . I've accepted that and began to
translate the texts.  And look what's the result: X months later when I had
the translation complete I found out that in fact the framework was not
ready for accepting new translations  - and all the work was in vain
because when/if  the everything will be ready  i'll have to check and
translate  every single page. Again.

Perhaps we could something like  :  every translated contains
a comment with the  date+time+revision  of the corresponding
English page ?    at least this would make it possible to
manually manage the corresponding English revision for a given
translated page.

Cheers,
Adrian Maier

---

From: Magnus Hagander <[email-masked]>
Date: Thu, 27 Mar 2008 13:06:32 +0100

> manually manage the corresponding English revision for a given

Perhaps we could use a custom svn property for that? That would likely
make it easier to process automatically to pull out differences and
such. And perhaps some wrapper scripts around svn to help you set
things automatically?

//Magnus

---

From: "Adrian Maier" <[email-masked]>
Date: Thu, 27 Mar 2008 15:38:46 +0200

>  such. And perhaps some wrapper scripts around svn to help you set

I haven't used such custom properties.  As long as such a property can
be set for individual files ,  it seems to be a better (more reliable) solution
compared to relying on comments.   Some scripts for using this facility
would be a significant progress .

-- 
Adrian Maier

---

From: Guillaume Lelarge <[email-masked]>
Date: Thu, 27 Mar 2008 15:06:44 +0100

> would be a significant progress .

There's something I don't quite understand. What's the interest in 
having these comments (or custom svn properties) ?

Tell me if I'm wrong. This is what we want to achieve :
   we want to know if a specific translated webpage is out of date

Problem is :
   * translator checks out a file
   * during the translation, someone else checks out the file
   * and commit its changes
   * then translator commits its new translation

Using the modification date doesn't help us because the translation file 
will be newer than the english one.
Using the svn revision does not help us because the translation file 
will have a revision newer than the english one.

So, your idea seems to put the revision of the english file in a comment 
or in a custom property. So, it's a manual change :

  * translator checks out a file
  * he updates the comment of the translation with the revision of the
    english file
  * he translates
  * he commits

Is that right ?

And when the web server needs to choose between the english file and the 
translated one, it parses the translated one to get the revision put in 
comments, it gets the current revision of the english files and it 
finally compares them ?

Could work, but we still rely on the translator good will. No automatic 
process here. Or am I wrong ?

---

From: Guillaume Lelarge <[email-masked]>
Date: Thu, 27 Mar 2008 15:22:31 +0100

> translate  every single page. Again.

Nope.

I'll have to do this also. But I won't translate everything. I'll print 
the english webpage and the french one, read them and check for 
differences. I agree this is a tedious process, but much less than 
translate once again every single webpage.

---

From: "Adrian Maier" <[email-masked]>
Date: Thu, 27 Mar 2008 17:29:54 +0200

>  translate once again every single webpage.

Checking differences visually is what we should try to avoid  :  it's tedious,
boring, and error-prone.  Obviously, we have no automatic tools capable
to compare an English text with its French translation and find the
differences .  But we can do 'diff'  between different versions of
the English text, provided that we know which revisions to compare.

If we knew "the revision of the English page that corresponds to the
latest translation "  we could do svn diff on the English page (between
that revision and HEAD) and see the differences , so that we wouldn't
need to visually compare anything !     All we'd have to do is to
look at the diff output and jump directly to the few phrases that were
modified.

That's what I'm after :  a mechanism that can show me the specific
modifications that were made on a specific English page since the
last time *I* have  translated that particular page.

The problem is that this variable "the revision of the English page that
corresponds to the latest translation"  can be different for every page and
is stored in only one place:  the translator's memory . If the translator is
offered a way to store it somewhere (anywhere : in svn as a property ,
or as a comment,  or in a PostgreSQL database, ... ) then it would be
possible to write some scripts for making the translation work much easier.

-- 
Adrian Maier

---

From: "Adrian Maier" <[email-masked]>
Date: Thu, 27 Mar 2008 17:46:10 +0200

>  process here. Or am I wrong ?

Actually , there are two different purposes :
1)  detecting which translated pages went out-of-date
2)  providing some reliable tools that help the translators to easily spot
the (English) modifications that they need to incorporate into their
translated texts.

I am primarily interested in (2) , because plain cvs/svn  is not helpful
enough when translating .

As for (1) : it will never be possible to automatically ensure that a
translation is truly up-to-date .  We'll need to trust the translators when
they claim they've updated some page .  Especially when you don't
understand their language ...

-- 
Adrian Maier

---

From: Guillaume Lelarge <[email-masked]>
Date: Thu, 27 Mar 2008 17:00:50 +0100

> the (English) modifications that they need to incorporate into their

You can already do this with HTML comments on the translation. To use 
custom SVN properties, all translators will need to have SVN write 
access to pgweb, which can be a problem.

If we agree on the HTML comments, we need to add the Revision tag on 
each "template/en" file (see 
http://svnbook.red-bean.com/en/1.4/svn.advanced.props.special.keywords.html 
for details). The PHP code will be able to compare the Revision tag 
automatically added by SVN in the english file with the Revision comment 
manually added by the translator on the translated file.

For example, we will have:

/* $Revision: 144 $ */

in the english file, and:

/* #Revision: 144 # */

in the translated one. I don't use dollar between "Revision: 144" in the 
translated file so that SVN doesn't automatically replace the english 
based revision with the new translated revision.

> understand their language ...

+1

---

From: Alvaro Herrera <[email-masked]>
Date: Thu, 27 Mar 2008 13:05:51 -0300

Adrian Maier escribió:

> is stored in only one place:  the translator's memory .

Unless the translator is smart and stores it in a comment in the file
(you don't really expect the translator to _remember_ the revision ID
that he last translated, right?)

A really good way to go about this would be to use gettext, but I guess
that's a bit removed from what's there currently :-)

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

---

From: Guillaume Lelarge <[email-masked]>
Date: Thu, 27 Mar 2008 17:21:42 +0100

> that he last translated, right?)

It seems to me a hard way to do it. But I don't find a better one. He 
doesn't have to remember that. He just needs to write it in the new file 
when he starts the translation. We can provide a shell script that finds 
the revision of the file to be translated.

> that's a bit removed from what's there currently :-)

pgAdmin's website is based only on gettext and it's way simpler.

---

From: Alvaro Herrera <[email-masked]>
Date: Thu, 27 Mar 2008 13:25:50 -0300

> pgAdmin's website is based only on gettext and it's way simpler.

Would it be too difficult to do that for postgresql.org?

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

---

From: Guillaume Lelarge <[email-masked]>
Date: Thu, 27 Mar 2008 17:36:31 +0100

> Would it be too difficult to do that for postgresql.org?

Not really. Only 220 files :)

The real hard part will be the 159 PGWN files.

---

From: Alvaro Herrera <[email-masked]>
Date: Thu, 27 Mar 2008 13:44:34 -0300

> The real hard part will be the 159 PGWN files.

But how much of that is automatable?

---

From: Guillaume Lelarge <[email-masked]>
Date: Thu, 27 Mar 2008 17:52:41 +0100

> But how much of that is automatable?

I'm not sure I understand your question. Adding the func_lang is not 
automatable, as far as I know, but I can do it (manually) if we choose 
to do it.

---

From: Alvaro Herrera <[email-masked]>
Date: Thu, 27 Mar 2008 13:57:59 -0300

> automatable, as far as I know, but I can do it (manually) if we choose  

Well, that manual work should be avoided if at all possible ... editing
5 or 10 files may be OK, but 370 files I would say is out of the
question.

---

From: Guillaume Lelarge <[email-masked]>
Date: Thu, 27 Mar 2008 18:03:10 +0100

> 5 or 10 files may be OK, but 370 files I would say is out of the

That's only 220 files (which is already a lot), the 159 PGWN files being 
a part of it.

---

From: "Adrian Maier" <[email-masked]>
Date: Thu, 27 Mar 2008 20:18:36 +0200

>  access to pgweb, which can be a problem.

In this case the comments are preferable  since they can start being used
immediately after choosing a certain format .

+1

-- 
Adrian Maier