Using gettext

An Internationalized Software Project With Auto Tools
Prev	Using gettext	Next

Using gettext

The usage of gettext has two aspects: The adding of a new target language (usually done by a developer or internationalization maintainer) and the translation workflow.

Initial Tasks / Adding a New Language

Creating the .pot File

Creating the pot file and merging it into the language dependant .po files (there are no so far) is not really necessary, as it is done during the translation workflow. But it shows here, that the previous steps were successful.

# gmake update-po

...

'make update-po' collects the names of all .cpp files and stores them in po/POTFILES, which is used to extract the translatable strings. The result of this file name collecting (as described above) is as it should be:

# cat po/POTFILES.in
./src/main.cpp
./src/testmodule/testfunc.cpp

Afterwards, 'make update-po' created the .pot file:

# cat po/testproj.pot
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR James T.
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR , YEAR.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: j.t.kirk@ncc-1701.ufp\n"
"POT-Creation-Date: 2006-06-28 23:24+0200\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=CHARSET\n"
"Content-Transfer-Encoding: 8bit\n"

#: src/testmodule/testfunc.cpp:7
msgid "hello world!\n"
msgstr ""

#: src/testmodule/testfunc.cpp:8
msgid "Press a key\n"
msgstr ""

Most lines here contain general properties and are of no interest at the moment. These have to be changed, if a new language is added. Interesting are the last few lines. Here the strings marked as translatable (e.g. "hello world") are listed. The strings itself serve as message id. Its translation (msgstr) is empty here, as the .pot file contains the english strings only. In the translated file, msgstr will take the translation.

Adding a New Language

Usually, msginit adds a new language. But as it gives some perl errors here, the new language 'de' is added here manually. As it is the first language, po/LINGUAS has to be ceated:

# touch po/LINGUAS

It is a white space separated list of language codes. As here just one language is added, it contains just this language:

po/LINGUAS

de

Next the .po file is created. The current .pot file is a template:

# cp po/testproj.pot po/de.po

po/de.po

...

#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: de\n"
"Report-Msgid-Bugs-To: j.t.kirk@ncc-1701.ufp\n"
"POT-Creation-Date: 2006-06-28 23:24+0200\n"
"PO-Revision-Date: 2006-06-28 23:25+0200\n"
"Last-Translator: p.chekov@ncc-1701.ufp\n"
"Language-Team: german\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms:  nplurals=2; plural=(n != 1);\n"

#: src/testmodule/testfunc.cpp:7
msgid "hello world!\n"
msgstr "hallo welt!\n"

#: src/testmodule/testfunc.cpp:8
msgid "Press a key\n"
msgstr "Drücken Sie eine Taste\n"

The replacements in the property part are done only once. The strange 'Plural-Forms' line is discussed in the next chapter.

The translation is added on each translation round for each added or modified source string. There are several tools (like KBabel) which support the translator in maintaining the .po files. Usually, this is done by the translator and belongs to the translation workflow. The translation is added here just to demonstrate some aspects of the translation workflow. In the real world, the new language is now added.

As the file now contains non-ascii characters, it has to be encoded in some way. The encoding used in the rest of the file is specified by the property 'charset'. In this example the de.po file was saved utf-8 encoded. It is important, that the actual encoding of the file matches the Content-Transfer-Encoding line!

Building and Installing the .mo Files

This steps again belongs to the workflow but is explained here: The german .po is converted into the .gmo file. The touch is necessary, as a new language was added. Otherwise, the autotools will detect no change necessary to trigger a .gmo file build!

# touch po/POTFILES.in
# gmake update-po

...

gmake[2]: Entering directory `/usr/home/he/develop/testproj/po'
de:
msgmerge de.po testproj.pot -o de.new.po
. done.
gmake[2]: Leaving directory `/usr/home/he/develop/testproj/po'
gmake update-gmo
gmake[2]: Entering directory `/usr/home/he/develop/testproj/po'
rm -f de.gmo && /usr/local/bin/msgfmt -c --statistics -o de.gmo de.po
2 translated messages.
gmake[2]: Leaving directory `/usr/home/he/develop/testproj/po'
gmake[1]: Leaving directory `/usr/home/he/develop/testproj/po'

'make update-po' runs among other things msgmerge, which merges the latest changes of the .pot file (if any) into the translated de.po file. This was in fact not necessary here. Afterwards it calls 'make update-gmo', which could also have been called directly here.

'make update-gmo' called msgfmt, which converts the .po into the binary .gmo file. This conversion also prints out a statistics: the .po file contained 2 translated strings. Other possible statistical results are shown in the next chapter.

Installing goes as always:

# gmake install

...

Making install in po
gmake[1]: Entering directory `/usr/home/he/develop/testproj/po'

...

/bin/sh .././mkinstalldirs /usr/local/share
installing de.gmo as /usr/local/share/locale/de/LC_MESSAGES/testproj.mo

...

'make install' copies among others this de.gmo into the 'locale' directory. The file is renamed to testproj.mo but placed in a directory, named 'de'. Therefore on runtime the programs (usually) look for their application names in this directory, if the language 'de' was choosen.

Running the Internationalized Program

# testproj
hello world!
Press a key

This is the english version, as the system is set to english. To test other languages, the language has to be changed. There are several way of doing that; one is to set the LC_ALL environment variable.

# setenv LC_ALL de_DE.ISO8859-1 ; testproj ; unsetenv LC_ALL
hallo welt!
Drücken Sie eine Taste

Obviously testproj found the german testproj.mo and used it. Possible values of LC_ALL are listed (on FreeBSD) by 'ls /usr/share/locale/'. Note that not all values will work correctly (e.g. de_DE.UTF-8) as the console used here does not work with utf-8.

On gentoo linux things are vice versa: iso-8859-1 does not work correctly, while utf-8 does:

# LC_ALL=de_DE.ISO8859-1 ; export LC_ALL ; testproj ; unset LC_ALL
hallo welt!
Drcken Sie eine Taste
# LC_ALL=de_DE.UTF-8 ; export LC_ALL ; testproj ; unset LC_ALL
hallo welt!
Drücken Sie eine Taste

Translation Workflow

This is now the usual translation workflow. During the development source files are added, changed and removed. That means that strings are added, changed and removed as well. As an example, one string is changed and another is added:

src/testmodule/testfunc.cpp

#include <stdio.h>
#include "testfunc.h"
#include "../i18n.h"

void printMessage()
{
	printf(i18n("Hello world!\n"));
	printf(i18n("Press a key\n"));
	getchar();
}

src/main.cpp

#include <stdio.h>
#include <locale.h>
#include "testmodule/testfunc.h"
#include "i18n.h"

int main(int)
{
	setlocale (LC_ALL, "");
	bindtextdomain (PACKAGE, LOCALEDIR);
	textdomain (PACKAGE);

	printMessage();
	printf(i18n("Bye\n"));
}

If the project has reached a state, where all string output works (e.g. all dialogs pop up) and where the strings will not change any more (except small fixes), the translators could start work. First, the internationalization maintainer (usually a developer) has to ship the current state to the translators.

Maintainer

Before shipping, the .pot file should be recreated and the .po files should be updated (just to see, if a problem pops up here).

# gmake update-po

...

Building all is also a good idea, so that the project at least builds.

# gmake

...

It might be a good idea to check the translation stage for all languages:

# gmake force-update-gmo
touch po/*.po
cd po && gmake  update-gmo
gmake[1]: Entering directory `/usr/home/he/develop/testpr/testproj-0.1/po'
rm -f de.gmo && /usr/local/bin/msgfmt -c --statistics -o de.gmo de.po
1 translated message, 1 fuzzy translation, 1 untranslated message.
gmake[1]: Leaving directory `/usr/home/he/develop/testpr/testproj-0.1/po'

The current state can be moved to the translators via cvs or via source tarball. Here the latter is described.

# gmake dist-bzip2

...

This tarball now goes to the translators.

Translator

First, the translator has to build the project.

# bunzip2 testproj-0.1.tar.bz2 
# tar -xf testproj-0.1.tar
# cd testproj-0.1
# ./configure

...

# gmake

...

To make shure, that the maintainer did not forget it, the .pot and .po files should be rebuild.

# gmake update-po 

...

Now 'make update-gmo' will print the translation statistics. If it is called again (to see the statistics again), it will show nothing, as the .po file has not been changed. So to enforce a rebuild of the .gmo files, 'make force-update-gmo-%' can be used, which is shown here ('make update-gmo' would have been okay as well):

# gmake force-update-gmo-de
gmake[1]: Entering directory `/usr/home/he/develop/testpr/testproj-0.1/po'
rm -f de.gmo && /usr/local/bin/msgfmt -c --statistics -o de.gmo de.po
1 translated message, 1 fuzzy translation, 1 untranslated message.
gmake[1]: Leaving directory `/usr/home/he/develop/testpr/testproj-0.1/po'

The translator should watch this statistics. It means in this example, that one string was not changed since the last translation, one was changed in the source and needs revision ("fuzzy") and one string is new.

Now the .po file has to be modified. This is usually been done with a help of a tool (like KBabel).

po/de.po

...

#: src/main.cpp:11
msgid "Bye\n"
msgstr "Auf Wiedersehen\n"

#: src/testmodule/testfunc.cpp:7
#, fuzzy
msgid "Hello world!\n"
msgstr "Hallo Welt!\n"

#: src/testmodule/testfunc.cpp:8
msgid "Press a key\n"
msgstr "Drücken Sie eine Taste\n"

If a fuzzy string was revised, the 'fuzzy' line has to be removed. To check the translation, the binary .gmo files are built:

# gmake update-gmo
gmake[2]: Entering directory `/usr/home/he/develop/testproj-0.1/testproj-0.1/po'
rm -f de.gmo && /usr/local/bin/msgfmt -c --statistics -o de.gmo de.po
3 translated messages.
gmake[2]: Leaving directory `/usr/home/he/develop/testproj-0.1/testproj-0.1/po'

Now, all 3 strings are translated and no fuzzy or untranslated strings are left. To test the translation, it should be installed

# gmake install

...

and the program should be started. All string displaying areas of the program should be tested to check the displayed strings.

# testproj
Hello world!
Press a key

Bye
# setenv LC_ALL de_DE.ISO8859-1 ; testproj ; unsetenv LC_ALL
Hallo Welt!
Drücken Sie eine Taste

Auf Wiedersehen

If everything is fine, the now complete .po file should be sent back to the maintainer or checked in into cvs.

Maintainer

The maintainer checks in the .po files, as they come in (if the translators havn't already done it). If all .po files are back, the translation build can start.

# gmake force-update-gmo

...

Afterwards the project is ready for package building and releasing.

It is important to note, that 'make' or 'make all' does not build the localization files. If it did, each source code change could trigger a full rebuild of all .gmo files of all languages. In large projects with lots of languages, this would be a waste of time. So the maintainer has to call 'make force-update-gmo', before the source tarball is released!

The result source code can be downloaded here.

The next chapter discusses some more subtile aspects of the internationalization.

Prev	Home	Next
Adding gettext Support		Internationalization Tips