modern javascript localization with c-3po and the good old gettext

Modern javascript localization with good

old gettext

https://c-3po.js.org/

Mostovenko Alexander

Company: EVO company

Twitter: @MostovenkoA

Github: github.com/AlexMostMe

About this talk

Localization process in general and more about gettext format.

About this talk

What is c-3po and how it helped us to improve existing process?

About this talk

What is c-3po and how it helped us to improve existing process?

Continuous translations.

Our initial tasks

1. Localize frontend on several projects:

(cabinet, company sites)

Bigl.ua

others...

Our initial tasks

1. Localize frontend on several projects:

(cabinet, company sites)

Bigl.ua

others ...

2. Setup translation process when devs and translators can work independently.

Gettext (.po)

...Localization

formats

ICU - International Components for Unicodehttp://userguide.icu-project.org/

ICU features >>> gettext features

ICU is cool, but ...

1. Poor tooling (editors for translators e.t.c).

ICU is cool, but ...

1. Poor tooling (editors for translators e.t.c).

2. Our backend already used gettext.

ICU - cool but ..."{gender_of_host, select, " "female {" "{num_guests, plural, offset:1 " "=0 {{host} does not give a party.}" "=1 {{host} invites {guest} to her party.}" "=2 {{host} invites {guest} and one other person to her party.}" "other {{host} invites {guest} and # other people to her party.}}}" "male {" "{num_guests, plural, offset:1 " "=0 {{host} does not give a party.}" "=1 {{host} invites {guest} to his party.}" "=2 {{host} invites {guest} and one other person to his party.}" "other {{host} invites {guest} and # other people to his party.}}}" "other {" "{num_guests, plural, offset:1 " "=0 {{host} does not give a party.}" "=1 {{host} invites {guest} to their party.}" "=2 {{host} invites {guest} and one other person to their party.}" "other {{host} invites {guest} and # other people to their party.}}}}"

GNU gettexthttps://www.gnu.org/software/gettext/

1. Simple format.

GNU gettexthttps://www.gnu.org/software/gettext/

1. Simple format.

2. Has good ecosystem for translation process.

gettext idea

‘Hello world’

Source code

gettext idea

gettext(‘Hello world’)

Source code

gettext idea

Source code

‘Hello world’

.pot file

gettext idea

Source code

msgid: ‘Hello world’msgstr: ‘’

.po file

‘Hello world’

.pot file

gettext idea

Source code

msgid: ‘Hello world’msgstr: ‘Здоровеньки були’

.po file

‘Hello world’

.pot file

gettext idea

Source code

msgid: ‘Hello world’msgstr: ‘Здоровеньки були’

.po file

‘Hello world’

.pot file

gettext idea

gettext workflow

Code .pot Extracting translation strings from code (AST traverse)extract

gettext workflow

Code .pot Extracting translation strings from code (AST traverse)

.pot .po Merge new extracted translations from .pot file with existing. msgmerge

extract

merge .po.po

gettext workflow

extract

Translator adds translationstranslation

.po.po

.po.po.po.po.po.po

gettext workflow

extract

Translator adds translationstranslation

.po.po

.po.po.po.po.po.po

resolve CodePlacing translations back to the code (by locale).po.po.po

.po file structure

Example .po file

msgid ""

msgstr ""

"Project-Id-Version: c 3po-webpack-start\n"

"PO-Revision-Date: 2017-02-01 22:22+0200\n"

"Language-Team: Ukrainian\n"

"Language: uk\n"

"MIME-Version: 1.0\n"

"Content-Type: text/plain; charset=UTF-8\n"

"Content-Transfer-Encoding: 8bit\n"

"Plural-Forms: nplurals=2; plural=(n!=1);\n"

#: app.js:6

msgid "Choose locale"

msgstr "Оберіть локаль"

#: app.js:10

msgid "${ 0 } second"

msgid_plural "${ 0 } seconds"

msgstr[0] "${ 0 } секунда"

msgstr[1] "${ 0 } секунди"

msgstr[2] "${ 0 } секунд"

#: app.js:14

msgid "webpack with c-3po localization demo"

msgstr "Демо локалізації з c-3po та webpack"

msgid ""

msgstr ""

"Language: uk\n"

#: app.js:6

#: app.js:10

#: app.js:14

Headers

msgid ""

msgstr ""

"Language: uk\n"

#: app.js:6

#: app.js:10

#: app.js:14

Translations

"Plural-Forms:

nplurals=2;

plural=(n!=1);\n"

Plural-forms header

"Plural-Forms:

nplurals=2;

plural=(n!=1);\n"

Plural-forms header

Plurals count

"Plural-Forms:

nplurals=2;

plural=(n!=1);\n"

Plural-forms header

Plurals formula

#: app.js:6

1 to 1 translations

#: app.js:10

msgid "${ n } second"

msgid_plural "${ n } seconds"

msgstr[0] "${ n } секунда"

msgstr[1] "${ n } секунди"

msgstr[2] "${ n } секунд"Plural forms example

#. Notes for the translator

#: app.js:6

#, flag

Comments

msgctx "email"

msgid "Hello"

msgstr "Привіт"

msgctx "main page"

msgid "Hello"

msgstr "Вітаємо"

Context example

c-3pohttps://c-3po.js.org/

Code .pot Extracting translation strings from code (AST traverse)extract

resolve CodePlacing translations back to the code (by locale).po.po.po

Library:

https://github.com/c-3po-org/c-3po

Core functionality.

Translation functions (tags):

t, ngettext, gettext, jt

Babel-plugin:

https://github.com/c-3po-org/babel-plugin-c-3po

Extract, Resolve, Validation

Key ideas

Key idea 1. Tagged template strings for formatting

c-3po translations are based on tagged template strings

import { t } from 'c-3po'

const name = 'Mike';

console.log(t`Hello ${name}`);

What is a template string?

What is template string?

console.log('Hello ' + name)

console.log(`Hello ${name}`)

What is a tagged template string?

const name = ‘Mike’

t`Hello ${name}`

function t(strs, ...exprs) {

// strs -> [‘Hello’, ‘’]

// exprs -> [‘Mike’]

Valid javascript

t `Hello ${name}`

t(5)`Hello ${name}`

t(5) `Hello ${name}`

Why just not simple functions but tags?

c-3po must work without transpilation

Why just not simple functions but tags?

// Without babel transpile !!!

const name = ‘Mike’

t(`Hello ${name}`)

function t(str) {

// strs -> ‘Hello Mike’

// no chance to get ‘Hello ${name}’ msgid here :(

Existing solutions

Force you to use sprintf formatting(i18next, jed e.t.c)

sprintf

gettext("Hello %s", user)

gettext("%2$s %3$s a %1$s", "cracker", "Polly", "wants")

gettext("Current timestamp: %d", Date.now)

sprintf

● Alien formatting for js

sprintf

● Alien formatting for js● No reason to use es6 template strings

sprintf

● Alien formatting for js● No reason to use es6 template literals● Extra CPU work on the client (sprintf parsing)

Key idea 2. Precompile translation on a build step

Traditional translations resolve flow

Load assets to the client (browser)

Resolve locale (from cookie, params e.t.c)

Fetch locale data

Apply translations

Wanted translations resolve flow

Wanted translations resolve flowLoad assets to the client

(browser)

Wanted translations resolve flowLoad assets to the client

(browser)

That’s it, they are already localized !!!

Benefits of precompiled translations:

● Smaller bundle size

● Smaller bundle size● Less work on the client

● Smaller bundle size● Less work on the client● No wait until translations are loaded and applied

Key idea 3.Works with and without babel

Demo on jsfiddle

Key idea 4. Can extract and resolve translations from

everything that works with babel (jsx)

Extract and resolve translations with simple config

Extract:

{ "extract": { "output": "extract.pot" } }

Resolve:

{ "resolve": { "translations": "uk.po" } }

https://c-3po.js.org/quick-start.html

Key idea 5. Efficient dev and prod setup

Requirements for the dev setup

Requirements for the dev setup● Faster builds

Requirements for the dev setup● Faster builds● Simple integration

Dev setup with c-3po

Dev setup with c-3po● Use whole c-3po lib (import c-3po).

Dev setup with c-3po

● Use whole c-3po lib (import c-3po).

● Babel plugin only for extraction translations

Requirements for the prod setup

Requirements for the prod setup● Smaller result assets

Requirements for the prod setup

● Smaller result assets

● Faster locale load

Prod setup with c-3po

Prod setup with c-3po● Use c-3po mock lib (alias in webpack e.t.c).

● Use c-3po mock lib (alias in webpack e.t.c).

● Babel plugin for transformations.

● Use c-3po mock lib (alias in webpack e.t.c).

● Babel plugin for transformations.

● Separate build for each locale.

Key idea 6.Validate translation strings

Validation problem

msgid "http://some.random.domain.link"msgstr ?

msgid ""msgstr ?

msgid "translate ${ test ? result1() : result2() }"msgstr ?

msgid "5"msgstr ?

c-3po validation (empty str)

Module build failed: SyntaxError: Can not translate empty string

> 7 | t``

c-3po validation (only numbers)

t`3243243`

ERROR in Error: Module build failed: SyntaxError: Can not translate '3243243'

> 4 | t`3243243`

c-3po validation (programs inside translations)

t`Translate ${someVar ? callFn1() : callFn2()}`

ERROR in Error: Module build failed: SyntaxError: You can not use

ConditionalExpression '${someVar ? callFn1() : callFn2()}' in localized

strings

> 4 | t`Translate ${someVar ? callFn1() : callFn2()}`;

c-3po validation (only var wrap)

t`${someVar}`

ERROR in Error: Module build failed: SyntaxError: Can not translate '$

{ someVar }'

> 4 | t`${someVar}`;

msgid:’${someVar}’

Validate not translated strings

Validation (no translated str)

t`non translated str`

ERROR in Error: Module build failed: SyntaxError: No "non translated str" in

"uk.po" file

> 3 | t`non translated str`;

https://c-3po.js.org/configuration.html#configresolveunresolved-string-one-of-fail-warn-skip

Key idea 7.Can use any default locale in sources

(standard gettext uses only English locale)

The problem

Standard ngettext accepts only 2 plural forms

char * ngettext(const char * msgid1, const char * msgid2, unsigned long int n);

Ukrainian/Russian strings in sources

How to be with plurals?

How to be with plurals? Standard ngettext accepts only 2 plural forms?

How to be with plurals? Standard ngettext accepts only 2 plural forms? What if we have more or less than 2 plural forms?

ngettext for other locales ( plural forms > 2 )

Doc - https://c-3po.js.org/ngettext.html

Change defaultHeaders setting to:

'plural-forms':'nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2);'

ngettext for other locales ( plural forms > 2 )

Demo on jsfiddle

Change defaultHeaders setting to:

'plural-forms':'nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2);'

ngettext(msgid`${n} банан`, `${n} банана`, `${n} бананів`, n)

c-3po tags and functions

t - tag gettext

Demo on jsfiddle

Doc - https://c-3po.js.org/tag-gettext--t-.html

t - tag gettext

t`Hello ${name}` < Usage

Demo on jsfiddle

t - tag gettext

t`Hello ${name}`

msgid "Hello ${ name }"msgstr "Привіт ${ name }"

< Usage

< Extract

Demo on jsfiddle

t - tag gettext

t`Hello ${name}`

msgid "Hello ${ name }"msgstr "Привіт ${ name }"

< Usage

< Extract

`Привіт ${name}` < Resolve

Demo on jsfiddle

ngettext - plural forms

ngettext(msgid`${n} time clicked`, `${n} times clicked`, n) < Usage

ngettext - plural forms

ngettext(msgid`${n} time clicked`, `${n} times clicked`, n)

msgid "${ n } time clicked"msgid_plural "${ n } times clicked"msgstr[0] "${ n } time clicked [translated]"msgstr[1] "${ n } times clicked [translated]"

< Usage

< Extract

Forgot msgid ?

ERROR in Error: Module build failed: SyntaxError: First argument must be tagged

template expression. You should use 'msgid' tag

> 4 | ngettext(`${hours} hour`, `${hours} hours`, hours);

Demo on jsfiddle

ngettext - resolve

function _tag_ngettext(n, args) {

return args[+(n != 1)];

_tag_ngettext(n,

[n + " time clicked [translated]", n + " times clicked [translated]"]));

Demo on jsfiddle

First contribution !!! jt (jsx-gettext)

jt - jsx gettext

const btn = <button key="btn">{ t`me`

}</button>;

<span>{jt`Click ${ btn }`}</span>

< Usage

jt - jsx gettext

}</button>;

< Usage

< Extractmsgid "Click ${ btn }"

msgstr "Click ${ btn } [translated]"

jt - jsx gettext

}</button>;

< Usage

< Extractmsgid "Click ${ btn }"

msgstr "Click ${ btn } [translated]"

['Click ', btn, ' [translated]'] < Resolve

gettext

gettext('simple gettext'); < Usage

gettext

< Extractmsgid "simple gettext"

msgstr "simple gettext [translated]"

gettext

< Extractmsgid "simple gettext"

msgstr "simple gettext [translated]"

'simple gettext [translated]' < Resolve

Why add gettext func if we have ‘t’ tag?

Legacy - *.coffee *.eco (100 K+ LOC)

1. Wrap string in gettext inside *.coffee *.eco

2. Add babel loader after coffee loader to the

webpack pipeline

3. Profit !!!

Even more features

Aliasinghttps://c-3po.js.org/aliasing.html

Babel plugin will extract aliased translation

import { t as i18n } from 'c-3po'

i18n`this translation will work`

Multilinehttps://c-3po.js.org/multiline-strings.html

import { t } from 'c-3po';

function test(name) { return t`multi line string with multiple line breaks and with formatting ${name}`}

Multiline (auto dedent)

Translator comments

Translator comments// translator: some description for the extracted string. t`some string`

#. some description for the extracted stringmsgid "some string"msgstr ""

Summary

Summary● Native js tagged template strings for

formatting

formatting● Can precompile translation on a build step

formatting● Can precompile translation on a build step● Works with and without babel

formatting● Can precompile translation on a build step● Works with and without babel● Extracts and resolves translations from

everything that works with babel (jsx)

everything that works with babel (jsx)● Efficient dev and prod setup

everything that works with babel (jsx)● Efficient dev and prod setup● Validation for translated strings

everything that works with babel (jsx)● Efficient dev and prod setup● Validation for translated strings● Can use any default locale in sources (not

only English)

Continuous translations process

Devs VCSrepo

Continuous translation process

Translator

Devs VCSrepo

Continuous translation processpush

Translator

Devs VCSrepo

Continuous translation processpush notify

Translator

Devs VCSrepo

translate

notify

Translator

Devs VCSrepo

translate

notify

update Translator

Tools for translators

Poeditor - https://poeditor.com/

Transifex - https://www.transifex.com/

Weblate - https://weblate.org

Devs VCSrepo

update Translator

notify

translate

Weblate example

Weblate - https://docs.weblate.org/en/latest/

● c-3po doc - https://c-3po.js.org/

● c-3po quick start - https://c-3po.js.org/quick-start.html

● gettext - https://www.gnu.org/software/gettext/

● ICU - http://userguide.icu-project.org/

● Weblate - https://weblate.org

Thx :)

modern javascript localization with c-3po and the good old gettext

Education

i18nize scala programs à la gettext

passive transport across a cell membrane, diffusion and...

javascript session 01 - introduction to javascript

javascript javascript introduction. q. what is javascript?...

· web viewr1-g4, r2-d2, it-o, a4-d, r-3po, 4-lom, r2-q5,...

javascript awesomeness: javascript for beginners

javascript (js) 3.pdffull stack javascript (หร อ...

gnu gettext tools, version 0.20 · 2020-04-14 · gnu...

the 3po model · what are the benefits of implementing the...

lego star wars c-3po papercraft minifig parts

odonnell.esc17.netodonnell.esc17.net/page/open/4755/0/star...

javascript i -...

javascript for beginners - daniel...

multi language for php with gettext

eiw: javascript the language1 the javascript language

javascript for · pdf...

lego star wars c-3po papercraft minifig...

javascript specialist. lesson 1: introduction to javascript

javascript guide - belajar pemrograman javascript

the 3po model - generis