Proposal: Translation

Toby

The example I gave is for Russian (from here) 🙂

Toby

amdad Bug 🙂 #101

Dakoom

That's great! Simple to understand!

changlon

Is this will include in July beta ?

Franz

changlon That is planned...

Toby

Update: I've finished laying the groundwork for translation. The implementation is slightly different to what I proposed in that language packs will not be extensions, but will rather be a part of the core + each individual extension.

This means that genderization will be an extension, and the locales in core will be able to take advantage of it if it is enabled. Just a heads up: This extension is low priority and will not be ready for beta, we'll get to it eventually though.

Allineer

Toby 😢

amdad

Toby I used to be like that in other systems I work with. That's ok.

Toby Low priority for beta and HIGH priority for final. If we need this extension enabled to fully working translations.

It's a little bit weird. Cause when we say, if you want install translation to language X you need first install and enable extension Y to support it.

I even would keep gender extension as part of core installation of Flarum just disabled by default. When you install a language that use it, then it would switch to ON, if added language needed it.

Updates

Qiaeru As a translator of different kind of software, 100% agree with this. Only default English should be shipped in core. Others as extensions to easier update. Or other form to update only translation/s.

Overrides

Flarum as others should have option to override language strings to make installation more custom/specific.

Toby

Allineer What's your issue with that decision?

Qiaeru

As an open source software translator, I like to manage my updates when I want to. If locales are in the core and that I want to update my translation, my changes will be only available when a new Flarum version will be released, right?

I think translation are like extensions and new versions should be available independently of the software (to fix a typo or to improve a translation for example).

PS: I'm an official phpBB translator (two languages packs which are ones of the most downloaded/used) ready to contribute to Flarum.

Toby

Thanks for the feedback guys.

Qiaeru Welcome! Thanks for your enthusiasm 🙂

Qiaeru Fair point.

My main concern with language packs being separate extensions is (a) the fact that they would need to contain translations for a broad but arbitrary selection of extensions (so everyone gets a whole bunch of code they don't need), and (b) the disparity it would create between versions of core/extensions. So for example: If someone updates core, but needs to hold back on upgrading an extension for whatever reason... but then the language pack has been updated for core and also for the extension, so now it's impossible for this person to have the correct combination of translations for core vs. extension.

One solution is to separate out the language packs into extensions per extension (e.g. a Core French Translation extension + a Categories French Translation extension). But I feel like this just introduces too much complexity, having to manage all those extensions individually.

Another solution (that I'm proposing) is to make translations a fundamental part of their parent package (core/extension). Translators work together with package authors to keep translations up-to-date. We would certainly ensure for core / first-party extensions that translation changes are released quickly as point updates (e.g. 1.0.x). This is the cleanest solution in my opinion. The correct version of the translations are tied directly to the version of the code. And another benefit: every Flarum instance is localizable by default, as a user you don't have to depend on the forum admin to install a language pack that you might want to use.

What do you think @Franz?

Qiaeru

Good point.

I think the core translation should not include translations of extensions, that would be unnecessarily too "heavy".

Toby That solution is nice and user-friendly.

However, it's important to know your policy about translations. I suppose you planned to have translation updates sent via PR on GitHub concerning the core? If so, how do you manage multiple contributors? There are multiple ways to translate something, but the key to have a good translation is consistency. So, in my point of view, a translation "manager" for each language (or a dedicated team) will be great to ensure that translation updates sent via PR are consistent. For example, in French, "Register" can be translated "S'enregistrer" et "S'inscrire". Both are good. But it will be so "unprofessional" if the French translation uses "S'enregistrer" in some places and "S'inscrire" on another.

On phpBB (and I don't say it's the perfect solution, I'm just giving an example here), each translation is managed by the author. He can add co-authors if he wants to. Authors and co-authors are the only ones who can send their translation package into a validation process managed by the Translations & International Support Teams Manager, a member of the official team who ensures that the translation does not contain additional code and meets the coding guidelines. This validation is now automatic thanks to a code checker and TravisCI. If everything is okay, the translation is validated and available for download on the official website.

The issue with this solution is that the manager is alone and when the software is updated, all translators send at the same time their updated translation. It may then take about ten days before your translation is validated when the software is already released (so the only translation available is outdated).

The translation in the core can solve that, because translators are supposed to work with developers. When there is a new language entry, translators are supposed to translate it ASAP and send their update. However, the real issue here is how can you handle abandoned translations? Kick it out of the core? Replace untranslated entries into default English ones?

Each solution has its advantages and disadvantages (I contribute to the translation of open source softwares for ten years so I'm used to that heh), and I hope we will find a solution to suit users, developers and translators. I'm pleased to talk about this with you. 🙂

Franz

I think I'm against including language packs in core. It's just near impossible to keep them up-to-date for all releases. Granted, we'd probably have foreign languages fallback to English anyway in case of missing strings, but it would still leave a bad feeling when an "official" language pack randomly has English strings all over the place.

The best solution in my eyes would be a translation repository (which extensions could also use) where appropriate language packs (for any combination of extension and language) could be downloaded. But that's a lot of ecosystem work that we probably couldn't afford to spend much time on right now.

Franz

Oh, by the way: why exactly do we need to make language a user setting? I'm not against making the platform translatable, but what's the point of making the interface another language when all the content is e.g. Korean?

Isn't that how WordPress does it, too? Have a translatable core platform, but basically ship different, localized versions?

amdad

Franz What about multi lang forums? Where for example you have MAIN English forum, and many subcategories for language specific topics. English is universal. Even here on this forum I would appreciate UI talks to me in my language even if I write here in English.

So installed languages should be options to select not only for whole system, but more for specific user.

Joomla have options like this:

Site language
Admin language

In global site config.

And in user profile:

Site language (for this user)
Admin language (for this user)

Dominion

Oops, here's a point I missed in my earlier response.

Franz Oh, by the way: why exactly do we need to make language a user setting? I'm not against making the platform translatable, but what's the point of making the interface another language when all the content is e.g. Korean?

I don't know if the "user setting" question has been settled one way or the other, but I thought that I should put in my two cents here just in case it hasn't.

As amdad pointed out, multilingual forums are one reason to make language a user setting. But the main reason is this: the user will not necessarily prefer to see the UI in the same language as the content.

I'm a case in point. I've been asked to set up and admin a forum so our Japanese staff can have conversations in Japanese. They obviously need a Japanese UI, so I spent a bit of time translating esoTalk for them (and plan to do the same for Flarum). But as an expat, I'd just as soon admin the site in my native language. Even I know the Japanese UI better than anyone else (being the one who translated it), having the ability to use the English UI for daily admin tasks makes my job that much easier. I probably wouldn't have chosen esoTalk for my site were that user setting not present.

By the same measure, it seems clear that you've got a lot of visitors here who aren't native speakers of English. Once you've got language packs available, you might want to install them on this site so foreign users can see the UI in their own language. It's a gesture of hospitality, really: "We can't answer your questions in Urdu, but at least we can give you Urdu buttons to click." (And in the case of this site, it would serve as a demo that might make it easier for folks to choose Flarum for their forums.)

amdad

Guys, I'm from Joomla world. Where translations (+64 languages!) are treated AS extensions.

Typical language package include all core and core extensions inside. Maintained by it's authors exactly how @Qiaeru described in phpbb community. Transifex are great tool to collaborate on translation work. Where we can setup manager/s and collaborators to active maintain updates as source/s (English) is changed or added.

Joomla have inbuild another (optional) step in installation process where we can quickly install needed language package/s which contains all core and core extensions files. Those packages can be uninstall as normal extensions and update separately from core. Any time and from ANY SOURCE. You can also install unofficial language pack if you want as another extension.

Also there is another tab in extensions installer to install/uninstall official lang packs from official branch.

At least, you can official strings with your own custom needs to have totally unique project-specific phrases on your site. If for example you don't want name people as users but as kids 🙂 You can do it.

So translations can be on github in separate repo, but it be much greater to have them on Transifex. See new Pagekit CMS. Or Joomla's oriented projects on Transifex where people not translate only core and core extensions but also 3rd party the same way.

So one ZIP to rule them all! COREa and COREs.

amdad

Please remember about gender problem, which is mandatory for support many languages properly. Without it, we just do what every project I know out here.

Which is improper & ungrammatical translations.

I really don't think such a simple thing as this should be extension. Rather disabled option in core. Enabled automatically or manually, when we add demanding it language pack.

Toby

I think I'm against including language packs in core. It's just near impossible to keep them up-to-date for all releases. Granted, we'd probably have foreign languages fallback to English anyway in case of missing strings, but it would still leave a bad feeling when an "official" language pack randomly has English strings all over the place.

True, but I imagine this would become less of a problem over time, as translations get more and more complete and interfaces change less and less (hopefully). There would certainly be instances where the translations lag behind a bit, but if there were enough people using them they'd be quickly fixed so I don't think it'd be a huge deal. And I guess we'd only include each language pack in core once they pass a certain level of coverage.

The best solution in my eyes would be a translation repository (which extensions could also use) where appropriate language packs (for any combination of extension and language) could be downloaded. But that's a lot of ecosystem work that we probably couldn't afford to spend much time on right now.

Yeah this might be ideal, but it's complex. I think you'd basically need a GitHub organisation for each locale, containing a repo per core/extension, and versions are then tagged within each repo. You could end up with hundreds of repos per locale which would be hard to manage. And then the mechanism to select and download and update all those files in Flarum would be hellish.

Regardless, this is basically where my thinking was leading anyway. If we were to assign an author to manage each locale, as @Qiaeru suggested, then it would make sense to keep each locale in its own GitHub repository under the Flarum organization and give the author write permission... similarly it wouldn't make sense to keep locales in the same repo as each extension, as it's a lot of responsibility for the extension dev to accept (or not accept) PRs. But I think that many repos is just too complex.

I think it boils down to two viable options:

Language packs are extensions, and contain translations for core and an arbitrary number of first- and third-party extensions. Ignore the possible disparity between installed extension/language pack versions; just encourage everyone to have the latest version of everything installed, and it should work well enough. Each language pack has its own repo managed by an author, who can accept PRs.
Language packs are a fundamental part of core + each extension, living directly inside the repos. Use something like Transifex to manage these translations (for core and first-party extensions) all in one place and assign authors/managers. Third-party extensions would still have to rely on the dev accepting PRs.

I've changed my mind and now I'm thinking option 1 gives the best balance. Thoughts?

amdad

Toby If you plan to keep Flarum extensions community small. Then YES. 😉 We can include them. But any other way NO, there should not be 3rd party translations in official language pack. There is many other way to support community extensions translation. From simple "3rd party extensions translations repo" or like I said, as separate Transifex project for each one of them.

3rd party extensions living own life. In many cases no translation is better then bad translation. Updates of 3rd party extension is up to it's author. Translator should follow those updates not Flarum updates.

By including 3rd party translation, you also brake rules of KISS and minimal core. Official translations should be as minimal as Flarum core (incl. core extension) is.

Look how Joomla community collaborate to translate 3rd party extensions: OpenTranslators.org

Man, I translated many 3rd party extensions which where many times bigger than Joomla itself.

Qiaeru

Again, I think that include translations of extensions into the core translation is a bad idea. It will unnecessary increase the file weight, and I think the default package of Flarum should be as light as possible, without useless/unused lines of code.

Also, I think extension developers should manage translations on their own, not Flarum developers. See an extension as a third-party product, Flarum should not manage them (unless the extension do not meets the coding guidelines or have security issue of course). Don't worry, if an extension is popular, translators will be there for it.

To illustrate my words above:

Example of a core translation architecture:
system/languages/(iso)/file1.yml
system/languages/(iso)/file2.yml
system/languages/(iso)/file3.yml
...

Example of the content of the "Hello World" extension package:
system/languages/(iso)/extensions/helloworld.yml
system/extensions/helloworld/file1.php
system/extensions/helloworld/file2.php
system/extensions/helloworld/file3.js
...

(iso) as ISO code, like fr, en-gb, en-us, de, pt, pt-br, etc.

Also, amdad got a point. Translations should be independant from the default package. I know it's more user-friendly to have them in core, but it will be a pain in the ass for developers to handle them (like abandoned translations, outdated translations, or translators asking some more time which delay the software release, etc.). It's sad to say, but you can't just trust your translators to update their translations with the latest version. I'm a former Translations Manager on the phpBB software, and everybody are not as serious as you want to. Do not forget it's a free and volunteer work, so people are doing that on their free time. I always kept my translations up-to-date, released them at the second a new software version is available, but that's because I'm passionate and involved. But trust me, it's not the case of everyone, some of them just want a software in their language in a specific priod of time, and do not reappear again, or very rarely.

Is it possible to just improve the admin panel to install and delete translations with a one-click feature? I'm not a developer, so I don't know how this will be hard (or heavy) to implement. I know WordPress handle one-click installations of extensions, styles and translations very nicely. This will solve the "user-friendly" thing.

amdad

Qiaeru If make translations as extensions. It should be easy, if we have extension manager to install extensions from web or local package. Enable/disable & install/uninstall. Modern software Wordpress, Joomla, Prestashop have inbuild (in admin) 'marketplace' to install extensions from official extension directory. Handling language extensions management is easiest to do because of it's simplicity. There is no need to have marketplace or even complicated local extension installer.