Toby jumped in before I was quite done. Then I was called away for dinner. And so now, newly refortified by a big bowl of bibimbap, I give you the final chapter (ta-dah!):
Stumbling block 6: content-related issues
... which isn't really a stumbling block, so much as a grab-bag of things that caused me a bit of trouble (or at least made me scratch my head) while translating esoTalk. I suspect that some of these may already be non-issues in Flarum, but it might be worth jotting my comments down here in case they'll come in handy while planning new features or extensions.
I was pleased that esoTalk had very few problems turning Japanese thread titles into pretty URLs. But there was a bit of oddness that puzzled me. For example, the URL for a thread titled
which means, "Read this before using this 'DCS' conference room!" would end up looking something like this:
It seems the Japanese in the URL has been adjusted by some processing intended to render it compatible with the ASCII standard. The 「 and 」 (which are quotation marks, incidentally) have been replaced by hyphens, and the final full-width exclamation point has been dropped. Also the capital letters have been made lower-case, even though they remain full-width and so are not ASCII compliant. All in all, it's a fascinating display of how processing intended for Latin strings can have the intended effect on Unicode. And it's not a huge problem, since the URLs are still pretty readable. But since we're talking about a Japanese URL in any case, I can't help but think: "Why bother?"
If current plans call for the inclusion of similar URL processing in Flarum (as appears to be the case), perhaps it would be a good idea to allow admins to turn it off when it's not needed? Of course, the same would apply to channel slugs (er, I mean tags): it would be nice to be able to use capital full-width letters there as well.
This isn't exactly a "content-related" issue, but...
I had fun trying to translate all the search gambits properly, and think I more or less succeeded. But because some of the gambits rely on non-Japanese glyphs like half-width colons, spaces, and question marks to determine how much of the text is selected by default, the results aren't quite as polished as I would have liked. Also, it was getting a bit hard to avoid Yoda-speak on the more complex gambits, mainly due to the need to pack a lot of information into a small space.
Since Flarum won't be able to search for text strings in Japanese (not to mention Chinese, Korean, etc.) unless the admin can supply the necessary infrastructure (which will require, at minimum, a VPS or dedicated server with root privileges), the ability to search using a combination of other criteria will be especially important for applications on shared or managed hosts. That being the case, I'd like the advanced search function to be as polished and useful as possible.
I don't how the plans for advanced searching in Flarum currently stand, but I've been wondering if it wouldn't be better to go with a more conventional "advanced search" page with the various criteria laid out in some slick fashion. That would pose less of a challenge for translators; and ultimately, I think users may find it easier that way to keep track of which criteria they've turned on for a specific search.
Flarum doesn't seem to have a member list page yet, so I can't be sure whether you're planning on adapting this feature from esoTalk. But just in case you are, here's a few thoughts on the subject:
Generally speaking I think it's a cool feature and a spiffy bit of code; but it's pretty much wasted on a forum whose members will mostly have user names in non-Latin characters. So ... I don't know if it'll make much difference, but if you're going to include the alphabetical scrubber, perhaps it would be a good idea to add an option to turn it off? On a forum where hardly any names are alphabetize-able, it would only be in the way.
The alternative would be to get fancy and make the scrubber itself localizable. The language pack could include a "namescrubber" file that would list the characters to be used by the scrubber, in the proper order. When you install the language pack, poof! The scrubber changes. But to make this work, there's a couple obstacles that need to be dealt with.
First, you wouldn't want the name scrubber to change depending on the user's language setting. If most of the users' names are in Japanese, for example, then a user who changed the UI language to English would be rendering their scrubber unusable. So you'd need to include a "scrubber language" setting in the admin options, which would set it for all users, period.
Second, while the "name scrubber" file might be enough to handle most languages, there are some languages that will need even more. Japanese is one of those. I'll go into a bit of detail in the next item.
Japanese user names can include characters from either of two syllabaries (of around 50 kana each) and a couple thousand or so Chinese characters. And while it's possible to do some processing and sort names using just the kana, the pronunciation of all those Chinese characters isn't fixed: most have at least two pronunciations, and many have several possibilities. So there's no easy way for a program to look at a Japanese name and decide how it is pronounced, or how to sort it with other names.
Most Japanese websites get around this problem by adding a "reading" field, which is restricted to characters from one of the two syllabaries. Users enter the reading of their name in this field, and this data allows the program to sort their name correctly.
So in order to enable localization of the name scrubber for Japanese, we would need to give admins some way to define custom profile fields, including a way to constrain a field so it only allows characters from a specified syllabary, as well as a way to specify that a given custom field is to be used instead of the username as the sort criterion. All told, that's a pretty tall order. In fact, in consideration of this, I'm beginning to think it might be best to put the name scrubber in an optional extension.
But even if we go that route, we still need to consider what happens to the member list when sorting is left up to the database. We can be pretty certain that many names will be out of the expected order. If the forum user base is large, names will be hard to find. A good username search function is clearly going to be needed, but it seems some way of specifying how a name is read would probably be a good idea in any case. Is this the sort of thing that can be done with an extension? And what is the relationship of the extension to the language pack? Would it be part of the language pack? or a recommended add-on?
This is a peculiarly Japanese issue, but I'd guess there are at least a few languages out there with similar issues.
Localization of editor buttons
Okay, this is not a major issue, but ... while translating esoTalk, I found myself wondering if it would be possible to replace a couple of the editor buttons with more Japanese alternatives. For example, it might be nice to have a bold 太 ("fat") character instead of a B, or an italic 斜 ("slanted") character instead of an I. And so on.
This isn't absolutely necessary, since Japanese users have become accustomed lately to seeing the standard buttons, but I thought it might be a slick touch to add. Would it be possible to create such buttons and package them with a language pack?
Of course, it would be necessary to provide some way for the buttons to change when the user changes the UI language setting. And there are interactions with alternate markup methods to consider. On the other hand, localized editor buttons might be a nice addition in more languages than Japanese. Just a thought ...