So the only people exposed to apocalypse are those who (a) change language mid-game; (b) know enough about the game to do so by editing the ini file; (c) don't realise that doing something radical like that with mods installed is a bad idea. The interection of (a)-(c) is quite small.
Like David and GeN1e mention, players are unlikely to be switching languages much, and I'd say it's on them if they do so.
I do not agree with that. With old games, the only way to change language was to overwrite the dialog.tlk file. So you knew what you did, as a player, even if you didn't realize that mods were adding texts to that file. With BGEE, all you need is go in a menu and select another language. And that just works, so people may get used to it for some reason (I am using this when a translation does not look right). Since this behaviour is currently broken by installing mods, I strongly support Wisp's idea of updating all languages at the same time. The player knows he/she installed the mod in a specific language so we can assume there won't be complaints that swithcing language didn't apply to the mod content.
However there is a pitfal to updating all languages : currently the english dialog.tlk is always ahead of other languages. Typically it has more strings than other languages. For instance, version 1.0.2014 goes up to @32149 in english and only @32140 in French (and the last two string are even empty, contrary to English).
If you want to allow switching between languages, I would suggest considering that the English language is always the reference to determine what is the first available StrRef. Then WeiDU should fill the missing strings in other languages, if needed, before adding the mod strings.
I think there is another "Oh, crap" here - what happens to string patching? Some of BG2Tweaks' and Item Revisions' components rely heavily on reading a string, REPLACE_TEXTUALLY-ing something within, and then SAY_EVALUATED-ing it back.
And with some imagination, worse stuff may happen when using REPLACE_EVALUATE. E.g. I use it in a couple of IR's components to log encountered bugs in descriptions, but in theory it may as well affect the resulting installation depending on what language has been chosen. An exaggeration it is, most likely, but better safe than sorry?
This issue is not specific to BGEE and its multiple languages. People who use mods that rely on REPLACE_TEXTUALLY in other languages than the one the mod was written are already likely to face such issues. Because, in another language, there may be various ways of writing something compared to English, so French may require a regexp to catch something while it wasn't anticipated by the mod author. Besides, people who install IR in English because there is no translation in their language, even if the game exists in that language, don't get the text updated either.
BGEE will not bring specific issues regarding these cases, I think.
Should new options be added to WeiDU language related instructions in order to handle character encoding, I believed adding the ability to specify the original encoding of the tra files for each language would help implementing an automated way to convert existing tra files, that use various Windows encoding, to UTF-8 on the fly at install time. Something like
LANGUAGE_EE ~Francais~
~CP1252~
~french~
~bg1npc/tra/french/setup.tra~
CP1252 being the code used in iconv for the corresponding encoding. I am assuming that, given the history of mods, the reference encoding for the tra files included in the mod would be the "old" 8 bits one and that WeiDU would only convert if the game is BGEE.
For reference,
here is what I experimented with the encoding issues and how to overcome them.