Post reply

Warning: this topic has not been posted in for at least 120 days.
Unless you're sure you want to reply, please consider starting a new topic.
Name:
Email:
Subject:
Message icon:

Verification:
Type the letters shown in the picture
Listen to the letters / Request another image

Type the letters shown in the picture:
What color is grass?:
What is the seventh word in this sentence?:
What is five minus two (use the full word)?:

shortcuts: hit alt+s to submit/post or alt+p to preview


Topic Summary

Posted by: jcompton
« on: July 21, 2014, 08:33:29 AM »

I saw HANDLE_CHARSETS and it's really handy. But you have to face reality that most of the old mods will never be updated due to "you can't host fixed mod archive without signed permission from the author that is absolutely beyond any way of reaching".

It's probably not as bad as you think, actually. Although numerically you may be right that a maintainer cannot be feasibly reached for "most" mods (as there are many, many mods out there), I submit that a responsible party can be reached to bless a straight-up EE fix for the vast majority of mods that people actually play.

People can contact me to submit EE fixups of all PPG and WeiDU.org mods, for example. (And I believe at this point that basically everything on PPG which anybody plays has been fixed.) I'd be surprised if other sites didn't have a similar policy among their remaining overlords and maintainers.
Posted by: Kaeloree
« on: July 21, 2014, 05:52:48 AM »

IE modding is already a legal grey area. It does pay to be careful in our case.
Posted by: AL|EN
« on: July 21, 2014, 05:22:00 AM »

I saw HANDLE_CHARSETS and it's really handy. But you have to face reality that most of the old mods will never be updated due to "you can't host fixed mod archive without signed permission from the author that is absolutely beyond any way of reaching".

Even if I feel that it's a lost case already, I'll show you what it looks like from my perspective. Thats an non-english user who install mods via BWS.

First is the license problems. It's exaggeration of the problem. Period. License for weidu is one thing and license terms for (lib)iconv is another thing and
zipping two tools and two license files into one zip file should be no problem for anyone. especially that is so valuable for this community.

Second:

1. Yes, It can fail in 0.01% cases. This particular mod needs to be updated anyway and renaming one directory is very easy. For the rest of 99.99% it would be fine.
2. Old and not updated mods have ANSI+local codepage character set. I've just checked for eg: Chloe and it's install perfectly fine on BG2:EE when you just convert polish files to UTF-8 using Windows-1250 character set. Same goes for almost all old mods. If there will be a problem, then the mod should get an update. All mods that can be converted automatically deserve it.
3. You can never avoid all of the bugs/crashes. No need to update the old mod is for me something worth every effort and that "something can go wrong sometimes"

Since you already put huge amount of work to wiedu I won't nag you about this anymore.
Posted by: AL|EN
« on: July 19, 2014, 11:40:29 AM »

I'm dying today from my first ever allergic reaction. I will reply when I feel better.
Posted by: Wisp
« on: July 19, 2014, 10:32:10 AM »

Not that you are likely to miss it anyway, but HANDLE_CHARSETS (as it came to be called) is expounded upon here.
Posted by: Wisp
« on: July 18, 2014, 02:49:12 PM »

I expect I'll present what findings I have at some point (spoiler: the outlook's bleak. Mostly due to licencing.)
Okay, that's probably enough suspense.
The thing is, I can't do this because of setup-mymod.exe. WeiDU is licenced under modified GPL terms that permit you to distribute unaltered binaries without any further conditions. There appears to be two possible conversion libraries available (libiconv and Camomile) and they are both licenced under regular LGPL terms, which do not permit the distribution of binaries without additional conditions. I can't very well say "From now on, additional terms and conditions apply to the distribution of WeiDU that did not apply before. I abide by them. Whether you do the same is your business *wink wink, nudge nudge*". Hence, I can't include conversion abilities in WeiDU due to setup-mymod.exe.

However, at the risk of giving things away, perhaps I should mention that I am working on something that will hopefully make setup-mymod.exe obsolete. Once you've gathered around to sing Ding-Dong! The shell menus are dead I'll be able to revisit this problem and most likely implement a nice solution. Until then, I guess I'll try to make the HANDLE_* function user-friendly.

1. Setup-oldmod.exe is executed, user selected "Polish" translation,
2. WeiDU detects XX:EE and also read folder with translated files are. Path contain "polish".
3. WeiDU reads "Conversion map:"

It is possible, but there are a few problems.
1. The name of the language directory is completely arbitrary. It will probably work for most cases, but when it doesn't, it'll get it wrong.
2. There are no guarantees about the encodings (the actual encoding may be different from what we think it is) or the sane-ness of the implementation (e.g., some implementations mix different encodings in the same file.) and WeiDU, by itself, can't know if this is a TRA file that contains strings that should be added to the game, or if it is a TRA file that contains strings that are displayed during the installation. The latter should be converted, the former should not be and if the two kinds of strings exist in the same file, all bets are off.
3. I hate automatic systems that can't be reasonably relied upon to get things right. (In this particular case, the possible treats include unexplained crashes and other problems because no one vetted the compatibility and patch after patch after patch to try to get the automatic-ness right.)
Posted by: AL|EN
« on: July 18, 2014, 03:42:55 AM »

I have no doubt that HANDLE_* funcion would be useful. But I have question:

It should be easy for WeiDU to read folder name of user selected translation. With this information how about this approach:

1. Setup-oldmod.exe is executed, user selected "Polish" translation,
2. WeiDU detects XX:EE and also read folder with translated files are. Path contain "polish".
3. WeiDU reads "Conversion map:"

IF %tra_folder_name% contains "polish"/"polski" > set encoding to Windows-1250
IF %tra_folder_name% contains "German" > set encoding to ...

and use some sort of open-source character converter or build-in function to convert content of those files on the fly (or eventually make converted copy of them with "_utf8" appendix) and use them for current installation.

This way an old mod can be successfully installed using all translations. Is this approach is possible ?
Posted by: Kulyok
« on: July 18, 2014, 12:30:38 AM »

Wish we modders could buy ourselves a license(if it's affordable financially), but given our grey status copyright-wise, that probably isn't going to work. Thank you so much for looking into this!
Posted by: Kaeloree
« on: July 17, 2014, 04:42:07 PM »

A HANDLE_* function would be awesome.
Posted by: Wisp
« on: July 17, 2014, 12:08:45 PM »

Okay, your choice.

When I asked about "handle this by weidu completely" what I mean is that weidu will convert all ANSI files to UTF-8 on the fly when the game will be XX:EE and without any modification to original mod.
I have been looking into adding a more transparent method of dealing with this. I expect I'll present what findings I have at some point (spoiler: the outlook's bleak. Mostly due to licencing.) But mods would still have to be updated. You have to know which encoding to convert from (and no, "ANSI" is not an encoding) and this information cannot be known or inferred from mods in their current state. It has to be added.
Posted by: AL|EN
« on: July 17, 2014, 07:48:20 AM »

But even if you provide HANDLE_* function, it would still require to update mod so in this case we can follow BG1NPC and you don't have to code anything. When I asked about "handle this by weidu completely" what I mean is that weidu will convert all ANSI files to UTF-8 on the fly when the game will be XX:EE and without any modification to original mod.

If this this is to much to ask then please forgive me, but you know ... it take years to update mods. And because there are no ANS-UTF8 conversion in 90% of XX:EE mods, it makes BWS completely useless for any non-English installation on XX:EE games :/
Posted by: Wisp
« on: July 16, 2014, 02:08:11 PM »

I can implement something along the lines of a HANDLE_* function for charset conversions. I have my reservations, but I suppose they are inconsequential, and don't let perfect be the enemy of good and all that. There would probably be a list of constraints you would need to operate by, because I can't do magic and this isn't like converting Ogg V. Since it'd be a function, you could use it without having to wait for a new WeiDU.
Posted by: AL|EN
« on: July 16, 2014, 06:45:02 AM »

BG1 NPC does not use two sets of files... BG1 NPC still utilizes only old files, and includes its own utf-8 converter... if I understand correctly.
Exactly. Hawing to deal with two set of files is a nightmare for translators, even if it is only "run my Copy-AND-Convert-ANSI-UTF8.bat file before he will send files to include in official mod archive. And then he just fix missing ~ and forgot to make copy and conversion ...
Posted by: Kulyok
« on: July 16, 2014, 02:17:29 AM »

Forgive me for keeping pestering you, but here's why I want to know so much whether to adapt BG1 NPC's way or not:

BG1 NPC does not use two sets of files. I looked up Edwin's Romance, and it uses /normal tra files and /utf-8 tra files in two different subfolders. BG1 NPC still utilizes only old files, and includes its own utf-8 converter... if I understand correctly. It doesn't seem to take up much space, either. So I'm wondering - maybe it's so good it could become a part of Weidu and a part of a natural mod-converting process for BG2EE translations?
Posted by: Wisp
« on: July 15, 2014, 01:58:02 PM »

I may or may not have something coming down the line. In the mean time, or in the event I don't, I would suggest the methods outlined and/or explained in this thread as the least crummy way of dealing with compatibility with both the regular BGs as well as the EEs. I can't vouch for the implementation in BG1 NPC, as I haven't looked at it. The implementation used in Kelsey (as well as in the Worldmap mod) works. IIRC, Edwin Romance uses an approach that is compatible with AUTO_TRA (something Kelsey does not need).

Edited for 90+ % less commitment.