Author Topic: Inverse HANDLE_CHARSETS?  (Read 2466 times)

Offline AstroBryGuy

  • Planewalker
  • *****
  • Posts: 154
  • Gender: Male
Inverse HANDLE_CHARSETS?
« on: February 04, 2016, 10:25:23 AM »
I don't know if this would be an in-demand feature, but how hard would it be to create an inverted version of HANDLE_CHARSETS, i.e. distribute .tra files in UTF-8 encoding and convert at install to Windows codepages?

UTF-8 is the default encoding for many modern text editors, and when mod authors receive translations via email or PM, the translated lines will usually be in UTF-8 (unless the translated files is a CP-encoded attachment). So, this could be a handy feature.

If this is already possible with HANDLE_CHARSETS, please accept my humble apologies and I'll go slink off to the corner and RTFM.  ;)

Offline Wisp

  • Moderator
  • Planewalker
  • *****
  • Posts: 1176
Re: Inverse HANDLE_CHARSETS?
« Reply #1 on: February 09, 2016, 01:11:16 PM »
I haven't tested it but it should be trivial to adapt HANDLE_CHARSETS to switch from_charset with to_charset. However, setup text would still need to be encoded in stupid-format or you'd print possible UTF8 to the Windows command window (which happens before HANDLE_CHARSETS can run). There might be a few more problems I am not thinking of now.

Offline AstroBryGuy

  • Planewalker
  • *****
  • Posts: 154
  • Gender: Male
Re: Inverse HANDLE_CHARSETS?
« Reply #2 on: February 11, 2016, 08:15:59 PM »
Good point about setup.tra. One more reason to write that in ASCII-only.  :P

Offline AL|EN

  • Planewalker
  • *****
  • Posts: 391
  • Gender: Male
Re: Inverse HANDLE_CHARSETS?
« Reply #3 on: March 06, 2016, 05:17:50 PM »
However, setup text would still need to be encoded in stupid-format or you'd print possible UTF8 to the Windows command window (which happens before HANDLE_CHARSETS can run). There might be a few more problems I am not thinking of now.


I've tested CMD chcp 65001 (cmd /k chcp 65001)

http://i.imgur.com/UZAaNyt.png

and while first appearance of the unicode chars worked fine, second appearance seems to be bugged and weidu hangs. This problem appears from some time, maybe we could put some effort to testing/fixing it?
« Last Edit: March 06, 2016, 05:24:40 PM by AL|EN »
Project Infinity public BETA - mod manager for Infinity Engine games
Modder's Guide to Github - you cannot have progress without committing changes

Offline Wisp

  • Moderator
  • Planewalker
  • *****
  • Posts: 1176
Re: Inverse HANDLE_CHARSETS?
« Reply #4 on: March 12, 2016, 03:27:42 AM »
As far as I can tell, WeiDU prints the exact same substring in both cases. Your problem is likely with the Windows cmd prompt.

Offline AL|EN

  • Planewalker
  • *****
  • Posts: 391
  • Gender: Male
Re: Inverse HANDLE_CHARSETS?
« Reply #5 on: March 13, 2016, 01:36:52 PM »
Since first appearance of the Unicode chars worked fine, there is no problem with  Windows cmd prompt - I've tested it also with powershell(it doesn't have any problems with UTF-8) but it still appears. Could you take deep look at this problem? Having one 'setup.tra' is notable improvement atleast for windows vista+
« Last Edit: March 13, 2016, 01:38:26 PM by AL|EN »
Project Infinity public BETA - mod manager for Infinity Engine games
Modder's Guide to Github - you cannot have progress without committing changes

Offline Wisp

  • Moderator
  • Planewalker
  • *****
  • Posts: 1176
Re: Inverse HANDLE_CHARSETS?
« Reply #6 on: March 16, 2016, 01:29:19 PM »
Done. Intercepting the text before it's printed to the command prompt shows it's all UTF-8. Your problem is with Windows and its bullshit implementation of coding systems.

Offline AL|EN

  • Planewalker
  • *****
  • Posts: 391
  • Gender: Male
Re: Inverse HANDLE_CHARSETS?
« Reply #7 on: March 20, 2016, 10:29:28 AM »
Thanks! I guess I have to wait until next weidu to test it.
Project Infinity public BETA - mod manager for Infinity Engine games
Modder's Guide to Github - you cannot have progress without committing changes

Offline AL|EN

  • Planewalker
  • *****
  • Posts: 391
  • Gender: Male
Re: Inverse HANDLE_CHARSETS?
« Reply #8 on: April 18, 2016, 03:21:45 PM »
Well, I've tested it with the new beta and the results are the same: second appearance of the unicode string is malformed and weidu console hangs at "Press ENTER to exit."
Project Infinity public BETA - mod manager for Infinity Engine games
Modder's Guide to Github - you cannot have progress without committing changes

Offline Wisp

  • Moderator
  • Planewalker
  • *****
  • Posts: 1176
Re: Inverse HANDLE_CHARSETS?
« Reply #9 on: April 23, 2016, 04:21:34 AM »
I'm not sure why you would expect a new version of WeiDU to solve a problem that is not caused by WeiDU. Given a setup.tra encoded in UTF-8, all text taken from setup.tra remains encoded in UTF-8 when WeiDU emits it. You asked me to look into whether this was really so, I did and it is really so.

 

With Quick-Reply you can write a post when viewing a topic without loading a new page. You can still use bulletin board code and smileys as you would in a normal post.

Warning: this topic has not been posted in for at least 120 days.
Unless you're sure you want to reply, please consider starting a new topic.

Name: Email:
Verification:
Type the letters shown in the picture
Listen to the letters / Request another image
Type the letters shown in the picture:
What color is grass?:
What is the seventh word in this sentence?:
What is five minus two (use the full word)?: