Post reply

Warning: this topic has not been posted in for at least 120 days.
Unless you're sure you want to reply, please consider starting a new topic.
Name:
Email:
Subject:
Message icon:

Verification:
Type the letters shown in the picture
Listen to the letters / Request another image

Type the letters shown in the picture:
What color is grass?:
What is the seventh word in this sentence?:
What is five minus two (use the full word)?:

shortcuts: hit alt+s to submit/post or alt+p to preview


Topic Summary

Posted by: Isaya
« on: July 11, 2013, 04:56:10 PM »

Would telling folks to use Notepad++ help with this issue?
Unfortunately, the point is not the editor people would use to write the tra files but the way the games handle character sets. And the fact that WeiDU shall operate with both older games and BGEE.

In older games, they only want ANSI, so UTF-8 characters wil become garbage in the games. Besides a few Eastern Europe languages use ANSI codes from Windows code pages (equivalent to their ISO-8859-X counterparts) to get their special characters displayed. So there is not a single ANSI encoding, but several. So conversion from ANSI to UTF-8 or from UTF-8 to ANSI is not a simple thing either, as it depends on the language.

For more on this topic, you may read this thread regarding a proposal to handle this with WeiDU and that topic explaining the issue with ANSI characters in BGEE.
Posted by: asdfasldkf
« on: July 11, 2013, 06:19:17 AM »

The issue is languages. x86 Windows systems still use local code pages for display of text. While you can use UTF-8 for Mac/Linux stuff, it special characters will display funny in Windows.

I may be on the wrong page here - but is the issue how UTF-8 is displayed (or not) in some editing programs?

Because if that's the issue, Notepad++ handles UTF-8 - and it's free.  I use it for all my scripting/coding and have done for probably decades.

Would telling folks to use Notepad++ help with this issue?
Posted by: cmorgan
« on: May 03, 2013, 10:53:14 PM »

Oh bother.

OK, so ANSI for encoding, and let weidu do the conversion.
Posted by: Ascension64
« on: May 03, 2013, 06:40:52 PM »

The issue is languages. x86 Windows systems still use local code pages for display of text. While you can use UTF-8 for Mac/Linux stuff, it special characters will display funny in Windows.
Posted by: cmorgan
« on: May 03, 2013, 02:04:54 PM »

Hey, is there any reason *not* to distribute (.d|.baf|.tp2|.tpa) files for weidu for all target i.e. games using UTF=8 encoding, rather than ANSI? i know you are working on coversion, but if a modder throws down in UTF-8 it should be good for BG/Tutu/BG:EE/BG2/ToB/BG2:EE/IWD1|IWD2/PST, right?
Posted by: Wisp
« on: December 20, 2012, 02:49:32 PM »

This is my BG2 TBD list (includes both WeiDU stuff and other mods). Hopefully that's useful.
Thank you.

Small suggestion: if the game is BGEE and no override folder exists, create one?
Yes, I should put that in.

Also, TLK location is still an issue.
Yeah, and it will likely remain a problem for a while longer (but in the mean time, hardlinking the file is easy). Aside from adjusting the code to deal with multiple output TLKs without breaking existing functionality, there is also the matter of which TLK strings should be sourced from (for, e.g., the German who wants to decompile a DLG and get German text), the matter of the character encoding and possibly a few other issues that need to be, or should be, sorted out at the same time. All in all, it's a fair bit of work.
Posted by: Avenger_teambg
« on: December 19, 2012, 02:58:34 AM »

Okay, I built a new beta. Link.
Among other tricks, 231.06 can cope with Beamhauled biffs, unbiff the new file types and recognise BGEE for purposes of (GAME|ENGINE)_IS.

I didn't have time to go over the standard library, I'm afraid (and probably won't have until Thursday).

Thank you, Wisp. I will be testing it.
Posted by: Kaeloree
« on: December 18, 2012, 05:44:20 PM »

Small suggestion: if the game is BGEE and no override folder exists, create one?

Also, TLK location is still an issue.

Confirming that dialogue tokens now work with the new beta, thanks Wisp!
Posted by: Wisp
« on: December 18, 2012, 04:01:36 PM »

Okay, I built a new beta. Link.
Among other tricks, 231.06 can cope with Beamhauled biffs, unbiff the new file types and recognise BGEE for purposes of (GAME|ENGINE)_IS.

I didn't have time to go over the standard library, I'm afraid (and probably won't have until Thursday).
Posted by: GeN1e
« on: December 18, 2012, 02:44:03 PM »

Can you give an example? I've found the current .d syntax incredibly powerful for making changes.
ALTER_TRANSing in a state that may have been top- or bottom- appended with new transitions, i.e. when you may want to check what the trigger/action/text/etc. is before altering it blindly.
Also when you need to patch rather than overwrite the trigger/action - REPLACE_TRANS_ACTION/TRIGGER are quite good, but I recall lacking in freedom with it somewhere.

Basically, it's a compatibility thing. I mostly need it for the eventual Quest Revisions.
Posted by: CamDawg
« on: December 18, 2012, 12:03:07 PM »

much less when people use a cracked copy of MS Word 2003 as their editor of choice
Shows what you know. Mine's a cracked coy of WordPerfect, so there!

We may need a tp2 macro to patch DLGs. The existing .D syntax is just not enough when you want to match and compare the script string line by line before applying a patch. Gonna be quite complex, however, if it were to be any useful.
Can you give an example? I've found the current .d syntax incredibly powerful for making changes.
Posted by: GeN1e
« on: December 18, 2012, 10:14:48 AM »

Anyway,
what's on people's wishlist for 232?
We may need a tp2 macro to patch DLGs. The existing .D syntax is just not enough when you want to match and compare the script string line by line before applying a patch. Gonna be quite complex, however, if it were to be any useful.
Posted by: Avenger_teambg
« on: December 18, 2012, 02:56:47 AM »

Filetypes
What are the new file types? I've only got the numbers and they don't tell me much, I am afraid. (And if there was schooling to be had on the subject, I slept through the class.) Aside from being made recognisable types, is there anything else to be done?
0x400 .fnt     font format (bitmap? didn't care)   
0x403 .sql    text format (semi script/some parts are csv), replace_textually is enuff for now
0x402 .gui    text format (semi script), replace_textually is enuff for now
0x401 .wbm standard movie format
0x404 .pvrz   compressed binary format (textures)

Of these, only .sql and .gui are something i would touch with weidu (but all should be known as resource type).

Btw, i was self taught in the subject, there is not school for this.
Posted by: the bigg
« on: December 17, 2012, 05:31:13 PM »

This is my BG2 TBD list (includes both WeiDU stuff and other mods). Hopefully that's useful.

There's no reasonable way to support ASCII, UTF-8 and whatnot, no matter the language and libraries used (not when you're dealing with disciplined people who know enough to use a proper text editor and save all their files in the same encoding, much less when people use a cracked copy of MS Word 2003 as their editor of choice). If anything, I'd embed an ad-hoc C++ utility and use it to overwrite local .tra files (doing ASCII -> UTF-8 for mods coded under Tutu and installed under BGEE, and viceversa).
Posted by: Wisp
« on: December 17, 2012, 05:13:11 PM »

Wisp - I'm not sure whether this makes a difference when it comes to WeiDU, or whether that's what you mean with "UTF-8 conversions", but the BG:EE game text makes use of non-ASCII characters (especially in foreign languages, but also in English). If this has an impact on WeiDU at any rate, could you add support for them?
Well, background for everyone's benefit: BGEE uses UTF-8 across the board, whereas the original editions use a multitude of different local encodings. The only commonality between UTF-8 and these other encodings is ACSII, so anything that is not expressible as ASCII will be interpreted incorrectly if it is in the wrong encoding. In practice, this means hassles aplenty for languages other than English, since BGEE requires that the translation is encoded in UTF-8, whereas the original editions require that the translation is encoded in, e.g., CP1252 (in the case of western-European languages). English doesn't escape entirely either, since some mods use dashes or ellipses or other non-ASCII characters.

The tentative plan is for WeiDU to automagically convert text into UTF-8 when the platform is BGEE, thus avoiding the nasty prospect of having to maintain multiple sets of TRA files whenever non-ASCII is used (and compatibility with both the EEs and the original editions is desired). The most likely hitch would be that the OCaml UTF-8 library (-ies) need to know which encoding the text should be converted from. Since TP2 language declarations are completely fluid, it would be very hard (and error prone) to deduce the encoding from that information. There is also the ugly practice of mixing several encodings in the same file and probably a number of additional complicating factors. If it goes in, I imagine this feature will require some sort of non-ambiguous declaration (e.g., you explicitly declare which encoding is used for the TRA file inside the TRA file itself, and if the file originally used multiple encodings, you need to split it apart into one file for each).

what's on people's wishlist for 232? Unconventional biffs , (GAME|ENGINE)_IS and an update of the standard library of macros and functions are already in.
Does this include my patch functions? I'll live either way, I just want to know if I should start with documentation.

edit: fixed broken link
I don't think the latter submissions in that thread have yet been included. But I can probably do it tomorrow.

Filetypes
What are the new file types? I've only got the numbers and they don't tell me much, I am afraid. (And if there was schooling to be had on the subject, I slept through the class.) Aside from being made recognisable types, is there anything else to be done?