Continuation from plainAB's thread at G3

[the bigg]

Continued from http://forums.gibberlings3.net/index.php?showtopic=15712 as per PlainAB's request.

For #1, #2 and #3 there's --change-log, you know wink.gif

create a results directory, then
weidu --change-log _davaeo.cre --out results --out results/log.txt
results/log.txt will contain the mods which edit _davaeo.cre, and a copy of all of its stages.

No, I never knew that. Another case of weidu docs being cryptic to those who don't know the language that it was built in.

Dead horses is beaten, still twitching.

Can we make this little option a part of the standard install somehow?

If we can set it up right it would be useful to determine if files going to be changed by mod B have been changed in some way by mod A. Thus being able to give the user a warning that there is a possibility of either a technical or conceptual incompatibility between mod(s) already installed and the mod about to be installed.

It might have to be an entirely new option rather than the existing one, since we wouldn't need to have copies of the given files in every state. We know what files have been changed, be we don't know what files might be changed by a mod, especially if they use REGEXP. So a pre-parse of the tp2 that doesn't do any of the writing, but does do all of the reading to create the list of files that will be modified by the mod will need to take place. The two lists can then be compared and any potential conflicts will be displayed to the user. This would be helpful to modders who want to know if their mod might be incompatible with another. It would be helpful to players who want to play a specific mod without any changes from the original mod conception.

I don't know if it could be possible. I'm sure something clunky could be worked out by a modder for a specific mod (I've already got some ideas). But for it to be available for every mod it would have to be somehow included in the installation weidu.exe file.

Unfortunately, keeping track of all files ever touched (as what you're suggesting) will out of memory for people installing the full mega mod (link). The problem will be exacerbated if it was done during an install. The whole point of --change-log is to diagnose patch-rific mods which are failing on a file, or similar (the ideal course of action is to instruct the user to run a .bat file which creates the directory and runs weidu --change-log, and then have him zip the directory up and sed it to the modder for inspection).
If you wish for automatic compatibility prediction, you should contribute to Baronius' tool instead. Also, two mods writing to the same file aren't automatically incompatible, whereas two mods not editing the same file aren't automatically compatible.

[plainab]

If you wish for automatic compatibility prediction, you should contribute to Baronius' tool instead.

I did get him to start talking about it. However, I have no formal training in programming and have been quite lost in reading the threads regarding his tools. I do continue to read them as I do look forward to seeing the tools once completed. Not actually needing automatic compatibility prediction (although that would be nice) but rather more information provided upfront. The less modders need to go hunting when trying to find reported bugs/problems the more likely the reported bugs/problems can be resolved.

Also, two mods writing to the same file aren't automatically incompatible, whereas two mods not editing the same file aren't automatically compatible.

I never stated that the same file edited by two mods was automatically incompatible. I said that it was possible for them to be incompatible either technically or conceptually (see Baronius for the definition on those), which leaves the door open for the possibility that the mods will work fine together (i.e. the game doesn't crash).

Unfortunately, keeping track of all files ever touched (as what you're suggesting) will out of memory for people installing the full mega mod (link). The problem will be exacerbated if it was done during an install. The whole point of --change-log is to diagnose patch-rific mods which are failing on a file, or similar (the ideal course of action is to instruct the user to run a .bat file which creates the directory and runs weidu --change-log, and then have him zip the directory up and sed it to the modder for inspection).

So basically the current commands aren't up to the task then.

So how about something new then? Tell me what is possible, you know weidu better than me.

I'd like to see something that happens behind the scenes so that old weidu mods will utilize the new feature simply by having the exe file upgraded.

As an example what I'd like to see is the following:

//File_name, Offset, Old_Value, New_Value // ~TP2_File~ #language_number #component_number // [Subcomponent Name -> ] Component Name [ : Version]
_AROW02.ITM, 0x38, 20, 40 // ~SETUP-TUTUFIX.TP2~ #0 #3 // BG2 Ammo Stacks
_AROW02.ITM, 0x38, 40, 9999 // ~BG2_TWEAKS/SETUP-BG2_TWEAKS.TP2~ #0 #3080 // Unlimited Ammo Stacking

In this example tutufix was ran and a component installed which changed the standard file stack amount from 20 to 40 and is noted in a change-log file. Later BG2_Tweaks was ran and a component installed which changed the same entry to 9999 and so is logged.

But I am flexible. This doesn't have to be a completely new file. The information from each mod can be listed in it's own debug file. Comparison data can be pulled from the other debug files as needed by the mod developers. If you do the debug file route, then you could have the following:

SETUP-TUTUFIX.DEBUG file:

[./override/_AROW02.ITM] loaded, 170 bytes
override/_AROW02.ITM copied to tutufix/backup/3/override._AROW02.ITM, 170 bytes
Copied [_AROW02.ITM] to [override/_AROW02.ITM]
_AROW02.ITM at offset 0x38 changed from 20 to 40

SETUP-BG2_TWEAKS.DEBUG file:

[./override/_AROW02.ITM] loaded, 170 bytes
override/_AROW02.ITM copied to BG2_Tweaks/backup/3020/_AROW02.ITM, 170 bytes
Copied [_AROW02.ITM] to [override/_AROW02.ITM]
_AROW02.ITM at offset 0x38 changed from 40 to 9999
See SETUP-TUTUFIX.DEBUG for other changes made to this file.

In the cases where an existing file is simply overwritten by a new one just an entry that says something like

Existing FILE replaced by New FILE

would be sufficient.

[the bigg]

Unfortunately, keeping track of all files ever touched (as what you're suggesting) will out of memory for people installing the full mega mod (link). The problem will be exacerbated if it was done during an install. The whole point of --change-log is to diagnose patch-rific mods which are failing on a file, or similar (the ideal course of action is to instruct the user to run a .bat file which creates the directory and runs weidu --change-log, and then have him zip the directory up and sed it to the modder for inspection).

So basically the current commands aren't up to the task then.

So how about something new then? Tell me what is possible, you know weidu better than me.

I'd like to see something that happens behind the scenes so that old weidu mods will utilize the new feature simply by having the exe file upgraded.

As an example what I'd like to see is the following:

//File_name, Offset, Old_Value, New_Value // ~TP2_File~ #language_number #component_number // [Subcomponent Name -> ] Component Name [ : Version]
_AROW02.ITM, 0x38, 20, 40 // ~SETUP-TUTUFIX.TP2~ #0 #3 // BG2 Ammo Stacks
_AROW02.ITM, 0x38, 40, 9999 // ~BG2_TWEAKS/SETUP-BG2_TWEAKS.TP2~ #0 #3080 // Unlimited Ammo Stacking

In this example tutufix was ran and a component installed which changed the standard file stack amount from 20 to 40 and is noted in a change-log file. Later BG2_Tweaks was ran and a component installed which changed the same entry to 9999 and so is logged.

Keeping a full list of file changes at install time would likely eat disk and RAM space like breakfast for dubious benefits (true, you can diagnose when two mods are attempting to edit the same field, but this kind of thing is rare and usually irrelevant to real compatibility).

In any event, WeiDU has a weak form of binary diff via a command-line option:
weidu --cmp-from bg2_tweaks/backup/3080/_AROW02.ITM --cmp-to override/_AROW02.ITM --out diff

Code: [Select]

// patches to turn [bg2_tweaks/backup/3080/_AROW02.ITM] into [override/_AROW02.ITM]
WRITE_BYTE 0x38 15 // 0x0f
WRITE_BYTE 0x39 39 // 0x27 == '
(there are also --bcmp-from/to and --dcmp-from/to, but they currently are even less useful).

I can include it in the standard --change-log results, but it'd only be marginally useful (Too Long, Didn't Read effect for file formats like DLG and so, cryptic results for small binary ones), and obtaining meaningful results would imply copying the whole of IESDP into WeiDU. Currently, my favorite way for quickly obtaining list of changes is to use nidiggen and diff; unfortunately, doing so efficiently requires a non-trivial amount of scripting.