Author Topic: Allow for defining MOD_FOLDER - 'immutable' mod design without hacks  (Read 864 times)

Offline AL|EN

  • Planewalker
  • *****
  • Posts: 391
  • Gender: Male
This request is the result of the combination of several things happened at once. Let me start with the idea that 'mod data folder' and 'mod backup folder' are two completely different things. And IMHO, WeiDU should threat them internaly as two separate things.


design-A)
Quote
BACKUP ~MyMod\backup~

design-B)
Quote
BACKUP ~weidu_external\backup\MyMod~

1. Most mods uses design-A for the value of BACKUP keyword. But some mods are using 'immutable' mod design-B with workaround. First, set the backup outside of 'mod data folder' via 'BACKUP' keyword. But it also sets MOD_FOLDER variable thi the value of it, so they need to overwrite MOD_FOLDER variable back to proper 'mod data folder'. This has no consequences when installing such mod but it has when mod manager try to detect ''mod data folder'.

2. While WeiDU provide way to get proper 'mod data folder' by using --debug-assign (or future supported version of it), when the mod BACKUP keyword has design-A (MOD_FOLDER will be set to 'MyMod'),  it doesn't work for the design-B (MOD_FOLDER will be set to 'weidu_external').

My request would be:
- ability to set 'MOD_FOLDER' along with "BACKUP"
Code: [Select]
BACKUP ~weidu_external\backup\MyMod~
MOD_FOLDER ~MyMod~

BEGIN "MyMod"

and make it so WeiDU will handle backup path from 'BACKUP' keyword and if MOD_FOLDER keyword exist, set MOD_FOLDER variable to the value of this keyword. This should handle compatibility with older mods and allow new mods to use this feature. Lastly, --debug-assign (or future supported version of it) will output both values into two separated variables even if they match.

This would allow to have clear 'immutable' mod design without hacks around the fact that WeiDU currently has one variable for 'mod data folder' and 'mod backup folder'. Plus, I get the proper values for both of them.

Alternative would be:
Introduce another variable, let's call it 'BACKUP' for --debug-assign (or future supported version of it), where WeiDU will put raw value of 'BACKUP' keyword, without touching it. So I could get it when I'm launching WeiDU and then deal with weidu_external-thing.

Q: Why PI simply doesn't load tp2-file and extract the full value of BACKUP keyword?
A: It was at the begging. No matter how fast tp2 scanning is (not possible to use something like yeld return), the reality of having to scan 100+ mods makes this approach impossible for mod manager: people put crazy things into their tp2, some tp2 are huge and there are some tp2 which aren't mods at all. Launching WeiDU and getting the keyword values is the fastest and most error-safe way.
« Last Edit: October 12, 2020, 09:10:52 AM by AL|EN »
Project Infinity public BETA - mod manager for Infinity Engine games
Modder's Guide to Github - you cannot have progress without committing changes

Offline DavidW

  • Planewalker
  • *****
  • Posts: 316
Re: Allow for defining MOD_FOLDER - 'immutable' mod design without hacks
« Reply #1 on: October 12, 2020, 10:58:54 AM »
some mods are using 'immutable' mod design-B with workaround. First, set the backup outside of 'mod data folder' via 'BACKUP' keyword. But it also sets MOD_FOLDER variable thi the value of it, so they need to overwrite MOD_FOLDER variable back to proper 'mod data folder'.
This is actually one of the things WEIDU v247 fixes. If you look at how MOD_FOLDER is now defined, it's equal to the folder in which the tp2 file exists if there is one; it's set from the BACKUP variable only for old mods where the tp2 file lives outside the mod folder. It's no longer necessary to set MOD_FOLDER explicitly (if you look at SCS v33.5, it doesn't do this any more.)

Quote
No matter how fast tp2 scanning is (not possible to use something like yeld return), the reality of having to scan 100+ mods makes this approach impossible for mod manager: people put crazy things into their tp2, some tp2 are huge and there are some tp2 which aren't mods at all. Launching WeiDU and getting the keyword values is the fastest and most error-safe way.
I'm confused how this can be. WEIDU can't extract these variables without parsing the whole tp2 file, whereas you can extract the keywords just by parsing the beginning of the tp2 file. OCAML doesn't have some magic ability to parse way faster than another language.

Offline AL|EN

  • Planewalker
  • *****
  • Posts: 391
  • Gender: Male
Re: Allow for defining MOD_FOLDER - 'immutable' mod design without hacks
« Reply #2 on: October 12, 2020, 12:01:00 PM »
some mods are using 'immutable' mod design-B with workaround. First, set the backup outside of 'mod data folder' via 'BACKUP' keyword. But it also sets MOD_FOLDER variable thi the value of it, so they need to overwrite MOD_FOLDER variable back to proper 'mod data folder'.
This is actually one of the things WEIDU v247 fixes. If you look at how MOD_FOLDER is now defined, it's equal to the folder in which the tp2 file exists if there is one; it's set from the BACKUP variable only for old mods where the tp2 file lives outside the mod folder. It's no longer necessary to set MOD_FOLDER explicitly (if you look at SCS v33.5, it doesn't do this any more.)

Regardles of what 247 fixed, I'm still getting: "SET %MOD_FOLDER% = ~weidu_external~" from SCS when I use --debug-assign in order to get MOD_FOLDER value.

Quote
No matter how fast tp2 scanning is (not possible to use something like yeld return), the reality of having to scan 100+ mods makes this approach impossible for mod manager: people put crazy things into their tp2, some tp2 are huge and there are some tp2 which aren't mods at all. Launching WeiDU and getting the keyword values is the fastest and most error-safe way.
I'm confused how this can be. WEIDU can't extract these variables without parsing the whole tp2 file, whereas you can extract the keywords just by parsing the beginning of the tp2 file. OCAML doesn't have some magic ability to parse way faster than another language.
The 'BACKUP' keyword is not always at the begging of the file. You need to read it and apply regex. 100+ tp2 files = 20 seconds or more of waiting time during application startup before you can do anything.
« Last Edit: October 12, 2020, 12:02:07 PM by AL|EN »
Project Infinity public BETA - mod manager for Infinity Engine games
Modder's Guide to Github - you cannot have progress without committing changes

Offline DavidW

  • Planewalker
  • *****
  • Posts: 316
Re: Allow for defining MOD_FOLDER - 'immutable' mod design without hacks
« Reply #3 on: October 12, 2020, 12:10:09 PM »
Regardless of what 247 fixed, I'm still getting: "SET %MOD_FOLDER% = ~weidu_external~" from SCS when I use --debug-assign in order to get MOD_FOLDER value.
That's an oddity in --debug-assign and/or your install configuration, then. If you just install SCS via the standard installer, MOD_FOLDER gets set correctly.

Quote
The 'BACKUP' keyword is not always at the begging of the file. You need to read it and apply regex. 100+ tp2 files = 20 seconds or more of waiting time during application startup before you can do anything.
But the point stands. Whatever your parser is doing, WEIDU itself has to do that and more. Unless the code you're using is really inefficient relative to whatever WEIDU is doing, it shouldn't be able to be slower than WEIDU.

Offline AL|EN

  • Planewalker
  • *****
  • Posts: 391
  • Gender: Male
Re: Allow for defining MOD_FOLDER - 'immutable' mod design without hacks
« Reply #4 on: October 12, 2020, 02:30:27 PM »
Regardless of what 247 fixed, I'm still getting: "SET %MOD_FOLDER% = ~weidu_external~" from SCS when I use --debug-assign in order to get MOD_FOLDER value.
That's an oddity in --debug-assign and/or your install configuration, then. If you just install SCS via the standard installer, MOD_FOLDER gets set correctly.
Well, I've checked this by launching weidu 247 two-ways:

1) Direct
$weiduPath = "C:\Users\ALIEN\Downloads\PI-Test\Tools\WeiDU\247.00\weidu.exe"
$workingDirectory = 'E:\Mods-Extracted-0\SwordCoastStratagems'
$tp2 = 'E:\Mods-Extracted-0\SwordCoastStratagems\stratagems\setup-stratagems.tp2'

SET %MOD_FOLDER% = ~E:~

2) Relative
$weiduPath = "C:\Users\ALIEN\Downloads\PI-Test\Tools\WeiDU\247.00\weidu.exe"
$workingDirectory = 'E:\Mods-Extracted-0\SwordCoastStratagems'
$tp2 = 'stratagems\setup-stratagems.tp2'

SET %MOD_FOLDER% = ~stratagems~

So it's because direct vs relative tp2 argument.

The thing is, since I'm operating on the 'extracted mods' folder, where the correct $workingDirectory (and thus tp2-relative path) cannot be provided before getting 'mod data folder' value.

@wisp, does %MOD_FOLDER% can be set correctly when '--debug-assign' is used with direct path to tp2 ?
Project Infinity public BETA - mod manager for Infinity Engine games
Modder's Guide to Github - you cannot have progress without committing changes

 

With Quick-Reply you can write a post when viewing a topic without loading a new page. You can still use bulletin board code and smileys as you would in a normal post.

Warning: this topic has not been posted in for at least 120 days.
Unless you're sure you want to reply, please consider starting a new topic.

Name: Email:
Verification:
Type the letters shown in the picture
Listen to the letters / Request another image
Type the letters shown in the picture:
What color is grass?:
What is the seventh word in this sentence?:
What is five minus two (use the full word)?: