Author Topic: weidu next generation - change "backup" location - won't fix  (Read 4134 times)

Offline AL|EN

  • Planewalker
  • *****
  • Posts: 391
  • Gender: Male
In the current state of the weidu backup system, there is one really annoying thing: when the weidu create "backup" folder and create binary files which are needed for the uninstallation, weidu itself directly affecting the mod directory. I really don't like this design. In order to use the same extracted mod, players have to make sure that they will remove "backup" folder when they will copy for eg cdtweaks + setup-cdtweaks.exe from BG1EE to BG2EE. If the unaware player will forgot about this, it will cause problems for them because "backup" folder will have leftover from the previous installation. Not to mention countless re-uploading mod because author forgot to remove "backup" folder.

New proposed solution is simple: instead of "GameDir\ModName\Backup" use "GameDir\weidu-backup\ModName\"



« Last Edit: July 28, 2017, 08:04:03 AM by AL|EN »
Project Infinity public BETA - mod manager for Infinity Engine games
Modder's Guide to Github - you cannot have progress without committing changes

Offline Wisp

  • Moderator
  • Planewalker
  • *****
  • Posts: 1176
The backup path is stored in the TP2. I'm not having external processes or WeiDU alter the contents of the TP2, I'm not overriding the backup path at runtime because the resultant problems would make yours pale. If the mod maker wants to store backup files outside of the mod directory, there's already nothing stopping zim from doing so; just do it.

Offline AL|EN

  • Planewalker
  • *****
  • Posts: 391
  • Gender: Male
So is the thing from the past, I got it. Still, it's horrible design, it was ignored by everyone. Because nobody will update their mods,current weidu can provide quick and easy fix for what is fundamentally flawed. Can you please be more specific why weidu can't simply replace "BACKUP ~ModName/backup~" to "BACKUP ~weidu-backup/ModName~"? Or just ignore it and handle backup path at weidu side. It's trivial piece of string manipulation code. You won't even try to introduce code changes inside BETA, to see the outcome?
« Last Edit: May 31, 2017, 01:33:06 PM by AL|EN »
Project Infinity public BETA - mod manager for Infinity Engine games
Modder's Guide to Github - you cannot have progress without committing changes

Offline subtledoctor

  • Planewalker
  • *****
  • Posts: 131
I think the question is, should Weidu override a setting that the modder specifically designates?

Don't get me wrong, I think it would indeed be better if mods backed up to "weidu_backups/modname/"... and maybe I'll do that myself, and maybe it will catch on. (I didn't know that was even an option, way back when I created my mod - most mod sets probably write the backup line when they first make a mod, and at that moment they don't really know what they are doing.)

Then again, the current default has the virtue of uniformity; tell modders to move it outside the mod folder and some might use "weidu-backup/" while others use "weidu_backup/" while yet others use "weidubackups/"... it could create file clutter and bloat in the game directory.

Offline AL|EN

  • Planewalker
  • *****
  • Posts: 391
  • Gender: Male

subtledoctor posted yet another thing which favors weidu-handled backup location. I add another: authors can seriously screw it:
Code: [Select]
BACKUP ~\KaraTur\Kbackup~this code will create "KaraTur\Kbackup" inside root level of the drive which has IE game installed.
Project Infinity public BETA - mod manager for Infinity Engine games
Modder's Guide to Github - you cannot have progress without committing changes

Offline Magus_BGforge

  • Planewalker
  • *****
  • Posts: 75
In the current state of the weidu backup system, there is one really annoying thing: when the weidu create "backup" folder and create binary files which are needed for the uninstallation, weidu itself directly affecting the mod directory. I really don't like this design. In order to use the same extracted mod, players have to make sure that they will remove "backup" folder when they will copy for eg cdtweaks + setup-cdtweaks.exe from BG1EE to BG2EE. If the unaware player will forgot about this, it will cause problems for them because "backup" folder will have leftover from the previous installation. Not to mention countless re-uploading mod because author forgot to remove "backup" folder.

New proposed solution is simple: instead of "GameDir\ModName\Backup" use "GameDir\weidu-backup\ModName\"
I don't think leftovers will cause any problems. Weidu logs changes, and restores only what was changed. So it's really cosmetic.
Anyway, with BWS/Zeitgeist, soon most people won't do any moving at all. And those who will, will also probably proficient enough to handle it properly.

subtledoctor posted yet another thing which favors weidu-handled backup location. I add another: authors can seriously screw it:
Code: [Select]
BACKUP ~\KaraTur\Kbackup~this code will create "KaraTur\Kbackup" inside root level of the drive which has IE game installed.
Even if it happened (and it didn't in 15 years, did it?), it's not the end of the world.

I agree that weidu directory layout is archaic and somewhat inconvenient, though - to the point of irritation, sometimes. With many mods, game directory is severely polluted. It's just hard to find the required files. And override is polluted with loads of marker files, too.
But if we're talking about changing it, I wouldn't stop on half-measures. Allocating one single directory for everything mod-related, and underneath it, separate dirs for mods, backups, logs, markers - that would be an improvement.

Anyway, all this is pretty low priority. People don't really have problems with current system.

Offline AL|EN

  • Planewalker
  • *****
  • Posts: 391
  • Gender: Male
I't's not about cosmetic weidu.log, it's about leftovers from backup. Changes will eliminate possibility of the mistakes.

Quote
Allocating one single directory for everything mod-related, and underneath it, separate dirs for mods, backups, logs, markers - that would be an improvement.
That is even better but it require more work. For now, get rid of backup leftovers problem will be enough.
Project Infinity public BETA - mod manager for Infinity Engine games
Modder's Guide to Github - you cannot have progress without committing changes

Offline Magus_BGforge

  • Planewalker
  • *****
  • Posts: 75
I't's not about cosmetic weidu.log, it's about leftovers from backup. Changes will eliminate possibility of the mistakes.
I'm not talking about weidu.log. I don't think there's a possiblity of a mistake.

Offline AL|EN

  • Planewalker
  • *****
  • Posts: 391
  • Gender: Male
« Last Edit: July 18, 2017, 09:17:30 AM by AL|EN »
Project Infinity public BETA - mod manager for Infinity Engine games
Modder's Guide to Github - you cannot have progress without committing changes

Offline CamDawg

  • Infidel
  • Planewalker
  • *****
  • Posts: 859
  • Dreaming of a red Xmas
    • The Gibberlings Three
I think you'll find that many mods alter other things inside their mod folders aside from the backup directory. Tweaks, SCS, Fixpack, and many others all rely on some sort of 'working' directory/files as they install. Depending on how they're coded, these could also interfere if you just pick up and move the mod directory to a new install.
The Gibberlings Three - Home of IE Mods

The BG2 Fixpack - All the fixes of Baldurdash, plus a few hundred more. Now available, with more fixes being added in every release.

Offline AL|EN

  • Planewalker
  • *****
  • Posts: 391
  • Gender: Male
That would be even worse! I've just check cdtweaks using git/folder compare and this is what I found:

cdtweaks\lib\pnp_dual_core.tpa
cdtweaks\lib\pnp_dual_working.tpa

Regarding lib\pnp_dual_core.tpa and \lib\pnp_dual_working.tpa changes case, what would be the effect of coping cdtweaks folder (skipping backup) after it was installed, and use it for another installation?
« Last Edit: July 18, 2017, 03:23:58 PM by AL|EN »
Project Infinity public BETA - mod manager for Infinity Engine games
Modder's Guide to Github - you cannot have progress without committing changes

Offline GeN1e

  • Planewalker
  • *****
  • Posts: 267
  • Gender: Male
This thread made me wonder - should matches be banned worldwide too, because you may accidentally set your house on fire if you aren't careful?

Offline ikki

  • Planewalker
  • *****
  • Posts: 10
Quote
weidu itself directly affecting the mod directory
Is that not something that the Big World Fixpack should handle ?
It is just a matter of changing one line from "mymod/backup" to "backup/mymod" in the tp2 (and adding the corresponding directory?)

and i'm pretty sure that when modders see that things work fine with this new template (because everyone has copied the way weimer did the backup in his mod) things will change
« Last Edit: July 19, 2017, 07:06:04 AM by ikki »

Offline AL|EN

  • Planewalker
  • *****
  • Posts: 391
  • Gender: Male
ikki such proposition could only come from people who never have to deal with Fixpack. Only few mod authors care about it. You want to create patches for all existing mods and all future mods since nobody will read readme/IE modding best practices? Are you serious?

But in the light of the CamDawg post, we need to wait for his answer.
Project Infinity public BETA - mod manager for Infinity Engine games
Modder's Guide to Github - you cannot have progress without committing changes

Offline subtledoctor

  • Planewalker
  • *****
  • Posts: 131
That would be even worse! I've just check cdtweaks using git/folder compare and this is what I found:

You want to be horrified, look inside the '/deity' folder before and after installing Faiths & Powers. :P

Regarding lib\pnp_dual_core.tpa and \lib\pnp_dual_working.tpa changes case, what would be the effect of coping cdtweaks folder (skipping backup) after it was installed, and use it for another installation?

Why do that? It's easy enough to just grab a clean copy of the mod for that 2nd install.

More generally, lots of mods NEED workspace to do things before the final install into /override... surely it's better for such operations to be contained rather than polluting the player's game folder (where it would be easy to delete or alter them).

Rather than impose rules on how modders can treat their own mod folders (I mean come on, if that's not a space I can work with as I see fit, what is??), why not simply add an automatic function to Weidu to back up a clean copy of the mod before it installs? Then you can simply grab it from that backup location.

(Players can/should do this themselves, anyway. I have extracted copies of all mods I intend to use (in folders numbered according to install order, yeah I'm anal), and I only copy (not move) the mod folder into my game folder when I'm about to do an install. I know BWS doesn't do this... to its detriment IMO. It seems like it would be a trivial thing to add. And more generally the entire functions of 1) downloading mods and 2) installing mods, should be more distinct.

Offline AL|EN

  • Planewalker
  • *****
  • Posts: 391
  • Gender: Male
Quote
Why do that?
People are doing such thing since I've remember. I always think that the backup is only problem, but it seems that it's much more

Quote
Rather than impose rules on how modders can treat their own mod folders (I mean come on, if that's not a space I can work with as I see fit, what is??), why not simply add an automatic function to Weidu to back up a clean copy of the mod before it installs? Then you can simply grab it from that backup location.
Very good idea. In addition, I would also propose a solution (installed.flag etc) which will prevent installing any mod which was copied to another installation. The weidu.log can have some sort of "unique game/installation hash" and each mod would get such hash as a flag/other. So if the will mismatch after coping mod files, weidu will prevent installation.
« Last Edit: July 19, 2017, 10:30:31 AM by AL|EN »
Project Infinity public BETA - mod manager for Infinity Engine games
Modder's Guide to Github - you cannot have progress without committing changes

Offline CamDawg

  • Infidel
  • Planewalker
  • *****
  • Posts: 859
  • Dreaming of a red Xmas
    • The Gibberlings Three
That would be even worse! I've just check cdtweaks using git/folder compare and this is what I found:

cdtweaks\lib\pnp_dual_core.tpa
cdtweaks\lib\pnp_dual_working.tpa

Regarding lib\pnp_dual_core.tpa and \lib\pnp_dual_working.tpa changes case, what would be the effect of coping cdtweaks folder (skipping backup) after it was installed, and use it for another installation?
Oh, Tweaks isn't your problem. Go take a look at SCS and its initialization of SSL if you really want to give up.
The Gibberlings Three - Home of IE Mods

The BG2 Fixpack - All the fixes of Baldurdash, plus a few hundred more. Now available, with more fixes being added in every release.

Offline GeN1e

  • Planewalker
  • *****
  • Posts: 267
  • Gender: Male
why not simply add an automatic function to Weidu to back up a clean copy of the mod before it installs? Then you can simply grab it from that backup location.
I'm not sure it's gonna solve his problem. If there're players unpacking mod archives, installing them, then moving the unpacked mod folders with all their backups to another game, then I can bet all the money in the world they won't even know there's an automatic backup made. The point of which eludes me all the same, because you can just as easily re-unpack the archive.

In addition, I would also propose a solution (installed.flag etc) which will prevent installing any mod which was copied to another installation. The weidu.log can have some sort of "unique game/installation hash" and each mod would get such hash as a flag/other. So if the will mismatch after coping mod files, weidu will prevent installation.
But this actually might. Just prevent installation prompt of a component if a backup folder for it exists and has some stuff in it, and tell the user to go and clean it up. INSTALL_BY_DEFAULT should be excluded from the rule though, I think, and maybe some other flags too. Installing with shell scripts should also ignore it.
« Last Edit: July 19, 2017, 01:49:43 PM by GeN1e »

Offline Wisp

  • Moderator
  • Planewalker
  • *****
  • Posts: 1176
In addition, I would also propose a solution (installed.flag etc) which will prevent installing any mod which was copied to another installation. The weidu.log can have some sort of "unique game/installation hash" and each mod would get such hash as a flag/other. So if the will mismatch after coping mod files, weidu will prevent installation.
But this actually might. Just prevent installation prompt of a component if a backup folder for it exists and has some stuff in it, and tell the user to go and clean it up. INSTALL_BY_DEFAULT should be excluded from the rule though, I think, and maybe some other flags too. Installing with shell scripts should also ignore it.
I'll pencil it (prevent component installation if there already exists backup data for the component) in for when I rewrite the install code.
« Last Edit: July 20, 2017, 07:35:29 PM by Wisp »

Offline AL|EN

  • Planewalker
  • *****
  • Posts: 391
  • Gender: Male
Quote
Oh, Tweaks isn't your problem. Go take a look at SCS and its initialization of SSL if you really want to give up.
Yep, that make the "backup" problem trivial compared to the fact that the mod files itself are changed after one installation. We need a solution to prevent re-use of already installed mod directory.

« Last Edit: July 21, 2017, 05:23:46 AM by AL|EN »
Project Infinity public BETA - mod manager for Infinity Engine games
Modder's Guide to Github - you cannot have progress without committing changes

Offline DavidW

  • Planewalker
  • *****
  • Posts: 316
Re: weidu next generation - change "backup" location - won't fix
« Reply #20 on: September 19, 2018, 10:35:10 AM »
Coming very late to this thread, but in fact SCS's mod directory is completely immutable during the install process and has been since I merged scs and scsii into stratagems. I build lots of files, but I build them outside the mod folder, in stratagems_external/workspace, and backups live in stratagems_external/backup/stratagems. Longer discussion at http://gibberlings3.net/forums/index.php?showtopic=29677&hl=

Offline The Imp

  • Planewalker
  • *****
  • Posts: 288
  • Gender: Male
Re: weidu next generation - change "backup" location - won't fix
« Reply #21 on: September 19, 2018, 05:31:57 PM »
In the current state of the weidu backup system, there is one really annoying thing: when the weidu create "backup" folder and create binary files which are needed for the uninstallation, weidu itself directly affecting the mod directory. I really don't like this design. In order to use the same extracted mod, players have to make sure that they will remove "backup" folder when they will copy for eg cdtweaks + setup-cdtweaks.exe from BG1EE to BG2EE. If the unaware player will forgot about this, it will cause problems for them because "backup" folder will have leftover from the previous installation. Not to mention countless re-uploading mod because author forgot to remove "backup" folder.
You know, usually mods are archives, not random packets of files... aka, just EXTRACT the files a new. Is that so much to ask ?

Yeah, and the SCS takes two folders. I'll in my next mod throw things not in the game folder, but at the root directory and we'll see whose fun that will ruin. People, people. You could fix all this by actually doing what you ought to, have a back up of the game folder in a .rar archive, and use that as a clean install. Why, cause it extracts faster than the direct copy does, copying...
And you don't need fancy links, just delete the mod folder after you uninstall it. Yeah, remember, you'll have to test that too, not just install.

Offline DavidW

  • Planewalker
  • *****
  • Posts: 316
Re: weidu next generation - change "backup" location - won't fix
« Reply #22 on: September 19, 2018, 06:45:55 PM »
Yeah, and the SCS takes two folders.

I'm not sure I understand the point you're making.

Offline The Imp

  • Planewalker
  • *****
  • Posts: 288
  • Gender: Male
Re: weidu next generation - change "backup" location - won't fix
« Reply #23 on: September 19, 2018, 09:57:52 PM »
The point was ...
I build lots of files, but I build them outside the mod folder, in stratagems_external/workspace
.. you could have done so in: stratagems/internal/workspace -and you would have been fine. The only real reason I can think of shuffling files outside the mod folder is that you .bif them, as you do so from the whole folder, while using a folder size check.

Offline DavidW

  • Planewalker
  • *****
  • Posts: 316
Re: weidu next generation - change "backup" location - won't fix
« Reply #24 on: September 19, 2018, 10:25:57 PM »
The point was ...
I build lots of files, but I build them outside the mod folder, in stratagems_external/workspace
.. you could have done so in: stratagems/internal/workspace -and you would have been fine.
Of course I could have done, but then the mod folder wouldn’t be immutable, which is the whole point.

 

With Quick-Reply you can write a post when viewing a topic without loading a new page. You can still use bulletin board code and smileys as you would in a normal post.

Warning: this topic has not been posted in for at least 120 days.
Unless you're sure you want to reply, please consider starting a new topic.

Name: Email:
Verification:
Type the letters shown in the picture
Listen to the letters / Request another image
Type the letters shown in the picture:
What color is grass?:
What is the seventh word in this sentence?:
What is five minus two (use the full word)?: