lol

[the bigg]

Code:

MOVE ~source~ ~destination_with_%variables%~

With that (and a couple  of extra tweaks), gen_biff now can biff 26K files (http://pastebin.com/f134f2341) in eight minutes... On a 5400 hard disk:

Code:

WeiDU Timings
<...>
stuff not covered elsewhere     11.380
MOVE                           110.918
BIFF                           337.897
TOTAL                          480.549

(suitably enough, Winamp is playing this as I type this).

[Azazello]

Code:

MOVE ~source~ ~destination_with_%variables%~

With that (and a couple  of extra tweaks), gen_biff now can biff 26K files (http://pastebin.com/f134f2341) in eight minutes... On a 5400 hard disk:

Code:

WeiDU Timings
<...>
stuff not covered elsewhere     11.380
MOVE                           110.918
BIFF                           337.897
TOTAL                          480.549

(suitably enough, Winamp is playing this as I type this).

Yay?

[cmorgan]

Great taste in music, man!

Good breakthrough!

[Miloch]

The readme says about MOVE:

Safety notes: when uninstalling, MOVE is restored first, then all generic actions are restored, then AT_*!UNINSTALL is handled. As such, do your AT_NOW, then do your COPY, then do your MOVEs

As discussed elsewhere, we want to move buggy files out of the override before COPYing over proper ones. The main reason we can't just overwrite the old ones is because they don't all have the same names as the one's we're copying. We do this by batch files now but I'm considering recoding it to use MOVE. Is this going to cause issues at uninstall?

[the bigg]

Can't you just write a sample tp2 and try it out? 

If there is no filename collision this is a non-issue; if there is a collision, the file is completely deleted from the override. I don't know the specifics of your mod, but on a general basis it's an issue.

[Miloch]

Can't you just write a sample tp2 and try it out? 

My goal was to get some sort of answer (or informed opinion at least) before coding something that may or may not be useful. Mainly because I spent loads of time writing the batch files to begin with but would prefer to dump those if feasible. The collision thing I'm not sure of. Certainly, before Infinity Animations, several mods may have overwritten these BAMs but after it we would hope not (since it should be installed near the end).

[the bigg]

The problem is if a mods installed before IA have filename collisions with IA, not with those installed after.

Code:

BACKUP ~test/backup~
AUTHOR ~tb~

BEGIN ~0~
<<<<<<<< .../newfoobar
old
>>>>>>>>

COPY ~.../newfoobar~ ~test/prod/foobar~


BEGIN ~1~

MOVE ~test/prod/foobar~ ~test/prod/oldfoobar~
<<<<<<<< .../newfoobar
new
>>>>>>>>

COPY ~.../newfoobar~ ~test/prod/foobar~

Install 0 and 1, uninstall 1 (simulate the case where a mod installed before IA collides with IA). Correct uninstall behavior would be to have foobar contain "old". However, the file is removed.

UNINSTALL_MOVE_AFTER_COPY tp2 flag incoming. This is the predicted end result.

[Miloch]

UNINSTALL_MOVE_AFTER_COPY tp2 flag incoming.

Serious or a joke? Either way, sounds like I should stick with the batch files until something like that exists.

[the bigg]

It's going in.

[Miloch]

Apparently some folks and memory-resident programs have been whinging about the MOVE command. Personally, I think it's antivirus (typically Avast to be specific), since it almost always goes away when disabling it. Moreover, this is rarely reproduceable in the same way, in that it hangs up on *different* files with some vague, bogus error message (such as "error copying somefile.bam" or "Patching failed (COPY)"). But some folks claim otherwise, in some very specific situations (such as if RoT and Aurora are installed). Since I'm not familiar with the code, is it at all possible a trifling code change such as specifying a destination file rather than folder would make a difference, as claimed here?

[the bigg]

Yes, you need to specify the full destination (including the filename). I thought nobody would use it, so I coded it in the quickest possible way (and obviously the underlying OCaml library is the usual piece of shit and doesn't add the filename if given a directory). 214 will allow you to specify just the destination folder.

[Miloch]

214 will allow you to specify just the destination folder.

What about just the source folder too? I'm thinking maybe MOVE ~mymod/bams~ override might be a less space-intensive (but possibly more bug-prone) way of transferring several GB of animations to the game than COPYing the same. Maybe it does this already, dunno.

As for UNINSTALL_ORDER (for which apparently I'm to provide documentation), this strikes me as a bit wack:

ALL operations must be specified (if any isn't, a warning is printed). Please note that this means your mod might start printing warnings three years down the line if a new uninstall action is coded

Why not just let it take something like UNINSTALL_ORDER COPY MOVE etc, where "etc" is the remaining arguments in the default order, whatever those happen to be at the time.

[the bigg]

What about just the source folder too? I'm thinking maybe MOVE ~mymod/bams~ override might be a less space-intensive (but possibly more bug-prone) way of transferring several GB of animations to the game than COPYing the same. Maybe it does this already, dunno.

Sure.

As for UNINSTALL_ORDER (for which apparently I'm to provide documentation)

Those who request actions that are complex to use or understand are supposed to write a mod using them and then provide a tutorial. This is both because I'm the usual lazy bum, and because I'd end up writing a lot of useless details and not enough information (see: READ_2DA_ENTRY_FORMER).

Why not just let it take something like UNINSTALL_ORDER COPY MOVE etc, where "etc" is the remaining arguments in the default order, whatever those happen to be at the time.

The warning is printed, but otherwise it's performing according to the behavior you suggest (unmentioned uninstall actions are performed after the explicitly sorted ones). And (potential) uninstall problems is something I'd really want to jump out.

[Miloch]

I'm guessing this is the right syntax?

Code:

UNINSTALL_ORDER ~STRSET~ ~COPY~ ~MOVE~ ~AT~

It fails with other variants, like without the quotes or putting all the arguments in one pair of quotes. If so, I'd say just put such an example of changing the default syntax in with the existing documentation, not a tutorial with a more detailed explanation, because the chances of a sane modder wanting to use it are fairly nil.

Workaround for MOVE ~mymod/bams~ ~override~ in the meantime seems to be:

Code:

ACTION_BASH_FOR ~mymods/bams~ ~^.+\..+$~ BEGIN
  MOVE ~%BASH_FOR_FILESPEC%~ ~override~
END

Small glitch seems to be the files get moved (and restored on uninstall) as uppercase regardless of the original casing, or is that a function of the BASH_FOR_FILESPEC variable being uppercase?

And before I test with some stupid amount of data, will this in theory need a MOVE_LARGE for massive files the same way COPY_LARGE is otherwise required?

[the bigg]

It fails with other variants, like without the quotes or putting all the arguments in one pair of quotes. If so, I'd say just put such an example of changing the default syntax in with the existing documentation, not a tutorial with a more detailed explanation, because the chances of a sane modder wanting to use it are fairly nil.

Quotes are needed because MOVE etc. are reserved keywords, and the effort to allow either strings or keywords is much greater than the usefulness (imagine allowing to use 'if' as a variable name in C++ or what have you).

Small glitch seems to be the files get moved (and restored on uninstall) as uppercase regardless of the original casing, or is that a function of the BASH_FOR_FILESPEC variable being uppercase?

It's BASH_FOR_FILEPSEC which is automatically uppercased. I'm not sure if it is by design or by accident, but I can convert it to standard case (and probably should).

And before I test with some stupid amount of data, will this in theory need a MOVE_LARGE for massive files the same way COPY_LARGE is otherwise required?

No.

[the bigg]

Done MOVE someDir someOtherDir. Do you need to be able to overwrite files with MOVE or not? I'd rather have it fail the installation, since this would break uninstallation, but if you need it I can leave it in.

[Miloch]

Do you need to be able to overwrite files with MOVE or not? I'd rather have it fail the installation, since this would break uninstallation, but if you need it I can leave it in.

Well, right now I'm generally MOVEing files from the override to the mod's backup folder if they already exist, then MOVE the new mod files to the override. But that seems a bit sloppy - can't WeiDU do that automatically?

[the bigg]

That still wouldn't produce valid uninstalling (as MOVEs are uninstalled in the same order they're performed, not in reverse order). The best solution would be to have MOVE fall back to COPY.

[Miloch]

That still wouldn't produce valid uninstalling (as MOVEs are uninstalled in the same order they're performed, not in reverse order). The best solution would be to have MOVE fall back to COPY.

Eh, ok. COPY overwrites anyway, but it also backs up overwritten files, right? Why can't it be done in reverse order - OCaml limitation?

[the bigg]

Why can't it be done in reverse order - OCaml limitation?

Laziness, and potential memory limitations (OCaml uses brain-dead French algorithms that surrender when facing lists of over 60k elements).

[Miloch]

I'm probably phasing out my use of MOVE anyway, since it's pretty useless if someone's biffed the files.

(Don't suppose I could talk you into EDIT_BIFF/EDIT_KEY commands along the same lines as EDIT_SAV_FILE? )

[the bigg]

biff files can be too big to be freely edited with WeiDU (> 16MB), as do files contained in them, so I can't do something as generic as EDIT_SAV_FILE. On the other hand, if you can give detailed specifications I can try to work something out.

[Miloch]

For the current purpose, I'd just want to delete certain files (BAMs mainly) from biffs and update them and the .key accordingly. I'm handling it now by dumping blank or duped BAMs to the override, but I'm not sure how well that's gonna work.

[the bigg]

How about

Code:

DISABLE_FROM_KEY ~file1.ext~ ~file2.ext~ ...
DISABLE_FROM_KEY_REGEXP ~^file[0-9]*\.ext$~ ...

They'd remain in the bif, but unreferenced from the chitin, thus functionally non-existent (deleting them from the bifs wouldn't result in reduced disk consumption, since I'd have to create a copy of the bif for backup purpose).

[Miloch]

Sure, I suppose that'd work. I'm guessing it could be coded such that it'll skip any files that aren't in the .key, or would you have to pass it a FILE_EXISTS_IN_GAME check?