Pocket Plane Group

Friends and Neighbors => Weimer Republic (WeiDU.org) => WeiDU => Topic started by: Miloch on March 17, 2010, 10:39:20 PM

Title: EDIT_SAV_FILE
Post by: Miloch on March 17, 2010, 10:39:20 PM
Could you provide at least some crude syntax for this? It's listed under Undocumented Features, and a grep of BWP shows no tp2 files where it's used. I was going to take a stab at some arcane loop of GET_DIRECTORY_ARRAY (also undocumented, but used in SConrad's wmp patching code) and DECOMPRESS/COMPRESS commands to achieve this but EDIT_SAV_FILE might be easier, if it works.

Speaking of undocumented features, didn't someone (GeN1e perhaps) offer to document these?
Title: Re: EDIT_SAV_FILE
Post by: the bigg on March 18, 2010, 03:03:39 AM
EDIT_SAV_FILE (test) BEGIN patch_list END

will iterate through the files in the SAV file, setting SAV_FILE to the file name and then apply patch_list only if 'test' is true.
Title: Re: EDIT_SAV_FILE
Post by: Miloch on March 18, 2010, 03:30:03 AM
Ok... does it include DECOMPRESS/COMPRESS or would I have to stack those in there somewhere to be able to sensibly edit any of the files?
Title: Re: EDIT_SAV_FILE
Post by: the bigg on March 18, 2010, 03:32:48 AM
Files are decompressed by EDIT_SAV_FILE.
Title: Re: EDIT_SAV_FILE
Post by: Miloch on March 30, 2010, 02:57:55 AM
Well, clearly I'm not doing something right with this command, or it's a bit hosed, or both.
Code: [Select]
GET_DIRECTORY_ARRAY save save ~~

ACTION_PHP_EACH save AS from => to BEGIN
  ACTION_IF FILE_EXISTS ~%to%/baldur.sav~ BEGIN
    PRINT ~Updating saved game %to% ...~
    COPY ~%to%/baldur.sav~ ~%to%/baldur.sav~
      EDIT_SAV_FILE (~%SAV_FILE%~ STRING_EQUAL_CASE ~ar0602.are~ = 1) BEGIN
        PATCH_PRINT ~Editing %SAV_FILE% ...~
//        WRITE_ASCII 0x94 ~none~ #8 //Area script (was ar0602)
      END
    BUT_ONLY
  END
END
I'm using an auto-save from the beginning of SoA for the test (I've moved others out of the save folder so it doesn't go through every bloody one). With the WRITE commented out, the code will execute, *but* it says it goes through not just ar0602.are, but default.toh and default.tot as well (the only files in the .sav).

With the WRITE uncommented, it fails on default.toh because it's smaller than my WRITE offset, but I don't want to patch that anyway, obviously.

Does SAV_FILE only get set after the EDIT_SAV_FILE ... BEGIN? From the way you described it, it happens before applying the patch_list, or should anyhow. I don't know what other "test" you'd want to use, other than files matching a certain pattern.

Also, I think it should DECOMPRESS the .sav only if it actually writes a change to the file, and COMPRESS again if it does. Otherwise, I'd have to keep track manually of whether it actually made a change, and reCOMPRESS everything if so. Unlike BAMs, SAVs don't indicate whether or not they're compressed (I'm guessing the game assumes they're always compressed).
Title: Re: EDIT_SAV_FILE
Post by: the bigg on March 30, 2010, 07:04:40 AM
*sound of me banging head against desk*

You need an explicit PATCH_IF to choose which files to patch (this allows you to patch multiple files in a single ESF). The parameter after EDIT_SAV_FILE is the (re)compression level - use 1 (BGII default) to make sure that BUT_ONLY does its job.

This is what happens when I read the grammar specification but not the actual implementation when answering questions.
Title: Re: EDIT_SAV_FILE
Post by: Miloch on March 31, 2010, 11:35:54 PM
There seems to be an extra trailing character (a null perhaps) in SAV_FILE making an INNER_PATCH such as the following necessary to trim it:
Code: [Select]
GET_DIRECTORY_ARRAY save save ~~

ACTION_PHP_EACH save AS from => to BEGIN
  ACTION_IF FILE_EXISTS ~%to%/baldur.sav~ BEGIN
    PRINT ~Updating saved game %to% ...~
    COPY ~%to%/baldur.sav~ ~%to%/baldur.sav~
      EDIT_SAV_FILE 1 BEGIN
        SPRINT t-bs ~%SAV_FILE%~
        TO_LOWER t-bs
        ln = STRING_LENGTH ~%t-bs%~
        INNER_PATCH ~%t-bs%~ BEGIN
          READ_ASCII 0 t-sav (ln - 1) //Trim rightmost character
        END
        PATCH_IF (~%t-sav%~ STRING_EQUAL ~ar0602.are~ = 1) BEGIN
          PATCH_PRINT ~Editing *%t-sav%* ...~
          WRITE_ASCII 0x94 ~none~ #8 //Area script (was ar0602)
          LPF REPLACE_AREA_ITEM
            STR_VAR
            old_item = ~bow05~ //Short Bow
            new_item = ~bow03~ //Long Bow
          END
        END
      END
    BUT_ONLY
  END
END
Title: Re: EDIT_SAV_FILE
Post by: the bigg on April 01, 2010, 05:49:22 AM
Fixed.
Title: Re: EDIT_SAV_FILE
Post by: Miloch on May 01, 2010, 01:39:17 PM
Fixed.
Eh, not really. In v216, EDIT_SAV_FILE now cuts off the last non-null character of every file within the .sav whether you edit the subfile or not, so you now get a .sav containing files like default.to and ar0602.ar etc. This is worse than the previous bug because I can't think of a workaround offhand.

Yeah I guess it's partially my fault since I should've tested the beta after the "fix" but I can't release the new components of Infinity Animations without forcing a regression to v215, which I don't think is even possible given the way WeiDU auto-updates.
Title: Re: EDIT_SAV_FILE
Post by: the bigg on May 01, 2010, 01:49:41 PM
Returns AR0602.ARE or whatnot for me.
Title: Re: EDIT_SAV_FILE
Post by: Miloch on May 01, 2010, 02:15:03 PM
SAV_FILE returns the full subfile names, but look at the resulting edited .sav file in NI and you'll see the two-character extensions.
Title: Re: EDIT_SAV_FILE
Post by: the bigg on May 01, 2010, 02:32:42 PM
Oh. Will fix.
Title: Re: EDIT_SAV_FILE
Post by: the bigg on May 02, 2010, 06:34:39 PM
Up.
Title: Re: EDIT_SAV_FILE
Post by: Miloch on May 03, 2010, 02:20:18 AM
Ok - tested, seems to work now.

Don't suppose I could talk you into a maintenance release just for this? It's either that or try to invent some nasty hackaround to make it work, and I'm not even sure that'd be possible.
Title: Re: EDIT_SAV_FILE
Post by: the bigg on May 03, 2010, 02:59:56 AM
Sure. The Win/Lin build will be up in half an hour or so from the date of this post, the Mac will have to wait for devSin's compile.
Title: Re: EDIT_SAV_FILE
Post by: devSin on May 03, 2010, 12:53:31 PM
No. Let's fix ADD_CRE_ITEM first.

Icewind 2 is targeting half-slot of what it should (e.g., INV16 is actually writing to +40, not +80).
Planescape fails (Invalid_argument("String.blit")) outright (is get_item_number in cre returning the right counts for non 1.0?).
Title: Re: EDIT_SAV_FILE
Post by: cmorgan on May 03, 2010, 01:09:47 PM
If you have a build up within about 3 hours, I can test for about two hours running out individual mods.
Title: Re: EDIT_SAV_FILE
Post by: the bigg on May 03, 2010, 02:40:52 PM
IWD2: in git (I don't have IWD2 installed so you'll have to test it).

PST: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAARGH
IESDP contains incorrect data - the BG2 section counts the magical weapon as a slot, the PST section doesn't. Fixed, in git (I tested it on a random creature, if you have more extensive tests please run them too).
Title: Re: EDIT_SAV_FILE
Post by: devSin on May 03, 2010, 03:14:00 PM
Looks fine. I'm not sure what you're talking about IESDP, though.

If you want to update the PST fallback to INV19 (instead of pick-a-slot-and-just-say-it's-15), then push the final source zip to FTP, I'll do the Mac build.
Title: Re: EDIT_SAV_FILE
Post by: the bigg on May 03, 2010, 03:34:36 PM
It's 15 because it's 15 in bg2 and it didn't feel significant enough to warrant yet another match..with. Do you still want it?
Title: Re: EDIT_SAV_FILE
Post by: devSin on May 03, 2010, 04:37:29 PM
It says 15 but it's not actually 15, duh. It would be 15 if it were V1.0 CRE, but it's not. :-)

It's 15 in BG2 only because that's max-1.
Title: Re: EDIT_SAV_FILE
Post by: the bigg on May 03, 2010, 05:48:01 PM
Now I get what you're saying. Done, and reuploaded as 217 (no version bump versus this morning's version).
Title: Re: EDIT_SAV_FILE
Post by: devSin on May 03, 2010, 06:23:43 PM
Mac build uploaded; thanks.
Title: Re: EDIT_SAV_FILE
Post by: devSin on May 12, 2010, 04:17:56 PM
Oops, the list is jumbled. For ID2, it's Quiver1 thru Quiver4 then Cloak (not Cloak then Quiver). And add the final quiver slot in (but you can leave 2E shield slot out), and for PST too. They're no less valid in these than in BG2. cre.ml

Hey, Jason, is there someplace on the server somewhere to put precompiled binaries of Near Infinity? There are like five billion versions floating around in various stages of decay; I always have the latest, but I never like linking to my personal site, so nobody else has access to them. (I don't think we need a page or anything, just something to link to in forum replies for people who say "Where do I get NI to test out the latest file corruption features?" or "My NI says it was last modified in 2004, is it the latest?")
Title: Re: EDIT_SAV_FILE
Post by: the bigg on May 13, 2010, 11:50:07 AM
My hate for the IESDP is unbounded.

What is Quiver4/Quiver6 for? They're not accessible by GUI AFAIK (I'd rather comment it out in BG2 too).
Title: Re: EDIT_SAV_FILE
Post by: devSin on May 13, 2010, 12:13:33 PM
They're just quiver slots. The engine likes them just fine.

Why all IE has one extra quiver slot than presented in the GUI (not even to ask why PST has more than one quiver slot at all) is a question for the ages.
Title: Re: EDIT_SAV_FILE
Post by: the bigg on May 13, 2010, 04:52:58 PM
not even to ask why PST has more than one quiver slot at all
There is a decent variety of enchanted bolts, although it isn't like you're going to find a battle where the standard ammo doesn't cut it.
Title: Re: EDIT_SAV_FILE
Post by: devSin on May 13, 2010, 05:48:35 PM
There is a decent variety of enchanted bolts, although it isn't like you're going to find a battle where the standard ammo doesn't cut it.
For the one character who can even shoot a fucking bow? Five slots to spend in only one place in the game?

There are plenty of enchanted projectiles in normal IE, and yet they make do with only THREE SLOTS. FOR ALL YOUR CHARACTERS.
Title: Re: EDIT_SAV_FILE
Post by: devSin on May 15, 2010, 06:41:06 PM
I'm really unhappy with the handling of BIS string concat when decompiling BCS. (I don't remember they did this in PST, so why the hell did they even add it?)

Currently, something like BitGlobal() will decompile to "SCOPE:VARNAME","" (in IWD); in NI, we decompile to ":VARNAME","SCOPE" (and leave the colon, simply because we don't treat it different from Global()). Which isn't a big deal, except that compiling ":VARNAME","SCOPE" with WeiDU will put SCOPE into the second (worthless) string parameter (and will break the check in-game). Ideally, I'd like it to handle NI's ":VARNAME","SCOPE", its own "SCOPE:VARNAME","" (although, decompiling like this should probably be a bug because that's what the freaking scope string is there for), and the way it was always supposed to be "VARNAME","SCOPE" (and let the compiler handle adding the colon where it's needed), although doing it the right way is pretty optional at this point. (I'm pretty sure we can compile WeiDU's junk output in NI without complaint, but I don't want to test.)

When it comes to ID2, it's all just broken. Decompiling any concatenated string strips the colon permanently (so BitGlobal() or SetGlobal(Timer)?Random() will decompile to "VARNAME","SCOPE", permanently losing the colon--i.e., compiling to invalid "SCOPEVARNAME" instead of "SCOPE:VARNAME", and SpellCastEffect() decompiles to WHO,"SOUND","RETARD1","RETARD2",WHAT,SPEED,SEQUENCE, and RETARD1 and RETARD2 aren't even handled correctly unless they're both specified--e.g., "RETARD1:" will decompile to "","RETARD1" and then recompile to an empty "" string). Ideally, all the forms should work and be equivalent (IWD broken "SCOPE:NAME","", NI good-enough ":NAME","SCOPE", and your try-but-totally-fail ID2 "NAME","SCOPE") and SpellCastEffect() needs really special love to handle "S1","S2" & "S1:S2","" & "","S1:S2" & "S1:","S2 & "S1",":S2" (as well as all the completely empty or colon-only equivalents).

Although, honestly, getting rid of the "right" way (no explicit colon in BAF) seems like the easiest shot. I know somebody asked for it (and apparently couldn't even tell that you just ended up breaking it all), but get over it. It's not a big deal to just put the colon in your own damn self (and NI will never support auto-colon, at least not by my hand).
Title: Re: EDIT_SAV_FILE
Post by: the bigg on May 15, 2010, 06:45:31 PM
Sorry, I have long since given up on BAF <-> BCS inconsistencies.
Title: Re: EDIT_SAV_FILE
Post by: devSin on May 15, 2010, 06:48:45 PM
It's fine if you don't want to make it work better, but it shouldn't be left breaking all the code where a colon is required (BitGlobal(), SetGlobalRandom(), and SpellCastEffect() can't be used as decompiled anymore because of the way it's manipulating the string params when decompiling). That's not an inconsistency; that's a bug.
Title: Re: EDIT_SAV_FILE
Post by: devSin on May 15, 2010, 07:29:01 PM
I've been blaming you, but it looks like this may be Fred's fault? What the Fruck were you thinking, Fred? Although, it makes your statement a little odd...

Worst case scenario, can't these be made to just decompile what they're supposed to be in the BCS (like IWD currently does, outputting the whole concatenated string in s1)? I just want it to decompile something that it can actually recompile into a working script, which means not screwing with the colon.
Title: Re: EDIT_SAV_FILE
Post by: devSin on May 16, 2010, 12:14:13 AM
Also, if possible and easy, maybe a MODDER strlen() <= 32 check for string params (>32 will never be valid for any IE) to dump a simple warning (it should still compile, since there are some scripts with too-long strings).

Apparently, we don't even check this is NI, doh (I'll see about adding in a test in checkString() since we already check for everything under the sun, but we have much bigger compiler issues to be honest).
Title: Re: EDIT_SAV_FILE
Post by: devSin on May 17, 2010, 09:20:54 PM
Apparently, we don't even check this is NI, doh (I'll see about adding in a test in checkString() since we already check for everything under the sun, but we have much bigger compiler issues to be honest).
Turns out it took like two seconds to add this check in for us, so don't bother in WeiDU.

In ID2, the correct object spec seems to be EA GENERAL RACE CLASS SPECIFIC GENDER ALIGNMNT SUBRACE AVCLASS CLASSMSK (WeiDU currently swaps CLASS and SUBRACE). Subrace, however, is the byte value (0 or 1 or 2) and not the race/subrace dword from Subrace.ids. Still no luck with AVCLASS, but it and CLASSMSK are actually stored in the CRE file (word + dword at 0x3B4). AVClass looks like just a clone of class, but the engine doesn't seem to want to check (it likes any value for any object).
Title: Re: EDIT_SAV_FILE
Post by: the bigg on May 23, 2010, 08:41:32 AM
Check that strlen(first) <= 32 done.

Subrace <-> class done. I don't understand what I should do about that SUBRACE.IDS or AVCLASS nonsense.

Still have to do that colonoscopy. Can you give me script name(s) to work on?
Title: Re: EDIT_SAV_FILE
Post by: devSin on May 23, 2010, 10:58:04 AM
Check that strlen(first) <= 32 done.
Is this before the concat of the scope and name for vars (i.e., just checking strlen(name))? You can test with 53MALAVC in ID2.

Subrace <-> class done. I don't understand what I should do about that SUBRACE.IDS or AVCLASS nonsense.
Nothing. It was just me saying what they were.

Still have to do that colonoscopy. Can you give me script name(s) to work on?
65CSTATU is good for SpellCastEffect() nonsense. 50CARIT0 and 50LEEVI0 have some TimerRandom() calls, and AR50(19|20|30) have some GlobalRandom() calls.
Title: Re: EDIT_SAV_FILE
Post by: the bigg on May 23, 2010, 01:52:12 PM
Is this before the concat of the scope and name for vars (i.e., just checking strlen(name))? You can test with 53MALAVC in ID2.
SetGlobal("12345678901234567890123456789012","GLOBAL",1) will not whine, SetGlobal("123456789012345678901234567890123","GLOBAL",1) will.

I'll install ID2 for the rest.
Title: Re: EDIT_SAV_FILE
Post by: devSin on May 23, 2010, 09:00:30 PM
Still have to do that colonoscopy. Can you give me script name(s) to work on?
Ouch. It looks like dialogues shouldn't have colons at all, though (the engine must be smart enough to compile it correctly, even though none of our compilers could), so decompiling to colon-free does have some use (have to be able to recompile 11OSWALD). AH HA HA HA HA!
Title: Re: EDIT_SAV_FILE
Post by: the bigg on May 24, 2010, 03:00:52 PM
Code: [Select]
WARNING: cannot verify action ~StartCutsceneMode()
StartCutscene(#53cKulda#)~: Parsing.Parse_error
WTF
Title: Re: EDIT_SAV_FILE
Post by: devSin on May 24, 2010, 03:44:52 PM
Yeah, some of the dialogues have # in place of " quotes. I fixed them locally, but I never played the game; the engine probably muscles through a mistake like that regardless of the fact that it's bullshit (or it's an orphaned node that they just left dumb).
Title: Re: EDIT_SAV_FILE
Post by: the bigg on May 24, 2010, 03:53:26 PM
All should be in git: the *.bcs files you listed are unaltered by DECOMPILE_BCS_TO_BAF/COMPILE_BAF_TO_BCS; 11oswald.dlg (after fixing # -> ") doesn't spontaneously add colons to actions or triggers by DECOMPILE_DLG_TO_D/COMPILE_D_TO_DLG (it changes because WeiDU adds all those newlines between actions/triggers).
Title: Re: EDIT_SAV_FILE
Post by: devSin on May 24, 2010, 05:18:34 PM
Cool. Since fixup3 is just for SpellCastEffect, can it cat s2,s3 rather than reverse? (The others switch around to pull out the scope and position it, but this is just literal s2:s3 mashed together.) And then just fix up splitc to dump "S1:","S2", I guess (or better yet, just screw the split and return "S1:S2","" and let it concat normally on compile).

Also, for Load.IWD1, can you add to is_concat1
Code: [Select]
141l (* GivePartyGoldGlobal(S:Name*,S:Area) *)
165l (* AddexperiencePartyGlobal(S:Name*,S:Area) *)
247l   (* BitGlobal(S:String1*,S:String2*,I:Value,I:Mode*BitMode) *)
0x40A5l (* BitGlobal(S:String1*,S:String2*,I:Value,I:Mode*BitMode) *)
and
Code: [Select]
248l   (* GlobalBitGlobal(S:String1*,S:String2*,S:String3*,S:String4*,I:Mode*BitMode) *)
0x40A6l (* GlobalBitGlobal(S:String1*,S:String2*,S:String3*,S:String4*,I:Mode*BitMode) *)
to is_concat2?

It looks like this will fix up the Scope:Name in BAF issue I mentioned.
Title: Re: EDIT_SAV_FILE
Post by: the bigg on May 24, 2010, 06:12:03 PM
Done.