Post reply

Warning - while you were reading 16 new replies have been posted. You may wish to review your post.
Warning: this topic has not been posted in for at least 120 days.
Unless you're sure you want to reply, please consider starting a new topic.
Name:
Email:
Subject:
Message icon:

Verification:
Type the letters shown in the picture
Listen to the letters / Request another image

Type the letters shown in the picture:
What color is grass?:
What is the seventh word in this sentence?:
What is five minus two (use the full word)?:

shortcuts: hit alt+s to submit/post or alt+p to preview


Topic Summary

Posted by: K4thos
« on: August 12, 2017, 01:59:04 PM »

Quote
Hey, @K4thos, you're a maintainer of big world setup (or at least a person that occasionally modifies code there) right?
not really, I have no idea about the way BWS works. I'm part of the "BiGWorldProject organization" on Github just to have full access to mods hosted there, not to mess with BWS (which I don't think is even hosted on Github but on bitbucket.org to which I don't have access). Currently BWS is not in active development (when it comes to new functionality) anymore since there is no one left to maintain it. Maybe AL|EN will accept a commit if you know how to fix it on Wine. I can't help with this task.
Posted by: SCO
« on: August 12, 2017, 01:35:02 PM »

Hey, @K4thos, you're a maintainer of big world setup (or at least a person that occasionally modifies code there) right?

Because i think i know why it's not working on wine.

Many times the program will start a external process, get its standard output into a string a check if that string has a substring (to check for success). This happens to check for successful extraction checking for "Everything is Ok" from 7z and i think it also does when it needs to copy inner folders out into the root of the game (fails finding the tp2 file).

Anyway, this fails because for some reason autoit (not wine, i tested this by using the 7z.exe bundled on BWS on a mod archive) is corrupting the inputstream, probably by assuming a different codepage or charset.

"Everything is Ok" turns into "øvørythingàisàOk" on the BWS text frame (every output of a external process is corrupted).
I could hack it for extraction by either ignoring the check or trying to compare against the corrupted string, but that won't work for the tp2 I think.
Do you have any insights how to fix this? LANG=C wine ... didn't work.
Posted by: K4thos
« on: August 12, 2017, 01:13:28 PM »

Quote
A open question in my mind is EET and BGT. These need to apply mods to *both* games, and either need to combine files before or after mods install. I'm actually rather confused on if or how Zeitgeist will support these. I suspect it wont, and more hackery will be needed.
I don't think BGT allows mods to be installed on BG1 before conversion. In EET this feature is more a workaround for the fact that many BG:EE mods don't have EET support. Since they work just fine when installed on BG:EE before EET starts conversion (so they are treated like vanilla game files) I added such option because why not. Ideally the list of mods installed on BG:EE (section 1 of this post) will (hopefully) shrink over time. This was meant as a temporary solution, so designing Zeitgeist around it sounds a bit excessive. There's no that much mods which still need this approach and biggest ones like BG1 NPC Project already uses cpmvars so adding native EET support is not much of a problem (there is some progress on it actually last time I heard). The situation is worse for mods designed exclusively with BG:EE in mind since those don't use multi-platform specific code but still I don't think EET compatibility state with other mods should dictate the way weidu is developed. As for normal compatibility checks there is GAME_IS ~eet~ parameter already.

edit: I think I misunderstood your post, sorry. You're probably talking about detecting game installation as either BGT or EET before any mod is installed on BG2/BG2:EE. This indeed may need some additional hackery to implement conflict resolve feature directly in Zeitgeist.
Posted by: SCO
« on: August 11, 2017, 07:14:23 PM »

Github apparently disables git archive so the 'simple' way to get single files - for the tp2 - is disabled, but there is a alternative:

https://stackoverflow.com/a/4605068/214260

for example, to get the mod 'tales of Anegh' tp2, you can do

https://raw.githubusercontent.com/whiteagnus/ToA/master/ToA/setup-toa.TP2

and to get the whole dir, unziped you can either do

git clone --depth 1 https://github.com/whiteagnus/ToA.git /path_of_game (doesn't need unzip, but has .git gunk)

There is a command to download a zip, but that leaves out the possibility to update the repo, which is valuable not to download again, if we're reinstalling or updating (right?). It would be better if WEIDU supported having the mods be outside of the game prior to install/uninstall.

Anyway, so i think a 'mod list' would have at least the relative path it expects to the tp2, in this case:
ToA/setup-toa.TP2

From this, another list for the download/checker tool would have
path                        user+repo            branch
ToA/setup-toa.TP2   whiteagnus/ToA    master

this would permit to build both urls above and still be flexible to change branches.
Posted by: SCO
« on: August 11, 2017, 05:23:24 PM »

Though i have to say i'm unsure about not having ordering. I think you should have a ordering rule, even if it's not used for sorting. Some mods *have* to be installed before others, for consistency or conflict reasons, and work perfectly fine in the opposite order. As such, it's another check that the provided list is sane.

Of course, some mods have to be (optionally, nearly) first or  (optionally, nearly) last, but i think it's not absolutely necessary to care about those absolutes, since every joe and jane would figure that out quickly when sharing lists. Random mod #23 having to go before random mod #54 is another matter.
Posted by: SCO
« on: August 11, 2017, 05:16:50 PM »

Umm, with github and a standard form for the repositories, this could be done already as long as WEIDU had some conflicts, depends, depends-one-of (to components hopefully) and we don't need to care about compression and uncompression.

Git allows download a single file or the 'repository' at depth 1 (that is just the last version). Everything comes zipped and unzips into a directory automagically. It even checks consistency.

All that it's required is that WEIDU gains those conflicts, depends, depends-one-of rules, and a

--check-requirements [list of tp2] single-tp2 [optional component number]

This would allow external tools to get all this info and do the rest, if mod order is specified with curated ordered lists of components (like today really).

It just needs doing that, getting the fixpack to put in all the conflict and requirements knowledge they already have on the tp2 - by means of the diff files, and this could be solved relatively quickly, except translating all the code necessary for BGT, EET and IWD-in-EET (but this should be relatively well documented i presume).

Zeitgeist will still provide a great impetus for 'sane' packaging with its new format, but i think the great advantage is if we can get those same sane folder structures in github pages.[/list]
Posted by: SCO
« on: August 11, 2017, 09:27:08 AM »

No, i'm pretty much convinced the central (without extra metadata) is better now (as long as the conflict language of weidu is rich enough, ie: oneof requirments, conflicts etc). This is because unlike metadata, updates don't have to care about mod versions.

Because if updates follow the same scheme as the fixpack; with diff files, they either apply, and the mod was meant to have those new conflicts or requirements, they're either already applied, and everything is fine, or they fail and the mod should be disabled from the list.


Now this doesn't necessarily mean that it must be Zeitgeist applying fixpacks before checking, just that they 'should' before running the check and in order to run the check we want at least the main mod tp2.


Fortunately, github pages allows to get not only the archive, but the latest version of a file easily. We could do a scheme where a plugin or a script apart takes a mod order list and a list of mod download places, filters the second with the first to get the required mods links, downloads the tp2 from those places, and makes Zeitgeist check *only* the tp2. Then when the check is done, the mods chosen, and the 'install' button pressed another plugin 'downloads for real', applies the fixpack to *all* the files, not just the tp2, makes zeitgeist recheck (because timing) and if it doesn't work stops, if works, actually start the install.

This could also be a (more complicated for the user) script song and dance.

A open question in my mind is EET and BGT. These need to apply mods to *both* games, and either need to combine files before or after mods install. I'm actually rather confused on if or how Zeitgeist will support these. I suspect it wont, and more hackery will be needed.


In the end, it might end up that the only real interesting part of zeitgeist we would use is a command line --check-conflicts [listoftp2] singletp2 to report conflict and script the rest for convenience sake.
Posted by: subtledoctor
« on: August 11, 2017, 12:03:44 AM »

I think a community of curated mod orders could arise for 'thematics' with import/export.

Re: install order(s), that is precisely what I had in mind.

Regarding conflicts reporting: it doesn't need to be an either/or proposition. Right now there's nothing modders themselves can do to mark conflicts. A component flag could make that trivially easy - every time a player drops a note on a forums like "hey when I use your sphere system, the Sandrah NPC has no spells" I could quickly add this to that component:
Code: [Select]
CONFLICT ~sandrahnpc.tp2~ ~100~ ~Sandrah has no spells~
That would of course not be exhaustive. But it would be bi-directional, so it could help even with mods no longer being maintained. And it could be parsed and added to some metadata, supplemented by more metadata, which would then be processed by Zeitgeist whenever a player checks two boxes that shouldn't be checked at the same time.

(Of course I have no idea how Zeitgeist actually works, so I am very much speaking in the abstract.)
Posted by: SCO
« on: August 10, 2017, 09:47:27 PM »

Ok i'm pretty convinced: centralizing conflicts and dependencies on the mod itself is the right approach, because it allows the 'fixes' not to care about version numbers, only that the diff applies or not, and if it doesn't, presumably things are fixed (the maintainer checks obviously).

I have to request then, that Zeitgeist, please please allow a preprocessing step for a mod list before conflict checking (install a fixpack weidu or something completely other). Megamods lists *need* updates like this and responsiveness between mods releases or forceful updates to dead mods to work.
Posted by: SCO
« on: August 10, 2017, 09:39:29 PM »

Though if it would it would treat fixpacks special... it sort of solve the issue, until the diff file gets accepted by upstream (if ever). The people doing the actual conflicts would need to know WEIDU, but other random people could report issues on the github page.

I have no opinion on the component numbers, designated uiid discussion.


Version numbers for mods... assuming the dependencies and conflicts are specified on the tp2... are not very important except for the fixpacks ? To make sure it doesn't get applied to where it's fixed already. Though i suppose the diff would fail then, so no problem.
Posted by: SCO
« on: August 10, 2017, 09:15:15 PM »

But, playing devil advocate, assuming everything about conflicts and dependencies is centralized in .tp2 (centralization is a virtue i like), a way to 'mitigate' the loss of 'wisdom of the crowds' is to have conflict modifications on a 'fixpack'.

Problem is, it's both less effective (more specialized user knowledge) and i think it might bork how Zeitgeist checks. Fixpacks operate on diff patches, Zeitgeist is (presumably) not going to treat fixpacks 'special' and install them prior to checking for conflicts to uninstall the pack after.
Posted by: SCO
« on: August 10, 2017, 09:09:57 PM »

4) This is good, but i can't help but think that if Zeitgeist is going to punt on 1-3 it should have a easy way to be fed the output of a tool that does 3) because it's really uh.... kind of pointless to seperate this in 1-4 otherwise.
Posted by: SCO
« on: August 10, 2017, 09:03:51 PM »

Quote
1) downloading mods
2) establishing install order
3) resolution of conflicts
4) triggering Weidu install of everything

Right now, Zeitgeist encompasses #4.  I think it's not too much to ask for players to handle #1, and my idea for a compatibility component flag would shift #3 to modders

1) is good not only to players but to modders because that way they can feel confortable in depending on another mod stable features. But it's really really good for players obviously. It's not even that hard, wget will get you 90% of the way there, github repositories the other 10%. If zeigeist wants to create a 'file format', something like a the zip download of a github page being accepted by it as long as everything else is correct (manifest, tp2 on the right dir etc) would be very very good for auto updated mods.

2) Pretty difficult to do 'in general'. Mlox tries to this 'flying with no breaks' that is, there isn't a single install order, but a emergent one from a collection of precedences. It gets pretty messy. My preference for IE, that has much less mods might be 'user defined' install orders with import and exports. The obvious 'depends on' dependencies would be handled by the mods (so if that changes the mod order acquires a error), but things like this first , then this one , then that by the absolute order. I think a community of curated mod orders could arise for 'thematics' with import/export.

3) pretty obvious that this should *not* be left to modders, or not only. No modder has the time to check that his mod has no conflicts with hundreds of other mods, or a modder can *easily* abandon a mod after a few years. A user finding a conflict, can report it fast or edit it right on the github page if it's good enough (it opens a pull request). Unlike learning WEIDU, this is actually possible for barbarian lusers. There is also the obvious point that modders like to delay things, and a conflict rule update download can be much faster than a new release.

4) This is good.
Posted by: AL|EN
« on: August 03, 2017, 03:46:43 PM »

You basically describe the reasons why a system which is based on the DESIGNATED won't work. It might work in most of the cases until someone will use outdated "best mod compilation" exported list and he will get wrong components and wrong install order. The failure is inevitable and the "until the next failure" is no acceptable solution. I've encounter this issue countless of times during BWS maintenance: it can'e even properly store current selection of the mods which you have selected a month ago. Image ongoing frustration when you have all you desired tweaks and suddenly it all go to hell during installation of after two weeks of progress in the game. I don't want to see such bullshit anymore.
Posted by: subtledoctor
« on: August 03, 2017, 03:38:13 PM »

What do you mean "it won't work if it's based on DESIGNATED?"  If I do a successful install and then give you my weidu.log, and you use that as a template for your install, the only way it will fail is if a mod changes component numbers within those 10 seconds.  The window for failure is very small.  And in the event of failure, someone will do an install and then there will be a valid template again and people can pass it around etc. until the next failure.