Pocket Plane Group

Friends and Neighbors => Weimer Republic (WeiDU.org) => WeiDU => Topic started by: ikki on July 12, 2017, 01:25:09 AM

Title: weidu and out of memory error
Post by: ikki on July 12, 2017, 01:25:09 AM
When weidu process a certain amount of file, it crash with this error
It happens frequently on moderately modded ETT install when installing SCS with the BWS

There are two solutions for this problem
- installing SCS in 2 or 3 steps and closing weidu in-between
- or enabling the Large Address Aware flag on weidu

Is it possible to enable the LAA flag on weidu by default ?
Title: Re: weidu and out of memory error
Post by: Wisp on July 14, 2017, 10:28:29 AM
I don't think I could do this without a ridiculous amount of QA. Setting the flag for an unaware program can reportedly cause subtle bugs and I don't know if the 32-bit OCaml stack produces aware programs.
Title: Re: weidu and out of memory error
Post by: enderandrew on January 08, 2018, 05:13:05 PM
I've been using HLA patches on games for 15 years and literally never ran into a single bug because of it.

If you simply need QA, I'm willing to use a HLA patch in Weidu and then do a full BWS install with everything to test it out. But from what I'm reading, plenty of other people have already been testing it to get SCS to install properly.

If you're wary of a HLA patch, another solution is to make a 64-bit version of Weidu.
Title: Re: weidu and out of memory error
Post by: The Imp on January 10, 2018, 12:41:40 AM
It happens frequently on moderately modded ETT install when installing SCS with the BWS
The problem here is SCS, as it tries to memorize too many variables.
It's hard to QA a larger than usual moded game for this, as the trouble gets worse with more components ... aka the variables are never cleared, which the SCS should do, but doesn't. Simple clear memory command that weidu has, does this. Not that I know that any mod uses that, but it's there - http://www.weidu.org/%7Ethebigg/README-WeiDU.html#CLEAR_MEMORY or the more sophisticated --clear-memory (http://www.weidu.org/%7Ethebigg/README-WeiDU.html#hevea_default97) command promp flag at autoinstall, that the BWS probably could use for this.
ETT, :sniggers:
Title: Re: weidu and out of memory error
Post by: enderandrew on January 10, 2018, 12:58:27 AM
BWS doesn't have a maintainer and isn't going to be updated. SCS could potentially be updated, but the scripting looks pretty lengthy and complex. Weidu is still actively maintained and developed. A 64-bit version would not only fix SCS, but enable other large and complex mods. It seems to have the highest upside.
Title: Re: weidu and out of memory error
Post by: The Imp on January 10, 2018, 01:57:07 AM
BWS doesn't have a maintainer and isn't going to be updated.
That's your words. I am pretty sure that if one were to investigate and make a good pull request featuring the fix into say 09_Install.au3 (https://bitbucket.org/BigWorldSetup/bigworldsetup/src/d2c1c75861ea90ace2131717d88b7cf5b548500f/BiG%20World%20Setup/Includes/09_Install.au3?at=master&fileviewer=file-view-default), it could be approved, but as there NOT A SINGLE pull requests for the project, it kinda hard to do low-level maintenance like that. Not that I know what two plus what was it, two is, somehow they came up with five, even though they just used a single number... but I just can't figure how.
Title: Re: weidu and out of memory error
Post by: AL|EN on January 11, 2018, 12:47:42 PM
Imp, those commands wont help. And even if, putting those into BWS is no-go because BWS is not the source of the problem. Times when BWS take care about every possible problems are definitely over. Besides, BWS stare is feature/code freeze, without chance for takeover. Since weidu cannot be released as 64-bit application, the only possible solution for this problem is rewriting SCS macros/replace them with lua.
Title: Re: weidu and out of memory error
Post by: enderandrew on January 11, 2018, 01:25:56 PM
Why couldn't Weidu get a 64-bit version?

Edit: That being said, updating SCS to better make use of newer Weidu features is probably a good idea either way.
Title: Re: weidu and out of memory error
Post by: The Imp on January 11, 2018, 01:30:20 PM
Why couldn't Weidu get a 64-bit version?
Sure, let's program a 64-bit program with an ancient 32-bit one that can also cover all the Windows'es, MacOs's versions, and Linux too. Ouh, and then have it still backwards compatible with the 32 ones. Yeah, not going to happen man.

putting those into BWS is no-go because BWS is not the source of the problem.
You sure ? Cause if one were to install all the components separately, one would succeed, so the BWS refusing to do that is what ? Not the source - nor the wigged that overcomes the obstacles you tell me, while it can install multiple mods doing the exact same thing. I do understand that you don't want to make the BWS install every one of the components this way... but maybe the SCS can be the special case. ..not that I actually know the involvement it would take from you to do that.
Yes, we all know the SCS is damaged, and has been for so long it's not ... well, many still think it's the best cause it has been for so good for so long.
Title: Re: weidu and out of memory error
Post by: enderandrew on January 11, 2018, 01:47:45 PM
Why couldn't Weidu get a 64-bit version?
Sure, let's program a 64-bit program with an ancient 32-bit one that can also cover all the Windows'es, MacOs's versions, and Linux too. Ouh, and then have it still backwards compatible with the 32 ones. Yeah, not going to happen man.

putting those into BWS is no-go because BWS is not the source of the problem.
You sure ? Cause if one were to install all the components separately, one would succeed, so the BWS refusing to do that is what ? Not the source - nor the wigged that overcomes the obstacles you tell me, while it can install multiple mods doing the exact same thing. I do understand that you don't want to make the BWS install every one of the components this way... but maybe the SCS can be the special case. ..not that I actually know the involvement it would take from you to do that.
Yes, we all know the SCS is damaged, and has been for so long it's not ... well, many still think it's the best cause it has been for so good for so long.
Porting an app from 32-bit to 64-bit has varying levels of complexity. Depending on how it is coded, it is sometimes an easy port and mainly an issue of just compiling (which is most most every Linux app simply can just be recompiled 32-bit or 64-bit with no effort involved).

Mac, Linux and Windows all have 64-bit support. Providing a 64-bit Weidu doesn't hurt Linux or Mac support in the least.

It doesn't really hurt backwards compatibility given that 64-bit operating systems have been around for 15 years now, and nothing would prevent Weidu from having both 32-bit and 64-bit compiled versions. That is what most people have been doing for ages.

So none of those are reasonable objections.

If there are mods that need more than 2 GB of memory and if Weidu wants to better support other large mods in the future, it makes sense for Weidu to get 64-bit support.
Title: Re: weidu and out of memory error
Post by: Wisp on January 11, 2018, 02:28:22 PM
64-bit WeiDU on Windows is problematic because Windows is or was quite the second-class citizen in the OCaml world, and the 64-bit support is reportedly weak. More (http://forums.pocketplane.net/index.php/topic,24120.msg338275.html#msg338275). I also don't have a build system for 64-bit Windows, but that's more a practical limitation than a real one.
Title: Re: weidu and out of memory error
Post by: enderandrew on January 11, 2018, 02:46:08 PM
64-bit WeiDU on Windows is problematic because Windows is or was quite the second-class citizen in the OCaml world, and the 64-bit support is reportedly weak. More (http://forums.pocketplane.net/index.php/topic,24120.msg338275.html#msg338275). I also don't have a build system for 64-bit Windows, but that's more a practical limitation than a real one.
There are updated 64-bit Windows OCaml packages from November, 2017. I don't know if they fix whatever issue people ran into before however.

https://fdopen.github.io/opam-repository-mingw/installation/