Author Topic: tolower breaks the new BG1EE Linux Edition  (Read 12547 times)

CarrKnight

  • Guest
tolower breaks the new BG1EE Linux Edition
« on: October 11, 2014, 07:40:56 PM »
Hello all, thanks for working on the WEIDU


I am playing on the bg1ee beta. I am also trying to install a mod using weidu. The problem is that weidu only works on linux after I run tolower. The problem with that is that the bg1ee beta doesn't like its files to have different cases and so if I run tolower bg1 stops working.

I was wondering if there is a way to compile weidu to be case-sensitive and/or a way to undo tolower.

Thanks!

Offline Wisp

  • Moderator
  • Planewalker
  • *****
  • Posts: 1071
Re: tolower breaks the new BG1EE Linux Edition
« Reply #1 on: October 13, 2014, 11:42:49 AM »
Oh, this is going to suuuck. You have no idea.

To fix your game you'll need to reinstall, or something to the same effect. There is no untolower.

For the uninitiated:
No mod anywhere cares one whit about case (outside of string comparisons, at least). This works fine on Windows and OS X, because IO is not case sensitive. But on Linux it suddenly matters. The solution devised in the olden days was to settle on a standard case (lower case). Tolower would lowercase any game files and WeiDU would convert any IO into lower case and things would kind-of work in a general fashion.
« Last Edit: October 13, 2014, 11:46:48 AM by Wisp »

CarrKnight

  • Guest
Re: tolower breaks the new BG1EE Linux Edition
« Reply #2 on: October 15, 2014, 12:55:21 AM »
Yeah, had to reinstall everything.

The second try went better though. From stackexchange I got a decent script to lowercase and then uppercase again: http://askubuntu.com/questions/535999/lowercase-to-all-files-in-a-directory-expecting-to-undo-it-at-some-point

lowercased my BG folder in steam
installed BG1NPC
uppercased it back

and it worked like a charm

sams

  • Guest
Re: tolower breaks the new BG1EE Linux Edition
« Reply #3 on: January 14, 2015, 07:47:24 PM »
The rabbit hole goes deeper yet...

For the Linux edition of IWDEE, CarrKnight's lowercase all files + run weinstall + restore original case of all files trick worked for some mods... But the BG2 Tweak Pack recompiled chitin.key, which broke the game because the .bif paths coded into the chitin.key were now all uppercase, instead of the mixed-case that they were before (like they are in the actual filesystem).
For example (as I found out using strace), the game would look for DATA/DEFAULT.BIF because that's the path that's now stored in the recompiled chitin.key, but could't find it because the actual path that file was installed to is data/DEFAULT.BIF. And so on for all .bif's.

It can be fixed manually, either by renaming the files to match the new chitin.key content (recommended), or by binary-patching chitin.key to match the existing files again (I wouldn't recommend it for actual use, but I confirmed it works too). Luckily there seem to be only two things that need renaming to get everything to work again:
  • the data directory itself - if this doesn't have the correct case, the game can't find any .bif files and crashes on start-up.
  • the IWDEE.bif file inside the data dir - if this doesn't have the correct case, all buttons and pictures in the main menu are invisible.
The other .bif files referred to by chitin.key already have uppercase filenames, so need no further renaming.

Long story short, if you installed a mod that recompiled chitin.key (you'll notice the game crashing on startup), then go to the game/ directory and run the commands:
Code: [Select]
mv data DATA
mv DATA/IWDEE.bif DATA/IWDEE.BIF

With that work-around, everything seems to work for me again (so far).

No idea if the same applies to BGEE and BG2EE though, because I don't own those games.

Sleepy

  • Guest
Re: tolower breaks the new BG1EE Linux Edition
« Reply #4 on: March 02, 2015, 09:12:49 AM »
Hmm... as anybody investigated this further?

I just got the Steam version Enhanced Edition of BG2 for Linux and while the game runs fine on its own, I can't install any mods. I strongly suspect "tolower" being the culprit, as the bif-files in /data are all CamelCase. Runing "tolower" lets you install the mods (via "weinstall"), but then nothing works. There's not even a window or any error message.

What to do?

Offline Wisp

  • Moderator
  • Planewalker
  • *****
  • Posts: 1071
Re: tolower breaks the new BG1EE Linux Edition
« Reply #5 on: March 03, 2015, 03:28:50 PM »
Yes, this is being looked at.

stormfield

  • Guest
Re: tolower breaks the new BG1EE Linux Edition
« Reply #6 on: July 08, 2015, 06:02:36 PM »
I have the same problem as Sleepy.

Is there any solution?


thehug0naut

  • Guest
Re: tolower breaks the new BG1EE Linux Edition
« Reply #7 on: July 29, 2015, 08:24:28 AM »
What you might want to try is to mount the game folder using ciopfs (a case insensitive filesystem for linux) - I've done this then installed both Rogue Rebalancing and Sword Coast Stratagems and they run without a hitch.  As a plus you don't even need to use tolower!  I'll provide some instructions for you guys, assuming the use of Ubuntu and the enhanced edition because that's what I have.  # are my instructions and the lines below are the commands to run.

# ==========
# First open a terminal and then install ciopfs:
sudo apt-get install ciopfs

# Then move to the level above the install directory and create a hidden foilder for ciopfs to use:
cd ~/.local/share/steam/steamapps/common
mkdir .BGEE

# Copy the game folder to another hidden folder and then clear it (so ciopfs can mount to it):
cp Baldur's\ Gate\ Enhanced\ Edition .oldBGEE
rm -r Baldur's\ Gate\ Enhanced\ Edition/*

# now mount the hidden folder with ciopfs and copy back the original data
ciopfs .BGEE Baldur's\ Gate\ Enhanced\ Edition
cp .oldBGEE/* Baldur's\ Gate\ Enhanced\ Edition
#==========

Now you should be able to test the game works, then extract your mod files to the "Baldur's\ Gate\ Enhanced\ Edition" folder as normal and install all the mods you want without the need to use tolower before weinstall.

#==========
# IMPORTANT
#==========
Once you  have tested the game runs then there is one more step to make the ciopfs baldurs gate permanent; edit your /etc/fstab file (try the command "sudo gedit /etc/fstab" without quotes) and copy in the following to mount the ciopfs folder to the game directory on login (assuming you have the default steam install).  Make sure to modify "USERNAME" to be your USERNAME.

/home/USERNAME/.local/share/steam/steamapps/common/.BGEE /home/USERNAME/.local/share/steam/common/Baldur's\040Gate\040Enhanced\040Edition\ ciopfs allow_other,default_permissions,use_ino,attr_timeout=0 0 0

And thats it - enjoy!   ;D

Note:  I only know about this because of a similar issue faced by the CIV5 modding community after the linux version came out.  See the steam community discussion http://steamcommunity.com/app/8930/discussions/0/540744299996631016/ for the original source of these instructions.

Decius

  • Guest
Re: tolower breaks the new BG1EE Linux Edition
« Reply #8 on: November 05, 2015, 02:25:14 AM »
I just got BG2EE + BG2 Tweaks to work on linux. I followed the procedure outlined by CarrKnight and sams.

The only difference was that I had to rename all files in the DATA directory to uppercase to get the game to start. From a terminal in this directory, I ran
Code: [Select]
for file in *; do mv -- "$file" "${file^^}"; doneto uppercase all files (taken from https://stackoverflow.com/questions/20253584/linux-rename-files-to-uppercase).

So far, the game runs great.

Offline khelban12

  • Planewalker
  • *****
  • Posts: 8
Re: tolower breaks the new BG1EE Linux Edition
« Reply #9 on: December 21, 2015, 10:40:19 AM »
In case it is forgotten, is it possible to make the next version of weidu to produce lowercase entries in key/bif files ? Would it cause any problems on windows ?

Code: [Select]
000008f0  64 61 74 61 2f 4f 42 4a  41 6e 69 6d 2e 62 69 66  |data/OBJAnim.bif|
00000900  00 64 61 74 61 2f 45 66  66 65 63 74 73 2e 62 69  |.data/Effects.bi|
chitin.key as it comes in the EE versions (and the vanilla bg2+tob) has mixed case entries.

Code: [Select]
000008f0  44 41 54 41 2f 4f 42 4a  41 4e 49 4d 2e 42 49 46  |DATA/OBJANIM.BIF|
00000900  00 44 41 54 41 2f 45 46  46 45 43 54 53 2e 42 49  |.DATA/EFFECTS.BI|
After running weinstall on linux, weidu changes the entries to uppercase.

So in order to install mods, you need to run tolower and lowercase all files but then the game cannot access them because the binary tries to open uppercase files (or mixed case files with the vanilla chitin.key). I have written a C program that changes all the relevant entries of chitin.key to lowercase and i can run the game perfectly, but the program needs to be ran after (almost) every mod installation.

If weidu is changed to produce lowercase entries then everything will work fine after running tolower without any steps from the user.

Offline Wisp

  • Moderator
  • Planewalker
  • *****
  • Posts: 1071
Re: tolower breaks the new BG1EE Linux Edition
« Reply #10 on: December 21, 2015, 01:09:36 PM »
In case it is forgotten, is it possible to make the next version of weidu to produce lowercase entries in key/bif files ? Would it cause any problems on windows ?
Doing something about the key file will be required, yes. But it is not quite so straightforward. WeiDU upcases the keyfile (for consistent case) and I can't change that without going over a hell of a lot of code to check for case-sensitive comparisons against upcased strings literals. Tolower can be changed into toupper, but the game has a lot of hardcoded mixed-case paths (like $GAME/lang/en_US/) and won't run if they are differently cased. On top of that, mods need to be completely insulated (or Linux users are limited to Linux-compatible mods; population: 0) regardless of whatever stupid crap they are doing. In short, I stand by my initial exclamation and the issue has not so much been forgotten, as it's something I sit down with, facepalm and turn to something else.
« Last Edit: December 21, 2015, 01:14:17 PM by Wisp »

Offline Argent77

  • Planewalker
  • *****
  • Posts: 173
Re: tolower breaks the new BG1EE Linux Edition
« Reply #11 on: December 21, 2015, 01:52:47 PM »
How about a third option: Don't change filename cases at all.

The correct case of biffed resource paths can be found in the chitin.key. For files in filesystem folders you could implement a simple cache mechanism (similar to what I have done in NI) and compare (real and cached) files case-insensitively to find matches. The performance hit should be negligible on moderns systems. Adding proper support for case-sensitive filesystems is time-consuming for either option.

Offline khelban12

  • Planewalker
  • *****
  • Posts: 8
Re: tolower breaks the new BG1EE Linux Edition
« Reply #12 on: December 21, 2015, 02:01:38 PM »
Doing something about the key file will be required, yes. But it is not quite so straightforward. WeiDU upcases the keyfile (for consistent case) and I can't change that without going over a hell of a lot of code to check for case-sensitive comparisons against upcased strings literals.
Thank you for taking the time to look at the problem. I am not familiar with the source code so it didn't occur to me that there are many checks against upcased literals.

Tolower can be changed into toupper, but the game has a lot of hardcoded mixed-case paths (like $GAME/lang/en_US/) and won't run if they are differently cased.
That problem is that the linux binary (at least the bg2ee one because i don't have the other games) has all the hardcoded paths as lowercase so it will need a lot of hex edits in order to work with uppercase paths. tolower has the benefit that the only hardcoded path that isn't full lowercase is the language code (e.g en_US as you mentioned) which can be solved with a simple symbolic link.

Is changing tolower to toupper a lot less work than changing weidu ? If yes, could the beamdog team (or the relevant party that produces the linux versions) be persuaded  to change the hardcoded paths to uppercase with the next patch ? Maybe we are too few running linux to justify the manpower needed to implement either solution.

Anyway, thank you for looking at this.

Offline koteko

  • Planewalker
  • *****
  • Posts: 4
Re: tolower breaks the new BG1EE Linux Edition
« Reply #13 on: April 19, 2016, 03:00:41 AM »
This is a relatively old topic, but I think this is still an issue for many Linux users.

The solution, to be honest, is extremely trivial. I don't know how I couldn't see it before.

To have a case insensitive BG (the linux version), you just need to mount a case insensitive partition (or loopback file, if you don't have a spare partition). Say, you have a partition at sda5. You format it as ntfs using gparted or other tools. You create a directory you want to mount it to (in my case, data20). You do:

Code: [Select]
sudo lowntfs-3g -o windows_names -o ignore_case /dev/sda5 /data20/

Then you copy your BG folder within data20 (even with steam: you just copy it). This is the shortest and less painful workaround.

It gets better: Steam can be configured to use a separate partition for certain games. So if you have a big enough partition formatted as ntfs, you can tell Steam to install all Infinity Engine games you have inside that partition. You now have an easily moddable BG setup with the native linux binaries.

 

With Quick-Reply you can write a post when viewing a topic without loading a new page. You can still use bulletin board code and smileys as you would in a normal post.

Warning: this topic has not been posted in for at least 120 days.
Unless you're sure you want to reply, please consider starting a new topic.

Name: Email:
Verification:
Type the letters shown in the picture
Listen to the letters / Request another image
Type the letters shown in the picture:
What color is grass?:
What is the seventh word in this sentence?:
What is five minus two (use the full word)?: