Reading Infinity files in Python

[Riklaunim]

To read a binary file in python we need a basic code:

Code:

import struct
binfile = open('FILENAME', 'rb')
try:
        bin = binfile.read()
finally:
        binfile.close()

Where bin variable contains content of the whole file. On IESDP we have IE File Formats where each file type has full description. We can read part of the file using given OFFSET - print bin[OFFSET:OFFSET+SIZE]

IF the data is binary - we have to convert it.
- resref, char array - direct acces, non binary
- dword, strref - convert with struct

Code:

print struct.unpack('i', bin[0x0014:0x0014+4])

- word - convert with struct

Code:

print struct.unpack('h', bin[0x0080:0x0080+2])

- byte, unsigned byte, char - convert with struct

Code:

print struct.unpack('b', bin[0x000b:0x000b+1])

note 1: To write binary data from a given text data use struct.pack with the same format (i, h, b).
note 2: struct.unpack returns a tuple. To acces a value:

Code:

a = struct.unpack('i', tekst[0x0014:0x0014+4])
print a[0]

Each tuple element has a number (from zero). This code will work everywhere python works. Should also work on jython (python in java) and IronPython (python in .NET). If someone wants to make a GUI app - on windows py2exe can help make an *exe from the Application, and for GUI libs: python tkinter is good for Mac and Windows, looks terible on Linux/Unix. pyGTK+ - good for all 3 OS. pyQT - easy, good for Unix/Linux and Mac, on Windows - from PyQT4 (in development)
For compressed biffs there is zlib module


CRE example:

Code:

import struct
plik = open('beheld01.cre', 'rb')
try:
        tekst = plik.read()
finally:
        plik.close()

print 'File Signature: ' + tekst[0x0000:0x0000+4]
print 'Version: ' + tekst[0x0004:0x0004+4]
print 'Long Creature name: ' + str(struct.unpack('i', tekst[0x0008:0x0008+4])[0])
print 'Short Creature name: ' + str(struct.unpack('i', tekst[0x000c:0x000c+4])[0])
print 'Creature flags: ' + str(struct.unpack('i', tekst[0x0010:0x0010+4])[0])
print 'XP for killing: ' + str(struct.unpack('i', tekst[0x0014:0x0014+4])[0])
print 'CRE Level: ' + str(struct.unpack('i', tekst[0x0018:0x0018+4])[0])
print 'Current HP: ' + str(struct.unpack('h', tekst[0x0024:0x0024+2])[0])

[Riklaunim]

Zlib compressed data
SAV and BIFC files have data compressed with zlib. In Python we have zlib module so we can decompress the data. This code decompress everything from a sav file and save it to files:

Code:

import struct
import zlib
# we need a SAV file in the same dir
plik = open('baldur.sav', 'rb')
try:
        tekst = plik.read()
finally:
        plik.close()

a = 0x0004 + 4
# decompress everything until end of file
while a < len(tekst):
lenght = struct.unpack('i', tekst[a:a+4])[0]
print 'Lenght of filename: ' + str(lenght)
# get the filename - it has various lenght
filename = str(tekst[a+4:a+16])
# split into 2 parts using dot as splitter
file_name = filename.split('.')
#get the extension+rubbish
ext = file_name[1]
# kill the rubbish, read only 3 chars :)
file_name = file_name[0] +'.'+ ext[0:3]
print 'Filename: ' + file_name
decomp_size = struct.unpack('i', tekst[a+0x0004+lenght:a+0x0004+lenght+4])[0]
print 'Decompressed size: ' + str(decomp_size)
comp_size = struct.unpack('i', tekst[a+0x0008+lenght:a+0x0008+lenght+4])[0]
print 'Compressed size: ' + str(comp_size)
if decomp_size != 0:
code = tekst[a+0x000c+lenght:a+0x000c+lenght+comp_size]
open(file_name, 'wb').write(zlib.decompress(code))
print 'Saved Raw data as ' + file_name
a = a+0x000c+lenght+comp_size
print "----------------------------"

(http://images3.fotosik.pl/31/7n27ad1nkg77pslvm.png)

[jcompton]

Okay, I'll give in and ask. What are you developing in python that's led you to work all this out?

[Riklaunim]

just learning python and maybe I'll make a pyQT based Infinity browser/editor in the near future.
Using cherryPy or Zope or other Python servers some IE "tools" could become online web based tools

[Riklaunim]

If someone is interested: Reading Infinity Engine files in Django - a python web framework