Post reply

Warning: this topic has not been posted in for at least 120 days.
Unless you're sure you want to reply, please consider starting a new topic.
Name:
Email:
Subject:
Message icon:

Verification:
Type the letters shown in the picture
Listen to the letters / Request another image

Type the letters shown in the picture:
What color is grass?:
What is the seventh word in this sentence?:
What is five minus two (use the full word)?:

shortcuts: hit alt+s to submit/post or alt+p to preview


Topic Summary

Posted by: aqrit
« on: August 27, 2017, 12:24:57 PM »

This issue prevents the widescreen mod from operating at 4K resolutions.

OCaml allows much larger string sizes in a 64-bit process, which may be another possibility to resolve this issue.
The last attempt at that failed though:
http://forums.pocketplane.net/index.php/topic,24120.msg338275.html#msg338275
Posted by: Wisp
« on: August 04, 2017, 11:44:04 AM »

Your approach sounds even better. There is no need to look further into PATCH_BUFFER if your idea can be realized.
Then I shall consider this and COPY_EXISTING_LARGE a possible plan B.
Posted by: Sam.
« on: August 01, 2017, 09:27:38 PM »

I've been working on some functions to add/delete/replace tiles in a tileset on-the-fly (instead of shipping the entire modified tileset with the mod just to append a few door tiles, making the mod incompatible with any other mod trying to do the same).  TIS V1 is fully supported with limited TIS V2 support in the works.  The major obstacle for my effort being particularly useful is WeiDU's filesize limit.  If you need a functional example illustrating why support for large files would be helpful to motivate you, I offer this one.  Here is another:
Quote
The new implementation needs to COPY BG:EE TLK file in order to read 0xa offset (total amount of strings). It's possible that weidu won't be able to do it if BG:EE TLK is very large (there is COPY size limit, although there's still about 8 MB left when it comes to vanilla file). If someone will report that in heavily modded BG:EE the size is reached we will switch to Lua code in order to read this value.
Posted by: Argent77
« on: August 01, 2017, 03:10:06 PM »

Your approach sounds even better. There is no need to look further into PATCH_BUFFER if your idea can be realized.

Although I can imagine that implementation of string arrays would be non-trivial. In addition to the points you have outlined, there are also the problems of reshuffling data in case INSERT_BYTES exceeds buffer limit or DELETE_BYTES removes data from consecutive buffers to solve.
Posted by: Wisp
« on: August 01, 2017, 02:15:52 PM »

My own thought-experiments along these lines have been that the large-file problem could be solved by changing the implementation of file buffers from being simple strings, to being arrays of strings. You'd then be able to load arbitrarily large files into memory with the standard actions and COPY_LARGE would become obsolete. Technically, I can't see a real obstacle. You need some code for figuring out what position in which string corresponds to a given offset, and you need some code for handling byte insertion and deletion, and reads that cross from one string to the next, but those should be solvable problems. But there might be artificial limitations I'm not aware of, and, of course, there is the actual (and not inconsiderable) work involved.

Do you want me to look into PATCH_BUFFER, as well? I'm afraid I can't give an ETA for either.
Posted by: Argent77
« on: July 30, 2017, 02:59:01 PM »

It is currently not possible to use COPY action with files that are larger than 16 MB. We have to use COPY_LARGE instead, which doesn't allow patching files. To circumvent this restriction I would like to suggest a functionality that allows you to request partial buffers for read and write access. That way it will be possible to patch files even if they exceed the file size of 16MB.

I thought of something like this:

PATCH_BUFFER offset size BEGIN patch-list END when-clause

where:
- offset indicates start offset within file.
- size indicates buffer size in bytes. Cannot exceed maximum buffer size (around 16MB).
- patch-list defines zero, one or more patch commands.
- when-clause controls whether the buffer is written back to the destination. It should include "BUT_ONLY", and possibly "IF regexp", "UNLESS regexp" and/or "IF_SIZE_IS bufferSize". I don't think "IF_EXISTS" would make sense though.

Patch functions that can modify buffer size, such as INSERT_BYTES or DELETE_BYTES, should also be supported.

Example:
Code: [Select]
COPY_LARGE ~%MOD_FOLDER%/tis/largefile.tis~ ~override~
  // reading buffer of fixed size
  PATCH_BUFFER 0 0x18 BEGIN
    READ_LONG 0x08 numTiles
    READ_LONG 0x0c tileSize
    READ_LONG 0x10 startOfs
  END BUT_ONLY

  // reading sequence of buffers with dynamic size
  SET ofs = startOfs
  FOR (idx = 0; idx < numTiles; ++idx) BEGIN
    PATCH_BUFFER ofs tileSize BEGIN
      READ_ASCII 0 palette (1024)
      // ...
      WRITE_ASCIIE 0 ~%palette%~ (1024)
    END BUT_ONLY
    SET ofs += tileSize
  END

  // appending data
  PATCH_BUFFER ofs 0 BEGIN
    INSERT_BYTES 0 tileSize
    // ...
    WRITE_ASCIIE 0 ~%palette%~ (1024)
    WRITE_ASCIIE 1024 ~%pixels%~ (4096)
  END BUT_ONLY

I don't know whether it is even possible to add such a feature, but it would greatly help modding if it could be implemented.