QuickBMS errors [programming, scripting, quickbms.exe tool... NOT games]

[aluigi]

Report any quickbms bug and error here.

The problems must affect quickbms and not specific scripts, for which you can use the Game Archive section.

Before reporting ANY problem be sure to have the latest version of quickbms and the latest version of the script that is causing the problem.

*edit* added tags in the title to avoid OT

[MotherFucker]

Hey Luigi, data type FILEPATH is broken!

Code: Select all

X:\Temp>quickbms -v -s "get path filepath" "" Temp.zip .

QuickBMS generic files extractor and reimporter 0.6.1d
by Luigi Auriemma
e-mail: (avoiding spam)
web:    aluigi.org
        (Dec 28 2014 - 21:19:39)

                  http://quickbms.aluigi.org
               http://twitter.com/luigi_auriemma
                       http://zenhax.com

- command-line arguments:
  quickbms
  -v
  -s
  get path filepath

  Temp.zip
  .
- current_folder: X:\Temp
- bms_folder:     X:\Temp
- exe_folder:     X:\Temp
- file_folder:    X:\Temp
- output_folder:  X:\Temp
- temp_folder:    X:\Temp
- open input file X:\Temp\Temp.zip
- open script
READLINE 1   get path filepath
             >set path (0) to "path"
  ARG0  "get"
  ARG1  "path"
  ARG2  "filepath"

- set output folder .

  offset   filesize   filename
--------------------------------------
             .start_bms start: -1 0 0

00000000 06  1   get path filepath
             >set path (0) to ""   <-- Huh?

- 0 files found in 0 seconds
  coverage file 0     0%   0          63579648

Can I suggest FULLPATH as a data type too? Thanks!

[aluigi]

Thanks for the feedback, I will fix it in the next version of quickbms.
This is the second time that "filepath" gives me problems
Also fullpath is a good name, maybe I can add it as alias.

[MotherFucker]

Also fullpath is a good name, maybe I can add it as alias.

With "full path" I mean a fully qualified path, i.e. "X:\Temp\Temp.zip", mainly for calling picky external apps that dislike relative paths.

[aluigi]

filepath in fact should report the full path.
Basically it should be just like fullname but without the filename (my mistake was taking filename instead of fullname as input).

[NullRd]

Why QuickBMS eating leading zeros from strings?

Code: Select all

 set test string "000111"
 print %test%   // - SCRIPT's MESSAGE: 111

 string test += "000222"
 print %test%   // - SCRIPT's MESSAGE: 111222

 string test p= "%s%s" test "000333"
 print %test%   // - SCRIPT's MESSAGE: 111222333

[aluigi]

The reason is explained in quickbms.txt from line 3389.

If you want to assign a numeric string to a variable use binary but it's not a real solution:
set VAR binary "0001234"

[Ekey]

Example part script code

Code: Select all

get OFFSET long
get SIZE long
getdstring NAME 0x200
putarray 0 i OFFSET
putarray 1 i SIZE
putarray 2 i NAME

got error -> 0.6.4

Code: Select all

- error in src\var.c line 778: variable_copy()
Error: No error

Last script line before the error or that produced the error:
  29  putarray 2 i NAME

0.6.1d works fine

[aluigi]

Yeah I found this error just 5 minutes ago while writing the Luna Moonlight script.

[dirox]

Hi, I wanted to ask, has the situation with the lz4 files has improved ? Or maybe there is a workaround ?
You mentioned something about this previously:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"I have an update.
I have just used the latest lz4 (the one in quickbms is just few weeks old) and I have better results:
The file has a size of 1178586 bytes and we must shrink it to 27910 bytes.

Code:
LZ4HC 27910
LZ4HC2 27813
LZ4 52918
This is an excellent news, the problem is solved

I guess I will release the new quickbms quite soon so you can start the modding."
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I try to export the files from "r4gui.bundle" using quickbms.exe or quickbms_4gb_files.exe and get the following error:
Info: algorithm 395
offset 01e7d000
input size 0x0003b137 241975
output size 0x0003b1b9 242105
result Oxfffffffe —2

Error: the uncompressed data (-2) is bigger than the allocated buffer (14700725)

Last script line before the error or that produced the error:
35 clog NAME OFFSET ZSIZE SIZE

Press RETURN to quit

[aluigi]

That one is not an error in recompression, it looks like the chosen compression algorithm is not the correct one for that specific file.
If you refer to witcher3, there were 2 versions of the script that I made and I think that you have the older one.
Check if you have the 0.1.1a version:
http://aluigi.org/papers/bms/others/witcher3.bms

[dirox]

Thank you for the quick response! I'm afraid that is the version I am using and I still get the error. The issue is that I can't extract the files, I haven't gotten to the re-compression part yet.
I can extract and re-compress other files, like xml.bundle. :s
Thank you for your time!

[aluigi]

I need the archive to check it, can you upload it?

[dirox]

Hey, I hope this is what you mean :S
https://www.dropbox.com/s/dnkiudb9ub87n ... undle?dl=0

[aluigi]

Interesting, apparently it's not an "AND" but just a switch with various compression algorithms.
It's still not clear why both 4 and 5 are lz4 but it's no longer important.
Script updated to version 0.1.2 with ZIP 2 which is Snappy.

[ThalonMook]

Hi Aluigi,

at first thx for your cool tool and the script without that modding of Witcher 3 and other games was not possible.

I have some problems with reimporting some files for The Witcher 3.
Im using quickbms 6.5 and witcher3.bms 1.2
I also used the 6.4 and bms 1.1a same problem.
It workes with the most xml files but I have 2 the will not reimport.

Code: Select all

QuickBMS generic files extractor and reimporter 0.6.5
by Luigi Auriemma
e-mail: me@aluigi.org
web:    aluigi.org
        (Jul  8 2015 - 12:15:00)

                  http://quickbms.aluigi.org
               http://twitter.com/luigi_auriemma
                       http://zenhax.com

- REIMPORT mode enabled!
- open input file M:\The Witcher 3 Wild Hunt\content\content0\bundles\xml.bundle

- open script J:\Witcher 3Modding\quickbms\witcher3.bms
- set output folder J:\Witcher 3Modding\Thalons Mod\ModOut\xml bundle

  offset   filesize   filename
--------------------------------------
- compressed size too big, I try using the uberflate/kzip method (may be very slow!)

Error: file "gameplay\items\def_item_trophies.xml"
       the reimport option acts as a reimporter and so you cannot reinsert a
       file if it's bigger than the original otherwise it will overwrite the
       rest of the archive or cannot be loaded correctly:

         new size: 1970 (36810 uncompressed)
         old size: 1953 (36810 uncompressed)

As you can see the size is the same but when compressed it's to big.
I can't find out whats the problem
I attached a file with the one that works and the other the brings that error.

I hope you can help.

Thx
Thalon

[aluigi]

This problems is not so uncommon.
Many libraries that compress using the deflate algorithm (known as zlib if there is a header and a crc before and after it) sometimes are not able to match the ratio or other libraries in some conditions.

For example zlib is not so good and for that def_item_trophies.xml file you provided you get 1996 bytes.
7zip compresses it to 1984 bytes.
The uberflate/kzip solution reaches 1970 bytes.

A good candidate is zopfli but it's unable to offer good results with various types of files.
In this case it's good and I can reach 1939 bytes so I will for sure add it to the next version of quickbms (I already evaluated it months ago but I rejected it just due to its results) but I guess I have to write a sort of "options fuzzer" for finding those with the best results.

[aluigi]

I have reached 1924 bytes by using the following options:

Code: Select all

    opt.numiterations = 15;
    opt.blocksplitting = 1;
    opt.blocksplittinglast = 1;
    opt.blocksplittingmax = 0;

[aluigi]

I'm performing additional tests just now and I confirm the problems I said before.
zopli is good only with small files like under one megabyte or even less.

[ThalonMook]

Hey cool.

Thx for your fast response.

I there anything I can do.

How can I use the options?

Cu
Thalon