QuickBMS errors [programming, scripting, quickbms.exe tool... NOT games]

[StreamThread]

Does this mean that also will fixed a allocation error, appearing when 'log' many files into Memory file with final size more then 1 gb and next try to 'log' this?

[aluigi]

It depends by the error because it looks like you are trying to decompress (clog) a big file and therefore quickbms must allocate a double amount of memory.
Let's say you have already extracted various files and the current one has a compressed size of 900Mb and an uncompressed size of 1.1Gb, that means:
- current memory (let's say 500Mb) + 900Mb + 1.1Gb

You can try using the -9 option of quickbms but it shouldn't change much.
Anyway it's not a bug or similar.

For the log command (without encryption) quickbms doesn't allocate memory.

[StreamThread]

It once was in my one packing script with log command, no clog. Was used 3 memory files: for files table, for files data, and for process loaded files in some order. At the last stage, table and data paste together and export. No one file was compressed, but yes, to memory file with files table encryption was assigned.

[aluigi]

So the problem was related to 3 big memory files that weren't able to be allocated in memory.
Unfortunately the memory is very limited, let's say less than 3Gb (it's a 32bit application).

[bugmenot]

I think I found a bug with the -P command or the Rpa Renpy Nopython script:

When extracting an RPA file that contains Cyrillic characters in some of the file names they did not extract correctly so tried adding -P UTF8 (unless I'm wrong RPA files are always UTF-8 encoded) but the result was still wrong, next I tried every other UTF and Cyrillic option from this table using both Identifier and .NET Name values, I also tried adding the codepage "utf8" and codepage 1251 string in the BMS script with the same result.

For example the correct file name is: Armature-трет руки_0 as can be seen in the attached file table, but the files are extracted as: Armature-трет СЂСѓРєРё_0, Armature-трет руки_0 or Armature-трет руки_0.

This bug is the same reported in viewtopic.php?f=11&t=4797 and I confirm that it will be fixed in the upcoming 0.8.1 version that will be released this week

It seems the problem is still present in v9.0, any suggestions?

For example the HEX string 70 68 6F 6E 65 20 2D 20 D0 BA D0 BE D0 BF D0 B8 D1 8F 2E 70 6E 67 is saved as phone - РєРѕРїРёСЏ.png instead of phone - копия.png
Tested with GDPC and RGSSAD scripts on files containing Russian and Japanese file names.

[aluigi]

It's an utf8 name which is correctly handled by quickbms, in fact my output name is perfect.

Script used for the test, and tested also with reading of name from file:

Code: Select all

set NAME binary "\x70\x68\x6F\x6E\x65\x20\x2D\x20\xD0\xBA\xD0\xBE\xD0\xBF\xD0\xB8\xD1\x8F\x2E\x70\x6E\x67"
log NAME 0 0

You can get that wrong filename you specified only if you use -P 1251 which indeed is wrong because the correct charset here is utf8.

[bugmenot]

In other words you are saying quickbms is running as non-Unicode program that's assuming it's running under 1250 encoding instead of checking the local system non-Unicode settings and taking them in account?

P.S. After testing the provided sample I get a file named "phone - копия.png" which probably means there is some other problem when reading the file name from the archive.

I've attached a small test GDPC sample file.

[aluigi]

Eh? No, quickbms uses utf8 by default, which is the codepage used for the string you provided.

Both the gdpc and rgssad have no hardcoded codepage in them and they don't even use any "unicode" conversion like "get NAME unicode" or "set NAME unicode NAME".
The scripts with hardcoded codepage (932) are: brave_soul.bms, ddworks.bms, hunex.bms and norn9.bms.

If you are experiencing that problem, please tell me how to replicate it step-by-step and upload the files you are using.

There is no problem reading filenames from archive, you can test it with a simple script like the following on a file containing the raw filename like the one you provided:

Code: Select all

get SIZE asize
getdstring NAME SIZE
log NAME 0 SIZE

[bugmenot]

Steps to reproduce:
Drag the GDPC.bms over quickbms.exe v9.0
select the asteroids.pck file from previous post
click save

Testing on a Win7 VM with non-Unicode language set to 1251. Switching to English (USA) produces a file named "копия.png".

P.S. The second sample is working fine as well.

P.P.S. It seems the problem is related to filenames that contain a directory in them. If I use a file containing "копия.png" the second example is working fine, but if it contains "res://копия.png" I get the wrong name.
"1/копия.png" produces the correct name but "1копия/копия.png" results in "1РєРѕРїРёСЏ\РєРѕРїРёСЏ.png"

[aluigi]

I confirm it's a bug because if you remove "res://" the name is correctly dumped.
I'm investigating.
Thanks for the feedback.

[aluigi]

The problem is not directly related to quickbms but in Windows not liking multiple consecutive path delimiters.

For example:
- WORKS: "res/file.png"
- DOES NOT WORK: "res//file.png" or "res\\\\\\file.png"

Basically it's a problem happening probably only the 0.1% of times but I'm happy you found it

It will be fixed in 0.9.1

Ah I have seen your last edit, the problem with the utf8 folder is known and will be fixed too.

[Shadow]

Report any quickbms bug and error here.

The problems must affect quickbms and not specific scripts, for which you can use the Game Archive section.

Before reporting ANY problem be sure to have the latest version of quickbms and the latest version of the script that is causing the problem.

*edit* added tags in the title to avoid OT

[bugmenot]

The problem is not directly related to quickbms but in Windows not liking multiple consecutive path delimiters.

For example:
- WORKS: "res/file.png"
- DOES NOT WORK: "res//file.png" or "res\\\\\\file.png"

Basically it's a problem happening probably only the 0.1% of times but I'm happy you found it

It will be fixed in 0.9.1

Ah I have seen your last edit, the problem with the utf8 folder is known and will be fixed too.

Thanks

[usernameqwerty]

Hi, I'm trying to extract embb's from Shadow of Mordor with shadow_of_mordor.bms 0.4 and I'm getting this error.

Code: Select all

quickbms.exe shadow_of_mordor.bms Patch_1.arch05 som

QuickBMS generic files extractor and reimporter 0.9.0
by Luigi Auriemma
e-mail: me@aluigi.org
web:    aluigi.org
        (Jun 17 2018 - 11:49:55)

                   quickbms.aluigi.org  Homepage
                            zenhax.com  ZenHAX Forum
                     @zenhax @quickbms  Twitter & Scripts

- open input file C:\Users\limited\Downloads\quickbms\Patch_1.arch05
- open script shadow_of_mordor.bms
- set output folder som

  offset   filesize   filename
--------------------------------------
  00000000 0          bundles/player_load_skin01.bndlxml05
+ 0000304c 65536      bundles/player_load_skin01.bndlxml05

-------------------
*EXCEPTION HANDLER*
-------------------
An error or crash occurred:

*EH* ExceptionCode      c0000005 access violation
*EH* ExceptionFlags     00000000
*EH* ExceptionAddress   016E73E9
                        01210000 + 004d73e9 quickbms.exe
*EH* NumberParameters   00000002
*EH*                    00000000
*EH*                    0F62EA45

Last script line before the error or that produced the error:
  183 clog NAME OFFSET CHUNK_ZSIZE CHUNK_SIZE

Code: Select all

quickbms.exe -f "*.embb" shadow_of_mordor.bms Patch_1.arch05 som

QuickBMS generic files extractor and reimporter 0.9.0
by Luigi Auriemma
e-mail: me@aluigi.org
web:    aluigi.org
        (Jun 17 2018 - 11:49:55)

                   quickbms.aluigi.org  Homepage
                            zenhax.com  ZenHAX Forum
                     @zenhax @quickbms  Twitter & Scripts

- filter string: "*.embb"
  *.embb
- filter   1: *.embb
- open input file C:\Users\limited\Downloads\quickbms\Patch_1.arch05
- open script shadow_of_mordor.bms
- set output folder som

  offset   filesize   filename
--------------------------------------
  00000000 0          bundles/player_load_skin01.embb
+ 00003c80 65536      bundles/player_load_skin01.embb

-------------------
*EXCEPTION HANDLER*
-------------------
An error or crash occurred:

*EH* ExceptionCode      c0000005 access violation
*EH* ExceptionFlags     00000000
*EH* ExceptionAddress   016E73E9
                        01210000 + 004d73e9 quickbms.exe
*EH* NumberParameters   00000002
*EH*                    00000000
*EH*                    0E87330F

Last script line before the error or that produced the error:
  183 clog NAME OFFSET CHUNK_ZSIZE CHUNK_SIZE

I extracted loose files from the same arch05 file with arch05explorer, but it can't extract whole embbs.
Do you still have an old shadow_of_mordor.bms script, like 0.2?

[chrrox]

zstandard with the -e option worked great on most files in star ocean it fails the last chunk of this file.

never mind they used a 00 size for uncompressed data.

[aluigi]

@usernameqwerty
It looks like the identified compression is not xmemdecompress and so it crashes.
That's OT here ("not games") but feel free to post a sample file here:
viewtopic.php?f=9&t=5080&start=60

@chrrox
ok I have updated the slz script to version 0.2.5.
quickbms 0.9.1 will no longer need -e

[aluigi]

0.9.1 has been released

[HPAndroid]

I encountered this error when extracting game's archive:

Code: Select all

QuickBMS generic files extractor and reimporter 0.9.1
by Luigi Auriemma
e-mail: me@aluigi.org
web:    aluigi.org
        (Nov 21 2018 - 14:40:39)

                   quickbms.aluigi.org  Homepage
                            zenhax.com  ZenHAX Forum
                     @zenhax @quickbms  Twitter & Scripts

- GUI mode activated, remember that the tool works also from command-line
  where are available various options like folder scanning, filters and so on

- select BMS script. type ? for using the content of clipboard like a script
- select input archives/files, type * for the whole folder and subfolders
- select output folder where extracting files
- open input file C:\Users\Q.Viet\Downloads\Compressed\mod\main.136.com.ea.game.fifa14_row.obb
- open script C:\Users\Q.Viet\Downloads\Compressed\mod\fightnight.bms
- set output folder C:\Users\Q.Viet\Downloads\Compressed\mod\136

  offset   filesize   filename
--------------------------------------
  19c5fe60 182724     data/gui/locale/matchday/JPN_JP.loc
*** stack smashing detected ***:  terminated

-------------------
*EXCEPTION HANDLER*
-------------------
An error or crash occurred:

*EH* ExceptionCode      c000001d illegal instruction
*EH* ExceptionFlags     00000000
*EH* ExceptionAddress   013927A5
                        00C90000 + 007027a5 quickbms.exe
*EH* NumberParameters   00000000

Last script line before the error or that produced the error:
  92  log FNAME OFFSET SIZE

Press ENTER or close the window to quit

Sorry! But you can delete this my topic: viewtopic.php?f=9&t=8938
I was confused, you had this topic. Thank you!

[aluigi]

There are few users who are reporting this exact problem with "stack smashing detected" and I'm investigating.
Here on all my testing platforms (linux, mac, win98, win7, win10, win2003 and so on) I have not been able to replicate it even with the same samples used by who reported the problem, so I hope only a small number of people are affected.
I will give you some updates soon.

[aluigi]

I will also check why 0.9.1 now takes 100 Mb of RAM more than 0.9.0 (about 280 Mb vs 180 Mb).
It's probably related to some new libraries (maybe the Capstone disassembler).
Investigating...