QuickBMS errors [programming, scripting, quickbms.exe tool... NOT games]

[BCGhost]

Issue with the "continue" instruction:

Code: Select all

set Flag long 0
set i long 0
set j long 0
for i = 0 < 3
   print "i = %i%"
   if Flag == 0
      math Flag = 1
      continue
   else
      math Flag = 0
   endif
   print "*** Message before embedded loop ***"
   for j = 0 < 4
      print "\tj = %j%"
   next j
next i

Of course it's easy to bypass it by placing the code withing an If...EndIf.

[BCGhost]

The problem is related to the pre-allocation of the arrays like "putarray 0 totalFileNum 0 # fileID" that indeed adds a new element in the array.
...
If you remove pre-allocation everything is ok...

Usually pre-allocation is not necessary for arrays.

Sorry that I didn't see this in time. It's just that the doc of QuickBMS says "it's highly suggested to pre-allocate the array if you know the max value, example: PutArray 0 FILES 0". I've tested that if you allocate (totalFileNum - 1) elements and adjust the counter afterward it can produce the correct result. So for performance reasons is this routine still necessary?

[Nameless]

Issue with the "continue" instruction:

Code: Select all

set Flag long 0
set i long 0
set j long 0
for i = 0 < 3
   print "i = %i%"
   if Flag == 0
      math Flag = 1
      continue
   else
      math Flag = 0
   endif
   print "*** Message before embedded loop ***"
   for j = 0 < 4
      print "\tj = %j%"
   next j
next i

You can use this solution: break to label.

Code: Select all

set Flag long 0
for i = 0 < 3
   print "i = %i%"
   if Flag = 0
      math Flag = 1
      break here
   else
      math Flag = 0
   endif
   print "*** Message before embedded loop ***"
   for j = 0 < 4
      print "\tj = %j%"
   next j
here:
next i

Though labels are also broken in some cases, for example, when you use them in functions then jump backwards doesn't work properly.

[aluigi]

@Nameless
"continue" and break to label are some sort of experiment, I know.

Instead regarding the problem you reported about CRC calculation, do you have any script for replicating the problem?
And is it related to NameCRC or to "Encryption crc"?

[aluigi]

@GHOST DEAD
MEMORY_FILEs aren't touched in reimport mode because they are usually used for moving data from the original archive to a temporary location.
Reimporting can only work from real file to archive and (experimentally) from some MEMORY_FILEs to archive.

[Nameless]

Instead regarding the problem you reported about CRC calculation, do you have any script for replicating the problem?
And is it related to NameCRC or to "Encryption crc"?

It is related to encryption. I didn't check namecrc, so I can't tell whether it is also affected.
And here is the test script you asked. Feed any standard text file to it, you will see that error rate is very high.

[Daftendirekt]

An error occurs while trying to unpack the .archive file using https://aluigi.altervista.org/bms/deus_ex_mankind_divided.bms script

$ quickbms_4gb_files.exe deus_ex_mankind_divided.bms "C:\Program Files (x86)\Steam\steamapps\common\Deus Ex Mankind Divided\DLC\runtime\DLC02.layer.0.all.archive" outdir

... some time later ...

- error in src\extra\xalloc.c line 703: xdbg_realloc()
Error: memory allocation problem

[aluigi]

@Daftendirekt
That's game-related, probably the script is not updated or doesn't support that specific archive. Or you are using an old version of quickbms.
In any case you should post in the topic of the game, not here.

[Shokoniraya]

i found two problem in QuickBMS

1: -e command will make quickbms slow (not a problem, but i reported it anyway)
2: xmath cant handle x as well, [xmath VAR "15 x 16]

[saeid0034]

recently i use a script name assassin_creed_raw.bms for decompress assassin creed odyssey data file (i export data from forge with Ubisoft_Forge_Tool)
script Successfully extracts DATA files and give me 2 DAT file
but when i try to import dat file into data again (with reimport.bat) it failed

. can anyone make a new script (for decompress and compress data file again) for assassin creed odyssey?
A tool already built for this task (Ubisoft_DATA_Tool_By_Delutto) but it cant import file again into data (also it looks like its developer has stopped working on this program)

[aluigi]

@saeid0034 and @greenlemonade1:
Off-topic

@sergop
Regarding tar.bms you don't need to use quickbms for that job, it's just the classical tar supported by tons of utilities and obviously tar itself

The technical reason behind quickbms refusing to reimport the files is that tar.gz (tgz) is a compressed tar archive, so quickbms must first decompress it in memory and then extract the files.
In reimport mode you need to directly work on the archive on the disk, no MEMORY_FILE, or the changes can't go on the original archive.

[aluigi]

@Shokoniraya

1)
the only two things done by the -e command is zeroing the memory of the output buffer and then returning the output size in case of errors.
I suspect that the slowness you are experiencing is caused by zeroing the output memory which may be time consuming with some large files.

2)
'x' can't be directly used in xmath because alphabetic characters may be misinterpreted with variables, that's why exist an alternative way for specifying tons of operators by using the '?' prefix, for example ?add instead of +.
What I noticed is that ?align and the other alternative operators don't seem to work correctly for reasons I will investigate.
Thanks for reporting.

[aluigi]

Instead regarding the problem you reported about CRC calculation, do you have any script for replicating the problem?
And is it related to NameCRC or to "Encryption crc"?

It is related to encryption. I didn't check namecrc, so I can't tell whether it is also affected.
And here is the test script you asked. Feed any standard text file to it, you will see that error rate is very high.

Sorry for the late reply.
What I can say in the meantime is that this issue happens only when using the "e/E" operators of the String command.
Using "log" works perfectly.
Even using "string CHSTR E CHSTR" (VAR1 same as VAR2) produces the correct result, even when calling MYCRC32 before it (just to avoid issues).

I will better investigate when I will work on the next version but it's clearly a bug in the length used for performing the encryption since it uses the length of the first variable instead of the second one:

Code: Select all

            case 'E': {
                if(len1 < len2) {
                    len1 = len2;
                    ...
                }
                memcpy(var1, var2, len1);
                fixed_len = perform_encryption(var1, len1);

[Nameless]

Even using "string CHSTR E CHSTR" (VAR1 same as VAR2) produces the correct result, even when calling MYCRC32 before it (just to avoid issues).

Interesting... Good to know. Too bad that readme doesn't specify it, I always thought that VAR2 will be destroyed in such case.

[aluigi]

Interesting... Good to know. Too bad that readme doesn't specify it, I always thought that VAR2 will be destroyed in such case.

Because it's a bug
The plan was to copy the encrypted content of VAR2 in VAR1 but the bug took the lenght of VAR1 instead of VAR2

[Nameless]

Glad that it will be fixed.

[Shokoniraya]

when i want to run a exe with a none-standard cstring name like ( or ) that should used " in beging and end of name, in execute command in bms script, command not working fine

"zlib(v2.4).exe -c #INPUT# #OUTPUT#"
or even this one: "\"zlib(v2.4).exe\" -c #INPUT# #OUTPUT#"
or this: ""zlib(v2.4).exe" -c #INPUT# #OUTPUT#"

[Shokoniraya]

sir aluigi

there is a problem in QuickBMS that always making some problem in text importing
there is some string like \n or \f or \e
but can you skip none cstring error? like \:

Error: cstring() failure, your input string has some wrong escape sequences or
it's not a valid escaped string

i know i should use \\:
but \: not used in cstring, right? becuase there is not any \: or \k too, so error and stop the process can't helping, then simply ignore unavailable escapes

and i know if i export that text again, it will turned to \\:
but there is no problem with that, so can you please ignore wrong escapes in next version?

thank you

[Shokoniraya]

sir aluigi, about cstring errors, i know there is some problems for that if you ignore it as default, but it's better to make a command option to ignore incorrect \* cstring

and about unicode converting that skips at zero, normaly if we want to read a 64 bytes that stored a name in it, it will stop at zero (null). so there must be a option to ignore zero bytes too
because of such cases

Code: Select all

63 6F 6D 6D 6F 6E 5C 00 00 00 00 00 75 69 5C 00 common\.....ui\.
00 00 00 00 00 00 00 74 65 78 74 00 00 00 75 72 .......text...ur
00 00 65 00 00 00 5C 00 00 00 66 6E 74 5F 00 00 ..e...\...fnt_..
00 00 00 30 2E 00 00 00 00 00 66 6E 74 00 00 00 ...0......fnt...

[Shokoniraya]

another problem, but i don't know that if it's a QuickBMS feature or not

append -1 not working as well, i think it should add binary to start of file but has problem

please test it on a 40 byte file (you can see it better in a small file)

Code: Select all

get SIZER asize

for i = 0 < 5
append -1
log FILE_APPEND 0 SIZER
append -1
next i