Possible next features of QuickBMS

[Shokoniraya]

sir aluigi, i have an idea for next version of QuickBMS

in -s "" "", we have to replace many escapes and replace \x0D\x0A to |
but how about file_compressed_var.bms? can you add a feature to use compressed var text like -s without replace any escape?
for example: -s 0xSWzdkGr/WfWtK-VIaEJwd+wGlQJKVzevre+vkIJKLXJwJWtImUOWEI

[spiritovod]

I wonder if it's possible to add extension guessing for files, where magic value is not header, but the footer. For example, in UE4 files "uasset" magic is 0xC1832A9E for first bytes, while for "uexp" files the same magic will be last bytes in the file.

[HenryEx]

I know this is meant for new features, but i haven't really found any closer info on it, so i thought i'd ask here.

The latest update 0.10.1 mentions implementing a few 010-like commands, but doesn't actually mention which ones. I briefly searched the manual for any mentions of 010 but didn't find any, and i haven't checked the entire thing closely again because it's so massive by now.

What exactly do these commands entail? Since i'm also working with 010, it might speed up my workflow a bit.

[aluigi]

@Shokoniraya
I guess you can already do that with:
-s "set VAR compressed CONTENT"
Not sure what you mean about escapes but check if they work when you use \\ instead of \

@spiritovod
ok I added it to sign_ext.c just after the code that handled the tga "TRUETYPE" footer but I'm not sure if it will work well.

@HenryEx
If 010 editor uses some internal commands, they may be referenced in the parse_bms function of bms.c, they are not documented and just partially implemented.
While the C structures, which are the main part of 010, are implemented with some work-arounds.
quickbms will say something like "- c_structs (arguments)" when it parses these fields.


Hopefully tomorrow I should be able to release a beta with the things fixed till now.

[Shokoniraya]

sir aluigi, i mean compress a script file to use it in bat file instead making whole script to a single line and replace \x0D\x0A to ; or replace " to \" or % to %%

normally, we have to use it in this way

Code: Select all

quickbms.exe -o -s "get SIZE asize ; log file 0 SIZE" "" file.bin

but how about just compressing a script file to use it? without any replaced escape and less size
just compressing bms file with file_compressed_var.bms

Code: Select all

quickbms.exe -o -s eNoBHwDg/2dldCBTSVpFIGFzaXplDQpsb2cgZmlsZSAwIFNJWkWfTgmc file.bin

it could be useful for including huge scripts in command line

[aluigi]

Ok I added that feature too.

This is the current beta:
https://aluigi.altervista.org/beta/quickbms_beta.zip

It's just quickbms.exe with the main issues fixed and main features added, it already contains the current version of the various external libraries.

Please let me know how it works and what other things remain to be fixed.

I still have stuff in my TODO list, for example:
- reimporting of data.win doesn't work as in version 0.9
- apparently there is a crash if you select thousands of files with quickbms (via gui)
- checking if append -1 works as expected
- adding more compression algorithms, maybe from garbro
- findarray / searcharray
- 64bit dll support

[Shokoniraya]

1) append -1
append -1 doesn't work correct yet

Code: Select all

append -1
log NAME OFFSET SIZE
append

2) EXECUTE
EXECUTE problem still not fixed
http://www.mediafire.com/file/ew70myc2t ... m.zip/file
note: zip file has 0.10.0 quickbms.exe, please replace it with beta

3) lzo1x_999_compress
this script should compress input file to a compressed file, but give me this error: the variable index is invalid (-1), there is an error in QuickBMS (403)
is that possible to use lzo1x_999_compress or its just a function name?

Code: Select all

comtype lzo1x_999_compress
get FILE_SIZE asize
clog compressed_file.compressed 0 FILE_SIZE FILE_SIZE

4) codepage problem and unicode names for input file
download/file.php?id=8840
those "Ianei" folder folder still created in output folder

about unicode names for input files, they are still has problem
- error in src\file.c line 597: fdnum_open()
Error: Invalid argument

5) xmath command
xmath still has problem with x

Code: Select all

xmath VAR "15 x 16"

[aluigi]

I just updated the beta.

1)
Fixed. I had to just read the whole file and move its content, this operation can't be done with a buffer, just whole or byte per byte (slow).

2)
It's not an error of quickbms, the script is messy and not clear what you want to do.
the script must have no access to the temporary files because their content is already moved to the output file and immediately deleted.
or do you mean another type of problem?

3)
you must use lzo1x_compress, because lzo1x_999_compress is the name of the function

4)
fixed, the error was related to how mkdir works, in short it's necessary to check both the return value and errno

5)
xmath doesn't support x because x can be a variable
xmath only supports non-alphanumeric operatorrs so what you want to do can be accomplished with && or ?align.
I added the possibility to use ?x too.

[spiritovod]

Thanks for the additions and fixes. Could you please clarify about extension guessing for uexp - did you integrated it to the latest beta or it was about extension guessing in a whole for other formats. Because guessing for uexp doesn't work with the latest beta.

[aluigi]

I added it to sign_ext.c but as far as I remember the signature checking is only limited to the first few bytes of the file, therefore no check on the tail is possible.
Sorry.

[spiritovod]

I see. Then maybe you should remove uexp definition from the code to avoid possible problems, since uasset is using the same magic value, but for first bytes, and guessing for uasset works fine (though not sure if overlapping is possible in case of the same magic for multiple extensions).
Thanks for checking it anyway.

[aluigi]

Ok.

In case someone is interested in why reimporting data.win in quickbms 0.10 gave a different result than 0.9:
viewtopic.php?p=58729#p58729

[Shokoniraya]

append 1- works fine
codepage problem solved, so there is no Ianei folder
and about EXECUTE, yes it fixed, i don't know why bms files was messed up, maybe UltraEdit broked the file, because that was not real one, no problem with EXECUTE anymore

here my correct EXECUTE example

Code: Select all

math INPUT_NUMBER = 2

set MEMORY_FILE1 binary "0"
comtype EXECUTE "calc\\calc.exe %INPUT_NUMBER% / 32 >#OUTPUT#"
clog answer.txt 0 1 1 MEMORY_FILE1

and both && or ?x do the job, so maybe keep using ?x in xmath will be fine, right? thank you


all of this problem solved, but there is one problem, unicode names in input file
i tested with last beta version
but i get this error, but there is no problem in GUI mode, i get this error when i use it in command line (example zip uploaded)
- error in src\file.c line 602: fdnum_open()
Error: Invalid argument

for /r "input\" %%x in ("*.bin") do (
)
%%x could be used on any file with unicode name

[aluigi]

Perfect, I just updated the beta.

There is also a "diff" file in case someone is interested in the work-in-progress with the source code, but it doesn't include garbro.c (just finished today).

[Shokoniraya]

thank you, but still i get this error in with a input file that has unicode name in batch file

- error in src\file.c line 602: fdnum_open()
Error: Invalid argument

exmaple file: download/file.php?id=9198

[aluigi]

My fault, I didn't move the beta zip to the folder ...
Now it's fixed.

[Shokoniraya]

thank you, works fine

[aluigi]

I have updated the beta with the latest version which I hope is also the one that will be released on the website.
It's already compiled with optimizations and with both the executables just like a normal release.
It includes the full source code, the changelog and also a revisioned quickbms.txt (on which I may work more).

Please test it as much as you can and tell me if you spot any bug or have any questions.

The link is the same (let's use the direct https one btw, copy the URL in the browser if you want to skip the redirect page) and the size is 30Mb:
https://aluigi.altervista.org/beta/quickbms_beta.zip

I plan to release quickbms 0.11 at the beginning of October.

[Shokoniraya]

it's really better to set names for MEMORY_FILE too

Code: Select all

set MEMORY_FILE_sewew string "1"
set MEMORY_FILE_wcfow string "2"

log memory 0 1 MEMORY_FILE_wcfow

and this one too

Code: Select all

set MEMORY_FILE1 string "1"
set MEMORY_FILE2 string "2"

math COUNT = 2
string MEMORY_SLOT p "MEMORY_FILE%s" COUNT
print "%MEMORY_SLOT%"
log memory 0 1 MEMORY_SLOT
#or even log memory 0 1 MEMORY_FILE%COUNT%

[Shokoniraya]

a problem in downloading file with -n option

most of links using / (slash) for passing directories
but if i use \ (back-slash), url not working and give me (404) as a remote error

exmaple
woring: http://web.com/image.png
not working: http://web.com\image.png