Possible next features of QuickBMS

Doubts, help and support about QuickBMS and other game research tools
michalss
Posts: 320
Joined: Sun Aug 10, 2014 12:49 pm

Re: Possible next features of QuickBMS

Post by michalss »

aluigi wrote:Currently I have opted for writing ? or a char like <>|: in the entry. They are invalid chars for a filename, perfect for forcing this "special" clipboard mode.

Ah, the IPC mode is complete, it's probably totally useless since I think I'm going to release a dll but who cares:
- named pipe in message mode \\.\pipe\quickbms
- named pipe in byte mode \\.\pipe\quickbms_byte
- mailslot (not tested) \\.\mailslot\quickbms
- web IPC

They support just compression and encryption since there is nothing else interesting to offer.


Nice, any samples pls ?
Top
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Possible next features of QuickBMS

Post by aluigi »

This is a portion of quickbms.txt that I'm still writing for quickbms 0.8.5 and it explains quite well how the IPC interface works:

Code: Select all

D] web API and named pipe/mailslot IPC interface
------------------------------------------------

The -W command-line option starts the IPC mode which includes:
- web api running on the port specified with the -W option
- named pipe IPC in byte mode on \\.\pipe\quickbms_byte
- named pipe IPC in message mode on \\.\pipe\quickbms
- mailslot IPC on \\.\mailslot\quickbms\send with
  \\.\mailslot\quickbms\recv open in write mode (create it on your tool)

These interfaces have been successfully tested on both Windows and
Linux and the following is a quick set of examples for how using them
for decompressing data, those 302 and 1028 are only an example of input
and output size:

Example of web API:
POST http://127.0.0.1:1234/compress?algo=zlib&size=1028
attached Content is compressed input "as-is" (application/octet-stream)

Example of Named pipe (byte mode):
CreateFile      \\.\pipe\quickbms_byte
send: "comtype zlib\n"
send: "302\n"
send: 302 bytes of compressed data
send: "1028\n"
recv: "1028\n"
recv: 1028 bytes of decompressed data

Example of Named pipe (message mode):
CreateFile      \\.\pipe\quickbms
send: "comtype zlib"
send: "302"
send: 302 bytes of compressed data
send: "1028"
recv: "1028"
recv: 1028 bytes of decompressed data

Example of Mailslot:
CreateFile      \\.\mailslot\quickbms\send  GENERIC_WRITE
send: "comtype zlib"
send: "302"
send: 302 bytes of compressed data
send: "1028"
CreateMailslot  \\.\mailslot\quickbms\recv
recv: "1028"
recv: 1028 bytes of decompressed data

The IPC interface supports the encryption command too and other
features and commands may be added in future.
Currently the web API supports also /script and /file that are meant
mainly for debugging an input script and an input file based on the
script previously provided. In the latter case there will be no output
file generated just like with the -0 option (the TEMPORARY_FILE may be
the only exception).

If you need "code" it depends by your programming language, there are plenty of examples online since they are just a web API and a simple CreateFile+ReadFile+WriteFile(+CreateMailSlot for mailslots) which are the basis of Windows programming
Top
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Possible next features of QuickBMS

Post by aluigi »

Just a note about filexor/filerot/filecrypt because the current quickbms.txt documentation lacks the FILENUM field after OFFSET:
filexor VAR [OFFSET] [FILENUM]

filexor is a command that works universally on all the files, that FILENUM is ONLY referred to the optional OFFSET field since it's used to calculate the position in the key you are using depending by the offset of the file.
Now there is a very rare event (happened yesterday) in which you may need to use a MEMORY_FILE or a file different than the main one (0) and after you set OFFSET you notice that the decryption is a mess... that's caused by the missing FILENUM field which is mandatory in this rare situation, example:

Code: Select all

savepos CURRENT_OFFSET MEMORY_FILE
filexor KEY CURRENT_OFFSET MEMORY_FILE
get VAR long MEMORY_FILE

Don't think that being the author of quickbms makes me remember all the commands, options, features, caveats and bugs of the tool :D
I check quickbms.txt and even the source code of quickbms (damn "encryption random"...) very often...
Top
michalss
Posts: 320
Joined: Sun Aug 10, 2014 12:49 pm

Re: Possible next features of QuickBMS

Post by michalss »

aluigi wrote:This is a portion of quickbms.txt that I'm still writing for quickbms 0.8.5 and it explains quite well how the IPC interface works:

Code: Select all

D] web API and named pipe/mailslot IPC interface
------------------------------------------------

The -W command-line option starts the IPC mode which includes:
- web api running on the port specified with the -W option
- named pipe IPC in byte mode on \\.\pipe\quickbms_byte
- named pipe IPC in message mode on \\.\pipe\quickbms
- mailslot IPC on \\.\mailslot\quickbms\send with
  \\.\mailslot\quickbms\recv open in write mode (create it on your tool)

These interfaces have been successfully tested on both Windows and
Linux and the following is a quick set of examples for how using them
for decompressing data, those 302 and 1028 are only an example of input
and output size:

Example of web API:
POST http://127.0.0.1:1234/compress?algo=zlib&size=1028
attached Content is compressed input "as-is" (application/octet-stream)

Example of Named pipe (byte mode):
CreateFile      \\.\pipe\quickbms_byte
send: "comtype zlib\n"
send: "302\n"
send: 302 bytes of compressed data
send: "1028\n"
recv: "1028\n"
recv: 1028 bytes of decompressed data

Example of Named pipe (message mode):
CreateFile      \\.\pipe\quickbms
send: "comtype zlib"
send: "302"
send: 302 bytes of compressed data
send: "1028"
recv: "1028"
recv: 1028 bytes of decompressed data

Example of Mailslot:
CreateFile      \\.\mailslot\quickbms\send  GENERIC_WRITE
send: "comtype zlib"
send: "302"
send: 302 bytes of compressed data
send: "1028"
CreateMailslot  \\.\mailslot\quickbms\recv
recv: "1028"
recv: 1028 bytes of decompressed data

The IPC interface supports the encryption command too and other
features and commands may be added in future.
Currently the web API supports also /script and /file that are meant
mainly for debugging an input script and an input file based on the
script previously provided. In the latter case there will be no output
file generated just like with the -0 option (the TEMPORARY_FILE may be
the only exception).

If you need "code" it depends by your programming language, there are plenty of examples online since they are just a web API and a simple CreateFile+ReadFile+WriteFile(+CreateMailSlot for mailslots) which are the basis of Windows programming




Thx im gonna try it out, but do u have any idea when DLL will be released ? I sure u dont know exactly but roughtly :) ?
Top
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Possible next features of QuickBMS

Post by aluigi »

I guess I will release 0.8.5 (and its dll version) before the end of the month.
All the things in my todo list have been done.
Top
michalss
Posts: 320
Joined: Sun Aug 10, 2014 12:49 pm

Re: Possible next features of QuickBMS

Post by michalss »

aluigi wrote:I guess I will release 0.8.5 (and its dll version) before the end of the month.
All the things in my todo list have been done.



Yeaah best news ever :) Thx man this will be a new rising day for modders :D
Top
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Possible next features of QuickBMS

Post by aluigi »

Is it really so useful to use the compression algorithms of quickbms?

Most of them are known libraries (zlib, lz4, lzo and so on) or small algorithms easy to port to other languages.
The encryption algorithms are all known libraries.
Top
michalss
Posts: 320
Joined: Sun Aug 10, 2014 12:49 pm

Re: Possible next features of QuickBMS

Post by michalss »

aluigi wrote:Is it really so useful to use the compression algorithms of quickbms?

Most of them are known libraries (zlib, lz4, lzo and so on) or small algorithms easy to port to other languages.
The encryption algorithms are all known libraries.



Yes it is for programmer like where i dont need to look for something, simply everything on 1 place and i believe that usage of the functions will be the same for all algos or compressions or at least very similar , so its also make thing easier.
Top
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Possible next features of QuickBMS

Post by aluigi »

Anyone with new ideas?
Basically quickbms 0.8.5 is ready and I can release it when I desire, so any last-minute fix/improvement/feature is welcome.

P.S.: just fixed a lame bug in SortArray. Now the sorting time passed from N sections (even minutes) to 0 seconds.
Top
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Possible next features of QuickBMS

Post by aluigi »

In the meantime this is the output generated by a new feature of quickbms that can be invoked with the -t option and may be implemented in other fields in future (for example with the web api interface):

Code: Select all

|- 
    |- file.txt
    |- files2
        |- folder1
            |- file7.txt
    |- files
        |- test_file.txt
        |- folder1
            |- file1.txt
            |- file2.txt
            |- file3.txt
        |- folder2
            |- file4.txt
            |- file5.txt
            |- file6.txt
        |- folder3
            |- file8.txt
        |- folder4
            |- file9.txt
            |- file10.txt
    |- a
        |- b
            |- c
                |- d
                    |- e
                        |- f
                            |- g
                                |- test.txt

Code: Select all

| 
|__ file.txt
|__ files2
|____ folder1
|______ file7.txt
|__ files
|____ test_file.txt
|____ folder1
|______ file1.txt
|______ file2.txt
|______ file3.txt
|____ folder2
|______ file4.txt
|______ file5.txt
|______ file6.txt
|____ folder3
|______ file8.txt
|____ folder4
|______ file9.txt
|______ file10.txt
|__ a
|____ b
|______ c
|________ d
|__________ e
|____________ f
|______________ g
|________________ test.txt

Code: Select all

{
"name":"",
"type":"folder",
"children": [
    {
    "name":"file.txt",
    "type":"file",
    "offset":0",
    "zsize":123456",
    "size":12345678",
    },
    {
    "name":"files2",
    "type":"folder",
    "children": [
        {
        "name":"folder1",
        "type":"folder",
        "children": [
            {
            "name":"file7.txt",
            "type":"file",
            "offset":0",
            "zsize":123",
            "size":123",
            },
        ]},
    ]},
    {
    "name":"files",
    "type":"folder",
    "children": [
        {
        "name":"test_file.txt",
        "type":"file",
        "offset":0",
        "zsize":123",
        "size":123",
        },
        {
        "name":"folder1",
        "type":"folder",
        "children": [
            {
            "name":"file1.txt",
            "type":"file",
            "offset":0",
            "zsize":123",
            "size":123",
            },
            {
...

Code: Select all

 Directory of 

30-May-18  09:29    <DIR>          files2
30-May-18  09:29    <DIR>          files
30-May-18  09:29    <DIR>          a
30-May-18  09:29    12345678       file.txt

 Directory of files2

30-May-18  09:29    <DIR>          folder1

 Directory of files2/folder1

30-May-18  09:29    123            file7.txt

 Directory of files

30-May-18  09:29    <DIR>          folder1
30-May-18  09:29    <DIR>          folder2
30-May-18  09:29    <DIR>          folder3
30-May-18  09:29    <DIR>          folder4
30-May-18  09:29    123            test_file.txt

 Directory of files/folder1

30-May-18  09:29    123            file1.txt
30-May-18  09:29    123            file2.txt
30-May-18  09:29    123            file3.txt

 Directory of files/folder2

30-May-18  09:29    123            file4.txt
30-May-18  09:29    123            file5.txt
30-May-18  09:29    123            file6.txt

 Directory of files/folder3

30-May-18  09:29    123            file8.txt

 Directory of files/folder4

30-May-18  09:29    123            file9.txt
30-May-18  09:29    123            file10.txt

 Directory of a

30-May-18  09:29    <DIR>          b

 Directory of a/b

30-May-18  09:29    <DIR>          c

 Directory of a/b/c

30-May-18  09:29    <DIR>          d

 Directory of a/b/c/d

30-May-18  09:29    <DIR>          e

 Directory of a/b/c/d/e

30-May-18  09:29    <DIR>          f

 Directory of a/b/c/d/e/f

30-May-18  09:29    <DIR>          g

 Directory of a/b/c/d/e/f/g

30-May-18  09:29    123            test.txt

Code: Select all

Index of /

Icon  Name                                                             Last modified              Size  Description
[PARENTDIR] Parent Directory
[DIR] files2/                                                          30-May-18  09:29              -  files2
[DIR] files/                                                           30-May-18  09:29              -  files
[DIR] a/                                                               30-May-18  09:29              -  a
[txt] file.txt                                                         30-May-18  09:29       12345678  file.txt

Index of /files2

Icon  Name                                                             Last modified              Size  Description
[PARENTDIR] Parent Directory
[DIR] folder1/                                                         30-May-18  09:29              -  files2/folder1

Index of /files2/folder1

Icon  Name                                                             Last modified              Size  Description
[PARENTDIR] Parent Directory
[txt] file7.txt                                                        30-May-18  09:29            123  files2/folder1/file7.txt

Index of /files

Icon  Name                                                             Last modified              Size  Description
[PARENTDIR] Parent Directory
[DIR] folder1/                                                         30-May-18  09:29              -  files/folder1
[DIR] folder2/                                                         30-May-18  09:29              -  files/folder2
[DIR] folder3/                                                         30-May-18  09:29              -  files/folder3
[DIR] folder4/                                                         30-May-18  09:29              -  files/folder4
[txt] test_file.txt                                                    30-May-18  09:29            123  files/test_file.txt


These outputs have been generated from the following files information:

Code: Select all

"file.txt",                 0, 123456, 12345678
"files/folder1/file1.txt",  0, 123, 123
"files/folder1/file2.txt",  0, 123, 123
"files/folder1/file3.txt",  0, 123, 123
"files/folder2/file4.txt",  0, 123, 123
"files/folder2/file5.txt",  0, 123, 123
"files/folder2/file6.txt",  0, 123, 123
"files2/folder1/file7.txt", 0, 123, 123
"files/folder3/file8.txt",  0, 123, 123
"files/folder4/file9.txt",  0, 123, 123
"files/folder4/file10.txt", 0, 123, 123
"files/test_file.txt",      0, 123, 123
"a/b/c/d/e/f/g/test.txt",   0, 123, 123

Currently this feature is totally useless, it's one of the many unknown features that are available (or are going to be available) in quickbms and even I don't remember them :)

P.S.: yes the json output needs some small fixes, this was just a quick test.
Top
AnonBaiter
Posts: 1125
Joined: Tue Feb 02, 2016 2:35 am

Re: Possible next features of QuickBMS

Post by AnonBaiter »

so i did this to my script

Code: Select all

get unix_timestamp time
and i want to implement this "timecode" into the extracted files

perhaps it would be better if quickbms had this kind of feature in which "log"/"clog" gets a "timecode" argument, such as if the "unix_timestamp" variable has this 32bit time value then said argument would need at least that variable so it can implement a "timecode" into the extracted file
if the "timecode" argument has nothing else to stick into then it'll be just assigned as "" or something
Top
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Possible next features of QuickBMS

Post by aluigi »

It's something that is rare and not useful in my opinion.
Additionally there are tons of factors that would make it a hell to use and creating a simple "syntax": creation, modification or access time or all or a combo? unix 32bit time (from what year? many implemenations), 64bit Windows FILETIME, floating point timestamps, other variants of timestamps...

I try to avoid to add new commands for keeping the language as simple as possible.
Top
AnonBaiter
Posts: 1125
Joined: Tue Feb 02, 2016 2:35 am

Re: Possible next features of QuickBMS

Post by AnonBaiter »

oh okay then
Top
michalss
Posts: 320
Joined: Sun Aug 10, 2014 12:49 pm

Re: Possible next features of QuickBMS

Post by michalss »

Hi Luigi,

Is this dll done pls i did not find it in release notes this change..

thx
Top
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Possible next features of QuickBMS

Post by aluigi »

I already said that it will be part of quickbms 0.8.5 which is not available yet.
Top
michalss
Posts: 320
Joined: Sun Aug 10, 2014 12:49 pm

Re: Possible next features of QuickBMS

Post by michalss »

aluigi wrote:I already said that it will be part of quickbms 0.8.5 which is not available yet.



ok but u said it will be done before end of last month, not a problem, was not sure..
Top
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Possible next features of QuickBMS

Post by aluigi »

Right, I have continued to fix and implement new stuff so I decided to postpone the new release.
I guess it's better if I will release it in less than 2 weeks, just the time for checking if everything works correctly and no bugs have been introduced :)
Top
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Possible next features of QuickBMS

Post by aluigi »

This is a feature of the upcoming quickbms:

Code: Select all

get BACKUP_NAME filename
savepos BACKUP_OFF
open "." TEMPORARY_FILE
... do your stuff ...
open FDSE2 BACKUP_NAME
goto BACKUP_OFF

Code: Select all

open "." TEMPORARY_FILE 1
open 1
... do your stuff ...
open 0
These 2 scripts do the exact same thing in two different ways:
  • saving the current offset, closing the file, open the new one, reopen the original file (FDSE2 is new), restore the offset
  • redirect all the operations of the current file to another one (file 1 in the example, it can work with MEMORY_FILEs too)

It's a feature that is useless in 99.9% of the scripts, but extremely useful when needed and it's also a good alternative (double alternative) to using a FILENUM variable for every instruction.

In case you are asking what is FDSE2, well it's the forced original input folder which is opposed to FDSE that is the folder of the current file, so if you open (for example) the TEMPORARY_FILE you will not be able to open files in the original folder, with FDSE2 you can.
Top
Dima Bilan
Posts: 25
Joined: Tue Jul 26, 2016 12:07 am

Re: Possible next features of QuickBMS

Post by Dima Bilan »

And you can add a function so that when you reimport files, it skips all files that are larger than the original size at a time so that you do not press a button y every time. I know that the files should be smaller than the original size, but still I ask to add it. I would also like to trim a file if it is larger than the original size, but I think you do will not want to add this function.
Last edited by Dima Bilan on Sat Jun 09, 2018 5:07 pm, edited 1 time in total.
Top
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Possible next features of QuickBMS

Post by aluigi »

In that case how can you know what files have been skipped?
Maybe it has more sense to allow the existent -0 option (it's used in extraction) to be used with the reimport feature for "testing" the injected files, and it would tell you "file1.txt" is ok, "file2.txt" can't be reimported and so on.
That would be easy to implement.
Top

Return to “Help with QuickBMS & tools”