Function

[chrrox]

I am trying to do this
''.join(chr(~ord(msg[i]) & 0xff) for i in xrange(4)) + msg[4:]
which is taking a string
taking the first 4 character of that string
and replacing them with the inverse ord of that character.
The place i got to in quickbms is
if i set the value manually like
set var int 97
it seems to work fine
but when i try
String TMP B= "b"
print "%TMP%"
the value it prints is correct but its returning decimal not hexidecimal
so in the case of a
it returns decimal 61 instead of hex 61

[puggsoy]

You can print hex like this:

Code: Select all

print %TMP|h%

This will print 3D instead of 61. This is explained by the Print command in quickbms.txt

[chrrox]

No i mean quickbms is returning the incorrect value.
it is returning 61 decimal when the value should be 0x61

[puggsoy]

I'm not understanding what you mean exactly. Do you mean it should print 61 but is printing 97 instead?

Can you please paste in your script, and the result you get and the result you want.

[aluigi]

@chrrox
Can you show some examples of input and output you expect?
Because I don't understand if you want to make just inverting the first 4 bytes (easy with getvarchr+reverselong+putvarchr) or something else.

[chrrox]

Ok here is an example
filename 64:61:74:61:2f:61:69:2f:70:61:63:6b:65:64 data/ai/packed
inverted file name 9b:9e:8b:9e:2f:61:69:2f:70:61:63:6b:65:64

so what i need is to convert "data/ai/packed" to hex
then
the first 4 characters of the string
need this operation applied to them
~ord(d) & 0xff - ord of this should be 0x64 - inverted would be -0x65 then xored with 0xFF would be 0x9B
~ord(a) & 0xff - ord of this should be 0x61 - inverted would be -0x62 then xored with 0xFF would be 0x9E
~ord(t) & 0xff - ord of this should be 0x74 - inverted would be -0x74 then xored with 0xFF would be 0x8B
~ord(a) & 0xff - ord of this should be 0x61 - inverted would be -0x62 then xored with 0xFF would be 0x9E
then when its all done
i need to perform a crc32 calculation on 9b:9e:8b:9e:2f:61:69:2f:70:61:63:6b:65:64
http://www.fileformat.info/tool/hash.ht ... 61636b6564
and the result should be
CRC32 - 0x9cd62589

[aluigi]

Code: Select all

set STR string "data/ai/packed"
getvarchr TMP STR 0 long
math TMP ^= 0xffffffff
putvarchr STR 0 TMP long

print "HEX %STR|x%"

[chrrox]

That works perfect.
What would you use to calculate the crc32 values using that hex output.

[aluigi]

This is the full example:

Code: Select all

set STR string "data/ai/packed"

# ~ 4 bytes
getvarchr TMP STR 0 long
math TMP ^= 0xffffffff
putvarchr STR 0 TMP long

print "HEX %STR|x%"
print "STR %STR%"

# calculate checksum
encryption crc "" 32
string STR E= STR
encryption "" ""
print "CRC %QUICKBMS_CRC|x%"

QuickBMS supports a huge number of crc types so be sure that this one (the default one "" 32) is the one you need.
You can even use the crc scanner in case you don't know what crc settings to use.

[chrrox]

Ok one last question.
I noticed you had an auto name replacer using a text document.
Is it possible to paste ¢₧ï₧/ai/packed into it or a hex string to use?
if not I can think of other ways just asking.

[aluigi]

I guess you are referring to the NameCRC command.

It can be used in various way, but let's consider the following usage.

Create a file called list.txt containing the following:

Code: Select all

0x9cd62589 data/ai/packed

Now in your script it's enough to add the following after having read the NAME_CRC field from the archive:

Code: Select all

...
            get NAME_CRC long # your instruction
            namecrc NAME NAME_CRC "list.txt" 32 # the new instruction
            log NAME OFFSET SIZE # your instruction
...

In this case you must already have the full list of crc and relative filenames because this is a customized algorithm but there is an alternative that "may" work:

Create a file called list.txt containing the following:

Code: Select all

¢₧ï₧/ai/packed

Yes the name with the ~4 bytes.
Now in your script it's enough to use the following:

Code: Select all

...
            get NAME_CRC long # your instruction
            namecrc NAME NAME_CRC "list.txt" 32 "" 32
            getvarchr TMP NAME 0 long
            math TMP ^= 0xffffffff
            putvarchr NAME 0 TMP long
            log NAME OFFSET SIZE # your instruction
...

Note that I have tested both the situations and it works perfectly so it's up to you to choose the preferred one.

[chrrox]

This works very well
is there a good example of a nested archive.
This is what i am working with

Code: Select all

00 00 00 00 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00    
00 00 00 00 11 00 00 00 40 9B 03 00 30 19 10 00 A6 B4 ED 53 01 1E 00 00    ai/
20 00 00 00 15 00 00 00 20 9B 03 00 10 19 10 00 A6 B4 ED 53 02 1A 00 00    fighter/
40 00 00 00 1E 00 00 00 17 10 00 00 70 45 00 00 A6 B4 ED 53 03 1A 00 00   captain/
60 00 00 00 2C 00 00 01 AB 03 00 00 40 09 00 00 1A 19 DE 53 04 08 00 00    attack_data
40 04 00 00 38 00 00 01 8E 06 00 00 A8 29 00 00 99 4F D2 53 04 08 00 00    param
00 0B 00 00 3E 00 00 01 57 05 00 00 10 12 00 00 2B 8B D0 53 04 08 00 00    script
80 10 00 00 45 00 00 00 24 24 00 00 24 A1 00 00 A6 B4 ED 53 03 1A 00 00    common/
A0 10 00 00 38 00 00 01 D8 04 00 00 A8 29 00 00 2C 8B D0 53 04 08 00 00    param
A0 15 00 00 4D 00 00 01 F6 01 00 00 C0 06 00 00 2B 8B D0 53 04 08 00 00    reference
C0 17 00 00 3E 00 00 01 39 1C 00 00 A0 6F 00 00 AF 18 DA 53 04 08 00 00    script
20 34 00 00 57 00 00 01 84 00 00 00 84 00 00 00 2C 8B D0 53 04 08 00 00    system
E0 34 00 00 5E 00 00 00 62 14 00 00 D0 56 00 00 A6 B4 ED 53 03 1A 00 00    dedede/

file paths
ai/fighter/captain/attack_data
ai/fighter/captain/param
ai/fighter/captain/script
ai/fighter/common/param
ai/fighter/common/reference
ai/fighter/common/script
ai/fighter/common/system
ai/fighter/dedede/

the 8th byte in this structure indicates a folder or a file
and the first byte indicates the folder level
00
01
02
03
04
04
04
03

How would you handle this or is there a good example of this for me to look at.

[aluigi]

With nested entries like that you must use a function that calls itself.
Sometimes it's easy while other times it isn't because it depends by the format used by the archive.

The following is the smaller script I have that uses this method:
http://aluigi.org/papers/bms/others/voyage_century.bms

The generic code is basically the following:

Code: Select all

set NAME string ""
set PATH string ""
callfunction EXTRACT

startfunction EXTRACT
    string PATH += NAME
    string PATH += /
    ...
    # file
        set FNAME string PATH
        string FNAME += NAME
        log FNAME OFFSET SIZE
    ...
    # folder
        callfunction EXTRACT
    ...
endfunction

[chrrox]

How would i go up and down multiple levels
for example
if my folder level went from 3 to 1
how would i go from
ai/fighter/captain/
to
ai/

this is the file i am working with

Structure is
0x134868 file table size
divide this by 0x18
to get the file count
then at offset
0x134880 we have the string chunk count
then we save the position 0x134884
as the base string offset
we skip 0x2000 * chunk count
to get to
0x168884
we read the extension count
then the offset to each extension relative to the string start.
each file entry follows this format
off_in_chunk, name_offset_etc, cmp_size, dec_size, timestamp, folder depth and flags

off_in_chunk - offset from chunk start that data begins (the chunk info is in a separate file)
name_offset_etc - the first 2 bytes are the name offset from string base
the 3rd byte here is a flag to read 2 bytes in front of the string then go back that many spaces read the string then subtract the other value and join them togather
example
01 0E jr/
go back 0xE read string
koopa/
subtract 1
koopa
combine
koopajr/
and the last byte is the extension id indexed into the last table
cmp_size is file compressed size
dec_size is file decompressed size
timestamp - creation time
folder depth and flags - first byte is folder depth
localized = bool(depth and flags & 0x800)
final = bool(depth and flags & 0x400)
compressed = bool(depth and flags & 0x200)


Only thing i can think of is in python doing something like this
I could do this in python
http://www.dotnetperls.com/list-python
names = ["Tommy", "Bill", "Janet", "Bill", "Stacy"]
# Delete all except first two elements.
del names[2: ]

then when an entry is a file i could join the array
then I can do
x = ['a', 'b', 'c', 'd']
print ''.join(x)
'abcd'

[aluigi]

This format is confusing but not impossible.
The following is a work-in-progress script for resource.dec to understand what's the format:

Code: Select all

math FILES = 0x134868
math FILES /= 0x18
math NAMES_OFF = 0x134884

for i = 0 < FILES
    get OFFSET long
    get NAME_OFF long
    xmath NAME_FLAGS1 "(NAME_OFF >> 20) & 0xf"
    xmath NAME_FLAGS2 "(NAME_OFF >> 24) & 0xf"
    xmath NAME_FLAGS3 "(NAME_OFF >> 28) & 0xf"
    math NAME_OFF &= 0x000fffff
    get ZSIZE long
    get SIZE long
    get TSTAMP long
    get FLAGS long
    xmath FOLDER_DEPTH "FLAGS & 0xff"
    xmath LOCALIZED "FLAGS & 0x800"
    xmath FINAL "FLAGS & 0x400"
    xmath COMPRESSED "FLAGS & 0x200"

    savepos TMP
    math NAME_OFF += NAMES_OFF
    goto NAME_OFF
    if NAME_FLAGS1 != 0
        get STR_PREV byte
        get STR_SUB byte
        get NAME string
        math NAME_OFF -= STR_PREV
        get NAME2 string
        string NAME2 -= STR_SUB
        string NAME += NAME2
    else
        get NAME string
    endif
    goto TMP

    print "%NAME%"

next i

Fix it with the fields you have already understood.

[chrrox]

Ok now all that is left is the directory navigation.

Code: Select all

math FILES = 0x134868
math FILES /= 0x18
math NAMES_OFF = 0x134884

goto 0x168884
get EXT_COUNT long
savepos tmp
for i = 0 < EXT_COUNT
get EXT_OFF long
savepos tmp
math EXT_OFF += 0x134884
goto EXT_OFF
get EXT string
putarray 0 i EXT
goto tmp
next i

goto 0
for i = 0 < FILES
    get OFFSET long
    get NAME_OFF long
    xmath NAME_FLAGS1 "(NAME_OFF >> 20) & 0xf"
    xmath NAME_EXT "(NAME_OFF >> 24)"
    math NAME_OFF &= 0x000fffff
    get ZSIZE long
    get SIZE long
    get TSTAMP long
    get FLAGS long
    xmath FOLDER_DEPTH "FLAGS & 0xff"
    xmath LOCALIZED "FLAGS & 0x800"
    xmath FINAL "FLAGS & 0x400"
    xmath COMPRESSED "FLAGS & 0x200"
    #print "%NAME_FLAGS1% %NAME_EXT%"

    savepos TMP
    math NAME_OFF += NAMES_OFF
    goto NAME_OFF
    if NAME_FLAGS1 != 0
        get NAME2_OFF short
        xmath NAME2_SIZE "(NAME2_OFF & 0x1F) + 4"
        xmath NAME2_VAR "(NAME2_OFF >> 8)"
        math NAME2_OFF & 0xE0
        math NAME2_OFF >> 6
        math NAME2_OFF << 8
        math NAME2_OFF | NAME2_VAR
        get NAME2 string
        math NAME_OFF -= NAME2_OFF
        goto NAME_OFF
        getdstring NAME NAME2_SIZE
        string NAME -= STR_SUB
        string NAME += NAME2
    else
        get NAME string
    endif
    goto TMP
    getarray EXT 0 NAME_EXT
    string NAME += EXT

    print "%NAME%"

next i

[chrrox]

I did this not sure if there is a better way.

Code: Select all

math FILES = 0x134868
math FILES /= 0x18
math NAMES_OFF = 0x134884

goto 0x168884
get EXT_COUNT long
savepos tmp
for i = 0 < EXT_COUNT
get EXT_OFF long
savepos tmp
math EXT_OFF += 0x134884
goto EXT_OFF
get EXT string
putarray 0 i EXT
goto tmp
next i

goto 0
for i = 0 < FILES
    get OFFSET long
    get NAME_OFF long
    xmath NAME_FLAGS1 "(NAME_OFF >> 20) & 0xf"
    xmath NAME_EXT "(NAME_OFF >> 24)"
    math NAME_OFF &= 0x000fffff
    get ZSIZE long
    get SIZE long
    get TSTAMP long
    get FLAGS long
    xmath FOLDER_DEPTH "FLAGS & 0xff"
    xmath LOCALIZED "FLAGS & 0x800"
    xmath FINAL "FLAGS & 0x400"
    xmath COMPRESSED "FLAGS & 0x200"
    #print "%NAME_FLAGS1% %NAME_EXT%"

    savepos TMP
    math NAME_OFF += NAMES_OFF
    goto NAME_OFF
    if NAME_FLAGS1 != 0
        get NAME2_OFF short
        xmath NAME2_SIZE "(NAME2_OFF & 0x1F) + 4"
        xmath NAME2_VAR "(NAME2_OFF >> 8)"
        math NAME2_OFF & 0xE0
        math NAME2_OFF >> 6
        math NAME2_OFF << 8
        math NAME2_OFF | NAME2_VAR
        get NAME2 string
        math NAME_OFF -= NAME2_OFF
        goto NAME_OFF
        getdstring NAME NAME2_SIZE
        string NAME -= STR_SUB
        string NAME += NAME2
    else
        get NAME string
    endif
    goto TMP
    set NAME3 string ""
    putarray 1 FOLDER_DEPTH NAME
    for a = 0 < FOLDER_DEPTH
    getarray NAME2 1 a
    string NAME3 += NAME2
    next a
    getarray EXT 0 NAME_EXT
    string NAME += EXT
    string NAME3 += NAME
    if NAME_EXT != 0
    print "%NAME3%"
    endif

next i

[aluigi]

It seems to work perfectly
Well done

[chrrox]

The only thing i am missing now is a way to use a dictionary.
I need to use a hash as an index key to an offset and a size
in python i would do
mydict = {}

mydict[HASH] = 10, 20
then i could call those values back later with
mydetails = mydict[HASH]

[chrrox]

I ended up doing this if there is anything you see that might make the script better let me know.
is the break command ok to use in the hash look up?

Code: Select all

open FDSE dt.dat 1
open FDSE ls.dat 2
idstring "RF"
get VERSION short
get START long
get NULL long
get START long
get FILES long
math FILES /= 0x18
get UNK long
get ZSIZE long
get SIZE long
get NAMES_OFF long
math NAMES_OFF - 0x80
clog MEMORY_FILE START ZSIZE SIZE
goto NAMES_OFF MEMORY_FILE
get CHUNK_COUNT long MEMORY_FILE
savepos NAMES_OFF MEMORY_FILE
xmath EXT_OFF "(NAMES_OFF) + (0x2000 * CHUNK_COUNT)"

get UNK long 2
get TBL_COUNT long 2
putarray 2 TBL_COUNT 0
putarray 3 TBL_COUNT 0
putarray 4 TBL_COUNT 0
for i = 0 < TBL_COUNT
get HASH long 2
get OFFSET long 2
get SIZE long 2
putarray 2 i HASH
putarray 3 i OFFSET
putarray 4 i SIZE
next i


goto EXT_OFF MEMORY_FILE
get EXT_COUNT long MEMORY_FILE
savepos tmp MEMORY_FILE
for i = 0 < EXT_COUNT
get EXT_OFF long MEMORY_FILE
savepos tmp MEMORY_FILE
math EXT_OFF += NAMES_OFF
goto EXT_OFF MEMORY_FILE
get EXT string MEMORY_FILE
putarray 0 i EXT
goto tmp MEMORY_FILE
next i

goto 0 MEMORY_FILE
for i = 0 < FILES
    get OFFSET long MEMORY_FILE
    get NAME_OFF long MEMORY_FILE
    xmath NAME_FLAGS1 "(NAME_OFF >> 20) & 0xf"
    xmath NAME_EXT "(NAME_OFF >> 24)"
    math NAME_OFF &= 0x000fffff
    get ZSIZE long MEMORY_FILE
    get SIZE long MEMORY_FILE
    get TSTAMP long MEMORY_FILE
    get FLAGS long MEMORY_FILE
    xmath FOLDER_DEPTH "FLAGS & 0xff"
    xmath LOCALIZED "FLAGS & 0x800"
    xmath FINAL "FLAGS & 0x400"
    xmath COMPRESSED "FLAGS & 0x200"

    savepos TMP MEMORY_FILE
    math NAME_OFF += NAMES_OFF
    goto NAME_OFF MEMORY_FILE
    if NAME_FLAGS1 != 0
        get NAME2_OFF short MEMORY_FILE
        xmath NAME2_SIZE "(NAME2_OFF & 0x1F) + 4"
        xmath NAME2_VAR "(NAME2_OFF >> 8)"
        math NAME2_OFF & 0xE0
        math NAME2_OFF >> 6
        math NAME2_OFF << 8
        math NAME2_OFF | NAME2_VAR
        get NAME2 string MEMORY_FILE
        math NAME_OFF -= NAME2_OFF
        goto NAME_OFF MEMORY_FILE
        getdstring NAME NAME2_SIZE MEMORY_FILE
        string NAME -= STR_SUB
        string NAME += NAME2
    else
        get NAME string MEMORY_FILE
    endif
    goto TMP MEMORY_FILE
    set NAME3 string ""
    putarray 1 FOLDER_DEPTH NAME
    for a = 0 < FOLDER_DEPTH
        getarray NAME2 1 a
        string NAME3 += NAME2
    next a
    getarray EXT 0 NAME_EXT
    string NAME += EXT
    string NAME3 += NAME

    if FINAL == 1024
            set FLDR string "data/"
            string FLDR += NAME3
                #print "%COMPRESSED% %FLDR%"
                if COMPRESSED == 512
                    string FLDR += "packed"
                endif
            #print "%FLDR%"
            getvarchr TMP FLDR 0 long
            math TMP ^= 0xffffffff
            putvarchr FLDR 0 TMP long
            encryption crc "" 32
            string FLDR E= FLDR
            encryption "" ""
            for b = 0 < TBL_COUNT
                getarray HASH 2 b
                if HASH == QUICKBMS_CRC
                    getarray TBL_OFF 3 b
                    getarray TBL_SIZE 4 b
                    log MEMORY_FILE2 TBL_OFF TBL_SIZE 1
                endif
            next b
    endif


    if NAME_EXT != 0
         if ZSIZE == SIZE
             log NAME3 OFFSET SIZE MEMORY_FILE2
         else
             CLOG NAME3 OFFSET ZSIZE SIZE MEMORY_FILE2
         endif
    endif


next i