My first quickBMS script

[Acewell]

i was just about to say "goto 0x40" should probably be outside your loop because you don't want to cycle that over and over

edit: also this file doesn't appear to be compressed so i don't think you need to call a compression type

[aluigi]

Wow lot of posts
I answer in order from the one of Getaway

1)
It's unknown if those sequence of strings are forced to 4 bytes (getdstring TYPE 4) or are just NULL terminated strings (get TYPE string), I may think to the former because fixed size strings are used in other places of the archive

2)
There are situations where you don't have the OFFSET fields or where you don't have the SIZE field.
Your situation is the lack of the SIZE field so you have to collect all the OFFSETs and dumping the files retrieving the SIZE using "NEXT_OFFSET - OFFSET". Currently the solution I like to use for these archvies is: store the info in an array, store the size of the archive in the last array, sortarray (no need to use it, skip it), SIZE = NEXT_OFFSET - OFFSET. Something like this:
http://aluigi.org/bms/winning_eleven_10.bms
0x3300 is the offset of the first file fanta1.ico!
Remember to perform the log operation if SIZE != 0 because the archive stores also the folders (senseless).

[AnonBaiter]

Okay, so I did it like this:

Code: Select all

get DUMMY long
get PKG_SIZE long
math BASE_OFF = 0x40
goto BASE_OFF
get FILES long
for i = 0 < FILES
#   getdstring NAME 0x3C # I don't know where are the correct information of the files so I'll just leave it like that
   get OFFSET long
   math OFFSET + BASE_OFF
   putarray 0 i OFFSET
next i
math PKG_SIZE + BASE_OFF
putarray 0 i PKG_SIZE
for i = 0 < FILES
   getarray OFFSET 0 i
   math i + 1
   getarray SIZE   0 i
   math SIZE - OFFSET
   if SIZE != 0
      log "" OFFSET SIZE
   endif
next i

But when I tested the script to extract the file, it gave me this error:

Code: Select all

. 00000000 getarr  OFFSET     0xc4840040 0:254
. 00000000 getarr  SIZE       0xc4b60040 0:255
  c4840040 3276800    00000000.dat

Error: incomplete input file 0: E:\Fantavision\FANTAVISION_JAPAN\DATA\COMMON.PKG
       Can't read 64 bytes from offset c4840040.
       Anyway don't worry, it's possible that the BMS script has been written
       to exit in this way if it's reached the end of the archive so check it
       or contact its author or verify that all the files have been extracted.
       Please check the following coverage information to know if it's ok.

  coverage file 0     2%   52236      1939296

Last script line before the error or that produced the error:
  22  log "" OFFSET SIZE

Am I doing something wrong?

[aluigi]

Yes, you must read the filename field.

Anyway this is the script:
http://aluigi.org/bms/fantavision.bms

[AnonBaiter]

Well, I should have gotten it right but I wasn't able to. Thanks for the script.

[AnonBaiter]

Here I am again.

I'm trying to write a script that can unpack .spd/.spt pairs used in Wii games, most notable the Table Tennis game that Rockstar developed. Here's the script:

Code: Select all

open FDDE "spt"
open FDDE "spd" 1

goto 0x03
get FILES long
goto 0x0A
get PARTS long
xmath INFO "0x14 * FILES + 0x30"
goto INFO
for i = 0 < FILES
   get PART1 long
   get PART2 long
   getdstring DUMMY 0x1C
   putarray 0 i PART1
   putarray 1 i PART2
next i

for i = 0 < FILES
   getarray PART2 0 i
   get OFFSET1 long
   savepos DAT1
   getdstring H1 0x2E
   savepos MYOFF
   get OFFSET2 long
   savepos DAT2
   xmath SIZE1 "OFFSET2 - OFFSET1"
   getdstring H2 0x2E
   if PART2 != 0xcdcdcdcd
      set CH 2
      savepos MYOFF
   else
      set CH 1
      set OFFSET2 0
      set DAT2 0
      set SIZE2 0
   endif
   xmath n "i + 1"
   if n != FILES
      get SIZE2 long
      goto MYOFF
   else
      get SIZE2 asize 1
   endif
   math SIZE2 -= OFFSET2
   xmath PSIZE "SIZE1 + SIZE2 + CH * 0x2E + 4"
   putvarchr MEMORY_FILE PSIZE 0
   log MEMORY_FILE 0 0
   putvarchr MEMORY_FILE 0 SIZE1 long
   putvarchr MEMORY_FILE 4    CH long
   append
   log MEMORY_FILE DAT1 0x2E
   if CH == 2
      log MEMORY_FILE DAT2 0x2E
   endif
   log MEMORY_FILE OFFSET1 SIZE1 1
   if CH == 2
      long MEMORY_FILE OFFSET2 SIZE2 1
   endif
   append
   get SIZE asize MEMORY_FILE
   get NAME basename
   string NAME p= "%s_%d.dsp" NAME 1
   log NAME 0 SIZE MEMORY_FILE
next i/

Am I doing something wrong with this script? Can any audio enthusiast point me out?

[aluigi]

Why opting for something so difficult?
It contains raw Gamecube adpcm audio with only references about the frequency and weird offset and size:
http://aluigi.org/bms/rockstar_table_tennis.bms

[AnonBaiter]

So I'm trying to reverse-engineer an archive file that has no offset references. However, it does reference something else...
00000190 00 00 00 00 00 00 00 00 01 00 00 00 08 00 00 00 ................
000001A0 23 D9 09 65 6E 67 6C 69 73 68 01 00 00 00 00 00 #Ù.english......

So instead of relying on offset references to extract a file, the best way would be to dump all the files that's available. Would you think of it as a good idea?

Another way I'm thinking of was to detect the filenames from between these spaces:
00000190 00 00 00 00 00 00 00 00 3C 00 00 00 11 62 01 30 ........<....b.0
000001A0 49 00 00 D8 DD F0 00 63 6F 6D 6D 65 6E 74 61 72 I..ØÝð.commentar
000001B0 79 5C 64 75 74 63 68 10 46 00 00 0B 08 84 00 B8 y\dutch.F....„.¸
000001C0 48 00 00 58 D1 E2 00 65 6E 67 6C 69 73 68 10 50 H..XÑâ.english.P
000001D0 00 00 12 04 13 00 BC 48 00 00 D8 F5 E2 00 2E 69 ......¼H..Øõâ..i
000001E0 6E 64 5A 00 00 0B 07 5A 00 0C 49 00 00 D0 20 C5 ndZ....Z..I..Ð Å
000001F0 00 66 72 65 6E 63 68 10 64 00 00 0B 07 44 00 E8 .french.d....D.è
00000200 48 00 00 49 10 96 04 67 65 72 6D 61 6E 10 6E 00 H..I.–.german.n.
00000210 00 0B 08 2E 00 B0 49 00 00 79 43 36 0E 69 74 61 .....°I..yC6.ita
00000220 6C 69 61 6E 10 78 00 00 0B 08 17 00 B8 48 00 00 lian.x......¸H..
00000230 C8 09 C9 0D 73 70 61 6E 69 73 68 10 82 00 00 02 È.É.spanish.‚...
00000240 1C BE 00 2F 4D 00 00 58 5E 30 31 6E 66 69 67 5C .¾./M..X^01nfig\
00000250 61 75 74 6F 72 65 70 6C 61 79 73 74 61 74 65 6D autoreplaystatem
00000260 61 63 68 69 6E 65 03 8C 00 00 07 0A 93 00 10 20 achine.Œ....“..
00000270 0D 00 FF 57 FC 94 69 6E 74 72 6F 6E 74 73 63 01 ..ÿWü”introntsc.
00000280 31 02 00 0C 04 13 00 10 00 0F 00 3F 74 FE 94 70 1..........?tþ”p
00000290 61 6C 01 12 04 00 07 0F 67 00 3E 07 00 00 62 D0 al......g.>...bÐ
000002A0 0B 0D 6C 61 6E 67 75 61 67 65 63 6F 6E 66 69 67 ..languageconfig
000002B0 03 13 04 00 08 0F 31 00 10 C0 12 00 0D 10 64 EC ......1..À....dì
000002C0 6F 61 64 73 63 72 65 65 6E 5F 6E 74 73 63 01 6C oadscreen_ntsc.l
000002D0 06 00 12 04 13 00 10 00 0F 00 A5 F9 B0 04 70 61 ..........¥ù°.pa
000002E0 6C 01 4D 08 00 07 09 18 00 D3 28 00 00 AA 02 E4 l.M......Ó(..ª.ä
000002F0 E2 73 74 61 64 69 75 6D 73 âstadiums

On another archive I've found however, it does have some kind of different method in that the offset references are there but the way they store the filenames are all messy...:
000001B0 74 03 28 00 00 00 1A 89 FE 24 06 01 00 29 30 07 t.(....‰þ$...)0.
000001C0 DB 63 61 72 65 65 72 5C 61 69 72 70 6F 72 74 5C Ûcareer\airport\
000001D0 61 69 72 70 6F 72 74 5F 61 31 01 49 00 00 18 02 airport_a1.I....
000001E0 77 00 24 86 00 00 78 87 10 F1 32 01 5A 00 00 18 w.$†..x‡.ñ2.Z...
000001F0 02 66 00 24 46 00 00 A6 86 D7 46 33 01 63 00 00 .f.$F..¦†×F3.c..
00000200 18 02 55 00 24 06 01 00 5E B8 BA 8C 34 01 84 00 ..U.$...^¸ºŒ4.„.
00000210 00 18 02 44 00 24 16 00 00 79 56 BC CE 35 01 87 ...D.$...yV¼Î5.‡
00000220 00 00 18 02 33 00 24 06 01 00 14 F5 C2 C0 36 01 ....3.$....õÂÀ6.
00000230 A8 00 00 18 02 22 00 24 26 00 00 C0 39 BE 81 37 ¨....".$&..À9¾.7
00000240 01 AD 00 00 18 02 11 00 24 26 00 00 DC 7A B2 15 ........$&..Üz².
00000250 38 01 B2 00 00 17 03 13 01 24 46 00 00 D0 36 4A 8.²......$F..Ð6J
00000260 66 62 31 01 BB 00 00 18 02 11 00 24 86 00 00 4C fb1.»......$†..L
00000270 FC 39 78 32 01 CC 00 00 17 03 F0 00 24 06 01 00 ü9x2.Ì....ð.$...
00000280 0A 57 67 C1 63 31 01 ED 00 00 18 02 66 00 24 06 .WgÁc1.í....f.$.
00000290 01 00 FF 35 DD 2F 32 01 0E 01 00 18 02 55 00 24 ..ÿ5Ý/2......U.$
000002A0 06 01 00 68 68 9A A8 33 01 2F 01 00 18 02 44 00 ...hhš¨3./....D.
000002B0 24 06 01 00 5B 98 04 E3 34 01 50 01 00 18 02 33 $...[˜.ã4.P....3
000002C0 00 24 06 01 00 E3 BF F0 AD 35 01 71 01 00 18 02 .$...ã¿ð.5.q....
000002D0 22 00 24 06 01 00 2C 13 C0 B0 36 01 92 01 00 18 ".$...,.À°6.’...
000002E0 02 11 00 24 86 00 00 E1 58 99 EA 37 01 A3 01 00 ...$†..áX™ê7.£..
000002F0 17 03 78 00 24 06 01 00 92 77 8D 7B 64 31 01 C4 ..x.$...’w.{d1.Ä
00000300 01 00 18 02 66 00 24 06 01 00 AD 22 DD DF 32 01 ....f.$...."Ýß2.
00000310 E5 01 00 18 02 55 00 24 06 01 00 8A E6 4A DE 33 å....U.$...ŠæJÞ3
00000320 01 06 02 00 18 02 44 00 24 06 01 00 B9 D0 D3 09 ......D.$...¹ÐÓ.
00000330 34 01 27 02 00 18 02 33 00 24 06 01 00 DE 72 C4 4.'....3.$...ÞrÄ
00000340 C5 35 01 48 02 00 18 02 22 00 24 06 01 00 AF 5B Å5.H....".$...¯[
00000350 56 79 36 01 69 02 00 18 02 11 00 24 86 00 00 D0 Vy6.i......$†..Ð
00000360 1C EE 81 37 01 7A 02 00 0F 12 8A 00 00 05 00 00 .î.7.z....Š.....
00000370 79 0C 66 FC 70 6C 61 6E 65 30 31 5F 6C 61 79 65 y.füplane01_laye
00000380 72 2E 70 72 69 6D 7B 02 00 15 0C 51 00 40 01 00 r.prim{....Q.@..

[aluigi]

What you have there is a sort of compressed table of filenames (with probably also information about offset and size).
You see commentary/dutch, english, nfig\autoreplaystatemachine which probably uses the "co" of "commentary/dutch".
It's probably compressed with an algorithm or a custom scheme.

[AnonBaiter]

Hmm... do you know what this kind of algorithm is? I'm tempted on running a comtype_scan2 bat file on it just to see if the file really is compressed...

[aluigi]

The comtype scanner works well when you have the exact compressed data (no bytes more or less) and the exact decompression size.
Otherwise it's just a waste of time.

[AnonBaiter]

Well I'm tempted to give it a try...

[AnonBaiter]

So I made this script:

Code: Select all

open FDDE "DIR"
open FDDE "IMG" 1

for
   get UNKNOWN1 long
   get UNKNOWN2 long
   getdstring NAME 0x18
next

And I have this archive file that will serve as the basis for the script:

Code: Select all

00000000  00 00 00 00 14 00 00 00 41 49 2E 63 61 74 00 00  ........AI.cat..
00000010  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000020  14 00 00 00 04 00 00 00 41 49 5F 41 6C 6C 79 2E  ........AI_Ally.
00000030  63 61 74 00 00 00 00 00 00 00 00 00 00 00 00 00  cat.............
00000040  18 00 00 00 04 00 00 00 41 49 5F 41 75 74 68 6F  ........AI_Autho
00000050  72 69 74 79 2E 63 61 74 00 00 00 00 00 00 00 00  rity.cat........
00000060  1C 00 00 00 04 00 00 00 41 49 5F 42 69 6B 65 2E  ........AI_Bike.
00000070  63 61 74 00 00 00 00 00 00 00 00 00 00 00 00 00  cat.............
00000080  20 00 00 00 03 00 00 00 41 49 5F 42 6F 78 65 72   .......AI_Boxer
00000090  2E 63 61 74 00 00 00 00 00 00 00 00 00 00 00 00  .cat............
000000A0  23 00 00 00 02 00 00 00 41 49 5F 43 68 61 64 5F  #.......AI_Chad_
000000B0  33 5F 30 34 2E 63 61 74 00 00 00 00 00 00 00 00  3_04.cat........
000000C0  25 00 00 00 06 00 00 00 41 49 5F 44 41 52 42 59  %.......AI_DARBY
000000D0  5F 32 5F 42 2E 63 61 74 00 00 00 00 00 00 00 00  _2_B.cat........
000000E0  2B 00 00 00 05 00 00 00 41 49 5F 44 6F 67 2E 63  +.......AI_Dog.c
000000F0  61 74 00 00 00 00 00 00 00 00 00 00 00 00 00 00  at..............
00000100  30 00 00 00 0A 00 00 00 41 49 5F 45 44 47 41 52  0.......AI_EDGAR
00000110  5F 35 5F 42 2E 63 61 74 00 00 00 00 00 00 00 00  _5_B.cat........

The problem though is that there's no reference for the total number of the files. What should I do about it?

[chrrox]

from your screenshot looks like the name size should be 0x18 not 0x20
and you can just divide the file size by 0x20 to get a count.
or don't use a count and just do
for
do stuff here
next

[AnonBaiter]

Oh, my mistake. Fixed.
Now let's say that the script has reached its end of the file and receives this error:

Code: Select all

. 000023a0 get     OFFSET     0x000051ba 4
. 000023a4 get     SIZE       0x00000001 4
. 000023a8 getdstr NAME       "yd_ph4.dat" 24
    79 64 5f 70 68 34 2e 64 61 74 00 00 00 00 00 00   yd_ph4.dat......
    00 00 00 00 00 00 00 00                           ........
.

Error: incomplete input file 0: E:\ROCKSTAR_LEL\GRANDTHEFTAUTO_III\PS2_1.40\ANIM\CUTS.DIR
       Can't read 4 bytes from offset 000023c0.
       Anyway don't worry, it's possible that the BMS script has been written
       to exit in this way if it's reached the end of the archive so check it
       or contact its author or verify that all the files have been extracted.
       Please check the following coverage information to know if it's ok.

  coverage file 0   100%   9152       9152

Last script line before the error or that produced the error:
  5   get OFFSET long

What should I do to fix that error, especially when extracting the files?

[aluigi]

For reading an index file till the end without knowing the number of files you have at least two options:

Code: Select all

get INDEX_SIZE asize
for CURR_OFF = 0 < INDEX_SIZE
    ...
    savepos CURR_OFF
next

or

Code: Select all

get INDEX_SIZE asize
for
    savepos TMP
    if TMP >= INDEX_SIZE
        break
    endif
    ...
next

or

Code: Select all

get FILES asize
math FILES / 0x20
for i = 0 < FILES
    ...
next i

[AnonBaiter]

Okay so I did it like this:

Code: Select all

open FDDE "DIR"
open FDDE "IMG" 1

get INDEX_SIZE asize

for CURR_OFF = 0 < INDEX_SIZE
   get ID long
   get SIZE long
   math SIZE * 0x800
   getdstring NAME 0x18
   putarray 0 i ID
   putarray 1 i SIZE
   putarray 2 i NAME
   savepos CURR_OFF
next

The problem though is finding off the offsets...

[aluigi]

ID is the offset

[AnonBaiter]

Okay, now I finished up my script:

Code: Select all

open FDDE "DIR"
open FDDE "IMG" 1

get INDEX_SIZE asize

for CURR_OFF = 0 < INDEX_SIZE
   get OFFSET long
   get SIZE long
   math OFFSET * 0x800
   math SIZE * 0x800
   getdstring NAME 0x18
   putarray 0 i OFFSET
   putarray 1 i SIZE
   putarray 2 i NAME
   log NAME OFFSET SIZE 1
   savepos CURR_OFF
next

This script works on GTA 3D-era games, but it might work with the other games made by Rockstar too. Note that because this script is WIP, it won't work with duplicate files(#_1.IMG, #_2.IMG and so on).

[aluigi]

Do you mean that there are files like NAME.DIR, NAME.IMG and alos NAME_1.IMG and NAME_2.IMG?
If yes then it's possible that one of the fields assigned to the filename is the ID of the archive to use.