LEGO Star Wars II: The Original Trilogy (NDS) *.FIB

[Mygoshi]

Hey everyone,

I'm trying to extract the *.FIB archive from LEGO Star Wars II (Nintendo DS version).

To my knowledge there are no ways to do this ATM so I felt like I could look for help here.

Samples: (expired)

Thanks in advance!

[Mygoshi]

To add more information to my request, I noticed most of the DS games use a SoundData.sdat and in these cases it's easy to output playable music, not for this case.

[Mygoshi]

My method (for the music) is to use VGMToolbox, and I simply drag and drop the NDS rom into the window which will produce automatically for me playable *.2SF files, decoded with 2SF decoder.

[Mygoshi]

Now this case is a little bit strange because I've never seen something like StarWars.fib, and even if what it looks like is not important, even the content of it seems very different.

[aluigi]

I already checked it and I pass.

[m64m]

Aluigi, would you be willing to give the format another try? I've found two titles that have a .CSV file alongside the .FIB, which contains the path and some extra information for each file within the .FIB archive. Here is the .CSV file from Harry Potter Years 5-7 (3DS): http://puu.sh/dh7Yi/1f93a0e282.csv

If you're up for it, I'll PM you a link to the FIB.

[aluigi]

Sorry but I don't get the format of this archive.
I suspect it's obfuscated.

[m64m]

Ok, thanks for trying!

[FireFly]

I have been working on reversing this format (FUSE) recently for another person, and although I seem to have figured out the compression, I still can't piece it all together and get data that I expect out of it.

The overall structure of FUSE is: there's a header { char magic[8] = "FUSE1.00"; u32 num_files; u32 zero1 = 0; u32 filesize; }. This is followed by compressed data for the first file, followed by compressed data for the second file, etc for num_files files.

Each compressed data starts with u32 compressed_size | 0x40000000, followed by the actual data. The compression is very similar to LZ. Triples (to_copy, then_copy, offset) are encoded in a variable-length way:

Code: Select all

    Bytes                                 to_copy  then_copy            offset
  -----------------------------------------------------------------------------
    0yyyxxzz zzzzzzzz                      x        y + 3                z + 1
    10yyyyyy xxzzzzzz zzzzzzzz             x        y + 4                z + 1
    110xxyyy zzzzzzzz zzzzzzzz yyyyyyyy    x        (y1 << 7) + y2 + 5   z + 1
    111xxxxx                               (x+1)*4  0                    0
    (0xFD)                                 3        0                    0
    (0xFE)                                 2        0                    0
    (0xFF)                                 1        0                    0

Decompressing this way seems to work perfectly with text files in the archives I've tried (using the accompanying CSV to derive filenames, offset and decompressed filesize). However, as I mentioned I've stumbled upon some issues.

Some files seem to not be compressed at all, but instead just stored verbatim. The weird thing is that in this case there doesn't seem to be any reasonable indicator of this, either in the CSV or the actual FUSE archive, and the "compressed" filesize (which had a 0x40 high byte in the compressed case) seems to be completely absent. Otherwise, if the filesize was still present I'd think that the high bits just act as a bitfield to store info such as whether the file is compressed or not. Non-compressed files include very big files with quite a lot of repetition, so it seems like compression would've made sense. It also includes very small files, and some in-between. There's relatively few uncompressed files.

Next is the main problem I've been stumped by: although text files decompress just fine, many kinds of binary files (png and tga images as well as xls spreadsheets, judging by filenames) seem to look nothing like what they're supposed to do. Instead, all seem to have a common header format. However, the rest of the data looks structured/not compressed or encrypted at all, so I'm not sure what's going on here. In the case of xls, several strings are visible in plaintext.

[Mygoshi]

Thanks for your research on this.

[FireFly]

Oh, you don't happen to know if the format is documented in detail anywhere? What tool did you use to extract the fib archives?

[Mygoshi]

I already extracted the fib files. The extracted files are instruments for the sequenced music. They're .bin files.

[MrJPGames]

I already extracted the fib files. The extracted files are instruments/Sequenced musics. They're .bin files.

How did you do this? I'm trying to extract a fib file for Lego HP 5-7 for the 3DS which is probably the same as it also has the .cvs and .fib combo (and the games are pretty much the same anyway)

[AlphaTwentyThree]

Yes, this is a necro-post, BUT: I found two games for the 3DS which have a csv attached to the archive with the TOC! I hope this helps to finally reverse the format!
Here's a sample pair: http://www80.zippyshare.com/v/UXIaJ2rk/file.html
Maybe somebody could first write a script that works with the csv and then try to do the same with the archive but without the CSV? That would be amazing!

[aluigi]

Well done:
http://aluigi.org/bms/fuse_fib.bms

To be honest the format was not difficult so I think the main problem was the compression algorithm (rfpk) that has been implemented in quickbms only in the recent versions.

[AlphaTwentyThree]

Thanks a lot Luigi!
The version with the csv works like a charm.
However, there are some problems without csv:
http://www29.zippyshare.com/v/rM7jyW9i/file.html
http://www29.zippyshare.com/v/nWyvYhVR/file.html
First of all, the script terminates in an error.
Then, take look at file 33d - the file seems to contain several CWAV files which is strange in itself but then if you search for CWAV as a string you can see that the first three headers look correct and the fourth one is gibberish. Maybe there's something wrong with the SIZE/OFFSET pairs when extracting with no names? This could be what causes the crash at the end.

[aluigi]

There are two problems:

1) the files that cause the read error are those without the compression flags, in short in legoharrypotter_ipad.fib these files are NOT compressed and usually just a zero size while in project_silver_hh_3ds.fib they are compressed (even if they have no flag). Even in the same project_silver_hh_3ds.fib compressed and non compressed files are mixed! The result is that I have no way to know if they are compressed or not.

2) I don't know what's the compression algorithm in project_silver_hh_3ds.fib because no algorithm has been identified for the data after the 32bit compressed size field

The work-around I can adopt is just checking if the first 32bit of the non-compressed files (those without flags) is different than 0 and 1, in which case I will consider it a compressed file to dump "as-is" in a folder called "unknown_compression".
Script 0.1.2

Long story short: the script is 100% useless due to the files compressed with unknown compression, those encrypted with unknown encryption and moreover because the non-compressed files use their own format (with f1 ff ff ff or f2 ff ff ff at offset 0x10). What a waste of time...

[AlphaTwentyThree]

Oh man... this really sucks. Sorry for the expression.
The format is so widely spread that an extraction possibility would really have been nice. But alas, it's how it is. Maybe Ekey has the patience to experiment further...
Thanks for all your work nevertheless. :\

[AlphaTwentyThree]

Ok, this may sound odd, but can you modify the script to the point where you dump all data, no matter if it's compressed or not? I'm only after the music which I hope is not compressed.
Sorry to bother you again with this but could you modify the script so it extracts all the contents of this file for example:
http://www42.zippyshare.com/v/lgGlwKRh/file.html
http://www42.zippyshare.com/v/KaN8PQtp/file.html
At least we have a better basis for further investigation then.

[aluigi]

Script 0.1.3, open it and set DISABLE_COMPRESSION to 1