Star Wars The Force Unleashed - Z files compression

[q-k]

Hi folks,

as I haven't seen a part of the forum to introduce yourself, I'm going to take my first post as a chance to do so.
I'm highly interested in reverse engineering/research for quite some time now and things seem to go well most of the time.
I do have some motorola 68000 assembler background (and you might have guessed right, I'm a little bit older than 20 by now), so reading PPC menmonics is working for me ( most of the time ) somehow.

I'd also like to thank Luigi for setting up this forum, I'm somehow too cheap to pay for xentax

Right now, I'm somewhat stuck with some files that are allegedly compressed with an algo I've not seen up to now. I must admit, I'm too lazy to try every decompression/inflate algorithm on the data that quickbms has to offer. Not from the beginning on, but after trying 30 algos I've given up..
Maybe someone will recognize the stuff I'm looking at and will point me to the right direction.

I've extracted the files from gamedata.wad of the PS3 Game "Star Wars - The Force Unleashed" (BLES-00262).
The file names highly suggest a compression (their extension is .Z). But, I'm quite confident that zlib (like unix/linux uncompress uses) isn't involved in these files at all..

Some examples of the alleged compressed data streams (this is headlerss data, I've taken out alleged uncompressed size, stream size and a further 4 bytes from which I highly suspect them to be a CRC)... The following data is to be found in the files at offset 0x000C:

from a very short file:

Code: Select all

00 00 10 44 40 C3 AD 90 91 96 91 BB 9A 99 8C C1   ...D@Ã..‘–‘»š™ŒÁ
F2 78 00 F5 F2 F5 DF 00 C3 BE 8B 8B 8D 96 9D B8   òx.õòõß.þ‹‹.–.¸
00 00 8D 90 8A 8F DF 96 9B C2 DD BE AB AB AD B6   ....Š.ß–›Âݾ««.¶

a larger file (it's interesting that the stream's are quite similar here up to a certain offset):

Code: Select all

00 00 10 44 40 C3 AD 90 91 96 91 BB 9A 99 8C C1   ...D@Ã..‘–‘»š™ŒÁ
F2 00 00 F5 F2 F5 DF DF C3 AC 86 92 9D 90 93 8C   ò..õòõßß솒..“Œ
DF 91 9E 00 00 92 9A C2 DD 99 90 8D 9C 9A 8F 90   ß‘ž..’šÂÝ™..œš..

another short file (notice the C3AD and following pattern from the previous files?):

Code: Select all

00 00 C3 AD 90 91 96 91 BB 9A 99 8C C1 F2 F5 F2   ..Ã..‘–‘»š™ŒÁòõò
F5 00 00 DF DF C3 BE 8B 8B 8D 96 9D B8 8D 90 8A   õ..ßßþ‹‹.–.¸..Š
8F DF 96 00 00 9B C2 DD BE AB AB AD B6 BD A0 B8   .ß–..›Âݾ««.¶½ ¸

another large file: (the 0xC0 @ 0x01 is giving me a headache on this one):

Code: Select all

00 C0 C3 AD 90 91 96 91 BB 9A 99 8C C1 F2 F5 DF   .ÀÃ..‘–‘»š™ŒÁòõß
0F 00 00 05 C3 AC 86 92 9D 90 93 8C DF 00 00 91   ....솒..“Œß..‘
9E 92 9A C2 DD B2 9E 8B 97 B9 90 8D 92 8A 93 E0   ž’šÂݲž‹—¹..’Š“à

the largest file I've found (56 KB in alleged compressed size):

Code: Select all

00 00 C3 AD 90 91 96 91 BB 9A 99 8C C1 F2 F5 F2   ..Ã..‘–‘»š™ŒÁòõò
F5 00 00 DF DF C3 AC 86 92 9D 90 93 8C DF 91 9E   õ..ßß솒..“Œß‘ž
92 9A C2 00 F0 DD 9B 9E 92 9E 98 9A AB 86 8F 9A   ’šÂ.ðÝ›ž’ž˜š«†.š

I'd highly appreciate any help/suggestions about this data. I might provide more data, but due to law not in public

Thanks, q-k

[aluigi]

Post moved in the appropriate section in its own new topic, so anyone can look at it.

The files are quite small as far as I can see, can you upload a couple of samples?
Currently there are no restrictions on the uploads and links.

There is no section in the forum where you can introduce yourself, just open a new thread in the section that is closer to the topic of the issue

[q-k]

Thanks for moving my thread into the proper section. I'm fully aware of the fact that I was somehow "abducting" the tutorial thread for my needs. Shame on me

I'm attaching 4 full extracted files. I expect them to be real XML files (ascii text data).. Things I'm assuming on them for now are:

Offset:
0x0000 -> 0x0003: alleged uncompressed size
0x0004 -> 0x0007: stream/data size
0x0008 stream/data start, consisting of:
0x0008 -> 0x000B: alleged crc32 of the file name (internal game handling with an alternate CRC32 method, I somehow assume it's safe to ignore it but who knows)
0x000C -> ...: alleged compressed data
EOF-4:
0xFFFFFFFF stream/data end marker

And thanks for anyone having a look at the files

*edited* some typos..

*edit2* PS: I haven't looked into the eboot of the game at all for now, extracting the WAD and the LevelPack files went totally fine without doing so. Chances are, I might have missed hints about that specific compression by doing so. I had hope that someone would recognize this data by just looking at it

[aluigi]

No luck with my scanner, I tried with the whole Z file, from offset 0xc and from 0x10 but no xml result.

[q-k]

Thanks for having a look at the files. I'm going to dig around inside the eboot in the near future, I'll report back if anything turns out.