EA Games [SKATE 3] XSF File Archive Unpacking Script [REQUEST]

[GHFear]

In The Skate games there are file packages called XSF files with 3D models, Collision data and Textures used for asset streaming.
If you have a minute to look at the format and see if there is anything you can do to get the files out of there intact, I would be very grateful

Here is a couple samples: https://mega.nz/#!K0gEjQzQ!DjvdfpVugiwJ ... yT-LqkMJXM

Many Thanks,
/GHFear

[aluigi]

http://aluigi.org/bms/skate3_xsf.bms

[GHFear]

http://aluigi.org/bms/skate3_xsf.bms

Thank you aluigi, you have no idea how much it means. =] Getting the level textures for this game is getting so close right now thanks to you!
The files seem to start a bit before they should and end a bit before they should tho. The header says the texture data should be 30 000 long and they are only about 26 000 - 29 000.
If you need more samples to look at, I can provide that?

Edit: It seem like the files are split up in the archives... It looks to me like the rest of the texture files are inside the cPres files haha.
Maybe that line at the end of each file is a reference to the other files?

Edit 2: So the files all have their own header with their own size... and the header says the data will be one length, yet the files are always WAY shorter and doesn't look like the normal rx2/ renderware4 files with the exact same headers. hmm...
Since the data looks so different, maybe it's compressed or something?

/GHFear

[aluigi]

Regarding the beginning and ending point I'm sure that it's all correct because the 0x80 bytes before the file are the header which is just 0x14 bytes plus all zeroes.
Coverage is 99% so it's all correct.

The extracted files have a 0x50 bytes header followed by 0x89 "RW4xb2" that seems to be a known texture format if you search it on Google.
I don't think there is any compression because there are various 32bit fields visible in these files, if they were compressed they would be a mess and in any case there is no decompressed size as far as I can see.

[GHFear]

Regarding the beginning and ending point I'm sure that it's all correct because the 0x80 bytes before the file are the header which is just 0x14 bytes plus all zeroes.
Coverage is 99% so it's all correct.

The extracted files have a 0x50 bytes header followed by 0x89 "RW4xb2" that seems to be a known texture format if you search it on Google.
I don't think there is any compression because there are various 32bit fields visible in these files, if they were compressed they would be a mess and in any case there is no decompressed size as far as I can see.

Yeah it's strange. I recently made a universal RW4xb2 texture exporter that works with a ton of EA games, but these files even tho the header looks 100% the same as the other files, it comes out as complete clutter with no visible patterns or colors at all. I can't even think of what it could be doing it. Looking at the files your script produces and looking at the same data inside the package, it ends where it should. So thanks a ton for making the process to get these textures WAY easier. I will work on the textures from here and see if I can find where the rest of the file (probably the mipmaps) are, how to combine them and such.

Thanks for everything aluigi
/GHFear

[aluigi]

Script 0.2

[GHFear]

Script 0.2

Haha omg, now it only gives me headers and nothing else xD
But now they start where the texture starts at least. So that's awesome!

[aluigi]

It worked with cSim* but yes you are right.
Script 0.3, please note that the names of the output files differ than those of the previous scripts (in case you want to use it with the previous output folder)

[GHFear]

It worked with cSim* but yes you are right.
Script 0.3, please note that the names of the output files differ than those of the previous scripts (in case you want to use it with the previous output folder)

Holy shit dude! It *SPAM* works! I love you dude. The textures show up just fine.
Make it name them rx2 instead of rw4 tho. :] I already changed that on my end, but for people who download it in the future it would be easier since that's what the xbox extension is for it. This is great. I almost have a rx2 -> OBJ script made now too. ^^ Finally this game can have it's 3D models and Textures preserved.

Thank you.

[GHFear]

btw aluigi, What compression does the file data have?

[aluigi]

RefPack algorithm (in quickbms it's used as dk2 or ea).
It's easy to recognize because the first 2 bytes are 0x10 0xfb.

[GHFear]

RefPack algorithm (in quickbms it's used as dk2 or ea).
It's easy to recognize because the first 2 bytes are 0x10 0xfb.

So after google'ing around for the last couple hours I can't find a tool that can compress with that algorithm. Do you know if there is such a tool publicly available?
Since I believe I need to compress the data again if I wanted to reimport the files =]

OK, so I found this on here posted by you:

Code: Select all

comtype dk2_compress
get SIZE asize
clog "new_file" 0 SIZE SIZE

I tried it and the file is ALMOST identical, there are some differences and the file is a bit longer, would there be another way to do it that could produce the same compression?

Edit: I made this compression script for those that would like to play around with it and find ways to make it work. File is attached.

/GHFear

[aluigi]

For compression I use qfs_compress_data from src\included\qfs.c:

Code: Select all

// New generation FSH/QFS decompressor/compressor
// Version 1.22 - copyright (c) Denis Auroux 1998-2002
// auroux@math.polytechnique.fr

[GHFear]

For compression I use qfs_compress_data from src\included\qfs.c:

Code: Select all

// New generation FSH/QFS decompressor/compressor
// Version 1.22 - copyright (c) Denis Auroux 1998-2002
// auroux@math.polytechnique.fr

Aah that may explain why it's slightly different. The first 20% is 100% the same but then it starts looking a bit different, which is very odd.
I have the Xbox 360 SDK and there is source for the Texture bundler, compressor and UnBundler and such in there, so I will look at that an see if it's something I can use to compress them with the absolute same format. There was also some Skate 3 source released that could be of help.

Edit:Maybe changing the QFS_MAXITER value could help this situation out? is that something I can do on my end?

Edit 2: If anyone knows how to reach the QFS_MAXITER parameter and change the value to test to see if you can make the files look identical or smaller in files size, I have the Decompressed file, the file with original compression and the file compressed and built with QuickBMS as samples.
I also attached the script I made for compressing Skate 2 and 3 rx2 files so you can use it for testing the changed levels of compression.

Thanks.

[aluigi]

MAXITER has been already edited because the original value gave ever worst results.
I guess it's just the implementation of that compression function that sucks.

[GHFear]

So, a follow up on this topic. The script exports 99% of the files correctly, but there are a few select ones that don't seem to have the same compression in the same file container. I will attach one of the files that did not get decompressed and seem to have another compression.
Maybe you can see what kind compression it is.

Edit: Scrap that ^ So it seems no important files come out wrong or compressed, But to make it easier to find what textures go to which model, do you think you could make it output the full name? The full name look something like this "0x2c70170a000d00e4". Right now it gives half the name or no part of the name maybe even, but it seems A LOT of files have the same name because of that, so it has to rename files and it makes it impossible to find what textures go on what 3d models when I have over 100 000 textures to choose from haha.

Edit2: The file name is at offset 1AC in the header of each file and is always 0x12 hex or 18 dec long.

Many thanks,
GHFear

[aluigi]

Edit2: The file name is at offset 1AC in the header of each file and is always 0x12 hex or 18 dec long.

I don't see any name in the sample you uploaded.
At offset 0x1ac there are only the bytes 00 02 00 e8

[GHFear]

Edit2: The file name is at offset 1AC in the header of each file and is always 0x12 hex or 18 dec long.

I don't see any name in the sample you uploaded.
At offset 0x1ac there are only the bytes 00 02 00 e8

Yeah that is just the file that didn't get decompressed. But I am not sure what it is since it seems like everything is there.
But yeah I made my own script real quick that puts textures in one folder and models in the other and renames them to the string in the file.
All I need to know now is how too append two string together.

All I am doing is this:

Code: Select all

# SKATE 3 RX2 RENAMER (script 0.1)

endian big
goto 1
idstring "RW4xb2"
get SIZE asize
goto 0x1AC
get NAME string
log NAME 0 SIZE

What should I add to make it say .rx2 at the end.

Thanks,
GHFear

[aluigi]

All I need to know now is how too append two string together.

string NAME + ".rx2"

[GHFear]

All I need to know now is how too append two string together.

string NAME + ".rx2"

Thanks, I figured it out pretty quick when I figured out you had documentation on QuickBMS in the text file in the QuickBMS download. After finding that it was soooo much easier to understand everything. Been making one script a day recently, to get some new knowledge about quickbms. Great tool man. It's so intuitive to write scripts for.