FIFA 18 (CAS) Files - ZSTD streams dumping help

[RageX]

Hi, previous FIFA 17 was known to contain ZSTD streams and it was pretty easy to pick them up by their headers. FIFA 18 also seems to have ZSTD streams but at many places the headers are not even complete (modified?) so its hard to pick ZSTD streams and dump them out.

Can anyone please find a solution to dump all the found ZSTD streams in a CAS file of this game ?

Sample file:

Code: Select all

https://www.mediafire.com/file/f5ovyzca66532ys/cas_01.cas

If you want a smaller sample please tell here.

[aluigi]

Can you provide a sample of FIFA17 (maybe the same cas_01.cas if available) so we can check what changed?

[RageX]

Can you provide a sample of FIFA17 (maybe the same cas_01.cas if available) so we can check what changed?

Here:

Code: Select all

http://www4.zippyshare.com/v/mwoIobM3/file.html

[aluigi]

Ok so the format is the one covered by frostbite.bms.
Do you have the "cat" of fifa18?

[RageX]

Ok so the format is the one covered by frostbite.bms.
Do you have the "cat" of fifa18?

Ok, I have attached the cas.cat file associated with that cas_01.cas.
Also here's another set of FIFA 18 samples in case you need it:

Code: Select all

http://www91.zippyshare.com/v/jC5IBBre/file.html

[aluigi]

I bet the problem is the finalSize 32bit field that apparently is a 24bit:

Code: Select all

01 00 04 60 0f 70 00 b0

So 0x460 is read as 0x01000460.

Are you using FB2Dumper.py?
If yes then find the following line:

Code: Select all

finalSize, magic, payloadLen = unpack(">IHH", cas.read(8))

and add this line after it:

Code: Select all

finalSize &= 0x00ffffff

[RageX]

Are you using FB2Dumper.py?

Actually I only want to dump out zstd compressed streams, both in compressed (raw,as-it-is) and decompressed forms. Can you help with it ?

[aluigi]

No, I can't.
First because it's not clear what type of zstd it's used in fifa18 since zstd06 gives just invalid files and then some chunks use oodle or are no compressed at all.

What I did was updating my frostbite script to version 0.3.1, I will upload it in the next hours

[RageX]

No, I can't.
First because it's not clear what type of zstd it's used in fifa18 since zstd06 gives just invalid files and then some chunks use oodle or are no compressed at all.

What I did was updating my frostbite script to version 0.3.1, I will upload it in the next hours

Hmmm maybe thats why in most of the cas files I wasnt even able to detect zstd unlike in FIFA 17 where I could find zstd everywhere, in FIFA 18 along with very few legit zstd chunks, only part of zstd headers were there so maybe they are not even zstd, they were oodle instead like you said.

[aluigi]

What's strange is that these invalid zstd chunks have the same 0x0f70 flag of the good ones, the only difference is that 0x01000000 added to the decompressed chunk size which is just another flag (so the size is 24bit).
Trying all the legacy zstd gave the same invalid results and even scanning all the compression algorithms was the same.

[aluigi]

The new script is available:
http://aluigi.org/bms/frostbite.bms

[deepshit]

ZSTD files in FIFA 18 are using pre defined dictiomary.
The buffer size has changed from 64K to 256K but the compressed size in header is now 24 bits and because of that we see 0x70 , 0x71,0x72,0x73.
The lower bits should be added to compressed size.

And the pre defined ditionary is encrypted in initfs_Win32.

[aluigi]

Interesting.
So first of all we need the dictionary to use when the first 8bit flag is 1.
Anyone?

That thing regarding 0x7* is not very clear, for example here I have "00 04 00 00 00 74 00 00" so I must copy 0x40000 bytes without compression to destination and should I append 4 zeroes to it?

In the meantime there was a typo in my script (FLAGS >= 0 instead of != 0) so I have updated it.

[deepshit]

That thing regarding 0x7* is not very clear, for example here I have "00 04 00 00 00 74 00 00" so I must copy 0x40000 bytes without compression to destination and should I append 4 zeroes to it?

you should copy 0x40000 bytes without compression and you don't need to append anything.
I have decrypted initfs and obtained the dictionary and I have tested it.
Decompression works fine.

If there's no legal concerns I can share it.

[aluigi]

So why you said "The lower bits should be added to compressed size"?
That means 0x40004 based on what you said.

Yeah, share the dict

[deepshit]

So why you said "The lower bits should be added to compressed size"?

What I meant was :
00 04 00 00 15 71 A9 9C => ((71 & 0xF) << 16 ) | 0xA99C => 0x1A99C

Here's the Dictionary.

[aluigi]

Does the game still use zstd 0.6?
Because I still get invalid data using the dictionary

[aluigi]

I answer to myself: no, it uses the new one.
The bad thing is that there is no way to know if it's the old (fifa17) or new one (fifa18)

[aluigi]

Script 0.3.3
http://aluigi.org/bms/frostbite.bms

[deepshit]

The bad thing is that there is no way to know if it's the old (fifa17) or new one (fifa18)

Actually there is.
Fifa17 zstd compression starts with : 0xFD2FB526
Fifa18 zstd compression starts with : 0xFD2FB528

Of course if you mean the compression.