Mass Effect Andromeda

[swinei]

I got the MEA EA and want to extract the files from it. This is the first time messing around with Frostbite 3 for me and I'm having trouble.

During my research I found the following helpful resources:

• http://aluigi.altervista.org/bms/frostbite.bms
• https://github.com/mitsuhiko/frostbite2-stuff
• https://github.com/GreyDynamics/Frostbite3_Editor

Questions:

• .toc files, wtf? What are they used for? The .sb files look more like a table of content. Are the .sb files enough to unpack?
• .toc encryption? In other games if the file is encrypted it starts with x00D1CE. In MAE it doesn't but always starts with x??D1 and ends with
  

  Code: Select all

  B9 73 E9 D4 88 43 59 08 A5 9F 30 6A E4 D2 79 94
  B2 BB D1 7F 75 F4 D0 D1 01 00 00 40 65 21 61 64
  6E 58 64 24 5E 21 72 66 4F 73 72 44 79 49 72 49
  21 78 56 67 48 65 41 21 36 56 63

  are they encrypted? The D1 makes me think so.
• How does variable4 work? It says in the manual it's a Frostbite specific type.

I've uploaded a sample of .toc/.sb pairs here and a sample of .cas/.cat files here.

Any help is much appreciated

[aluigi]

These toc files look obfuscated yes.
Two weird things: the second byte is ever 0xd1 even if the others are ever different and that 0xd0 0xd1 0x01 0x00 0x00 "@e!adnXd$^!rfOsrDyIrI!xVgHeA!6Vc" at the end.

To know how the data is read in variable4 take a look at src\file.c searching "variable4":

Code: Select all

            i = 0;
            do {
                c = fgetxx(fdnum, 1, error);
                if(*error) break;
                retn |= ((c & 0x7f) << i);
                i += 7;
            } while(c & 0x80);

[apollon08170]

Hey i found this but don't work for me with extraction files =), if help you =)

http://files.1337upload.net/BF4_Python_ ... 18017e.rar

[apollon08170]

When i execute bf4dumper.py with files in path

Code: Select all

#This script runs through all toc files it can find and uses that information to extract the files to a target directory.
#Often the assets are actually stored in cascat archives (the sbtoc knows where to search in the cascat), which is taken care of too.
#The script does not overwrite existing files (mainly because 10 sbtocs pointing at the same asset in the cascat would make the extraction time unbearable).

#Adjust paths here.

bf4Directory    = r"C:\Users\Apollon\Desktop\EXTRACTION\DATA"
targetDirectory = r"C:\Users\Apollon\Desktop\EXTRACTION\extract" #do yourself a favor and don't dump into the Users folder (or it might complain about permission) 


###The following paths do not require adjustments (unless the devs decided to rename their folders). Note that they are relative to bf4Directory.

#As files are not overwritten, the patched files need to be extracted first.
#The script will dump all tocs it can find in these two folders+subfolders:
tocRoot  = r"Update" #patched and xpack files FIRST
tocRoot2 = r"Data"   #unpatched vanilla files SECOND

#Note: The "Update" tocRoot contains both patch (for vanilla AND xpack) and unpatched xpack files. The reason it still
#      works correctly is because it goes through the folders alphabetically, so the patch comes first.


#Feel free to comment out one or both cats if they don't exist (some Frostbite 2 games shipped without cats).
#Although in that case you could just as well use an invalid path, i.e. not change anything.
catPath       = r"\cas.cat" 
updateCatPath = r"\cas.cat"

.........
.....
...

The cas.cat file is "not found" ?

Code: Select all

Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> 
========= RESTART: C:\Users\Apollon\Desktop\EXTRACTION\bf4dumper.py =========
Unpatched cat not found.
Patched cat not found.
>>> 

Do you have an idea ?

[xrb936]

I tried to use the script for FIFA17, no good. Hope someone can find why!

[AnonBaiter]

I have never expected Bioware to get their hands on Frostbite 3 AGAIN.

If you want to dump just the .cas/.cat files then use my modified script. I want to know if it works on this "game" after you`ve done using it.

[Snowpiercer]

I have never expected Bioware to get their hands on Frostbite 3 AGAIN.

If you want to dump just the .cas/.cat files then use my modified script. I want to know if it works on this "game" after you`ve done using it.

Doesn't work for me, tries to open "cas_-446005142.cas" after pointing the script at a cas.cat file when the files are named cas_01.cas, cas_02.cas etc.

Code: Select all

- enter in folder C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage
  coverage file 0     0%   0          8637248    . offset 00000000
- open input file C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas.cat
- enter in folder C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage
- open input file C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas_-446005142.cas

- error in src\file.c line 247: fdnum_open()
Error: No such file or directory

Last script line before the error or that produced the error:
  429 open FDSE TMP 1

Press ENTER or close the window to quit

[AnonBaiter]

Oh god.

Try to run quickbms through command-line and use -0 -V with the script, then the file in which you want to extract and post the results here after you`re done. After all I need to fix this even if I refuse to get the game by myself.

[Snowpiercer]

Oh god.

Try to run quickbms through command-line and use -0 -V with the script, then the file in which you want to extract and post the results here after you`re done. After all I need to fix this even if I refuse to get the game by myself.

Code: Select all

C:\Users\Snowpiercer\Documents\QuickBMS>quickbms -0 -V "C:\Users\Snowpiercer\Documents\QuickBMS\Scripts\frostbite_mea.bms" "C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas.cat" "C:\Users\Snowpiercer\Documents\MEA_Extract"

QuickBMS generic files extractor and reimporter 0.8.0
by Luigi Auriemma
e-mail: me@aluigi.org
web:    aluigi.org
        (Mar 12 2017 - 15:03:46)

                   quickbms.aluigi.org  Homepage
                            zenhax.com  ZenHAX Forum
                               @zenhax  Twitter & Scripts

- current_folder: C:\Users\Snowpiercer\Documents\QuickBMS
- bms_folder:     C:\Users\Snowpiercer\Documents\QuickBMS\Scripts
- exe_folder:     C:\Users\Snowpiercer\Documents\QuickBMS
- file_folder:    C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage
- output_folder:  C:\Users\Snowpiercer\Documents\MEA_Extract
- temp_folder:    C:\Users\Snowpiercer\AppData\Local\Temp\
- open input file C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas.cat
- open script C:\Users\Snowpiercer\Documents\QuickBMS\Scripts\frostbite_mea.bms
- set output folder C:\Users\Snowpiercer\Documents\MEA_Extract

  offset   filesize   filename
--------------------------------------
. 00000000 get     EXT        "cat" -1002
.
- enter in folder C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage
  coverage file 0     0%   0          8637248    . offset 00000000
- open input file C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas.cat
. 0000022c idstr              "NyanNyanNyanNyan" 16
    4e 79 61 6e 4e 79 61 6e 4e 79 61 6e 4e 79 61 6e   NyanNyanNyanNyan
. 0000023c get     FILES      0x0003a923 4
. 00000240 getdstr ZERO       "" 20
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
    00 00 00 00                                       ....
. 00000244 getdstr HASH       "" 20
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
    00 64 1f 50                                       .d.P
. 00000258 get     FOFFSET    0xd288a5cd 4
. 0000025c get     FSIZE      0xf45b90c6 4
. 00000260 get     ZERO       0x2753e5e2 4
. 00000264 get     PACK_NUM   0xe56a806a 4
. 00000268 get     PACK_NAME  "cas" -1001
- enter in folder C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage
- open input file C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas_-446005142.cas

- error in src\file.c line 247: fdnum_open()
Error: No such file or directory

Last script line before the error or that produced the error:
  429 open FDSE TMP 1

[xrb936]

Oh god.

Try to run quickbms through command-line and use -0 -V with the script, then the file in which you want to extract and post the results here after you`re done. After all I need to fix this even if I refuse to get the game by myself.

Code: Select all

- open input file I:\Origin\Mass Effect Andromeda\Data\Win32\loc\en.toc
- open script C:\Users\Robin-PC\Desktop\quickbms\frostbite.bms
- set output folder O:\New folder

  offset   filesize   filename
--------------------------------------
- enter in folder I:\Origin\Mass Effect Andromeda\Data\Win32\loc
  coverage file 0     0%   0          115857     . offset 00000000
- open input file I:\Origin\Mass Effect Andromeda\Data\Win32\loc\en.TOC

- signature of 3 bytes at offset 0x00000000 doesn't match the one
  expected by the script:

  this one: "���"
  fd d1 e5                                          ...

  expected: ""
  00 d1 ce                                          ...

- 0 files found in 0 seconds
  coverage file 0     0%   3          115857     . offset 00000003

Press ENTER or close the window to quit

Here is what I got.

[AnonBaiter]

I`m working on it. Don`t count on me for .sb/.toc files though.

EDIT: Okay, on the frostbite.bms script, replace this line:

Code: Select all

   if ZERO != "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
      goto -16 0 SEEK_CUR
   endif

With this line:

Code: Select all

   if ZERO != "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
      goto -16 0 SEEK_CUR
   else
      goto 0 0 SEEK_CUR
   endif

See if it works.

[Snowpiercer]

I`m working on it. Don`t count on me for .sb/.toc files though.

EDIT: Okay, on the frostbite.bms script, replace this line:

Code: Select all

   if ZERO != "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
      goto -16 0 SEEK_CUR
   endif

With this line:

Code: Select all

   if ZERO != "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
      goto -16 0 SEEK_CUR
   else
      # do nothing
   endif

See if it works.

Getting the exact same output as before.

[swinei]

These toc files look obfuscated yes.
Two weird things: the second byte is ever 0xd1 even if the others are ever different and that 0xd0 0xd1 0x01 0x00 0x00 "@e!adnXd$^!rfOsrDyIrI!xVgHeA!6Vc" at the end.

To know how the data is read in variable4 take a look at src\file.c searching "variable4":

Code: Select all

            i = 0;
            do {
                c = fgetxx(fdnum, 1, error);
                if(*error) break;
                retn |= ((c & 0x7f) << i);
                i += 7;
            } while(c & 0x80);

Thanks, how can I deobfuscate the .toc files? And can you tell me what is in them that isn't in the .sb files?

Hey i found this but don't work for me with extraction files =), if help you =)

http://files.1337upload.net/BF4_Python_ ... 18017e.rar

I checked that out thank you.

I have never expected Bioware to get their hands on Frostbite 3 AGAIN.

If you want to dump just the .cas/.cat files then use my modified script. I want to know if it works on this "game" after you`ve done using it.

Can you tell me how to save the bundles? What's up with ebx, dbx, res, chunk, bmm?
And can you tell me what's usually in the .toc files? And why is there so many duplicates?

I can read cas.cat files and .sb files with the attached project, it's a VS C# program. Currently working on writing the bundles to disk.

[AnonBaiter]

Can you tell me how to save the bundles? What's up with ebx, dbx, res, chunk, bmm?
And can you tell me what's usually in the .toc files? And why is there so many duplicates?

What? I don't have the game, and I won't intend to have it anytime soon. You don't want to know why.

[swinei]

What? I don't have the game, and I won't intend to have it anytime soon. You don't want to know why.

Now I'm really curios why not?

You seem to know the Frostbite engine that's why I ask. I don't have any other Frostbite games to look at. I can also upload more sample files if you need anything specific.

[AnonBaiter]

You seem to know the Frostbite engine that's why I ask. I don't have any other Frostbite games to look at.

Well, to be honest though I'm just a beginner on this stuff.

I can also upload more sample files if you need anything specific.

Thanks for the offer, but I already have a Frostbite 3 game on my HDD. Which is enough since my Frostbite "research" is limited to games made by EA DICE with that engine. That habit of mine might change sometime though, just not today.

[xrb936]

You seem to know the Frostbite engine that's why I ask. I don't have any other Frostbite games to look at.

Well, to be honest though I'm just a beginner on this stuff.

I can also upload more sample files if you need anything specific.

Thanks for the offer, but I already have a Frostbite 3 game on my HDD. Which is enough since my Frostbite "research" is limited to games made by EA DICE with that engine. That habit of mine might change sometime though, just not today.

So is there any possible to extract files from toc and sb files?

[AnonBaiter]

No ideas at the moment.

[aluigi]

if ZERO != "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"[/code]See if it works.

That can't work because you can't use C escapes in conditions.
Strings containing escapes are allowed only in those few commands where it's written "cstring" in quickbms.txt (unfortunately this is an "heritage" of the original bms language).

If TOC is encrypted there is no way to extract data from it using the current script.

[swinei]

The CAS entries are compressed with Zstandard. Yay comptype scanner
compression type = 0F 70

The resId for textures is 1809719482

So now I have the res entry with the DDS header info and the chunk entry with the texture data.

The DDS header info has the 16 byte chunk id at position 32.
The last 16 bytes is maybe the name, but multiple entries can have the same name hm.
Width and height shorts start at 22 (I think).

I've attached an example pair, any help figuring out the header would be much appreciated!

edit: I started on a 010 template with info from here and here

Code: Select all

//------------------------------------------------
//--- 010 Editor v7.0.2 Binary Template
//
//      File: 
//   Authors: 
//   Version: 
//   Purpose: 
//  Category: 
// File Mask: 
//  ID Bytes: 
//   History: 
//------------------------------------------------

int mipOneEndOffset;
int mipTwoEndOffset;
int textureType;
int pixelFormat;
short unknown;

short depth;
short sliceCount;

short width;
short height;

byte numSizes;
byte firstMip;

int dummy;

char chunkID[16];

local int i;
for(i=0;i<15;i++){
    int mipSize;
}

int chunkSize;

byte dummyy[16];

int nameHash;
char name[16];

Well, to be honest though I'm just a beginner on this stuff.

Ha me too

Thanks for the offer, but I already have a Frostbite 3 game on my HDD. Which is enough since my Frostbite "research" is limited to games made by EA DICE with that engine. That habit of mine might change sometime though, just not today.

Could you upload some decrypted .toc files please? I wanna know whats in them