Need Help with figuring out Decompression algorithm for Madden NFL 08 dats

[violaboy13]

First, thank you for this amazing site and the incredible work that's already been done with QuickBMS. In trying to figure out the decompression algorithm for EA Sports Madden NFL 08 I stumbled onto QuickBMS and this forum and have been trying to absorb it as quickly as possible.

I unsuccessfully ran the comtype_scan2.bat on a file that I know is compressed. The good news is that I have the compressed and uncompressed version of the same file and I know at least part of the decompression algorithm from break and tracing on the mainapp.exe. Over at footballidiots.com we have developed an ugly technique by which we use the mainapp.exe to decompress a specific file. However, this is time consuming and clunky. What we need is a BMS script that can decompress these files without going through the mainapp executable. We do not need to recompress the files as our only concern is decompression.

I've included both the compressed and decompressed version of the same file as well as at least part of the decompression algorithm in assembly code using break and trace while debugging from Cheat Engine. Download files here: https://www.sendspace.com/file/r1rsqo

Thanks so much for your time in taking a look at these files and whatever advice you can give me.

[violaboy13]

OK, after doing more research into the mainapp.exe I see now that the decompression algorithm is quite complex and includes calls to many functions. The best way to proceed as far as I can tell is to dump these functions and use calldll from quickbms.

The problem of course is these functions use lots of static addresses. Is there a way to rewrite the assembly code to get around this problem. I'm fairly adept with assembly so I'm just looking to be pointed in the right direction.

Here's an overview of the assembly code of the base function of the decompression algorithm. I tried to add a spoiler tag so I wouldn't clog up space but that apparently doesn't work on this forum.

mainapp.exe+45D390 - push esi
mainapp.exe+45D391 - push 01 { 1 }
mainapp.exe+45D393 - call mainapp.exe+45C260
mainapp.exe+45D398 - add esp,04 { 4 }
mainapp.exe+45D39B - test al,al
mainapp.exe+45D39D - jne mainapp.exe+45D452
mainapp.exe+45D3A3 - push mainapp.exe+AB023C { [06FBEA00] }
mainapp.exe+45D3A8 - call mainapp.exe+31FA90
mainapp.exe+45D3AD - mov eax,mainapp.exe+507EF0 { [00EAF3C8] }
mainapp.exe+45D3B2 - mov [mainapp.exe+AB07B4],mainapp.exe+45C260 { [0085C260] }
mainapp.exe+45D3BC - call mainapp.exe+45C3F0
mainapp.exe+45D3C1 - push mainapp.exe+507EF0 { [00EAF3C8] }
mainapp.exe+45D3C6 - call mainapp.exe+45C470
mainapp.exe+45D3CB - mov eax,mainapp.exe+507EF0 { [00EAF3C8] }
mainapp.exe+45D3D0 - call mainapp.exe+45C500
mainapp.exe+45D3D5 - mov eax,[mainapp.exe+507EF4] { [00EAF948] }
mainapp.exe+45D3DA - xor esi,esi
mainapp.exe+45D3DC - push eax
mainapp.exe+45D3DD - push esi
mainapp.exe+45D3DE - mov [mainapp.exe+AAE758],si { [00000000] }
mainapp.exe+45D3E5 - mov word ptr [mainapp.exe+AAF388],0001 { [00000027] }
mainapp.exe+45D3EE - mov [mainapp.exe+AB0234],eax { [00EAE1D8] }
mainapp.exe+45D3F3 - call mainapp.exe+45C600
mainapp.exe+45D3F8 - mov eax,mainapp.exe+507EFC { [00EAE160] }
mainapp.exe+45D3FD - call mainapp.exe+45C3F0
mainapp.exe+45D402 - push mainapp.exe+507EFC { [00EAE160] }
mainapp.exe+45D407 - call mainapp.exe+45C470
mainapp.exe+45D40C - mov eax,mainapp.exe+507EFC { [00EAE160] }
mainapp.exe+45D411 - call mainapp.exe+45C500
mainapp.exe+45D416 - mov eax,[mainapp.exe+507F00] { [00EAE1D8] }
mainapp.exe+45D41B - push eax
mainapp.exe+45D41C - push esi
mainapp.exe+45D41D - mov [mainapp.exe+AAE758],si { [00000000] }
mainapp.exe+45D424 - mov word ptr [mainapp.exe+AAF388],0001 { [00000027] }
mainapp.exe+45D42D - mov [mainapp.exe+AB0234],eax { [00EAE1D8] }
mainapp.exe+45D432 - call mainapp.exe+45C600
mainapp.exe+45D437 - push mainapp.exe+AB023C { [06FBEA00] }
mainapp.exe+45D43C - call mainapp.exe+31FAA0
mainapp.exe+45D441 - add esp,20 { 32 }
mainapp.exe+45D444 - mov [mainapp.exe+AAE750],mainapp.exe+45DDF0 { [0085DDF0] }
mainapp.exe+45D44E - xor eax,eax
mainapp.exe+45D450 - pop esi
mainapp.exe+45D451 - ret

[aluigi]

Is that "UIS_GRP_ALL.dat" you provided the original file as-is?
That file doesn't have a header with information about the decompressed size or other details, it's a bit unusual.
Dumping the decompression function is a good solution when the function is simple and you correctly identify it

[violaboy13]

Is that "UIS_GRP_ALL.dat" you provided the original file as-is?

No I dumped just the compressed archived file. I've attached the file one level up that has the necessary header info. Basically Madden dats are archived like this:

UIS_GRP_ALL.dat, which I didn't include, contains 30 archived files. Each one of those 30 files contains more archived files most of which are compressed.

The file I just attached to this post is the 4th file in the archive. The file I posted earlier that has no header info is the 35th file of that archive.

So, the header info for the file I posted earlier is found in the file I just attached at 0x128 for file offset and file size and 0x2B8 for decompression size.

Sorry for the confusion.
I know that EA Sports often uses/used custom compression for their game archives. But because of their connection with the NFL and the copyright/trademark issues they seemed to lock these files down tighter than in their other game franchises.

[aluigi]

Yeah now it has more sense.
The first bytes of the compressed files look so weird like if there is a sort of obfuscation that hides the real compressed data.
In the meantime I have added the TERF format to my collection of scripts in case someone solves the problem with compression and for making additional tests, the files are dumped with their decompressed size in the filename:
http://aluigi.org/bms/madden_terf.bms

[aluigi]

P.S.: quite discouraging that searching "madden" and "terf" leads to a topic on xentax of 2005 without a solution about compression.
Discouraging only because it means nobody cared to work on that, not that it's difficult.

[violaboy13]

P.S.: quite discouraging that searching "madden" and "terf" leads to a topic on xentax of 2005 without a solution about compression.
Discouraging only because it means nobody cared to work on that, not that it's difficult.

Part of the reason for that is a member of our Madden Modding Community at Football Idiots (was Football-Freaks) developed an ugly workaround that included loading up a known compressed file in Madden and when Madden crashed you could dump the decompressed output. This is slow and cumbersome process and members on the forum are wondering if there's a better way. Using Cheat Engine I'm the one who knows the most about the inner workings of the mainapp.exe, which has allowed me to inject a ton of my own code to update the NFL rules and change the way the game plays. No NFL football game has been released on PC since Madden 2008 so people keep modding it by updating graphics etc.

[aluigi]

I understand.
The demo of Madden 2004 available on the EA FTP server contains this compression and it's not just a simple standalone function, it's called with blocks of 0x8000 bytes of data like a sort of "stream" mode (sort of prototype func([out,size],[in,zsize]) and it's integrated in the code, so you were right about the tons of constants and additional things.

[aluigi]

The decompression function is ready and works perfectly.
It will be available in quickbms 0.8.2 that I will release probably tomorrow.

[violaboy13]

The decompression function is ready and works perfectly.
It will be available in quickbms 0.8.2 that I will release probably tomorrow.

Seriously?????!!!!!! I thought this decompression was too complex for anyone to crack without significant more time and effort. Thank you aluigi; you sir are amazing!

[aluigi]

quickbms 0.8.2 and the updated madden_terf script are available:
http://quickbms.com

If you need the stand-alone C decompression function, it's available here:
http://aluigi.org/papers/ea_madden.c

[violaboy13]

There is great joy and elation over at footballidiots.com for the decompression script. Thanks for your help; the script works beautifully and the C code you sent me is elegant. I have no idea how long that would have taken me to create something similar. Thanks again aluigi!

[aluigi]

I'm happy of that, finally this mistery is solved after almost 15 years