God of War series - WAD files

Extraction and unpacking of game archives and compression, encryption, obfuscation, decoding of unknown files
Wulf
Posts: 49
Joined: Mon Oct 27, 2014 8:30 pm

God of War series - WAD files

Post by Wulf »

Edit: I've worked out the following script to extract data from WADs. It's not perfect and ignores all grouping/HEAP data, but it' most of the way there. Works on GoW1/2 for both PS2 and PS3, and GoWA. Untested with GoW3.

Thanks for the help aluigi.

Note: Skips CAT 0x18 to avoid deciphering HEAP in R_PERM.

Code: Select all

get ARCHIVE_SIZE asize
get DUMMY long
get ENDTEST short

if ENDTEST == 0
    endian big
    math ENDIPAD = 0x20
endif

for OFFSET = 0 < ARCHIVE_SIZE
    math valid = 1

    goto OFFSET
    get CAT short
    get DUMMY short
    get SIZE long
    getdstring NAME 0x18

    math OFFSET += 0x20
    math OFFSET += ENDIPAD

    if NAME == "EntityCount" || CAT == 0x18
        math SIZE = 0
   math valid = 0
    endif

    if valid != 0 && size > 0
        log NAME OFFSET SIZE
    endif

    math OFFSET += SIZE
    math OFFSET x= 0x10
next


List of non-standard WADs not covered by this script:
GoW2 - R_mcicon.wad_ps3
GoW3 - R_NETWORK.WAD
GoWA - R_MULTIPLAYER.WAD



And here's a 010 template I've started cobbling together. Seems pretty successful at parsing the data groups (Only tested against PS2 WADs thus far), but currently makes no attempt at further parsing the files that get exposed.

Code: Select all

//------------------------------------------------
//--- 010 Editor v13.0 Binary Template
//
//      File: God of War, *.WAD
//   Authors:
//   Version:
//   Purpose:
//  Category:
// File Mask:
//  ID Bytes:
//   History:
//------------------------------------------------

typedef struct {
   int16 chunkID <format=hex, hidden=true>;
   int16 unk1 <hidden=true>;
   int size <format=hex, hidden=true>;
   char chunkName[0x18] <hidden=true>;
   if (size > 0)
   {
      char data[size] <hidden=true>;
   }
   pad();
   parse();
} PUSHHW;

typedef struct {
   int16 chunkID <format=hex, hidden=true>;
   int16 unk1 <hidden=true>;
   int size <format=hex, hidden=true>;
   char chunkName[0x18] <hidden=true>;
   pad();   
} POPHW;

typedef struct {
   int16 chunkID <format=hex, hidden=true>;
   int16 unk1 <hidden=true>;
   int size <format=hex, hidden=true>;
   char chunkName[0x18] <hidden=true>;
   if (size > 0)
   {
      char data[size] <hidden=true>;
   }
   pad();
   parse();
} PUSHCONTEXT;

typedef struct {
   int16 chunkID <format=hex, hidden=true>;
   int16 unk1 <hidden=true>;
   int size <format=hex, hidden=true>;
   char chunkName[0x18] <hidden=true>;
   pad();   
} POPCONTEXT;

typedef struct {
   int16 chunkID <format=hex, hidden=true>;
   int16 unk1 <hidden=true>;
   int size <format=hex, hidden=true>;
   char chunkName[0x18] <hidden=true>;
   if (size > 0)
   {
      char data[size] <hidden=true>;
   }
   pad();
} CLIENTPARM;

typedef struct {
   int16 chunkID <format=hex, hidden=true>;
   int16 unk1 <hidden=true>;
   int size <format=hex, hidden=true>;
   char chunkName[0x18] <hidden=true>;
   pad();   
   parse();
} GROUPSTART;

typedef struct {
   int16 chunkID <format=hex, hidden=true>;
   int16 unk1 <hidden=true>;
   int size <format=hex, hidden=true>;
   char chunkName[0x18] <hidden=true>;
   pad();   
} GROUPEND;

typedef struct {
   int16 chunkID <format=hex, hidden=true>;
   int16 unk1 <hidden=true>;
   int val <hidden=true>;
   char chunkName[0x18] <hidden=true>;
   pad();
   Printf("%s = %d\n", chunkName, val);
} DYNASTRING;

typedef struct {
   int16 chunkID <format=hex, hidden=true>;
   int16 unk1 <hidden=true>;
   int size <format=hex, hidden=true>;
   char chunkName[0x18] <hidden=true>;
   if (size > 0)
   {
      char data[size] <hidden=true>;
   }
   pad();
} DATABLOCK;

typedef struct {
   int16 chunkID <format=hex, hidden=true>;
   int16 unk1 <hidden=true>;
   int size <format=hex, hidden=true>;
   char chunkName[0x18] <hidden=true>;
   if (size > 0)
   {
      char unk[size] <hidden=true>;
   }
   pad();
} UNKCHUNK;


void parse()
{
   local byte deeper = 1;
   
   while (deeper == 1)
   {
      if (FEof()) {return;}
      switch(ReadShort(FTell()))
      {
         case 0x18:
            DYNASTRING DynaString <open=false, read=Str("%s = %d", chunkName, val)>;
            break;
         case 0x1E:
            CLIENTPARM ClientParm <open=true, read=chunkName>;
            break;
         case 0x28:
            GROUPSTART Group <open=true>;
            break;
         case 0x32:
            GROUPEND GroupEnd <hidden=true>;
            return;
            break;
         case 0x46:
         case 0x29A:
            PUSHCONTEXT PushContext <open=true, read=chunkName>;
            break;
         case 0x50:
         case 0x309:
            POPCONTEXT PopContext <hidden=true>;
            break;
         case 0x6f:
            DATABLOCK DataBlock <read=chunkName>;
            break;
         case 0x378:
            PUSHHW PushHW <open=true, read=chunkName>;
            break;
         case 0x3E7:
            POPHW PopHW <hidden=true>;
            return;
            break;
         default:
            UNKCHUNK unkChunk <open=true, read=chunkName>;
            break;
      }
   }
}

void pad()
{
   local int pos = FTell();
   local int pad = 0x10 - (pos % 0x10);
   if (pad < 0x10)
   {
      char padding[pad] <hidden=true>;
   }
}

struct FILE {
   parse();
} file <open=true>;
Last edited by Wulf on Wed Nov 30, 2022 7:26 am, edited 9 times in total.
Wulf
Posts: 49
Joined: Mon Oct 27, 2014 8:30 pm

Re: God of War series - WAD files

Post by Wulf »

Edit: Removed.
Last edited by Wulf on Fri Nov 07, 2014 5:45 am, edited 1 time in total.
Wulf
Posts: 49
Joined: Mon Oct 27, 2014 8:30 pm

Re: God of War series - WAD files

Post by Wulf »

From GoW1-PS3-r_shell.trimmed.wad_ps3:

At 0x1c1f8 there is the value 00 0A AA AC.
At 0x1c264 32-bit pixel data starts and carries on until 0xc6d10, which is 0xaaaac bytes long.




------

There are 180 "GroupStart"s, and only 122 "GroupEnd"s.
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: God of War series - WAD files

Post by aluigi »

I guess that the thread you opened in 2012 didn't give results, correct?

The format doesn't seem to be so simple to parse.
Wulf
Posts: 49
Joined: Mon Oct 27, 2014 8:30 pm

Re: God of War series - WAD files

Post by Wulf »

I guess that the thread you opened in 2012 didn't give results, correct?
Correct.

The WADs from the PS3 versions seem a little easier to understand because they have recognizable data in them (DDS, BMP, etc), but aside from picking out individual files from the data I haven't been able to get an idea of its overall layout.
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: God of War series - WAD files

Post by aluigi »

Yeah, it "seems" simple but it isn't.
I tried to make a very quick and simple test but it failed:

Code: Select all

endian big
math OFFSET = 0xd00
for
    goto OFFSET
    padding 0x10
    get DUMMY short
    get DUMMY short
    get SIZE long
    getdstring NAME 0x38
    get DUMMY short
    get DUMMY short
    getdstring NAME 0x38
    getdstring DUMMY 0x14
    savepos OFFSET
    math SIZE -= 0x50
    log NAME OFFSET SIZE
    math OFFSET += SIZE
next

Anyway the fact that it uses normal files it's good, you can try to rip them with DragonUnpacker (it has a simple ripper embedded as far as I remember).
Wulf
Posts: 49
Joined: Mon Oct 27, 2014 8:30 pm

Re: God of War series - WAD files

Post by Wulf »

Might take a crack at attaching a debugger and watching the game parse the WAD, but it'll probably be beyond my understanding of PPC.

Edit: Dragon Unpacker finds a single, giant DDS file in the GoWA Shell. Nothing in the others.
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: God of War series - WAD files

Post by aluigi »

Very strange the behaviour of Dragon Unpacker.
A ripper should rip everything, in fact there are tons of DDS in the sample you provided.
There are other file rippers but they are dead/old or maybe not safe.
Wulf
Posts: 49
Joined: Mon Oct 27, 2014 8:30 pm

Re: God of War series - WAD files

Post by Wulf »

The following could be incorrect in minor ways, it was late when I looked at it and I wasn't documenting. I'll take a better look tonight.

The game starts off by loading the entire WAD into memory without parsing it. Once it's in memory, one function will read up to "PopHeap" before passing it off.

The GroupStart/GroupEnd lines are read in 0x20 byte chunks, starting at even 0x10 multiples of the WAD file. Meaning, the following would be a single read:

Code: Select all

28 00 00 00 00 00 00 00 47 72 6F 75 70 53 74 61     (.......GroupSta
72 74 00 00 00 00 00 00 00 00 00 00 00 00 00 00     rt..............


The xxxX_R_Shell lines are read in 0x30 byte chunks, after the endianness of 0x20-0x24 is reversed. Meaning, the red bytes would be reversed, then the chunk would be read:
1E 00 00 00 30 00 00 00 47 46 58 58 5F 52 5F 53 ....0...GFXX_R_S
68 65 6C 6C 00 00 00 00 00 00 00 00 00 00 00 00 hell............
0C 00 00 80 00 00 00 00 00 04 00 00 00 04 00 00 ...€............

After hitting PopHeap, it would pass it off to another function which would already know the start and end bytes of the data it cared about, and read it in up to 0x20000 byte chunks. I have to trace that one back to see where it's calculating those.
Wulf
Posts: 49
Joined: Mon Oct 27, 2014 8:30 pm

Re: God of War series - WAD files

Post by Wulf »

Edit: Removed
Last edited by Wulf on Fri Nov 07, 2014 4:01 pm, edited 1 time in total.
Wulf
Posts: 49
Joined: Mon Oct 27, 2014 8:30 pm

Re: God of War series - WAD files

Post by Wulf »

Well, I'm successfully using a hex editor to rip and replace models in the game without it crashing. Not with 100% success, but a bunch of weird orbs running around in place of the player is a start.

Looks like model data changed between GoW1 and GoW2, no success swapping the Kratos costumes between games.
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: God of War series - WAD files

Post by aluigi »

Maybe you can try to put everything in a script to automatize the extraction.
When you understand the format, the script is simple.
Wulf
Posts: 49
Joined: Mon Oct 27, 2014 8:30 pm

Re: God of War series - WAD files

Post by Wulf »

Having a bit of trouble with the file ending offset seeming to be inconsistent... As in, jumping ahead by the amount shown will sometimes land me partway through the next header.

Also haven't fully figured out how to deal with GroupStart/Ends. I assume it relates to the WAD being treated as a heap, but I don't fully understand the concept as it would apply to an archive file.
Wulf
Posts: 49
Joined: Mon Oct 27, 2014 8:30 pm

Re: God of War series - WAD files

Post by Wulf »

Edit: Moved to first post.


Optional insert:

Code: Select all

string NAME p= "%08X-%s" OFFSET NAME

More work needs to be done to figure out how the grouping works, but this seems to extract the pieces.

Works on GoW 1 and 2 Shell WADs so far.
Last edited by Wulf on Fri Nov 07, 2014 2:43 pm, edited 3 times in total.
Wulf
Posts: 49
Joined: Mon Oct 27, 2014 8:30 pm

Re: God of War series - WAD files

Post by Wulf »

Edit: Removed, combined with script in first post.
Last edited by Wulf on Fri Nov 07, 2014 4:01 pm, edited 1 time in total.
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: God of War series - WAD files

Post by aluigi »

Well done, I have seen you made also a video.

I made just some small aestethic changes to your script, only to make (I hope) a bit more readable.
Maybe it's useful to know how to tweak the scripts for doing the same job with less instructions and to use indentation, the only thing I changed was "SIZE == 0x50" with "SIZE <= 0x50" (is it better?).

Code: Select all

endian big
get ARCHIVE_SIZE asize
for OFFSET = 0 < ARCHIVE_SIZE
    math valid = 1

    goto OFFSET
    get CAT short
    get DUMMY short
    get SIZE long
    getdstring NAME 0x18

    math OFFSET += 0x40

    if NAME == "EntityCount"
        math SIZE = 0x20
    elif NAME == "GoStreamInfo"
        math SIZE = 0x50
    endif

    if SIZE <= 0x50
        math valid = 0
    endif

    if valid != 0
        log NAME OFFSET SIZE
    endif

    math OFFSET += SIZE
    math OFFSET x= 0x10
next


Should this script work also with the sample file you provided?
Wulf
Posts: 49
Joined: Mon Oct 27, 2014 8:30 pm

Re: God of War series - WAD files

Post by Wulf »

That was a good starting off point. I tweaked some of it around, and the one script now works with GoW1/2 and GoWA. I'll dig out my disc later tonight and test with GoW3.

I'm pretty happy with it so far but still need to figure out the significance of GroupStart/End. Being able to throw all the models/textures/scripts/sounds into designated folders would make everything much simpler.
Wulf
Posts: 49
Joined: Mon Oct 27, 2014 8:30 pm

Re: God of War series - WAD files

Post by Wulf »

Alternate version of the script, to output files numbered in the order they're processed. Also outputs empty GroupStart/End files for analysis.

Code: Select all

get ARCHIVE_SIZE asize
get DUMMY long
get ENDTEST short

if ENDTEST == 0
    endian big
    math ENDIPAD = 0x20
endif

math count = 1

for OFFSET = 0 < ARCHIVE_SIZE
    math valid = 1

    goto OFFSET
    get CAT short
    get DUMMY short
    get SIZE long
    getdstring NAME 0x18

    string NAME p= "%04d-%s" COUNT NAME

    math OFFSET += 0x20
    math OFFSET += ENDIPAD

    if NAME == "EntityCount" || CAT == 0x18
        math SIZE = 0
   math valid = 0
    endif


        log NAME OFFSET SIZE


    math OFFSET += SIZE
    math OFFSET x= 0x10
    math COUNT += 1
next
Wulf
Posts: 49
Joined: Mon Oct 27, 2014 8:30 pm

Re: God of War series - WAD files

Post by Wulf »

Modified version to allow greater success with GoWA WADs.

Will be put into the first post once I pretty it up and confirm I didn't break the other GoWs.

Code: Select all

get ARCHIVE_SIZE asize
get DUMMY long
get ENDTEST short

if ENDTEST == 0
    endian big
    math ENDIPAD = 0x20
endif

math count = 1

for OFFSET = 0 < ARCHIVE_SIZE
    math valid = 1

    goto OFFSET
    get CAT short
    get DUMMY short
    get SIZE long
    getdstring NAME 0x18

   

    math OFFSET += 0x20
    math OFFSET += ENDIPAD

    if NAME == "EntityCount" || CAT == 0x18
        math SIZE = 0
        math valid = 0
    endif

  if NAME == "N_SCREENS" || NAME == "DELAY_00"
    math size = 0
  endif

  if NAME == "MSGS_COUNT" || NAME == "MSGS_LINES"
    math size = 0
  endif

  if NAME == "HERO_HEAP_SIZE" || NAME == "UPGRADE_HEAP_SIZE"
    math size = 0
  endif

  if NAME == "ContextNames"
    math valid = 0
  endif


if VALID == 1
  string NAME p= "%s.%06d" NAME COUNT
  log NAME OFFSET SIZE
endif

    math OFFSET += SIZE
    math OFFSET x= 0x10
    math COUNT += 1
next
cyberspeed
Posts: 104
Joined: Wed Mar 23, 2016 5:11 am

Re: God of War series - WAD files

Post by cyberspeed »

Hey guys, I know its an old thread, but was wondering, was anyone ever fully able to extract proper texture/meshes/animation from any of the GoW series?
Tried all 4 samples of the script here, on all four PS3 games, but there is a mess up on extraction, is there anything updated and usable?

cheers.