[PC] AREA-51 [2005] (.000/.DFS/.AUDIOPKG/.RIGIDGEOM/etc)

[Mortisanti]

Greetings, ZenHAX Users!

Introduction
Area 51 (stylized AREA-51) is a game that was developed by Midway Studios Austin (formerly Inevitable Entertainment) starting in 2002 using a proprietary engine/toolset/art pipeline (apparently didn't have any part of Unreal Engine) and it was published by Midway Games for the PS2/Xbox and was eventually ported to PC in 2005; this is not to be confused with the very original arcade version of Area 51. The game was released as freeware some time in 2008 and thus can be freely distributed; after GameSpy servers went down, online play was once again enabled by changing the gamespy URL to the openspy URL within the game's EXE using a hex editor. If someone would like a copy to play/examine, and if that determines whether or not it will peak their interest in helping with this project, feel free to get it here.

What We Would Like To Achieve

• A better understanding of Area 51's engine, archives, etc.
• The ability to properly extract/convert/edit the game's assets as well as successfully re-import them back, i.e. new multiplayer maps and models.
• Possibly modify the game's campaign after being granted such knowledge.
• Subject to be updated/changed.
  
  What We Know
  
  • This is the PC version.
    
    
  • The game uses RAD Game Tools: BINK for in-game videos and Miles Sound System for audio.
    
    
  • Despite this thread, I had no such luck with WATTO Game Extractor.
    
    
  • A link to my Google sheet with all(?) of the file formats and a potential description for each, here. Within the game's root folder lies five 000 archives (COMMON, HIRES, PRELOAD, RESOURCE, & STRINGS) and four of their respective DFS files (with the exception of RESOURCE.DFS), as well as a folder for the AUDIO and the campaign/multiplayer LEVELS (which both contain more 000/DFS files). I've packed up the root folder containing all of the 000/DFS files and it can be downloaded here.
    
    
  • Using a script, posted by Mr.Mouse here, in MultiEx Commander or with quickbms, I was able to successfully (I think) perform an initial extraction from each 000 archive within the game folders, which can be downloaded here.
    
    
  • It is somewhat possible to extract from the .AUDIOPKG files using this quickbms script, which extracts the files as .RAW audio files that can be imported into Audacity. Although, in most cases the script runs into an error - for example, using the script with "SFX_MENU_INGAME.AUDIOPKG" (audio archive extracted from "HOT.000" which is found in the "AUDIO" folder) it results in 24 audio files - at least 2 of which are broken - and gives this error:
    

    Code: Select all

    Error: impossible to write 0xffd8003d bytes (total 0xffd8003d)
       Check your disk space
    
    Last script line before the error or that produced the error:
      83  log NAME OFFSET SIZE

    
    Which leads me to believe it's an issue with the script itself(?)
    
    
  • The .RIGIDGEOM files are meshes - info about that here.
    
    
    Resources & Links
    
    • "Area51UnDFS and XBMP2TGA": Simple executables created by a user that goes by the name jTommy. One can extract from 000 archives without using MEX/quickbms scripts (I had better luck with the script by Mr.Mouse that I linked above.); the other EXE is used to convert XBMP textures to TGA (and TGA back to XBMP). The TGA files can be opened in Photoshop.
      
    • "GOBREAD": Apparently another tool that can be used to extract from 000/DFS archives ONLY if you're on a 32-bit system.
      
    • PCGamingWiki page for Area 51.
      
    • XeNTaXWiki page for Area 51 DFS files.
      
    • 3D Architecture, courtesy of PS23DFormat.
      
    • Other thread on XeNTaX relating to Area 51. (If someone still has this patch, or a working link, I'd love to see it.)
    • Other thread on XeNTaX relating to Area 51.
    • Other thread on XeNTaX relating to Area 51.
    • Other thread on XeNTaX relating to Area 51.
      
    • More game info, courtesy of Moby Games.
      
      
    • Hopefully more to come...

Any help is greatly appreciated.
Thank you,
Morti

[aluigi]

Regarding my area51_audiopkg.bms script I think I can't do much, the message "this file format is horrible, this script is experimental" in the header doesn't give much hopes

[Mortisanti]

Regarding my area51_audiopkg.bms script I think I can't do much, the message "this file format is horrible, this script is experimental" in the header doesn't give much hopes

I was afraid of that.

[AnonBaiter]

Regarding my area51_audiopkg.bms script I think I can't do much, the message "this file format is horrible, this script is experimental" in the header doesn't give much hopes

I was afraid of that.

I don`t mean to insult you, but now you`re even more afraid that you can`t analyze the file for yourself? Please...

[Mortisanti]

Regarding my area51_audiopkg.bms script I think I can't do much, the message "this file format is horrible, this script is experimental" in the header doesn't give much hopes

I was afraid of that.

I don`t mean to insult you, but now you`re even more afraid that you can`t analyze the file for yourself? Please...

Well, Sir Baiter, I was always aware of the header of the script, though I don't have the knowledge to analyze the archives via hex and write a quickbms script to extract them - nor would I be able to modify the current scripts properly. I wanted to find other people interested in the game as much as I am so we could work together in analyzing it. I don't expect people to do all the work for me, nor do I want them to.

Also, not sure if language barrier or you're joking with me.

[AnonBaiter]

It was just for humor.

[Mortisanti]

It was just for humor.

Apologies for making an ass out of myself.

[AnonBaiter]

It's okay.
So back to the topic, I'm not quite sure if I can help with you on this task... So where do I start?

[Mortisanti]

It's okay.
So back to the topic, I'm not quite sure if I can help with you on this task... So where do I start?

Well, considering it may be out of the question to modify the AUDIOPKG script to extract from the archives (without broken files) I would like to find a tool or two that can extract the .RAW audio files from them - Dragon UnPack's "Hyper Ripper" seems like it does the job for extracting the .MP3 files but that's about it.

There's a RESOURCE.000 file in the root folder of the game but no accompanying RESOURCE.DFS file - I assume the .DFS file is needed to extract from the .000. If not, If there's a workaround, I'd enjoy seeing what's in there - it could even be one of the hidden levels that were removed/unfinished.

[aluigi]

Well, let's start from the beginning.
I have just made a script from scratch for the DFS/000 archives:
http://aluigi.org/bms/area51.bms

Next step is having a list of ALL the file extensions and trying to understand their content.
For example the textual files require no knowledge (it's text), XBMP are probably headerless bitmaps (so I guess relatively easy to convert), audiopkg are painful, ANIM and FXO probably unknown formats and so on.

It's a big work so I think you should find someone really interested in the project or there is the risk of spending months with new bits of information retrieved by people here and there.
Anyway that's the starting point.

[Mortisanti]

Well, let's start from the beginning.
I have just made a script from scratch for the DFS/000 archives:
http://aluigi.org/bms/area51.bms

Next step is having a list of ALL the file extensions and trying to understand their content.
For example the textual files require no knowledge (it's text), XBMP are probably headerless bitmaps (so I guess relatively easy to convert), audiopkg are painful, ANIM and FXO probably unknown formats and so on.

It's a big work so I think you should find someone really interested in the project or there is the risk of spending months with new bits of information retrieved by people here and there.
Anyway that's the starting point.

Cool, thank you. I've been waiting over a year hoping to find more people interested, so I'm in no hurry. I've been meaning to get around to making a list of files, known and unknown - so that is one of my next steps. And, yes, XBMP files can be converted to TGA using one of the two programs I linked above "XBMP2TGA", created by a user that goes by the name jTommy. I'd like to contact him but both of his websites are gone and archive.org yields no direct contact information that isn't deactivated (it also doesn't help that the sites are Russian). I emailed http://www.ctpax-x.org/index.php?lang=en asking if they could help me get in contact with him; no response yet.

The research continues.

[eleix]

I'm currently on a mad man's quest to bring the game over to the unreal engine (4.14 to be exact) and slowly going through the painful process of extracting out each and every archive so I can begin recreating the game maps since I don't think we can simply "import" the maps sadly. I'll most likely be spending a good chunk of this year alone just in recombining the 3D models and reapplying their respective skins (importing the .3ds file in blender only gives a partial piece that lacks the texture details). While I was looking through some of the .TXT files I noted that much of the resources like dialogue and possibly other key components of the game are calling back to the level RESOURCE file with direct filenames.[1]

Unfortunately I suspect that the game engine was unique to this file specifically (ex: this file's missing .dfs was most likely embedded inside the game engine itself due to all levels and key resources calling back to it.) due to the sheer amount of core data it contained.

EDIT: After a few hours I was able to piece everything together and found everything I was looking for.

[1]Campaign dialogue is contained inside the resources archive of each map. All you have to do is find the loadfile.txt which actually tells you everything needed for the level which includes the filenames.

Truthfully I'm still interested in the root RESOURCE.000 file as for what it's worth I feel the file size alone makes it an important archive of data. A good test to run would be to delete the root RESOURCE.000 archive and play through the game and see if everything still works properly.

Also I have a quick request for a save that has completed the game. I'm going to begin reconstructing the map levels for the campaign and don't want to go through the entire game just so I can begin to recreate the levels.

[AlphaTwentyThree]

Hi there!

There's at least SOME kind of progress: I tried to reverse-engineer the .audiopk files for some hours (phew) and came quite far. So far I only tested the music packs from the PS2 version, the others MIGHT follow BUT that depends on how the issue I have will be resolved.
Here's what my script can do:
- extract the sound streams without names (:\)
- leave out duplicate streams (the TOC lists every file twice)
- for each ID, extract some obscure portion in the header that is linked to the ID
Why no file names you might ask? This is "the issue". The problem is, that the [number of files] variable is only accessed one single time which is on the TOC. Each file has indeed some kind of identifier variable which MIGHT link it to a name but I haven't been able to understand the system. Each file has a number in the TOC which doesn't make any sense because it is neither an offset nor an ID but some weird number. Looking at the number sizes it indeed looks like it links to a name (lower numbers correspond to lower IDs) but I need the final "translation". Without this I simply cannot link names.
The script DOESN'T yet work on MUSIC_FRONTEND.AUDIOPKG as the structure is somehow different. I will look at the file later as it's much more difficult to understand the structure when there are only two files to extract (as in this case).
The ENTRIES variable below is just a pseudo-variable as you can see (divide the section size by 4...) to jump over the section with the "CRCs" (I only assigned a fitting name but surely that's no CRC).
For now I've written the script to extract each file in a subfolder with the file name to avoid confuction when you extract multiple files.
If you have any question about the script (e.g. what some section does), feel free to post here.
Here's the script:

Code: Select all

# Area 51 (PC/PS2/Xbox, 2005), script v0.2
# extracts most of the MUSIC_*.AUDIOPKG:
# - streams have no names
# - each ID has some data which is also extracted
# 
# script for QuickBMS http://quickbms.aluigi.org
# written by AlphaTwentyThree of Zenhax 05/2018

get BNAME basename
getDstring VER 0x10
getDstring PLATFORM 0x10
if PLATFORM == "PlayStation II"
   set EXT "mib"
   set BIAS 0
elif PLATFORM == "Windows"
   set EXT "mp3"
   set BIAS 0x1b0
elif PLATFORM == "Xbox"
   set EXT "dat"
   set BIAS 0
endif
getDstring INTERNAL_NAME 0x10
getDstring DATE 0x10
goto 0xc0
get NAMES long
get NAMES long
get SECTION1 long
get SECTION_NAMES long
goto 0xd4
get SECTION2 long
xmath ENTRIES "SECTION2 / 4"
goto 0xe4
get FILES long
get BIAS long
goto 0xf0
get VAR_FILES long # number of different files?
set NAME_OFF 0x110
goto NAME_OFF
for i = 1 <= NAMES
   get NAME string
next i
xmath MYOFF "NAME_OFF + SECTION_NAMES"
goto MYOFF

for i = 1 <= ENTRIES
   get CRC long
next i
savepos MYOFF
for i = 1 <= NAMES # 0xc
   get NAMEPOS short
   get IDX short # name id
   get ZERO long
   savepos MYOFF
   xmath GETNAME "NAMEPOS + NAME_OFF"
   goto GETNAME
   get NAME string
   putArray 0 IDX NAME
   goto MYOFF
next i

savepos MYOFF
xmath GETSECTION_OFF "MYOFF + NAMES * 4"

for i = 0 < NAMES # i = IDX
   get OFFSET long
   savepos MYOFF
   xmath k "i + 1"
   if k != NAMES
      get SIZE long
   else
      set SIZE SECTION1
   endif
   math SIZE -= OFFSET
   xmath OFFSET "OFF_SECTION + GETSECTION_OFF"
   getArray NAME 0 i
   string NAME p= "%s/ID%i_%s" BNAME i NAME
   log NAME OFFSET SIZE
   goto MYOFF
next i

savepos MYOFF
math MYOFF += SECTION1
goto MYOFF

get TEST short   # there can be a number of zeros here
for            # just a workaround
   savepos MYOFF
   get TEST short
   if TEST == 0
      break
   endif
next
goto MYOFF
for i = 0 < BIAS
   get DUMMY long
next i
get BNAME basename
#savepos MYOFF
#print "%MYOFF%"
#cleanexit

for i = 1 <= FILES
   get ZERO long
   get OFFSET long
   get SIZE long
   get UNK long # 0xffffffff
   get UNK long
   get ZERO long
   get UNK long # ID???
   get FREQ long
   getDstring ZERO 8
   string NAME p= "%s/%i_%s_%i.%s" BNAME UNK BNAME i EXT
   if OFFSET != OLDOFFSET || i = 1 # duplicate files are omitted
      math OFFSET += BIAS
      math SIZE -= BIAS
      log NAME OFFSET SIZE
      set OLDOFFSET OFFSET
   endif
   set OLDOFFSET OFFSET
next i

[AlphaTwentyThree]

v0.2 update
- PC version supported (mp3 with correct file start)
- Xbox version has raw extraction with .dat extension for now
There's a BIG problem with the Xbox version looking at the raw stream. It's not possible to add a header right away because for some unknown reason the stream is broken in a way that the interleave changes from time to time. It's probably fixable by deleting incomplete samples but at the moment I'm not eager to do this as there's still the name issue. But I can tell that the streams are only mono.

[Warthog1336]

Hi! I need some help extracting the content of this archive which contains two PS ADPCM streams (32000Hz).

Download

Thanks!

[Warthog1336]

Awesome! Could you please update your script in order to extract the content of FRONTEND.AUDIOPKG ? That'll be amazing.

[Mortisanti]

I'm currently on a mad man's quest to bring the game over to the unreal engine (4.14 to be exact) and slowly going through the painful process of extracting out each and every archive so I can begin recreating the game maps since I don't think we can simply "import" the maps sadly. I'll most likely be spending a good chunk of this year alone just in recombining the 3D models and reapplying their respective skins (importing the .3ds file in blender only gives a partial piece that lacks the texture details). While I was looking through some of the .TXT files I noted that much of the resources like dialogue and possibly other key components of the game are calling back to the level RESOURCE file with direct filenames.[1]

Unfortunately I suspect that the game engine was unique to this file specifically (ex: this file's missing .dfs was most likely embedded inside the game engine itself due to all levels and key resources calling back to it.) due to the sheer amount of core data it contained.

EDIT: After a few hours I was able to piece everything together and found everything I was looking for.

[1]Campaign dialogue is contained inside the resources archive of each map. All you have to do is find the loadfile.txt which actually tells you everything needed for the level which includes the filenames.

Truthfully I'm still interested in the root RESOURCE.000 file as for what it's worth I feel the file size alone makes it an important archive of data. A good test to run would be to delete the root RESOURCE.000 archive and play through the game and see if everything still works properly.

Also I have a quick request for a save that has completed the game. I'm going to begin reconstructing the map levels for the campaign and don't want to go through the entire game just so I can begin to recreate the levels.

I'm surprised to come back to this thread, years later, and see some activity - it makes me happy. A friend and I have a theory that the stray RESOURCE.000 file could be for an extra multiplayer map named "Cavern" that didn't make it into the game (there's proof that such a level could have existed due to a load-screen image found in the game files). I have yet to start a play-through without the file in the folder though. I may also be able to provide you with a completed game save but real life is a bit busy for me at the moment.

Also, awesome work AlphaTwentyThree. I'm looking forward to trying your script when I have some free time.

[gabenga0er]

wow, I thought I was interested in this game alone, well then if you don’t mind, I will ask the question whether it is possible to somehow get the levels from this game and import into blender
(google translate russuan to english)

[Mortisanti]

wow, I thought I was interested in this game alone, well then if you don’t mind, I will ask the question whether it is possible to somehow get the levels from this game and import into blender
(google translate russuan to english)

I'm a year late to respond... but to answer your question, the closest we've come to "extracting" levels from AREA-51 is using 3D Ripper DX to rip them from the game while it's running.

[Lamantyne1]

@AlphaTwentyThree: Now that you're active again (happy new year!) it would be awesome if you could find a way to edit your script in order to be finally able to extract FRONTEND.AUDIOPKG and the files with proper filenames