Megatextures research (Doom, Wolfenstein, Evil Within, Rage)

Textures, recreate headers, conversions, algorithms and parsing of image files
id-daemon
Posts: 1040
Joined: Sun Mar 22, 2015 7:09 pm

Megatextures research (Doom, Wolfenstein, Evil Within, Rage)

Post by id-daemon »

After I did lots of research in sound formats, archives, models and animations I finally decided to get to most complicated task I've heard of: Megatextures. I was able to reverse the modified version of HDP they used and decompress megatextures. The devs improved the coding algorithm in compare to standard HDP.

Doom Tool released http://aviacreations.com/wraith. And here's The brief history of megatexture research.

March 22nd

The first look into mega2 files. I found they contain tons of small pieces - tiles. From xentax thread I got info that's probably HDP or JXR compression. After downloading HDP specs and sources it becomes clear why nobody was able to decode it before. Walls of code. Very complicated algorithm, even having the specs.

March 25th

I found decoding routine in Doom and got the first tile decoded by it from memory.

Image

Making standard headers for the raw data (with all available options tried) surely fails.

Image

March 26-30th

I needed to finish cloth extraction for Hitman 5, and D-Walker animations for MGSV, so nothing was done here.

April 3rd

After some days of studying the format specs and raw data I came to conclusion that it was modified, non standard HDP format. So the only way to decode it was to carefully study their routine step-by-step and find what exactly was changed. But it looked so tangled up even after some debugging and multiple reading through the doc, that I said: "no. its TOO complex. i think i wont be able to do this"

April 7th

After some more debugging and studying, and changing the code, the very first monochrome tile was decoded. I couldn't believe it first, but it couldn't be just by accident. This was correctly decoded tile.

Image

April 8th

More in-depth research showed that most changes were made deep inside of the adaptive variable-length bitstream reading which mostly concerned colored images. Some changes found, and I could decode about half of the image, and then it fails:

Image

April 9th

Lots of hard work to dig into the very core of this crazy algorithm, many more changes were found and at some point I got this:

Image

This was the very first 3-channel tile (not very colorful, but its colored) decoded from megatexture. After that, It was all history. Just some little tweaks and checks, was a little bit different in all games, but the problem was finally solved. You can count, it took 13 days.
Last edited by id-daemon on Sun May 14, 2017 3:00 pm, edited 1 time in total.
id-daemon
Posts: 1040
Joined: Sun Mar 22, 2015 7:09 pm

Re: Megatextures research (Doom, Wolfenstein, Evil Within, Rage)

Post by id-daemon »

Doom tool updated yesterday, now all export issues must be solved.

Meanwhile I'm back to Evil Within megatextures. Unlike Doom, they have 8 images, and all different format. So I need to do some tests. This is for example some weapon. Diffuse 3ch, normal 2ch, and specular 1ch. There are 2 more images for this one, working on those.

Image

Image

Image
Manoa
Posts: 15
Joined: Sun Sep 09, 2018 1:16 pm

Re: Megatextures research (Doom, Wolfenstein, Evil Within, Rage)

Post by Manoa »

is it possible to put the texture back into the game after modification ?