Mass Effect Andromeda

Extraction and unpacking of game archives and compression, encryption, obfuscation, decoding of unknown files
GOTBLZ
Posts: 16
Joined: Tue May 09, 2017 4:21 pm

Re: Mass Effect Andromeda

Post by GOTBLZ »

OClear wrote:
GOTBLZ wrote:WV's tool already exports the highest quality LOD, the LOD you think exists, doesn't. What ever texture size that is exported, is the highest LOD there is, because if every texture was the highest LOD that you think is there, the game would be like 400 GB... When they "release" a game, it gets cooked, and an automated process happens to reduce the size of all textures.


I believe in statistics. Prove that LOD 0 doesn't exist.

Write a plugin that scans all meshes, and keep a tally of whether a LOD exists or not. Print the results for all 5 LODs.
percentage of meshes with LOD 0 found
percentage of meshes with LOD 1 found
percentage of meshes with LOD 2 found
percentage of meshes with LOD 3 found
percentage of meshes with LOD 4 found


I don't know how else to explain it to you. Ok tell me, when you re-imported a texture into DAI, and went in-game, did the texture you edit show up only when you were far away from the object with the texture? Because if the LOD 0 existed it would take priority over the lower LOD that you edited. Just by roughly calculating the textures of the male customized faces, if all LOD0 textures existed there would be 500MB just in 10 items, that is not counting the females custom faces, or their hair, or the armor models etc etc etc...

Have you not ever noticed that all the Normal Maps that you edit are 2048x2048? Do you think WV would make his tool somehow export all 2048x2048 for Normal Maps, but 256x256 for face Diffuse? No, its because when a game is released it goes through a process called 'cooking' and it automatically goes through all textures, and works out what size it can reduce it down to, without losing too much detail, this is why Normal Maps aren't small, or else it'd make the meshes look horrible, but for a skin tone, it can reduce it a lot more, because there is little detail.... I cannot be bothered explaining this anymore, Google it... Or if you don't believe us because you only "believe in statistics" then make the plugin yourself and prove us wrong! We'll be waiting.
Renmiri
Posts: 3
Joined: Wed Apr 12, 2017 3:52 am

Re: Mass Effect Andromeda

Post by Renmiri »

OClear wrote:
warrantyvoider wrote:And you determine what is helpful on this research? Great, improvements for tools that dont even exist yet? This is the exact shit that stops any research, demands before anyone can even know if its possible. If you dont know how to re, yeah, pls leave and let it be. Dunno if you noticed, but in this is thread are ppl that do research and collect information for research, not make-a-wish foundation and collections for demands. Wheres your code? Your messing with files? Your testing of my tools? Your search for frostbite information? Your debugging results and obfuscation/encryption research? wheres your help with the texture formats? or creating dds? or talktables? or animations? or sounds? or finding structures? etc, etc, etc


Have a snickers bar dude, and try not to eat the hand that feeds you. See ya.


What exactly are you doing to be "the hand that feed ya" ? Are paying someone ? Helping with anything ?

Just the opposite, you are using someone's tool for free and demanding stuff just for you, while not helping with anything.

You are the one mooching on someone else's work and not giving anything back but attitude.

Wanna help ? code, test the tool, add some info about frostbite or file formats, help the other research he mentions above. Wanna use the tools ? feel free to but if you want it custom made for your wishes do it yourself or STFU. No one is here to be your personal coder
warrantyvoider
Posts: 236
Joined: Tue Apr 04, 2017 11:44 am

Re: Mass Effect Andromeda

Post by warrantyvoider »

Looking at the log, i see that the position for patching the uncompressed size in sb files is way too far away from the sha1 position, meaning its patching SOMETHING else. In theory if you reimport the original dds and restore only the sb file, it should stick. Now i sit in a bus, can someone test that theory? Thx

Greetz

Ps: to the question if you can replace data at its original position: no, because compression can lead to a bigger result and you would have to shift and patch everything that comes after... also what if you wanna import something bigger in general?
GOTBLZ
Posts: 16
Joined: Tue May 09, 2017 4:21 pm

Re: Mass Effect Andromeda

Post by GOTBLZ »

warrantyvoider wrote:Looking at the log, i see that the position for patching the uncompressed size in sb files is way too far away from the sha1 position, meaning its patching SOMETHING else. In theory if you reimport the original dds and restore only the sb file, it should stick. Now i sit in a bus, can someone test that theory? Thx

Greetz

Ps: to the question if you can replace data at its original position: no, because compression can lead to a bigger result and you would have to shift and patch everything that comes after... also what if you wanna import something bigger in general?



I will try that out now, aha, yeah your right, I didn't understand how MEAExplorer works enough, I thought it was almost like reading a zip/rar file, but stupid me for thinking that... I didn't realize MEAExplorer could import larger textures, I suppose I had only ever tried importing smaller ones, because some faces were 43kb and others 86kb, but now I see what you are saying, all good, thought it wouldn't hurt to throw ideas out there, will update when I find out how the texture import goes

Edit* I exported, and re-imported the same (original) face diffuse texture, but none of the .sb's had been modified, only cat's and cas'...

https://pastebin.com/fuk7Zdxr Here's my log, it says its patching the .sb, but its not modified, well at least according to Windows... or I shouldn't go by that?

*Never mind... I just attempted it second time, and its patched it... I will try it now

This is after I restored the original .sb
Image
warrantyvoider
Posts: 236
Joined: Tue Apr 04, 2017 11:44 am

Re: Mass Effect Andromeda

Post by warrantyvoider »

GOTBLZ wrote:https://pastebin.com/fuk7Zdxr Here's my log...
This is after I restored the original .sb ...


damn, so either your backup sb is already borked or this is yet another issue. because now, only the cat files are touched, the cas files dont matter, as its the cat file that says where something in it is. dunno, still have to fix the patching problem with the uncomp size and that sb files only get touched once

greetz
GOTBLZ
Posts: 16
Joined: Tue May 09, 2017 4:21 pm

Re: Mass Effect Andromeda

Post by GOTBLZ »

warrantyvoider wrote:damn, so either your backup sb is already borked or this is ...


I will give you everything I know. The .sb wasn't patched the first time I tried, because I used an older version (5 days ago) of MEAExplorer, when I used the new one you just put up, it patched the .sb file.

It's not my sb (that sounds rude for some reason? not meant intentionally) I know that, because I backed up all my modified files before I started modifying the face textures (well I didn't really, but after getting a black and blue squared face, and having to delete all my modified files that were last changed, and re-import all the changes I had, I learned to back it up... :roll: ), and also have two untouched copies of MEA, one so I can test single things like this, and the other as a backup with original files, (with all files over 300MB deleted, as MEAExplorer doesn't edit any big ones)

I'm still here to help test any ideas you have too.
warrantyvoider
Posts: 236
Joined: Tue Apr 04, 2017 11:44 am

Re: Mass Effect Andromeda

Post by warrantyvoider »

well nothing you can do for now, I will have to check this when home and fix the offset patching

PS: just watch out for the "now on github" line in my posts, if it appears, get a fresh copy from git :P
Last edited by warrantyvoider on Wed May 10, 2017 1:04 pm, edited 1 time in total.
GOTBLZ
Posts: 16
Joined: Tue May 09, 2017 4:21 pm

Re: Mass Effect Andromeda

Post by GOTBLZ »

That's cool, whenever you get some time, I understand you have a life to live too.
warrantyvoider
Posts: 236
Joined: Tue Apr 04, 2017 11:44 am

Re: Mass Effect Andromeda

Post by warrantyvoider »

GOTBLZ wrote:That's cool, whenever you get some time, I understand you have a life to live too.


well nvm, seems you dont need to wait for me , no need for me to do anything anymore, why do all the frustrating work alone if others seem to be able to work in teams or whatever, I shouldnt go on

https://www.youtube.com/watch?v=kPHsmm95fRg

iirc, by all requests, my tools gui's always suck, so just wait for his/their fancy tool
AnonBaiter
Posts: 1125
Joined: Tue Feb 02, 2016 2:35 am

Re: Mass Effect Andromeda

Post by AnonBaiter »

So, does this "next-generation modding tool" has the capacity to deliver what was promised? Because I`d rather mod Battlefield 1 than just Mass Effect Andromeda and Dragon Age Inquisition.

You know, just to bring back the stupid AI in an offline, AI-controlled match of Battlefield 1942 so I can enjoy people running off walls for no reason or running around in circles in a single-player campaign.
GOTBLZ
Posts: 16
Joined: Tue May 09, 2017 4:21 pm

Re: Mass Effect Andromeda

Post by GOTBLZ »

warrantyvoider wrote:
GOTBLZ wrote:That's cool, whenever you get some time, I understand you have a life to live too.


well nvm, seems you dont need to wait for me , no need for me to do anything anymore, why do all the frustrating work alone if others seem to be able to work in teams or whatever, I shouldnt go on

https://www.youtube.com/watch?v=kPHsmm95fRg

iirc, by all requests, my tools gui's always suck, so just wait for his/their fancy tool


Wha? You totally took my message out of context... Sorry, I meant it as though I didn't want you to think that you had to do it straight away... I'm thankful you're trying to work out the problem...

Oh... I see what you are saying... I should've clicked the youtube link first...
warrantyvoider
Posts: 236
Joined: Tue Apr 04, 2017 11:44 am

Re: Mass Effect Andromeda

Post by warrantyvoider »

original
Image

any import
Image

bundledSize, rangeStart and rangeEnd need patching too it seems

EDIT: well no glitching but also my texture doesnt show up, dafuq?

Code: Select all

 Patching D:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\ambsibling.sb
 Found Sha1 @0x00038BDF
 Patching Compressed Size @0x00038BF9 type Int32 with 0x8544
 Patching Range Start @0x00038C09 type Int32 with 0x0
 Patching Range End @0x00038C17 type Int32 with 0x8544
 Patching Bundled Size @0x00038C28 type Int32 with 0x8544


why?
Image

just as proof, it does show up for a second
Image

and instantly swaps to
Image

and after a while it reverts to the first pic

so no, absolutely no idea at this point, but

now on github

greetz WV
michalss
Posts: 320
Joined: Sun Aug 10, 2014 12:49 pm

Re: Mass Effect Andromeda

Post by michalss »

warrantyvoider wrote:
GOTBLZ wrote:That's cool, whenever you get some time, I understand you have a life to live too.


well nvm, seems you dont need to wait for me , no need for me to do anything anymore, why do all the frustrating work alone if others seem to be able to work in teams or whatever, I shouldnt go on

https://www.youtube.com/watch?v=kPHsmm95fRg

iirc, by all requests, my tools gui's always suck, so just wait for his/their fancy tool


Nope.. and Nope.. Yr gui is fine and clean, like it other then that that person never release it. That is mine opinion im afraid..
warrantyvoider
Posts: 236
Joined: Tue Apr 04, 2017 11:44 am

Re: Mass Effect Andromeda

Post by warrantyvoider »

michalss wrote:Nope.. and Nope.. Yr gui is fine and clean, like it other then that that person never release it. That is mine opinion im afraid..

dunno, im not a user, they are the ones with demands and complains. I dont force anyone to use my tools. I also dont like redundancy aswell as competition, especially if theres nothing to win other than more work. So if he has a team and whatnot, is so much liked by users, gets more views and attention (even with no release), I dont see why I should continue alone and "against" everyone. im not here to work against anyone, I collect puzzle pieces and do research, I make most stuff open source and invite ppl to write plugins, instead of trying to manage alot devs on a single project.

GOTBLZ wrote:...

btw, you got pm since a while
warrantyvoider
Posts: 236
Joined: Tue Apr 04, 2017 11:44 am

Re: Mass Effect Andromeda

Post by warrantyvoider »

uploaded a new version to github

I made the import alot faster (no redudant sb patching, no cat reloading on each edit, instead at the end)

now you can also put into settings.txt following line

Code: Select all

patchall = 1

and on import, it now looks through all sb files for that SHA1. This ofc slows the overall import down alot, also the better solution would be to make mods for each sb file instead, so I made this optional for now, but this is the flamethrower approach and should patch any left reference in other sb files. Please note that it does not look through patch folder files, because that somehow crashes the game.

Also now it patches the logicalOffset field.

some tests:
-importing the exact data back in, results in a bigger compressed chunk, so it has to be about the sb data, not the cas data
-tried to also patch Total(Chunk/Res/Ebx)Size field , aswell totalSize field in each affected bundle. no change, still glitchy
-as usual, any imported texture actually shows up, but reverts to glitchy on closer inspection
-patch-sb files do use this SHA1 too, but offer no deltaSha1, so no patches could interfer
-the chunk id is also used in idata, which seems to be just a copy of the res data, no size values
-normal other texture imports, like my "warranty voided*" mod, still work fine

so yeah, still no idea what it could be, especially when the textures actually show up for 2 seconds

greetz
Trip
Posts: 7
Joined: Thu Apr 06, 2017 12:50 pm

Re: Mass Effect Andromeda

Post by Trip »

warrantyvoider wrote:uploaded a new version to github
so yeah, still no idea what it could be, especially when the textures actually show up for 2 seconds


maybe it is displaying an incorrectly loaded LOD for that texture set?

(I know nothing about the engine I am just reading and guessing)
warrantyvoider
Posts: 236
Joined: Tue Apr 04, 2017 11:44 am

Re: Mass Effect Andromeda

Post by warrantyvoider »

if that would be the case, the glitch would revert as soon as you reach more distance. LOD=Level of Detail, defining which resolution to use depending on the distance to the object
Huili
Posts: 3
Joined: Fri May 12, 2017 9:39 am

Re: Mass Effect Andromeda

Post by Huili »

Warrantyvoider, нello, first of all, thank you very much for your work, and will it work with the version of the game 1.06? After the patch, any mod simply freezes the game on loading the save.
warrantyvoider
Posts: 236
Joined: Tue Apr 04, 2017 11:44 am

Re: Mass Effect Andromeda

Post by warrantyvoider »

I have updated to latest version aswell, so dunno what the problem is. does the console window pop up? what does it say? what OS do you have? have you tried others saves? is it corrupted maybe? does this happen without loading saves? btw, the tool creates a log now what gets edited, maybe something there to see? did you put the anselsdk64.dlls back in?
Huili
Posts: 3
Joined: Fri May 12, 2017 9:39 am

Re: Mass Effect Andromeda

Post by Huili »

warrantyvoider wrote:I have updated to latest version aswell, so dunno what the problem is. does the console window pop up? what does it say? what OS do you have? have you tried others saves? is it corrupted maybe? does this happen without loading saves?

The game is loaded, the console window pops up.
The new game also does not start.
Image Image
In mod, I replaced the face texture of the character.
Image
Here is the log of the plugin's work. https://pastebin.com/PFb0di1p
OS - Windows 7 64bit
My english is terrible, so I'm sorry.