The Witcher 3 Wild Hunt(*.cache*.bundle)

[aluigi]

I don't remember if the cache files are supported (I guess yes) but it's possible that some of them have small differences that avoid their extraction.

Maybe first try to use quickbms_4gb_files.exe because it's a big file.

If it still doesn't work maybe upload the beginning/end of the archive cutting it with this script http://aluigi.org/papers/bms/others/filecutter.bms

[Fig8man]

working on uploading the filecutter files, just trying to find a host i can upload to on my sorta crappy (tethered through my phone) internet.

mega.co.nz was loading okay'ish yesterday but now i just get respond timeouts

[aluigi]

Why don't you use this forum for the upload?
zip the first file obtained with filecutter (I don't think the ending part is useful at the moment), it should be about 2 megabytes.

[Fig8man]

i didn't think of that derp sorry

[aluigi]

I have checked that file and I think it's impossible that you get that error.
The reason is that witcher3.bms performs a sequential extraction of the files in the cache archives (even if there is a sort of complex table at the end which is ignored) so you can't get a 0% coverage.
You may get that error only when quickbms has extracted all the files and reaches the final header, but in that case you should have already an over 95% of coverage and not 0%, and you should have over 4gb of extracted files on disk.

For example here I can extract all the files available in the part you provided extracted almost 3 megabytes of files.

How many files have you extracted from texture.cache?

[Fig8man]

9,146 files totaling to .98 gb
and its all .dat files and files with extensions I've never heard of (.a8a,.b0b,.bxm,.bxp,.c1a,.cia,Etc)

[aluigi]

Ok that's a problem because it's not the whole file.
An alternative is using offzip that will be able to decompress the whole archive but the names of the files will be the same.

The fact that the script doesn't parse the final header and information is probably caused by differences in the structure of different cache files, I don't remember why I don't parse it but usually that's the reason.

[Fig8man]

someone from the nexus forums was kind enough to upload the extracted texture.cache files to mega

Code: Select all

Here: https://mega.co.nz/#fm/nVVxlJ4S

(But beware its BIG! over 13,000 files! the only way i was able to download the whole thing was by importing it into my MEGA Cloud and downloading it with MEGA's sync client for windows)
i found the gwent cards i was looking for but for some reason they're really blue in color so i guess the gwint_game.redswf module is doing some kind of color manipulation on its end after it calls the .dds

[hhrhhr]

But beware its BIG! over 13,000 files! the only way i was able to download the whole thing was by importing it into my MEGA Cloud and downloading it with MEGA's sync client for windows

did you try script for manual textures unpack?

i found the gwent cards i was looking for but for some reason they're really blue

can you write their filenames?

[Amioran]

Luigi, I tried to contact you by email but for now you have not replied me.

I have a problem with using QuickBMS and the witcher3.bms to reimport some specific files (and only them) as when you try to reimport them the file size is too large even when the character count and file size is the same as the extracted source (you can either reimport the source file intact and it will do the same). It is like when exported the file becomes larger than the source or the reimporting compressing algorithm doesn't work as it should.

One example is in patch.bundle inside gameplay\abilities\monster_abl_base.xml; if you export the file somewhere and then if you try to reimport it inside the bundle even with no change at all the reimport will say that the file is too large.

Only certain extracted files have this problem (usually large files) while for all the rest it works flawlessly. I used QuickBMS to reimport changed files in the Witcher 3 bundles without problems now for almost a month, but now I'm stucked because there are certain files (as I said usually large ones) that have this problem and cannot be reimported correctly.

Do you think there could be a solution to this issue?

[aluigi]

Hi, yes I remember your email and I also replied to it.
It was about lz4 and reimporting.
The problem is unclear because the other users didn't report issues with the latest quickbms and script.
Anyway the following is my original that hopefully is still valid for solving your problem:

Code: Select all

Hi,

can you check if you are using quickbms 0.6.4?

It may be possible that these specific files are compressed with lz4.
The lz4 code used in versions before 0.6.4 wasn't able to compress the
files good like the code used in the game (lz4 has been improved a lot
recently) and that caused a difference of few bytes that didn't allow to
reimport some files back.

Anyway there is a worst problem with lz4, the library returns an error
if the size of the new compressed data is minor than the original one
because it handles also the bytes between its end and the original
limit.
The full details are available here:
http://zenhax.com/viewtopic.php?p=5500#p5500

Just in case it may be helpful, recently I updated also the script to
version 0.1.2 for adding another compression algorithm used by the game:
http://aluigi.org/papers/bms/others/witcher3.bms

You may also like to post on the zenhax forum in case other people have
the same problems or have found a solution.

[Amioran]

Updating to 0.6.4 and the new witcher3.bms resolved the issue, thanks!

I didn't think about checking up if there was an update of quickbms to resolve the problem, my bad.

Anyway, thank you very much for taking the time to reply, now I can finally alter some specific things I couldn't before for my mod.

EDIT:

I did rejoice too soon. The reimporting doesn't give an error now, but it corrupts the bundle anyway. Using the aforementioned "monster_base_abl.xml" and reimporting it as it is (i.e. just extracting the file and reimporting it back) in the patch.bundle will corrupt it: all monsters will have level -1 with no health bar and unkillable. At first I thought I made an error somewhere while modifying the file, but then I made a test with the original source (and to be sure starting from an original backup of patch.bundle) and the error persisted. Not reimporting just that specific file fixed the issue (everything worked fine).

So, albeit now the reimporting process doesn't give errors there's still something wrong in the compression somewhere (or some other problem, you are surely much more knowledgeable than me on the matter to understand the possible cause) because when using those files (even without any change whatsoever from the extracted source) will corrupt the bundle when reimporting them back.

To be sure I tried 5 different times, with different files and starting always from an intact bundle source. The first two tries were with "monster_base_abl.xml" in patch.bundle (under gameplay\abilities). Extracted it, reimported, loaded save file with a group of wolves in the near vicinity, all of them had -1 level and unkillable. The third try was with "def_item_weapons.xml" in the same patch.bundle (under gameplay\items). Did the same thing, loaded save game and most weapons were vanished altogether from the inventory. The fourth and fifth try was with "def_item_weapons_secondary.xml" in the xml.bundle (under gameplay\items). Same procedure, loaded save game and all secondary weapons were gone from the inventory (as before because much probably unrecognized).

As for the problem with the file size that cannot be less than the original one, I did already know. Even a single character difference and reimporting the file will corrupt the archive. You must be sure that the file size is exactly the same (and it's an hassle many times).

[aluigi]

That's the problem explained in the second paragraph. It's how the lz4 library works and currently there is no solution, it's independent from quickbms.

[Amioran]

That's the problem explained in the second paragraph. It's how the lz4 library works and currently there is no solution, it's independent from quickbms.

Ok, I understand.

I guess that at this point the only solution is a rebuilder. There's already a way to create bundles using the Witcher executable but the way the metadata.store works is not yet fully understood and there's no way to modify it. Right now is a complete mess to recreate the metadata.store and then redistributing it would require from users to do that same mess.

EDIT:

I have a strange idea, actually. Right now Quickbms checks the uncompressed file size, but if you could also check the compressed one then you could, for example, add characters to the source in forms of comments to reach the same exact compressed length, hence resolving the issue of the garbage being added (in the previous versions you could actually see the compressed size of the file you were about to reimport). In little words: if you could add an option that instead of checking the uncompressed file size it checks the compressed one and, knowing how many bytes are needed to fill the gap, then you could easily add comments in the file to resolve the issue (a thing that you must do in many cases anyway when you have a file size that's lower than the original because you removed some line, for example). It would be the same exact thing only done for the compressed size instead of the uncompressed one and in an "exact" way (i.e. enabling the reimport only if the *compressed* file size is exactly the same).

If I'm saying an idiocy please excuse me, as I'm not very versed in compression algorithms, however it is similar to hex editing yourself the LZ4 compressed file to remove the garbage and simply add 00s to reach the same size. But I can understand that doing all this mess just for a single game is a bit nonsensical (this also in the case my idea would actually work to begin with).

[aluigi]

You can't add data to the new file before recompressing it because the game may not appreciate it.
It's like reading a text file with 4kb of zeroes at the end, and it's worst if it's a specific format that reads every single byte of the file including the new added ones.
Additionally it means that a sort of brute forcer is necessary.
quickbms is a general tool so it must follow rules that work in any situation.

That's also the reason why it's not possible to modify the original compressed size value, simply because nobody knows where it comes from. It can be a 32bit field read from the file or from a memory file or calculated with a math operation or just a fixed number... anything.

What quickbms already does in reimporting is adding zeroes after the new file to match the size of the original file (line 1801 of src\file.c).

[Doggystyle]

., can we translate the subtitles? Is it possible?

I want to translate this game to my own language but i can't find any tool about it..

[aluigi]

Personally I don't know where are located the subtitles, maybe someone else knows that.

[Haoose]

The Witcher 3: Wild Hunt now with the mod support
http://thewitcher.com/news/view/1094

Download Witcher_3-Modkit:
http://filedelivery.nexusmods.com/Misc/ ... da68d1a8af

[SLaYeR983]

Personally I don't know where are located the subtitles, maybe someone else knows that.

in -Game dir\content\contentx... just w3strings files...

Decode and encode systems similar like Gibbed Red Tool (for Witcher2). We waiting for your support...

[ner0]

You can't add data to the new file before recompressing it because the game may not appreciate it.
It's like reading a text file with 4kb of zeroes at the end, and it's worst if it's a specific format that reads every single byte of the file including the new added ones.
Additionally it means that a sort of brute forcer is necessary.
quickbms is a general tool so it must follow rules that work in any situation.

That's also the reason why it's not possible to modify the original compressed size value, simply because nobody knows where it comes from. It can be a 32bit field read from the file or from a memory file or calculated with a math operation or just a fixed number... anything.

What quickbms already does in reimporting is adding zeroes after the new file to match the size of the original file (line 1801 of src\file.c).

This is a really annoying problem...
For example, I'm trying to tweak an available mod to my personal taste by changing values related to an NPC inside def_item_weapons.xml
After reinjecting the files what happens is that usually the game doesn't like the values I've put in and the result is that the NPC loses it's weapon.
So I keep trying and trying until it works without actually understanding what makes it work or not work.
Here's an example of 2 lines from the original def_item_weapons.xml:

Code: Select all

<SlashingDamage min="250" max="250"/>
<SilverDamage min="220" max="220"/>

If I change the values in the second line from 220 to 250, the compressed and uncompressed byte count is the same but the game will let me know that something is not right by stripping the NPC of it's weapon. Now, if I change the same values to 240, then it will work. I can't figure out an exact logic but the repetition of numbers will influence the outcome.
If I do the opposite it also works, I change the values in the 1st line from 250 to 240 and the values in the 2nd line from 220 to 250. Another example, changing the values in the 2nd line from 220 to 210 will also fail, but keeping this change and changing some 2 other lines somewhere else in that xml from "0.10" to "0.40" will make it work again. This is pretty weird...
So the compressed/uncompressed size is not all, what else could there be? I mean, quickbms shows both sizes and they're always consistent both when it works and when it doesn't.