Diablo 3 console CPK format (not CriWare!)

[PASAf]

In console version of Diablo 3 Blizzard used own format of archives. Extension of archives is CPK and they located at CPKs folder, but it is not CriWare format.
Signature of archive is \xA1 \xB2 \xC3 \xD4 and there is also second part - I think it is version of format or at least could be part of signature: \x00 \x00 \x00 \x06.
At least archive where cutscenes stored goes without any compression/encryption, since video stored in ogv format bounds of files could be easily determined by signature (OggS, theora, etc).
But header format is complicated. I was not able to find any clear values except of files count @ 0x14.
There is also plain list of filenames which order doesn't match the order of files placement in archive (at least in Cutscenes archive files stored in alphabetical order).

Results of my research:

Code: Select all

// Header
DWORD dwSignature = A1 B2 C3 D4
DWORD // 00 00 00 06 always - version? part of signature?
QWORD qwSize // in Cutscenes it is size of archive from start of files (0x10000) in other archives - not
DWORD // 00 00 00 02 everywhere except Cutscenes (00) - level of compression? encryption indicator?
DWORD dwFilesCount // number of files in archive
DWORD // everywhere except cache archives = dwFilesCount
DWORD // some small number, in Act1-4 = 01
DWORD // another small number, in localization archives (xxXX_Act1-4, e.g. enGB_Act1) = 12-13
DWORD // in most cases = 01, cache archives = 08 / 07, Common = 02
DWORD // small number in range 16-1F
DWORD // equals previous
DWORD // small number in range 00-0F
DWORD // some value (not offset/size)
DWORD // always null (00 00 00 00)
DWORD // unknown value (not offset/size)
DWORD // unknown value (not offset/size)
DWORD // unknown value (not offset/size) seems like little endian
DWORD // unknown value, in cache small number in other archives - big
...

Next goes some structure or table or/of hashes, I don't know, size of this area depends on number of files stored in archive (more files - bigger size), which is separated from list of filenames with some set of dwords nearly similar to arithmetic series (arithmetic progression) with shift, differ from archive to archive (for example : 00, 2C, 5A, 89, B1...). Size of this set is equals to number of files stored in archive. After filelist there is blank area filled with zeroes (not fixed size, differ from archive to archive) and finally then goes the files themselves (till the EOF of archive).

Both known values (FilesCount and Size in Cutscenes) is big endian.

Here is the smallest archive example: plPL_CacheCommon.cpk (1 Mb)
Rest of archives (including Cutscenes (~1.1Gb) and CacheAct (smallest - 200 Mb) archives): PM me please
Default.xex (if anyone skilled in reversing Xbox binaries, I'm - not): Default.xex (17 Mb)

[cockafej]

Hi Aluigi,

I tried uncompress the enUS_CacheCommon.cpk with your diablo3_xbox.bms, but the script running on error.

See, the first file, offset 30006, and the filesize is 0, filename is Conv_Fol_Player_TP_Wrld_14.stl
The first 78 DA identifier (offset 30006) is not the starting identifier without data?
Doesn't the extraction slipping with one line, file?
Doesn't the data of the Conv_Fol_Player_TP_Wrld_14.stl file (first file) starting the offset 30014?


The number of compressed files: 2839 pcs
The number of 78 DA identifer: 2902 pcs, but the first and the last, i think are not valid identifier (maybe header and footer? ) (offsets are: 30006 and FDEEE), so remains 2900 identifiers
The difference is: 61

Maybe more 78 DA identifier belongs some files?

Please, help me, look at your bms script, and try to modify it.

Thank you

The cpk file is here:
https://www18.zippyshare.com/v/f9tobMqc/file.html

[aluigi]

I checked the file and the data at offset 0x33e11 is probably a corrupted zlib stream because even offzip can't get anything there at that offset, there is just a "hole" during the scan.
Are you sure your sample is correct?

The only thing I can do is using comtype zlib_noerror, check the new script.

[cockafej]

The sample is original.

I attach other sample, but french language:

https://www39.zippyshare.com/v/fQQPd2FB/file.html

Thank you

You told, you made a new script?

[aluigi]

Yes I did a little edit to the script, I just added "comtype zlib_noerror".
The extracted files coming from the corrupted zlib data look right so I guess that it's a success

[cockafej]

You are awesome!

Thanks
The script is working, but only with this one file.

I attach the other file, which the script is not working:

https://www46.zippyshare.com/v/hA87Mvmz/file.html

Can you look at this?

Thank you very much

Edit: the reimport is not working even without modification

[aluigi]

Script 0.1.1
The problem was caused by the difficulty in finding where the names offsets start.

[Ekey]

https://github.com/zeroKilo/CPKReaderWV

[cockafej]

Thank you guys.
Both programs extract the files with same method.

[RedMadKnight]

I've made some time ago a fork from ZeroKillo. Here is my version of CPKReader - still can't extract files, but I've cracked the header.
I've have no time to crack it all, but maybe someone will use it good

Code: Select all

https://github.com/RedMadKnight/CPKReaderWV

[swuforce]

Here is a dummy mix of aluigi's and RedMadKnight's work for unpack the files.

Updated.

[dev42]

Here is a dummy mix of aluigi's and RedMadKnight's work for unpack the files.

Updated.

Thanks for this! The format for PS4/Switch is slightly different and I was wondering if anyone here could take a look and see if it's a quick fix? Attached a small example file

[swuforce]

There is an updated script, to work also for the ps4 version. Some chunks are probably encrypted, so if you see a warning az end of extraction, your files might be corrupted.
The repack script works only if there is no warning.
Usage:
unpack.bat YOURFILE.CPK
repack.bat YOURFILE.CPK

The repacked file will be in the YOURFILE folder.

[dev42]

There is an updated script, to work also for the ps4 version. Some chunks are probably encrypted, so if you see a warning az end of extraction, your files might be corrupted.
The repack script works only if there is no warning.
Usage:
unpack.bat YOURFILE.CPK
repack.bat YOURFILE.CPK

The repacked file will be in the YOURFILE folder.

Thank you! My friends and I can reverse any encryption key needed to improve the script, can you point to the name of anything specific you need?

[swuforce]

Thank you! My friends and I can reverse any encryption key needed to improve the script, can you point to the name of anything specific you need?

Check your pm.