Surviving Mars .hpk

[hhrhhr]

hpk format:

Code: Select all

// header, 36 bytes
char magick[4]; // "BPUL"
uint unk1;      // 36 (header size or data offset??)
uint unk2[5];   // 1, -1, 0, 0, 1

uint entry_offset;  // absolute
uint entry_size;    // entry_offset + entry_size = EOF

// data
char filedata[xxx]; // FOURCC can be "LZ4 " or other

// file desc
struct dir_entry[xxx] {
    uint ptr;   // entry[ptr] -> next item
    uint attr;  // 0 - directory, 1 - file
    short sz;
    char name[sz]   // name of current dir or file
}

// entry_offset
struct entry[xxx] {   // xxx = entry_size >> 3
    uint offset;        // absolute, pointer to dir_entry (included dir, file or filedata)
    uint size;          // size of dir_entry or zsize of file
}

LZ4 packed file:

Code: Select all

char magik[4];          // "LZ4\x20"
uint size;              // unpacked size
uint block_size;        // 2^17
uint h_size;            // header size; if size==0, then EOF here
uint chunk_offset[xxx]; // xxx = (h_size - 16) >> 2, absolute offsets of chunk begins

struct chunk[xxx] {     // offset = h_size
    char lz_data[];     // full LZ4 block (unpacked == block_size)
}
char lz_data_tail[];    // partial LZ4 block (unpacked < block_size)

for unpack can be used LZ4_decompress_safe (src, dst, zsize, size). if (entry.size == size - h_size) then file content is not packed.

implementation in Lua here.

[MerlinSVK]

Or you can use HPK Archiver, modded by Atom0s

https://github.com/atom0s/HpkArchiver

Win build:

HPK_archiver_1_0_14_mod.7z

[hhrhhr]

"quick&dirty" BMS-script for unpack lz4 compressed files (I hope that someone optimizes this code ;)

Code: Select all

idstring "LZ4 "
comtype lz4

#### bad example, output is corrupt
get SIZE long
get BLOCK long
get H_SIZE long
get ZSIZE ASIZE
math ZSIZE -= H_SIZE

clog "milky_way_bad.dds" H_SIZE ZSIZE SIZE


goto 4 #### back to begin

#### good example, CRC32 1AA851A4
get SIZE long
get BLOCK long
get H_SIZE long
get ZSIZE ASIZE
xmath CHUNKS "(H_SIZE - 16) / 4"

math I = 0
set CHUNK_OFF[I] H_SIZE
for I = 1 <= CHUNKS
    get CHUNK_OFF[I] long
next I
set CHUNK_OFF[I] ZSIZE

math CHUNKS + 1
for I = 1 <= CHUNKS
    math VAR = CHUNK_OFF[I]
    math I - 1
    math VAR - CHUNK_OFF[I]
    set CHUNK_SZ[I] = VAR
next I + 2
math CHUNKS - 1

for I = 0 < CHUNKS
    clog "milky_way_good.dds" CHUNK_OFF[I] CHUNK_SZ[I] BLOCK
    if I == 0
        append
    endif
next i
xmath TAIL "SIZE - CHUNKS * BLOCK"
clog "milky_way_good.dds" CHUNK_OFF[I] CHUNK_SZ[I] TAIL
append

[zbychos]

Why does the script not save the file under the file's input name?
You can not do it?

rm_mist.tga
+ 0006fada 18 milky_way_good.dds
Info: algorithm 249
offset 0006fada
input size 0x00000012 18
output size 0x00000012 18
result 0xfffffffc -4

Error: the uncompressed data (-4) is bigger than the allocated buffer (131072)

=============================================
Most of the "tga" files work.
In "lua" it only cuts out a few bytes from the beginning of the file.

[hhrhhr]

Why does the script not save the file under the file's input name?

because this is a test case for a single file. in your case, the input size is equal to the output, that is, no compression is used.

In "lua" it only cuts out a few bytes from the beginning of the file.

can an example? the name of the hpk-archive and the "cutted" file?

[makc_ar]

I found decompressor

Code: Select all

idstring "LZ4 "
comtype lz4
get SIZE long
get CHUNK_SIZE long
get NAME filename
xmath TMP "SIZE % CHUNK_SIZE"
if TMP != 0
   math TMP = 1
else
   math TMP = 0
endif
xmath CHUNKS "(SIZE / CHUNK_SIZE) + TMP"
append
for i = 1 <= CHUNKS
   get OFFSET long
   savepos POS
   if i == CHUNKS
      get ZSIZE asize
   else
      get ZSIZE long
   endif
   math ZSIZE -= OFFSET
   goto POS
   if ZSIZE < CHUNK_SIZE
      clog NAME OFFSET ZSIZE CHUNK_SIZE
   else
      log NAME OFFSET ZSIZE
   endif
next i
append

[zbychos]

Why does the script not save the file under the file's input name?

because this is a test case for a single file. in your case, the input size is equal to the output, that is, no compression is used.

In "lua" it only cuts out a few bytes from the beginning of the file.

can an example? the name of the hpk-archive and the "cutted" file?

hhrhhr
I apologize for my joke but I could not resist ...: D.
I know it's just an example.
OK, the end of jokes.

I used HPK Archiver

graphics catalog - UI
file UI\Icons\Research\rm_mist.tga
It may be a bug in the file but I have hpk once again and it's the same.

Lua 'Data' catalog
It does not work with any * .lua file
see lua before.rar and lua after.rar

Such a request: can you give a way to store the directory structure in the 'output' directory?

get FNAME filename
get FFOLDER input_folder
string NAME += FFOLDER
string NAME += "/"
string NAME += FNAME

It works more or less, but not as I expected. I have never had contact with QuickBMS before and for now I find it difficult to understand the basics.

[zbychos]

I found decompressor

makc_ar
I do not want to worry you, but it's cleansed and improved the same script.
In action nothing changed

[makc_ar]

zbychos
Try this viewtopic.php?f=9&t=2630

[hhrhhr]

I used HPK Archiver

I'm not the author of this program.

Such a request: can you give a way to store the directory structure in the 'output' directory?

if you have in mind the finalization of the bms-script, then I'm not interested. for full unpacking there is the aforementioned HPK Archiver. my lua script on githab (hpk_unpack.lua) also unpacks all lz4 compressed files.

[zbychos]

OK no problem. Thanx

[nuke974]

Someone in the world have figured out how to convert the .hgm file?