lzovl -> Reverse LZ format

Extraction and unpacking of game archives and compression, encryption, obfuscation, decoding of unknown files
chrrox
Posts: 388
Joined: Thu Aug 07, 2014 10:28 pm

lzovl -> Reverse LZ format

Post by chrrox »

Can this be added to quickbms.
There is source code for it.
https://code.google.com/p/dsdecmp/
And i have attached a sample.
chrrox
Posts: 388
Joined: Thu Aug 07, 2014 10:28 pm

Re: lzovl -> Reverse LZ format

Post by chrrox »

it appears the tool miss calculated the size and did not add it to the header at the end.
But i am still having trouble with this file.
http://gbatemp.net/threads/recompressin ... le.329576/
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: lzovl -> Reverse LZ format

Post by aluigi »

dsdecmp contains various algorithms and quickbms already implements most of them.
You can check them in goldensun.c, luminousarc.c and nintendo.c

The list of comtypes should be the following:
lz77wii
lz77wii_raw10
lz77wii_raw11
lz77wii_raw20
lz77wii_raw30
lz77wii_raw40
goldensun
luminousarc

It's not clear in that pmo files where the compressed data starts.
chrrox
Posts: 388
Joined: Thu Aug 07, 2014 10:28 pm

Re: lzovl -> Reverse LZ format

Post by chrrox »

its this format
https://code.google.com/p/dsdecmp/sourc ... s/LZOvl.cs


Code: Select all

            #region Format description
            // Overlay LZ compression is basically just LZ-0x10 compression.
            // however the order if reading is reversed: the compression starts at the end of the file.
            // Assuming we start reading at the end towards the beginning, the format is:
            /*
             * u32 extraSize; // decompressed data size = file length (including header) + this value
             * u8 headerSize;
             * u24 compressedLength; // can be less than file size (w/o header). If so, the rest of the file is uncompressed.
             *                       // may also be the file size
             * u8[headerSize-8] padding; // 0xFF-s
             *
             * 0x10-like-compressed data follows (without the usual 4-byte header).
             * The only difference is that 2 should be added to the DISP value in compressed blocks
             * to get the proper value.
             * the u32 and u24 are read most significant byte first.
             * if extraSize is 0, there is no headerSize, decompressedLength or padding.
             * the data starts immediately, and is uncompressed.
             *
             * arm9.bin has 3 extra u32 values at the 'start' (ie: end of the file),
             * which may be ignored. (and are ignored here) These 12 bytes also should not
             * be included in the computation of the output size.
             */
            #endregion


this looks like the lz-0x10
https://code.google.com/p/dsdecmp/sourc ... ro/LZ10.cs
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: lzovl -> Reverse LZ format

Post by aluigi »

ok, basically it works from the end to the beginning of the compressed data.

The code of dsdecmp has been updated from the last time I checked it for quickbms, what I don't understand is why lz10.cs differs so much than the old code.
chrrox
Posts: 388
Joined: Thu Aug 07, 2014 10:28 pm

Re: lzovl -> Reverse LZ format

Post by chrrox »

Yes it seems to be reverse compression.
Do yo think its possible to add this to quickbms?
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: lzovl -> Reverse LZ format

Post by aluigi »

yes, it's in my TODO list.

My doubt now is about the other algoritms because I know that they work in quickbms so I would avoid to re-porting the dsdecmp code again to match the current code.