LZW Implementation

[Ekey]

There is a realization of LZW algorithm is similar to the code below?

Code: Select all

int LZWEncode(unsigned char* pScrBuffer, unsigned char* pDstBuffer) - It should return the compressed size
int LZWDecode(unsigned char* pScrBuffer, unsigned char* pDstBuffer) - It should return the decompressed size

I see only by file stream realizations. Like

Code: Select all

void LZWEncode(FILE *fin, FILE *fout)
void LZWDecode(FILE *fin, FILE *fout)

[aluigi]

Have you already checked quickbms?
I have seen many LZW* in various games and implemented lot of them so it's highly possible that it's already there.
And if it's there, there is also the source code for performing the memory->memory decompression.

[Ekey]

Yeah, I already checked QuickBMS source's. Correct algorithm is clzw for The Binding of Isaac compressed data, but it also works like FileIn, FileOut

Code: Select all

   lzw_dec_init(ctx, fout);
   while (len = lzw_readbuf(fin, buf, sizeof(buf)))
   {
      int ret = lzw_decode(ctx, buf, len);
      if (ret != len)
      {
         fprintf(stderr, "Error %d\n", ctx);
         break;
      }
   }

[aluigi]

100% memory2memory or quickbms doesn't use it
From src/perform.c:

Code: Select all

            lzw_dec_t  *clzw_ctx;
            clzw_ctx = calloc(1, sizeof(lzw_dec_t));
            if(!clzw_ctx) STD_ERR(QUICKBMS_ERROR_MEMORY);
            clzw_outsz = 0;
            lzw_dec_init(clzw_ctx, out);
            lzw_decode(clzw_ctx, in, zsize);
            size = clzw_outsz;
            FREE(clzw_ctx);

[Ekey]

Here is my attempt to do something like this

Code: Select all

int LZWDecode(unsigned char* pScrBuffer, unsigned char* pDstBuffer)

But in the output buffer I see incorrect data.

QuickBMS result with 1024 bytes is:

Code: Select all

<achievements gfxroot="resources/gfx/ui/achievement/">
   <!-- have 7 or more max red hearts at one time -->
   <achievement id="1" text='You unlocked "Magdalene"' gfx="Achievement_Magdalene.png" />
   <!-- hold 55 pennies at one time -->
   <achievement id="2" text='You unlocked "Cain"' gfx="Achievement_Cain.png" />
   <!-- kill Satan for the 1st time -->
   <achievement id="3" text='You unlocked "Judas"' gfx="Achievement_Judas.png" />
   <!-- beat Mom for the 1st time -->
   <achievement id="4" text='You unlocked "The Womb" -Chapter 4- Mother sleeps' gfx="Achievement_TheWomb.png" />
   <!-- beat Mom for the 1st time -->
   <achievement id="5" text='You unlocked "The Harbingers" The Horsemen are loose' gfx="Achievement_Harbingers.png" />
   <!-- beat Mom for the 1st time -->
   <achievement id="6" text='"A Cube of Meat" has appeared in the basement' gfx="Achievement_CubeOfMeat.png" />
   <!-- beat a horseman for the 1st time -->
   <achievement id="7" text='"The Book of Revelations" has appeared in the basement' gfx="Achi<achievements gfxroot="resources/gfx/ui/achievement/">
   <!-- have 7 or more max red hearts at one time -->
   <achievement id="1" text='You unlocked "Magdalene"' gfx="Achievement_Magdalene.png" />
   <!-- hold 55 pennies at one time -->
   <achievement id="2" text='You unlocked "Cain"' gfx="Achievement_Cain.png" />
   <!-- kill Satan for the 1st time -->
   <achievement id="3" text='You unlocked "Judas"' gfx="Achievement_Judas.png" />
   <!-- beat Mom for the 1st time -->
   <achievement id="4" text='You unlocked "The Womb" -Chapter 4- Mother sleeps' gfx="Achievement_TheWomb.png" />
   <!-- beat Mom for the 1st time -->
   <achievement id="5" text='You unlocked "The Harbingers" The Horsemen are loose' gfx="Achievement_Harbingers.png" />
   <!-- beat Mom for the 1st time -->
   <achievement id="6" text='"A Cube of Meat" has appeared in the basement' gfx="Achievement_CubeOfMeat.png" />
   <!-- beat a horseman for the 1st time -->
   <achievement id="7" text='"The Book of Revelations" has appeared in the basement' gfx="Achi

My crapy result with 511 bytes is:

Code: Select all

<achievements gfxroot="resource/gx/ui/aheeet/">
   <!-- hav 7 or moe ax rd eart at netim ->
<civmn id=1" text'Yo unlocked"Magdalee"' f=Acemt_Mgae.png"/>   !-hold55 penisa eie-
   aieni=2"tx='oulce "Cain" x=cve_Ci.n"><- kill Saan fo he1sttm-
aee d"3"etYu nokdJuds"g=hvn_Jd.g /
<-betMom oth 1ti ->avti"4"x'uuoe"ThWob"-Chptr4-Mtersleps'g"AhetTeWmb."
! baMmfrt s i-   ce =5"toncdT Habiges THose r los'f"cm_Hrbne. >!bto rh1 m-
aei6"=" Cub f Me"hsppaei ebasm'x"hmCbOfMa./
!e  oea   tm>cei7"'"hBok  Relaionsh per ntbse'=i0-Р

Based on this source's (clzw from QuickBMS)

Where I made a mistake lol?

[aluigi]

Why don't have you used lzw-dec.c in your code?
Your implementation of clzw is a bit different than the library.

[Ekey]

I do not see the difference, because give identical good results in 1024 bytes: QuickBMS library (lzw-dec.c) and (Google Code lzw.c)

[Ekey]

Ok seems found problem

After part

Code: Select all

        strlen = lzw_get_str(ctx, nc, buff, sizeof(buff));
        c = buff[sizeof(buff) - strlen];
        memcpy(pDstBuffer, buff+(sizeof(buff) - strlen), strlen);

Add

Code: Select all

       for(int i = 0; i < strlen; i++)
           pDstBuffer++;

Decompressed size:

Code: Select all

return isize * 2 + 1; // +1 mean first byte which is copied

[Ekey]

Well, the problem is still there, not all blocks can be decompressed, god damn! QuickBMS also give invalid dump

Chunk 1: Offset at 0xE > ZSize is 602 - Valid dump
Chunk 2: Offset at 0x26C > ZSize is 436 - Invalid dump

[aluigi]

I tried also to concatenate chunk1 and chunk2 but it fails too.