QuickBMS CRC

[Delutto]

Hi Aluigi.
I'm working on a Lords Of Fallen text tool and the localization files have a checksum verification, what can be done with this script:

Code: Select all

#------------------------------------------------
# Lords Of The Fallen language *.bin CRC fixer
# Script version: 0.2
# Author: merlinsvk
#------------------------------------------------
idstring "\xFC\x89\xC5\xA3"      # just a test if the input file is a supported .bin file

encryption crc 0x01800063 "32 0 0 19 1 1"
get SIZE asize
xmath TXTSIZE "SIZE - 0x18"      # 0x18 = size of header
log MEMORY_FILE 0x18 TXTSIZE   # 0x18 = start of the data block
encryption "" ""

get NAME basename
get EXT extension
string NAME p= "%s.%s_FIXED" NAME EXT
get SIZE asize
log MEMORY_FILE2 0 SIZE                     # create copy of file into RAM
putVarChr MEMORY_FILE2 0x04 QUICKBMS_CRC long   # write new CRC value on position 0x04
putVarChr MEMORY_FILE2 0x14 TXTSIZE long      # write text block size
log NAME 0 SIZE MEMORY_FILE2               # write new file on disk

But I don't understand what that two variables on "encryption crc" means: 0x01800063 "32 0 0 19 1 1".
I asked merlinsvk about it and he replied:

I have no idea. I just used Aluigi's CRC scanner and these values were in the results.

I would like to implement the crc update in my tool, can you explain me about it?

[aluigi]

Sure, this is the explanation from the quickbms.txt manual:

Code: Select all

                crc, a complete and powerful checksum function that can
                  be fully configured:
                  - key is the polynomial (use "" for the default crc32 0x77073096)
                  - ivec contains:
                    - the size of the crc (8/16/32/64)
                    - the initial value (like -1)
                    - the final xor value (-1, the complement)
                    - the type (various supported, check crc_calc in src/crc.c)
                    - the reverse/reflect mode during the generation of the table (0 or 1)
                    - the bitmask_side (0 or 1 where 1 is the most used one)
                  default values: 0xedb88320 32 -1 -1 0 0 1
                  if you need the classical crc16 (0xc0c1) use:
                    encryption crc 0xa001 "16 0 0 0 0 1"
                  or
                    encryption crc "" 16
                  the result is placed in the variable QUICKBMS_CRC
                  example for type 39:
                    encryption crc 0 "0 0 0 39 0 1"
                  for additional info:
                    http://aluigi.org/bms/quickbms_crc_engine.txt
                  for technical information about the operations check the
                  crc_calc function in crc.c, it's quite easy to understand
                  because it contains the simple operations performed in
                  each cycle.
                  note that some crc types use the polynomial value as
                  constant in each cycle
                  crc64 and 64bit crc work with quickbms_4gb_files.exe only

So the CRC engine will generate a crc 32bit table starting from the polynomial 0x01800063.

The next 2 values (0 and 0) are the initial and final values used for CRC calculation, usually the classical crc function is like the following:

Code: Select all

crc = 0xffffffff; // -1
for(i = 0; i < size; i++) {
    crc = ...stuff...
}
crc = ~crc; // which means crc ^= 0xffffffff;

In your case instead the initial value of crc is 0 and there is no xor/complement operation at the end.

19 is an internal number for selecting the crc operation since exist many types of crc:

Code: Select all

else if(ctx->type == 19) CRC_CALC_CYCLE(    MYBYTE + (MYCRC << 6) + (MYCRC << 16) - MYCRC) // sdbm 65599

which is converted to the following code without the "macros":

Code: Select all

    unsigned int crc = 0;
    for(i = 0; i < datalen; i++) {
        crc = data[i] + (crc << 6) + (crc << 16) - crc);
    }
    return crc;

Next 1 is used for generating the tables from the polynomial, I guess 0 is usually the default.
The final 1 is the default value and it's used for generating the table too.

No table is used in this crc so the code posted above is all you need

Anyway I'm going to add a variable called QUICKBMS_CRC_TABLE in quickbms 0.9.1 for dumping the (raw) generated table, it would be a lot more easy for users to port the code their programming language.

[Delutto]

Sure, this is the explanation from the quickbms.txt manual:

Shame on me, I totally forgot to check the quickbms.txt

So the CRC engine will generate a crc 32bit table starting from the polynomial 0x01800063.
The next 2 values (0 and 0) are the initial and final values used for CRC calculation, usually the classical crc function is like the following:

Code: Select all

crc = 0xffffffff; // -1
for(i = 0; i < size; i++) {
    crc = ...stuff...
}
crc = ~crc; // which means crc ^= 0xffffffff;

In your case instead the initial value of crc is 0 and there is no xor/complement operation at the end.

I get it.

19 is an internal number for selecting the crc operation since exist many types of crc:

Code: Select all

else if(ctx->type == 19) CRC_CALC_CYCLE(    MYBYTE + (MYCRC << 6) + (MYCRC << 16) - MYCRC) // sdbm 65599

which is converted to the following code without the "macros":

Code: Select all

    unsigned int crc = 0;
    for(i = 0; i < datalen; i++) {
        crc = data[i] + (crc << 6) + (crc << 16) - crc);
    }
    return crc;

Many thanks! Easy peasy... my pascal code:

Code: Select all

var
   Buffer: PByte;
   CRC: UInt32;
begin
   ...
   CRC := 0;
   for I := 0 to DataLength - 1 do
      CRC := PByte(Buffer + I)^ + (CRC shl 6) + (CRC shl 16) - CRC;
   ...
end;

Anyway I'm going to add a variable called QUICKBMS_CRC_TABLE in quickbms 0.9.1 for dumping the (raw) generated table, it would be a lot more easy for users to port the code their programming language.

This will be great! But please, don't give up of the quickbms.dll.

[extrme_sports]

Please share the .bms script.