LEGO Dimensions

[AlphaTwentyThree]

Hi there!

I've tried Luigi's ttgames.bms script on this game but it doesn't work. Does this use a variant? Can the script be updated? Here are some *.hdr samples: http://www101.zippyshare.com/v/1MhUWYEf/file.html
Thanks for your help!

Regards, Timo

[aluigi]

Encrypted

[AlphaTwentyThree]

Dammit :/

[Teancum]

Works for me. I used QuickBMS 0.7.1 and the latest ttgames.bms script. If I remember right it didn't work on 0.7.2[x]

[AlphaTwentyThree]

Huh, now I'm curious why this could be the case. Did you use it on the Xbox or PS3 version?

[Teancum]

Xbox 360.

[AlphaTwentyThree]

Do you still have this older version so I can test?

[Teancum]

http://aluigi.altervista.org/quickbms.htm

About 10% down on the page look for the text "Old versions of QuickBMS to download:". 0.7.1 should be in there. It's fastest just to CTRL+F the web page and search for "0.7.1". I used the 0.8 script, which I think is the newest, but in case it's not...

Code: Select all

# Traveller's Tales games DAT files extractor (script 0.8)
#  Lego Movie: The Video Game
#  LEGO Batman 1
#  LEGO Batman 2
#  LEGO Star Wars
#  LEGO Star Wars III
#  LEGO Indiana Jones
#  LEGO Harry Potter
#  LEGO Pirates of the Caribbean
#  LEGO Lord of the Rings
#  Transformers
#  LEGO Worlds
#  others, check them on http://www.ttgames.com
#
# In case of problems with the extraction try setting NAMELESS to 1
# thanx to who fixed the handling of the names!
#
# Note that the script may not work with the compressed files of
# Lego Movie for PS3, in case of crashes or other issues you can
# extract only the non compressed files by setting
# EXTRACT_COMPRESSED to 0.
#
# script for QuickBMS http://quickbms.aluigi.org

quickbmsver "0.7.1"
set NAMELESS long 0             # set to 1 to extract the files without names
set EXTRACT_COMPRESSED long 1   # set to 0 to extract ONLY the non-compressed files

getdstring SIGN 4
goto 0
math ONE_FILE = 0
  if SIGN == "LZ2K"
    math ONE_FILE = 1
elif SIGN == "DFLT"
    math ONE_FILE = 1
elif SIGN == "ZLIB" # currently doesn't exist
    math ONE_FILE = 1
elif SIGN == "LZMA" # currently doesn't exist
    math ONE_FILE = 1
endif
if ONE_FILE != 0
    math OFFSET = 0
    get ZSIZE asize
    math SIZE = ZSIZE
    callfunction UNPACK
    log "" 0 SIZE MEMORY_FILE2
    cleanexit
endif

get HDR_NAME basename
string HDR_NAME += ".hdr"
open FDSE HDR_NAME 1 EXISTS

if EXISTS != 0
    get NAME filename
    if NAME == HDR_NAME
        get NAME basename
        string NAME += ".dat"
        open FDSE NAME
    endif
    get HDR_SIZE asize 1
    log MEMORY_FILE 0 HDR_SIZE 1
    get DUMMY long MEMORY_FILE
else
    get INFO_OFF long
    if INFO_OFF & 0x80000000
        math INFO_OFF ^= 0xffffffff
        math INFO_OFF <<= 8
        math INFO_OFF += 0x100
    endif
    get INFO_SIZE long
    log MEMORY_FILE INFO_OFF INFO_SIZE
endif

get TYPE_BOH signed_long MEMORY_FILE
get FILES long MEMORY_FILE

math NEW_FORMAT = 1
if TYPE_BOH != 0x3443432e   # game.hdr
if TYPE_BOH != 0x2e434334
if FILES    != 0x3443432e   # game.dat
if FILES    != 0x2e434334
    math NEW_FORMAT = 0
endif
endif
endif
endif

if NEW_FORMAT == 1   # ".CC400TAD"

    goto 0 MEMORY_FILE
    endian big
    get HDR_SIZE long MEMORY_FILE
    idstring MEMORY_FILE ".CC4"
    idstring MEMORY_FILE "0TAD"
    get TYPE_BOH signed_long MEMORY_FILE    # -8
    get DUMMY long MEMORY_FILE  # 1
    get FILES long MEMORY_FILE
    get NAMES long MEMORY_FILE
    get NAMES_SIZE long MEMORY_FILE
    savepos NAMES_OFF MEMORY_FILE

    math OFFSET = NAMES_OFF
    math OFFSET + NAMES_SIZE
    goto OFFSET MEMORY_FILE

    putarray 10 NAMES ""    # necessary and useful
    putarray 1  NAMES ""    # necessary and useful
    get DUMMY long MEMORY_FILE
    math MYID = 0
    xmath LAST_NAME_WORKAROUND "NAMES - 1"
    for i = 0 < NAMES
        get NAME_OFF long MEMORY_FILE
        get FOLDER_ID short MEMORY_FILE
        get SOME_ID signed_short MEMORY_FILE
        get FILE_ID short MEMORY_FILE

        if NAME_OFF != 0xffffffff
            savepos TMP MEMORY_FILE
            math NAME_OFF + NAMES_OFF
            goto NAME_OFF MEMORY_FILE
            get NAME string MEMORY_FILE
            goto TMP MEMORY_FILE

            # I will find a solution another day maybe...
            if i == LAST_NAME_WORKAROUND
                math FILE_ID = MYID
            endif

            getarray PATH 1 FOLDER_ID
            string NAME p "%s\%s" PATH NAME
            if FILE_ID != 0
                putarray 10 MYID NAME
                math MYID + 1
            else
                putarray 1 i NAME
            endif
        endif
    next i

    get TYPE_BOH signed_long MEMORY_FILE    # -8
    get FILES long MEMORY_FILE

    for i = 0 < FILES
        get OFFSET long MEMORY_FILE
        get ZSIZE long MEMORY_FILE
        get SIZE long MEMORY_FILE
        get PACKED byte MEMORY_FILE
        get ZERO short MEMORY_FILE
        get OFFSET2 byte MEMORY_FILE
        math OFFSET u<< 8
        math OFFSET | OFFSET2
        putarray 2 i OFFSET
        putarray 3 i ZSIZE
        putarray 4 i SIZE
        putarray 5 i PACKED
    next i

    for i = 0 < FILES
        get CRC long MEMORY_FILE
        putarray 0 i CRC
    next i

    open FDDE "DAT" # in case HDR has been opened
    for i = 0 < FILES
        getarray FULLNAME 10 i
        callfunction GET_NAME 1

        getarray OFFSET 2 TMP
        getarray ZSIZE  3 TMP
        getarray SIZE   4 TMP
        getarray PACKED 5 TMP

        callfunction EXTRACT_FILE 1
    next i

else

    savepos INFO_OFF MEMORY_FILE

    math TMP = FILES
    math TMP *= 16
    math NAME_INFO = INFO_OFF
    math NAME_INFO += TMP

    goto NAME_INFO MEMORY_FILE
    get NAMES long MEMORY_FILE
    savepos NAME_INFO MEMORY_FILE

    math NAME_FIELD_SIZE = 8
    if TYPE_BOH <= -5
        math NAME_FIELD_SIZE = 12
    endif

    math TMP = NAMES
    math TMP *= NAME_FIELD_SIZE
    math NAME_OFF = NAME_INFO
    math NAME_OFF += TMP

    goto NAME_OFF MEMORY_FILE
    get NAMECRC_OFF long MEMORY_FILE
    savepos NAME_OFF MEMORY_FILE

    math NAMECRC_OFF += NAME_OFF
    goto NAMECRC_OFF MEMORY_FILE
    for i = 0 < FILES
        get CRC long MEMORY_FILE
        putarray 0 i CRC
    next i

    if TYPE_BOH <= -2
        get DUMMY1 signed_long MEMORY_FILE
        get DUMMY2 long MEMORY_FILE
    endif

    # print "files:       %FILES%"
    # print "names:       %NAMES%"
    # print "info_off:    %INFO_OFF%"
    # print "info_size:   %INFO_SIZE%"
    # print "name_info:   %NAME_INFO%"
    # print "name_off:    %NAME_OFF%"
    # print "namecrc_off: %NAMECRC_OFF%"

    set NAMEZ long 0
    set FULLNAME string ""
    set FULLPATH string ""

    for i = 0 < FILES
        callfunction SET_NAME 1
        callfunction GET_NAME 1

        math TMP *= 16
        math TMP += INFO_OFF
        goto TMP MEMORY_FILE
        get OFFSET long MEMORY_FILE
        get ZSIZE long MEMORY_FILE
        get SIZE long MEMORY_FILE
        get PACKED threebyte MEMORY_FILE
        get OFFSET2 byte MEMORY_FILE
        math OFFSET <<= 8
        math OFFSET += OFFSET2

        callfunction EXTRACT_FILE 1
    next i

endif

startfunction EXTRACT_FILE
    endian little   # necessary, they are little endian on big endian archives too
    goto OFFSET
    getdstring SIGN 4
      if SIGN == "LZ2K"
        set PACKED long 2
    elif SIGN == "DFLT"
        set PACKED long 3
    elif SIGN == "ZLIB"
        set PACKED long 3
    elif SIGN == "LZMA"
        set PACKED long 3
    elif SIGN == "ZIPX"
        set PACKED long 3
    else
        set PACKED long 0
    endif

    if PACKED & 2   # includes both 2 and 3
        if EXTRACT_COMPRESSED != 0
        callfunction UNPACK
        log FULLNAME 0 SIZE MEMORY_FILE2
        endif
    else
        if SIZE  != 0
        if ZSIZE != 0
        if SIZE  != ZSIZE
            print "SIZE (%SIZE%) and ZSIZE (%ZSIZE%) differ at offset %OFFSET|x%, contact me"
            cleanexit
        endif
        endif
        endif
        log FULLNAME OFFSET SIZE
    endif
endfunction

startfunction SET_NAME
    do
        goto NAME_INFO MEMORY_FILE
        get NEXT signed_short MEMORY_FILE
        get PREV signed_short MEMORY_FILE
        get OFF signed_long MEMORY_FILE
        if TYPE_BOH <= -5   # if NAME_FIELD_SIZE >= 12
            get DUMMY long MEMORY_FILE
        endif
        savepos NAME_INFO MEMORY_FILE

        if OFF < 0
            set NAME string ""
        else
            math OFF += NAME_OFF
            goto OFF MEMORY_FILE
            get NAME string MEMORY_FILE
        endif

        # used only for LEGO the game if you don't use the hdr file
        getvarchr TMP0 NAME 0
        if TMP0 >= 0xf0
            set NAME string ""
        endif
        #string NAME u= NAME # needed for the crc check, but doesn't improve performances so much

        if PREV != 0
            getarray FULLPATH 1 PREV
        endif
        putarray 1 NAMEZ FULLPATH   # putarray 1 NAMEZ NAME
        if NEXT > 0 # folder
            getarray TMP 1 PREV
            if TMP != ""    # long story to avoid things like 2foldername that gives problems to QuickBMS
                set OLDNAME string \    # do not use /
                string OLDNAME += TMP
                string OLDNAME += \     # do not use /
                string FULLPATH >>= OLDNAME
            endif
            if NAME != ""
                #string FULLPATH += \    # do not use /
                string FULLPATH += NAME
                string FULLPATH += \    # do not use /
            endif
        endif
        math NAMEZ += 1
    while NEXT > 0
    set FULLNAME string FULLPATH
    string FULLNAME += NAME
endfunction

startfunction GET_NAME
   getvarchr TMP0 FULLNAME 0
   if TMP0 == '\\'
      string FULLNAME <<= 1
   endif

    if NAMELESS == 0
        math CRC = 0x811c9dc5
        strlen NAMESZ FULLNAME
        string FULLNAME u= FULLNAME
        string FULLNAME R= / \
        for j = 0 < NAMESZ
            getvarchr CHR FULLNAME j
            math CRC ^= CHR
            math CRC *= 0x199933
        next j
        math CRC &= 0xffffffff
        for j = 0 < FILES
            getarray TMP 0 j
            if CRC == TMP
                break
            endif
        next j
        if j >= FILES
            print "Alert: the crc of the file %FULLNAME% has not been found, I extract the current file"
            #cleanexit
            # all wrongs!
            math j = i
            #math TMP = NEXT
            #math j = 0
            #math j -= NEXT
            #math j -= 1
        endif
        math TMP = j
    else
        math TMP = i
        set FULLNAME string ""
    endif
endfunction

startfunction UNPACK
    putvarchr MEMORY_FILE2 SIZE 0
    log MEMORY_FILE2 0 0
    append
    for TMPSZ = 0 < ZSIZE
        goto OFFSET
        getdstring SIGN 4
          if SIGN == "LZ2K"
            comtype lz2k
            get CHUNK_SIZE long
            get CHUNK_ZSIZE long
        elif SIGN == "DFLT"
            comtype dflt
            get CHUNK_ZSIZE long
            get CHUNK_SIZE long
        elif SIGN == "ZLIB" # currently doesn't exist
            comtype zlib
            get CHUNK_ZSIZE long
            get CHUNK_SIZE long
        elif SIGN == "LZMA" # currently doesn't exist
            comtype lzma
            get CHUNK_ZSIZE long
            get CHUNK_SIZE long
        elif SIGN == "ZIPX"
            comtype deflate # ??? not tested
            get CHUNK_ZSIZE long
            get CHUNK_SIZE long
            callfunction LONG_TO_DATA 1
            encryption rc4 TMP "" 0 4
        else
            print "Error: the compressing signature at offset %OFFSET% (%SIGN%) is not known, contact me"
            cleanexit
        endif
        savepos OFFSET
        if CHUNK_ZSIZE == CHUNK_SIZE
            log MEMORY_FILE2 OFFSET CHUNK_SIZE
        else
            clog MEMORY_FILE2 OFFSET CHUNK_ZSIZE CHUNK_SIZE
        endif
        encryption "" ""
        math OFFSET += CHUNK_ZSIZE
        savepos TMP MEMORY_FILE2
        math TMPSZ += 12
        math TMPSZ += CHUNK_ZSIZE
    next
    append
endfunction

# String TMP = CHUNK_ZSIZE will be fixed in quickbms 0.6.8
startfunction LONG_TO_DATA
    math TMP2 = CHUNK_ZSIZE
    set TMP string "0000"
    putvarchr TMP 0 TMP2
    math TMP2 >> 8
    putvarchr TMP 1 TMP2
    math TMP2 >> 8
    putvarchr TMP 2 TMP2
    math TMP2 >> 8
    putvarchr TMP 3 TMP2
endfunction

[aluigi]

quickbms 0.7.3 is the latest and it works like 0.7.1.
the latest script is 0.8b but it differs only in the way it works with files that don't have a crc in the database.

[AlphaTwentyThree]

I think my source files could be errornous. I will check again with another image.

[Teancum]

@Alpha -- is there anything particular you want to look at? I could PM you a few files (obviously not the whole thing).

[AlphaTwentyThree]

Nah, that's ok, I'll have another go myself, thanks.

[aluigi]

LEGO Dimensions is now supported (0.9).
Basically in this game TTGames uses a lzma compressed DAT that has the signature "CMP2CMP2CMP2CMP2".
Would be interesting to know if exist a "CMP1CMP1CMP1CMP1" version or similar in other games...

*edit* moved my post from the other LEGO thread

[pieterjee]

hi, since year 2 is released I tried to extract the dat files of the new dlc, but the script doesn't work on them. The file names are correct, but the offsets are incorrect. The old ones do extract properly.

[aluigi]

Can you provide a sample archive?

[pieterjee]

http://www.filedropper.com/dlc11

The previous year does work for me with the same extraction method: move from xbox to xbox hard disc with transfer cable, use xplorer360 to get the file, and then use wxpirs to extract the dat file. This file gives me incorrect file offsets.

[luna rom]

Same issue here it seems that they've updated the format, again.

[aluigi]

Sorry for the delay.
Script 0.9.4
http://aluigi.org/bms/ttgames.bms

[luna rom]

Hate to ask for help again but They've update the format again. Here is a file https://www.mediafire.com/?oo369fzj60fmk35 that has the most recent update.

[aluigi]

Script 0.9.6.
Luckily this time it was easy: 64bit offset instead of the previous 32bit one