Tokyo Xtreme Racer Drift 2 BUILD.DAT

Extraction and unpacking of game archives and compression, encryption, obfuscation, decoding of unknown files
Argonaut
Posts: 46
Joined: Sat Sep 27, 2014 10:24 pm

Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by Argonaut »

Files:

Sample of TXR Drift 2 BUILD.DAT + TOC: https://drive.google.com/open?id=0B9U7H ... 0FxdjhGYTg
Main TXR Drift 2 BUILD.DAT:

Sample of TXR 3 BUILD.DAT + TOC: https://drive.google.com/open?id=0B9U7H ... kNOLUJvNlE
Main TXR 3 BUILD.DAT: https://1drv.ms/u/s!AmM2j7pIas_EkUdes71nACOSDa8r



Also if I may express my thanks for a free community like this being available for free.. I'm not sucking up, I promise :lol:
Last edited by Argonaut on Mon Aug 08, 2016 5:09 pm, edited 2 times in total.
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by aluigi »

In my opinion there is an index file other than build.dat.
Anyway I used a work-around to get the files but it seems they are compressed using an unknown compression.
I leave the work-in-progress script but I think it's not useful:

Code: Select all

endian big
get DAT_SIZE asize
for OFFSET = 0 < DAT_SIZE
    findloc NEW_OFFSET string "\x00\xe9\x55\x43\x4c\xff\x01\x1a\x00\x00"

    if NEW_OFFSET != OFFSET
        xmath SIZE "NEW_OFFSET - OFFSET"
        log "" OFFSET SIZE
        goto NEW_OFFSET
    endif

    getdstring DUMMY 10
    get DUMMY long
    get XSIZE long
    get XSIZE long
    get SIZE long
    savepos OFFSET
    log "" OFFSET SIZE  # unknown compression
    math OFFSET += SIZE
    math OFFSET x= 0x800
    goto OFFSET
next
Argonaut
Posts: 46
Joined: Sat Sep 27, 2014 10:24 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by Argonaut »

aluigi wrote:In my opinion there is an index file other than build.dat.
Anyway I used a work-around to get the files but it seems they are compressed using an unknown compression.
I leave the work-in-progress script but I think it's not useful:

Code: Select all

endian big
get DAT_SIZE asize
for OFFSET = 0 < DAT_SIZE
    findloc NEW_OFFSET string "\x00\xe9\x55\x43\x4c\xff\x01\x1a\x00\x00"

    if NEW_OFFSET != OFFSET
        xmath SIZE "NEW_OFFSET - OFFSET"
        log "" OFFSET SIZE
        goto NEW_OFFSET
    endif

    getdstring DUMMY 10
    get DUMMY long
    get XSIZE long
    get XSIZE long
    get SIZE long
    savepos OFFSET
    log "" OFFSET SIZE  # unknown compression
    math OFFSET += SIZE
    math OFFSET x= 0x800
    goto OFFSET
next



Thank you very much for your input,

Yes there is a BUILD.TOC that I forgot to provide, Link very shortly being hosted! Stay tuned

EDIT: Here it is: https://www.dropbox.com/s/mtrrzdtymk57n ... D.TOC?dl=0

DOUBLE EDIT: Made the same mistake on Xentax forum when I first posted (goddammit), way to go :roll: Sorry for wasting your time.


I have to also add there are a variety of games in this series, one in particular that might use a different compression is TXR Zero which uses a CDDATA.000 and a .LOC index instead of .TOC.
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by aluigi »

With the TOC file the work is simpler, but remains the problem of the compression.
The second file in the archive is a TIM2, so I launched my scanner but all the results don't match the file format.
This is the script:

Code: Select all

open FDDE "TOC"
get FILES long
get DUMMY long
get DUMMY long
get DAT_SIZE long
for i = 0 < FILES
    get OFFSET long
    get ZSIZE long
    get SIZE long
    get ZERO long
    math OFFSET *= 0x800
    putarray 0 i OFFSET
    putarray 1 i ZSIZE
    putarray 2 i SIZE
next i

endian big
open FDDE "DAT"
for i = 0 < FILES
    getarray OFFSET 0 i
    getarray ZSIZE 1 i
    getarray SIZE 2 i
    if SIZE == 0
        log "" OFFSET ZSIZE
    else
        goto OFFSET
        getdstring SIGN 10
        get DUMMY long
        get XSIZE long
        get SIZE long
        get ZSIZE long
        get DUMMY long
        savepos OFFSET

        # unknown compression
        #clog "" OFFSET ZSIZE SIZE

        log "unknown_compression/*" OFFSET ZSIZE # ???
    endif
next i
Argonaut
Posts: 46
Joined: Sat Sep 27, 2014 10:24 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by Argonaut »

aluigi wrote:With the TOC file the work is simpler, but remains the problem of the compression.
The second file in the archive is a TIM2, so I launched my scanner but all the results don't match the file format.
This is the script:

Code: Select all

open FDDE "TOC"
get FILES long
get DUMMY long
get DUMMY long
get DAT_SIZE long
for i = 0 < FILES
    get OFFSET long
    get ZSIZE long
    get SIZE long
    get ZERO long
    math OFFSET *= 0x800
    putarray 0 i OFFSET
    putarray 1 i ZSIZE
    putarray 2 i SIZE
next i

endian big
open FDDE "DAT"
for i = 0 < FILES
    getarray OFFSET 0 i
    getarray ZSIZE 1 i
    getarray SIZE 2 i
    if SIZE == 0
        log "" OFFSET ZSIZE
    else
        goto OFFSET
        getdstring SIGN 10
        get DUMMY long
        get XSIZE long
        get SIZE long
        get ZSIZE long
        get DUMMY long
        savepos OFFSET

        # unknown compression
        #clog "" OFFSET ZSIZE SIZE

        log "unknown_compression/*" OFFSET ZSIZE # ???
    endif
next i


Honestly don't know whether my obvious lack of <10 brain cells is causing this issue, but after Selecting the script in a Notepad file (C&P's exactly) both identically named files (first the TOC then the DAT as per the script's process) and the folder, I get the Error no such file or directory. Ah well, I think the unknown compression issue pretty much kills 99.9% of the chance of getting stuff from this game without an offical SDK from Genki. Heck, I might try and contact them sometime. Ah well, onwards and upwards onto other games of which there are numerous opportunities.

Thanks again for your efforts, they are greatly appreciated.
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by aluigi »

My fault, I tought that BUILD.DAT.DAT was a mistake.
I have uploaded the script here:
http://aluigi.org/papers/bms/others/tok ... rift_2.bms
Argonaut
Posts: 46
Joined: Sat Sep 27, 2014 10:24 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by Argonaut »

aluigi wrote:My fault, I tought that BUILD.DAT.DAT was a mistake.
I have uploaded the script here:
http://aluigi.org/papers/bms/others/tok ... rift_2.bms


Perfect work+ great efficiency with splitting unknown Compressions into a different folder..
Argonaut
Posts: 46
Joined: Sat Sep 27, 2014 10:24 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by Argonaut »

Well, out of the entire extraction the only unknown compression files were all under 500kbs, and most of the obtain files are .DATs- two of which are 1GB a piece. Now I can take alittle inference to work out that either of these might contain 1) The maps 2)All the cars.

This will be the last thing I request on this specific game as I think no more could really be done to seperate the .DATs, so the sample files are listed below. I'd probably understand if nothing could really be grabbed from this (due to compression).

Samples:

https://www.dropbox.com/s/rlnk19fd89u4b ... e.zip?dl=0



EDIT: After some thought I've gone back to the original assumption of the 1GB files containing the car models+ extras and have given samples for that instead.
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by aluigi »

In my opinion that one is audio, like a headerless VAG.
There was a tool to play raw PS audio data but honestly I no longer remember the name.
Argonaut
Posts: 46
Joined: Sat Sep 27, 2014 10:24 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by Argonaut »

aluigi wrote:In my opinion that one is audio, like a headerless VAG.
There was a tool to play raw PS audio data but honestly I no longer remember the name.


That is one to consider actually- in my opinion, I saw the 4x156MB files as all the textures/minor models for each courses, but there are a pair of 208MB .DATs that also start with the .eUCLy header aswell, as does many of the .DAT files from this 208MB one down to the 1MB ones. Definite pattern here, this'll be the last sample as the pattern means the potential script (if anything) would not be just a single purpose job.. last one. Promise. (Have to be away for a few hours soon anyway)


Link: https://www.dropbox.com/s/jltb3tkphjbek ... e.zip?dl=0
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by aluigi »

So you have found these eUCLy outside the unknown_compression folder?
Because they are compressed.
Argonaut
Posts: 46
Joined: Sat Sep 27, 2014 10:24 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by Argonaut »

aluigi wrote:So you have found these eUCLy outside the unknown_compression folder?
Because they are compressed.


Yeah, I have. Ah well, that closes that then. Case closed
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by aluigi »

Those UCL files have been compressed with uclpack.
I have written a new script to support the Tokyo Xtreme Racer series (Drift 2, X and 1):
http://aluigi.org/papers/bms/others/tok ... _racer.bms

Note that the x360 files can't be extracted with the current version of quickbms because they use a modified version of uclpack.c where the ucl_decompress_8 is replaced with the ucl_decompress_32. I will add automatic support for it in the next version of quickbms.
Argonaut
Posts: 46
Joined: Sat Sep 27, 2014 10:24 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by Argonaut »

aluigi wrote:Those UCL files have been compressed with uclpack.
I have written a new script to support the Tokyo Xtreme Racer series (Drift 2, X and 1):
http://aluigi.org/papers/bms/others/tok ... _racer.bms

Note that the x360 files can't be extracted with the current version of quickbms because they use a modified version of uclpack.c where the ucl_decompress_8 is replaced with the ucl_decompress_32. I will add automatic support for it in the next version of quickbms.


Hi,

This is awesome work that I'm coming back to check on as I revisit 3DS Max work, however I've come across this issue with all games I've tested in the series (Drift 2 / Drift 1 / 3) which stops them from ever progressing past a few tiny files:

Image

I hope I'm doing this right- selecting the BUILD.DAT/TOC to be used- but otherwise I'm a tad clueless on what's up. Also curious on whether it could work for TXR 3- have my eyes on the Corvette C4 WEST prize :D
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by aluigi »

Well, the script is right because those 8 bytes are not a magic or a value or anything.
Anyway I have noticed that the uclpack decompressor changes the endianess and doesn't reset it so, I don't know if it may be a problem, but I preferred to fix the script with version 0.2c so can you try it?

If it doesn't work then I need the whole file to analyze.
Argonaut
Posts: 46
Joined: Sat Sep 27, 2014 10:24 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by Argonaut »

aluigi wrote:Well, the script is right because those 8 bytes are not a magic or a value or anything.
Anyway I have noticed that the uclpack decompressor changes the endianess and doesn't reset it so, I don't know if it may be a problem, but I preferred to fix the script with version 0.2c so can you try it?

If it doesn't work then I need the whole file to analyze.



Yes, this is seemingly working- in that a long list of DATs, TM2s, CL5 etc formats are being produced.

The main ones which I'm interested in will probably be the top few- one 1GB dat, and the other three around 200-100 MB. I can only assume that as the courses, cars, textures or music.

EDIT: Doesn't work for TXR 3. I'll upload the whole BUILD DAT/TOC ASAP.
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by aluigi »

00000bd3.dat is raw vag audio data.
00000bfb.dat is a set of files compressed with uclpack, for example at offset 0, 0x2000, 0x3800 and so on.
Argonaut
Posts: 46
Joined: Sat Sep 27, 2014 10:24 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by Argonaut »

aluigi wrote:00000bd3.dat is raw vag audio data.
00000bfb.dat is a set of files compressed with uclpack, for example at offset 0, 0x2000, 0x3800 and so on.


Fair does to that as something can be done with your UCLpack script I assume, but here's a link to the full TXR 3 BUILD DAT/TOC you requested.

https://1drv.ms/u/s!AmM2j7pIas_EkUdes71nACOSDa8r
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by aluigi »

Just curious, why the format of your files is different than the one provided in the first post of this topic?
I mean, do exist various versions of the same game or localizations or what?
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: Tokyo Xtreme Racer Drift 2 BUILD.DAT

Post by aluigi »

Script 0.3