Renowned Explorers .tim archive

[0wn3df1x]

Hi there! I need some help with .tim archives from Renowned Explorers.

Example: https://drive.google.com/file/d/0B4gOIO ... ZOVGc/view
Game: http://store.steampowered.com/app/296970/

[Ekey]

Fast info

Code: Select all

struct TIMHeader
{
   uint32_t   dwID; //IRUY
   uint32_t   dwVersion; //144
   uint32_t   dwUnknown;
   uint32_t   dwFiles; // Probably
   uint32_t   dwNTableOffset; // Offset for table with names
   uint32_t   dwZero;
   uint32_t   dwNTableSize; // Uncompressed size for names table (876726)
   uint32_t   dwNTableZSize; // Compressed size for names table (76598)
   uint32_t   dwITableOffset; // Offset for table with entry index
   uint32_t   dwZero;
   uint32_t   dwITableZSize; // Compressed size for entry index table (96529) (uncompressed size but it must be > 679464)
   uint32_t   dwUnknown; // 171
   uint32_t   dwXTableOffset; // Another table offset (dunno what this)
   uint32_t   dwZero;
   uint32_t   dwXTableZSize; // Compressed size for another table 1 (33401) (uncompressed size but it must be > 131072)
   uint32_t   dwZero;
   uint32_t   dwYTableOffset; // Another table 2 offset (dunno what this)
   uint32_t   dwZero;
   uint32_t   dwXTableZSize; // Compressed size for another table 2 (279943) (uncompressed size but it must be > 339732)
};

Used lzham_codec as compression.

[abbeybas]

Hello from AbbeyGames.

I have no intention of making the files in this archive invisible, so I'll provide the format of our archive format here. What do you want to use it for if I may ask? The file simply contains all our game content there is nothing of real interest in there.

Code: Select all

struct TIMHeader
  {
    // Archive id 'YURI'
    uint32_t id = 'YURI';

    // Size of the archive header
    uint32_t headerSize = sizeof(TIMHeader);

    // Version format of the archive (0 = default)
    uint16_t version = 0;

    // Number of files contained in the archive
    uint32_t fileCount;

    // Offset from the beginning of the archive to the directory listing
    uint64_t directoryListingOffset;

    // The uncompressed size of the directory
    uint32_t directoryListingSize;

    // The compressed size of the directory listing
    uint32_t directoryListingCompressedSize;

    // Offset from the beginning of the archive to the file registry
    uint64_t fileRegistryOffset;

    // Compressed size of the file registry
    uint32_t fileRegistryCompressedSize;

    // Offset from the beginning of the archive to the hashtable
    uint64_t hashtableOffset;

    // Size of the hashtable when compressed
    uint32_t hashtableCompressedSize;

    // Offset from the beginning of the archive to the bucket table
    uint64_t bucketTableOffset;

    // Size of the bucket table when compressed
    uint32_t bucketTableCompressedSize;

    // Directory listing MD5 before compression
    uint8_t directoryListingDigest[16];

    // File registry MD5 before compression
    uint8_t fileRegistryDigest[16];

    // Hash table MD5 before compression
    uint8_t hashtableDigest[16];

    // Bucket table MD5 before compression
    uint8_t bucketTableDigest[16];
  };

  struct TIMHashEntry
  {
    // Offset to the first bucket for the given hash
    uint32_t startBucketIndex;
  };

  struct TIMBucketEntry
  {
    // Hash of the filename with the default hashing algorithm
    uint32_t hash;

    // Hash of the filename with a different algorithm than used for the hash table
    uint32_t otherHash;

    // The index in the file table
    uint32_t fileIndex;
  };

  enum class TIMFileFlags : uint8_t
  {
    kNone = 0,
    kCompressed = 1 << 0,
  };

  struct TIMFile
  {
    // The offset of the file from the beginning of the archive
    uint64_t fileOffset;

    // Size of the file uncompressed
    uint32_t uncompressedSize;

    // Size of the file compressed
    uint32_t compressedSize;

    // Flags for the file (see TIMFileFlags)
    uint8_t flags;
  };

  struct TIMDirectoryInfo
  {
    // The size of the struct including its children
    uint16_t size;
    
    // The number of characters that make up the name of the directory
    uint16_t nameLength;

    // The number of sub directories
    uint16_t numDirectories;

    // The number of files in the directory
    uint16_t numFiles;

    // Followed by nameLength char's
    // Followed by numDirectories TIMDirectoryInfo's
    // Followed by numFiles TIMFileInfo's
  };

  struct TIMFileInfo
  {
    // The number of characters that make up the name of the file
    uint16_t nameLength;

    // Followed by nameLength char's
  };

Have fun! I'll leave the rest up to you as an exercise.. (But just let me know if you want a more thorough description..)

Cheers from Bas@AbbeyGames!

[Ekey]

Hello from AbbeyGames.

I have no intention of making the files in this archive invisible, so I'll provide the format of our archive format here. What do you want to use it for if I may ask? The file simply contains all our game content there is nothing of real interest in there.

Woah! Hi! .. Usually it is necessary for translating game to another languages or just for collecting models / textures etc. Thanks for the detailed information of archive

[0wn3df1x]

Hello from AbbeyGames.

I have no intention of making the files in this archive invisible, so I'll provide the format of our archive format here. What do you want to use it for if I may ask? The file simply contains all our game content there is nothing of real interest in there.

Cheers from Bas@AbbeyGames!

Hello, dear devs! I'm not coder, only translator.

We admire your "Renowned Explorers" project. Our team is looking forward to make a Russian localization for it. We hope you'll get wise to our ambition to make Russian version of your wonderful game. But in order to realize our desire, we need tools. For example, to get the fonts, textures and text from content.tim. (Lockit)

[Ekey]

Hmmm... the structure of the header does not match that in the archive

[abbeybas]

Ow shoot, yeah we have since moved to version 1 but I forgot to update our documentation. The fileCount however, should match. I parse the header in code simply by reading in the structure specified in the code as raw C++ memory. There should be around 33000 files in the archive atm.

We have always planned on translating the game and it's set up the be translatable. At this point however there is quite a lot of text in the game and it would be too expensive to do all translations at once. That's why we chose to only do english and in the future add new translations. The archive contains a binary encoded file that contains all the translations for the game. However we'd love it for people to help us out with the translations so if you want to help please send an email to info@abbeygames.com and let's set something up!

Cheers!
Bas

[Ekey]

Ow shoot, yeah we have since moved to version 1 but I forgot to update our documentation. The fileCount however, should match. I parse the header in code simply by reading in the structure specified in the code as raw C++ memory. There should be around 33000 files in the archive atm.

Oh. Ok

We have always planned on translating the game and it's set up the be translatable. At this point however there is quite a lot of text in the game and it would be too expensive to do all translations at once. That's why we chose to only do english and in the future add new translations. The archive contains a binary encoded file that contains all the translations for the game. However we'd love it for people to help us out with the translations so if you want to help please send an email to info@abbeygames.com and let's set something up!

You could make the individual archives with language data's like: language_en.tim, language_fr.tim, language_ru.tim etc. and load them first by priority depending on the selected language in Steam. I know one team that will help translate game to Russian language!

[aluigi]

The version field is padded to 32bit (perfectly normal for struct on 32bit systems without pragma pack) but there are a couple of new fields, one after fileRegistryCompressedSize (0xab) and another after hashtableCompressedSize (0).

The compression algorithm doesn't match because lzham produces a file of 0 bytes.
I tried comtype lzham of quickbms, I tried also older versions of quickbms with same result.
Ekey, what lzhma API did you use for the decompression?

[Ekey]

Ekey, what lzhma API did you use for the decompression?

They use 10.8.1

[aluigi]

ok, I guess I use an old version in quickbms. going to update it in the next quickbms

[aluigi]

No Ekey, the latest version is the same with DEST_BUF_TOO_SMALL while using lzham_decompress and -3 with lzham_z_uncompress.
clog "out.dat" directoryListingOffset directoryListingCompressedSize directoryListingSize

Can you show what are you exactly using?

[aluigi]

If I use LZHAM_DECOMP_FLAG_READ_ZLIB_STREAM I get error 11 (LZHAM_DECOMP_STATUS_FAILED_BAD_RAW_BLOCK) and the correct output size but the output is all zeroes.

[Ekey]

Well I did some tests.

Table_Compressed_Original.dat - Compressed table from TIM archive
Table_Decompressed_Original.dat - Decomrpessed table
Table_Compressed_lzhamtest.dat - Compressed by lzhamtest_x86 with comp level 4

As you can see Table_Compressed_lzhamtest.dat it differs from the original only in the existing header with 13 bytes.

[aluigi]

How you got Table_Decompressed_Original.dat ?

[Ekey]

How you got Table_Decompressed_Original.dat ?

Debugger

[aluigi]

That explains why I have this problems, I guess they use a fixed dictionary or similar.
I have made all the possible tests and only using the unbuffered option I get at least this:

Code: Select all

TEST 7 33 0 (76598 876726)
  b6 60 00 00 0b 00 1b 00 71 00 05 00 00 00 08 00   .`......q.......
  61 72 69 61 6c 09 ff ff ff ff ff ff ff ff ff ff   arial...........
  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................

The 0xff you see there mean that the output is only 0x16 bytes so there is something invalid and is not caused by the API or quickbms.

[aluigi]

@abbeybas any explanation?

[Ekey]

If lzham modified I do not see sense to continue

[aluigi]

I agree.
Considering that it was planned to be a 100% documented format, it's taking too much time and effort.

The only positive thing is that the next quickbms will have a correct implementation of lzham.