File structure from HEX codes

Programming related discussions related to game research
Qnoops
Posts: 39
Joined: Thu Aug 04, 2016 8:03 pm

File structure from HEX codes

Post by Qnoops »

I have a file with .dat extension, i check the file on hex editor and saw that "GAV" thingy.

https://prnt.sc/l52evm

Now, what should i do to reverse engineer this file? What should i understand from that "GAV" text? I cant find any information about gav file structure.

Can someone help me about that, please?

https://www49.zippyshare.com/v/Sm6KkqpY/file.html
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: File structure from HEX codes

Post by aluigi »

There are many things you can understand just from that screenshot.

You have "GAV" followed by a small 8bit number which is probably a sort of version.
Then you can clearly see a PNG image starting from offset 0x21.
Before that image there are various 32bit fields that can be easily identified by looking at the "zeroes" of the high bits.
These fields are not incremental and they don't contain the number 0 (relative offset) or 0x21 (absolute offset), therefore they can't be offset, more probably they are size fields.
The total number of size fields is 7 which is also the 8bit value at offset 4.

Without even downloading the file I can guess a format like the following:

Code: Select all

idstring "GAV"
get VER byte
get FILES byte
savepos OFFSET
xmath OFFSET "OFFSET + (FILES * 4)"
for i = 0 < FILES
    get SIZE long
    log "" OFFSET SIZE
    math OFFSET + SIZE
next i

Let me know if I was correct :)
Ah, please let us know the name of the game.
Qnoops
Posts: 39
Joined: Thu Aug 04, 2016 8:03 pm

Re: File structure from HEX codes

Post by Qnoops »

Wow, there are too many things to learn :-D I tried the script and it worked. Its exported some .dat files and 2 .png files and 1 .oy1 file.

Btw, what is that "0x21" you were talking about? Where should i look to see that? (Edit: Oh, i think i figured it out but not completely. offset = 0x21 = second row, first column. But what about very last offset? how do we name it? For example in the image, left bottom corner's hex value C4, how we can say which offset is it? 0x21000?? :D)

Game's name; Northgard Ragnarok

And what about that file? Can we export it to editable .csv or .txt?


https://www73.zippyshare.com/v/3d5dyZLp/file.html
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: File structure from HEX codes

Post by aluigi »

0x21 is the offset where the PNG image starts.
0x21 is 33 in decimal if you prefer it, but you must switch your mind to hexadecimal for this stuff :)

Regarding that other file you attached:

Code: Select all

get ZERO long
get STRINGS long
for i = 0 < STRINGS
    get OFFSET long
    get SIZE long
    get ID long
    slog "" OFFSET SIZE
next i
Qnoops
Posts: 39
Joined: Thu Aug 04, 2016 8:03 pm

Re: File structure from HEX codes

Post by Qnoops »

aluigi wrote:0x21 is the offset where the PNG image starts.
0x21 is 33 in decimal if you prefer it, but you must switch your mind to hexadecimal for this stuff :)

Regarding that other file you attached:

Code: Select all

get ZERO long
get STRINGS long
for i = 0 < STRINGS
    get OFFSET long
    get SIZE long
    get ID long
    slog "" OFFSET SIZE
next i


Thank you, it's extracted as text file but some characters are not supported, by the way when i want to reimport or convert the exported .txt file to .dat again i am getting this error

http://prntscr.com/l5cbkr

do you have any solution for that?
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: File structure from HEX codes

Post by aluigi »

Yes, use the current version of quickbms because what you have there is an older version not supporting the Slog "" command.
Qnoops
Posts: 39
Joined: Thu Aug 04, 2016 8:03 pm

Re: File structure from HEX codes

Post by Qnoops »

aluigi wrote:Yes, use the current version of quickbms because what you have there is an older version not supporting the Slog "" command.


I was already using version 0.9.0

And also i tried to reimport it without running reimport.bat (directly with quickbms.exe) it's giving me this error

http://prntscr.com/l5f3a9
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: File structure from HEX codes

Post by aluigi »

It looks like you selected the wrong files.
Reimport works exactly as extraction (same files and folders to select), read section 3 of http://aluigi.org/papers/quickbms.txt
Qnoops
Posts: 39
Joined: Thu Aug 04, 2016 8:03 pm

Re: File structure from HEX codes

Post by Qnoops »

aluigi wrote:It looks like you selected the wrong files.
Reimport works exactly as extraction (same files and folders to select), read section 3 of http://aluigi.org/papers/quickbms.txt


Yeah i think i was made a mistake while sleepy, now i can manage to make it work smoothly, thank you.

Can we add something to code for supportinf UTF-8 chars?

EDIT:

i have a problem about reimported file,

http://prntscr.com/l5mq67 this is raw exported

http://prntscr.com/l5mqrz this is reimported file

it doesn't reimport all texts and it's reimporting with some weird chars
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: File structure from HEX codes

Post by aluigi »

I bet you changed the format of the txt file generated by quickbms. It looks like it has a BOM now.
Open both the original and your edited file (before reimporting, just after you finish editing it) with a hex editor and check the difference.
Qnoops
Posts: 39
Joined: Thu Aug 04, 2016 8:03 pm

Re: File structure from HEX codes

Post by Qnoops »

aluigi wrote:I bet you changed the format of the txt file generated by quickbms. It looks like it has a BOM now.
Open both the original and your edited file (before reimporting, just after you finish editing it) with a hex editor and check the difference.


I just edited the file with notepad.exe, should i use something else to not change the structure? And by the way can you add anything to script for UTF-8 support?
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: File structure from HEX codes

Post by aluigi »

notepad is a toy, use a real text editor like notepad++
Qnoops
Posts: 39
Joined: Thu Aug 04, 2016 8:03 pm

Re: File structure from HEX codes

Post by Qnoops »

aluigi wrote:notepad is a toy, use a real text editor like notepad++


Okey i will, thank you

I will try my chance one last time by asking that.. :D Can you make it support UTF-8 with some kind of code?
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: File structure from HEX codes

Post by aluigi »

Notepad++
Encoding -> Encode in UTF8
sergop
Posts: 125
Joined: Mon Jul 31, 2017 2:28 am

Re: File structure from HEX codes

Post by sergop »

Can someone help me out? With the first script i get 6 files 2 of them are in .png format 1 is in .oy1 and the rest are in .dat. But when it comes to using the second script i get errors. I will add the files if someone wants to check them out.
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: File structure from HEX codes

Post by aluigi »

Settling.dat is the format for the first script, you can't use the second one and the dat files don't use that format too..
sergop
Posts: 125
Joined: Mon Jul 31, 2017 2:28 am

Re: File structure from HEX codes

Post by sergop »

aluigi wrote:Settling.dat is the format for the first script, you can't use the second one and the dat files don't use that format too..

Well thx for the answer. :)