ParaWorld .gsf container

[DryFun]

What should I do now?

Some info from devs:
The binary format is pretty much write-only and optimized to be shoved into DirectX with only little additional information (collision data) There was a 3DSMax plugin to export 3D data for paraworld and a resource builder to package it.(In resource builder source folder) If I had that plugin and were allowed to give it away there would still be the problem that it doesn't work with recent 3DSMax versions. The actual on-disk file format gives a whole new meaning to the word "complicated". There is an outer hull that is bascally a chunk-based container format similar to RIFF. GSF file contains compressed data blocks.

.GOD - General Object Data
.gsf - Graphic Set File

[DryFun]

Just realized that there was already a topic about gsf files
viewtopic.php?t=962

[BSM]

Just a few findings to help you a bit.

4 bytes: "GSF\0"
4 bytes: always seen as zeroes
4 bytes: offset to the TableOfContens
Variable length: Tree of objects and properties
Variable length: Pad with zeroes until the TableOfContents
44 bytes: TableOfContents
Variable length: ObjectsData

The table of contents is always located at a position multiple of 0x10000, depending on how much room is required to store the Tree of objects and properties
All the strings in the tree are null terminated (a byte = 0x00 after all chars), and also prefixed with 4 bytes indicating the string length, including the null char.
I will name this as P4bNtString (Prefixed with 4 bytes, Null terminated string).

The tree has this format:

Code: Select all

P4bNtString rootNode (it is the same as the .gsf file, without extension)
4 bytes childsCount
   repeat childsCount times NODE
4 bytes propsCount
   repeat propsCount times PROP

NODE has this format:

Code: Select all

P4bNtString name
4 bytes index
4 bytes childsCount
   repeat childsCount times NODE
4 bytes propsCount
   repeat propsCount times PROP

PROP is still not studied, but it shouldn't be hard

An example of Tree from your attached files, for icewaste_animals.gsf

Code: Select all

icewaste_animals, root, childs=5 {
   bothriolepis, index=0, childs=3 {
      swim, index=0, childs=0, props=0
      standanim, index=1, childs=0, props=0
      dying, index=2, childs=0, props=0
   } props=1 {
      
   }
   dunkleosteus, index=1, childs=8 {
      attack_front, index=0, childs=7 {
         0, 2, 3, 4, 5, 1, 6
      } props=0
      swim_1, index=1, childs=6 {
         7, 10, 8, 11, 9, 12
      } props=0
      swim_2, index=2, childs=6 {
         7, 14, 10, 13, 15, 16
      } props=0
      swim_3, index=3, childs=6 {
         17, 19, 20, 18, 21, 22
      } props=0
      swim_left, index=4, childs=0, props=0
      swim_right, index=5, childs=0, props=0
      standanim, index=6, childs=29 {
         23, 25, 26, 27, 28, 29, 30, 31, 32, 32,
         32, 32, 32, 32, 32, 32, 32, 32, 24, 33,
         33, 33, 33, 33, 33, 33, 33, 33, 33
      } props=0
      dying, index=7, childs=3 {
         34, 35, 36
      } props=0
   } props=1 {
      
   }
   henodus, index=2, childs=4 {
      swim, index=0, childs=2 {
         37, 38
      } props=0
      growup, index=1, childs=0, props=0
      standanim, index=2, childs=0, props=0
      dying, index=3, childs=0, props=0
   } props=1 {
      
   }
   quetzalcoatlus, index=3, childs=2 {
      standanim, index=0, childs=0, props=0
      dying, index=1, childs=0, props=0
   } props=0
   quetzalcoatlus_boden, index=4, childs=2 {
      standanim, index=0, childs=0, props=0
      dying, index=1, childs=0, props=0
   } props=0
} props=39 {
   00_npc/npc_dino_huge1_water_att1.wav
   02_battle/hit_water_medium2.wav
   00_npc/npc_dino_huge1_water_att2.wav
   00_npc/npc_dino_huge1_water_att3.wav
   00_npc/npc_dino_huge1_water_hit2.wav
   00_npc/npc_dino_huge1_water_hit3.wav
   02_battle/hit_water_medium1.wav
   04_step/step_dino_swim1.wav
   04_step/step_dino_swim2.wav
   04_step/step_dino_swim3.wav
   04_step/step_dino_swim4.wav
   04_step/step_dino_swim5.wav
   04_step/step_dino_swim6.wav
   04_step/step_dino_swim2.wav
   04_step/step_dino_swim3.wav
   04_step/step_dino_swim5.wav
   04_step/step_dino_swim6.wav
   04_step/step_dino_swim1.wav
   04_step/step_dino_swim2.wav
   04_step/step_dino_swim3.wav
   04_step/step_dino_swim4.wav
   04_step/step_dino_swim5.wav
   04_step/step_dino_swim6.wav
   00_npc/npc_dino_huge_carn_idle1.wav
   04_step/step_dino_swim2.wav
   00_npc/npc_dino_huge_carn_idle2.wav
   00_npc/npc_dino_huge_carn_idle3.wav
   00_npc/npc_dino_huge_carn_idle4.wav
   00_npc/npc_dino_huge_carn_idle5.wav
   04_step/step_dino_swim3.wav
   04_step/step_dino_swim4.wav
   04_step/step_dino_swim5.wav
   00_npc/npc_dino_huge1_water_die.wav
   02_battle/hit_water_medium2.wav
   02_battle/hit_water_medium1.wav
   04_step/step_fish_swim1.vaw
   04_step/step_fish_swim2.vaw
   Animals1
   Animals3
}

The table of contents has this format:

Code: Select all

00 00 00 00    Always zero?
05 00 00 00    Blocks1Count
0B 00 00 00    Blocks2Count
00 00 00 00    Always zero?
1C 00 00 00    SkipToBlocks1
05 00 00 00    Blocks3Count
B8 01 00 00    SkipToBlocks2
0B 00 00 00    Blocks4Count
06 00 00 00    Blocks5Count
4C A0 00 00    SkipToBlocks4
E8 03 00 00    Always 1000?

Counts, from 1 to 5, are the count for each data type stored for objects.
Skips, from 1 to 3, are values to add to the current stream pointer while reading data in order to reach the desired location (see examples).
There are 5 blocks of descriptors type1, and 11 blocks of descriptors type2. Each type1 uses 84 bytes, and each type2 uses 28 bytes, but both them has the same structure in the first bytes (see below), and includes offsets to the linked data in other block types.

As the TableOfContents for this gsf pack starts at 0x00010000, the stream pointer will be at 0x00010010 when we read "SkipToBlocks1". If we add 0x00010010 + 0x0000001C we have Blocks1Offset = 0x0001002C
When reading "SkipToBlocks2", the stream pointer will be at 0x00010018, and if we add the value 0x000001B8 we have the Blocks2Offset = 0x000101D0.
When reading "SkipToBlocks4", the stream pointer will be at 0x00010024, and if we add the value 0x0000A04C we have the Blocks4Offset = 0x0001A070. I hope you get the idea.
At 0x00010304, just after blocks2, we can find the Blocks3 (still not investigated enough).

Example of blocks type 1, the first entry:

Code: Select all

43 68 61 72    "Char"  (There are a bunch of types, including Char, Anim, Misc, Deko, Wall, Bldg...)
38 FE 00 00    Skip1
2C FE 00 00    Skip2
02 00 00 00    Count1
1C A0 00 00    Skip3
01 00 00 00    Count2
00 00 00 00    ???
FF FF 00 00    ???
00 01 00 00    ???
FF 00 00 00    ???
00 00 00 80    null
00 00 00 80    null
00 00 00 00    ???
48 AF 10 BF    CoordX1 = -0.565174 (It is a float, in IEEE single format)
FB 41 7E BF    CoordY1 = -0.993194
64 B8 37 BE    CoordZ1 = -0.179414
4A AF 90 3F    CoordX2 = 1.130349
DA B5 FD 3F    CoordY2 = 1.982112
57 E6 E1 3E    CoordZ2 = 0.441210
10 FE 00 00    ???
03 00 00 00    ???

Example of blocks type 2, the first entry:

Code: Select all

43 68 61 72   "Char"
30 01 00 00    Skip1
74 04 00 00    Skip2
01 00 00 00    Count1
00 00 00 80    Skip3   (null)
01 00 00 00    Count2
0A 00 00 00    ???

If some "skip" is 0x800000 it means a null value (this object has no data linked for that type). Otherwise, just add the value to the current stream position.

I also saw the last blocks structure, and it contains meshes (some size and coords, list of vertices and triangles, plus some times a filename for texture and an object name).

I hope this can help you.

[DryFun]

Many thanks

[DryFun]

"(There are a bunch of types, including Char, Anim, Misc, Deko, Wall, Bldg...)"
There is no Wall class in all files I upload
How did you know about it?

[BSM]

"(There are a bunch of types, including Char, Anim, Misc, Deko, Wall, Bldg...)"
There is no Wall class in all files I upload
How did you know about it?

I have downloaded also the files in the post you linked:

Just realized that there was already a topic about gsf files
viewtopic.php?t=962

and it is used in boosterpack1.gsf

[DryFun]

I know that model starts right after the last texture name of the previous model and ends with own last texture name
Can I get model at this stage?

[BSM]

I know that model starts right after the last texture name of the previous model and ends with own last texture name
Can I get model at this stage?

I think yes. Let me show you my way of thinking, step by step, by using the same example file "icewaste_animals.gsf"

I start at the position 0x00000000, and try to read data for one mesh, which I guess will correspond to icewaste_animals->bothriolepis->swim (from the tree).

Code: Select all

0x00000000      PACK FILE START
47 53 46 00    "GSF"
00 00 00 00    0
00 00 01 00    0x00010000      Offset to the Table of Contents

So I advance in the stream to 0x00010000

Code: Select all

0x00010000      TABLE OF CONTENTS
00 00 00 00    Always zero?
05 00 00 00    Blocks1Count = 5
0B 00 00 00    Blocks2Count = 11
00 00 00 00    Always zero?
1C 00 00 00    SkipToBlocks1      0x00010010 + 0x1C = 0x0001002C
05 00 00 00    Blocks3Count
B8 01 00 00    SkipToBlocks2      0x00010018 + 0x1B8 = 0x000101D0
0B 00 00 00    Blocks4Count
06 00 00 00    Blocks5Count
4C A0 00 00    SkipToBlocks4      0x00010024 + 0xA4C = 0x00010A70
E8 03 00 00    Always 1000?

Now I have 3 offsets to care for:
0x0001002C : Blocks1
0x000101D0 : Blocks2
0x00010A70 : Blocks4

Advance the stream to 0x0001002C and read the first block

Code: Select all

0x0001002C      BLOCKS1 DESCRIPTORS
43 68 61 72    "Char"
38 FE 00 00    Skip1      0x00010030 + FE38 = 0x0001FE68
2C FE 00 00    Skip2      0x00010034 + FE2C = 0x0001FE60
02 00 00 00    Count1
1C A0 00 00    Skip3      0x0001003C + A01C = 0x0001A058
01 00 00 00    Count2
00 00 00 00    ???
FF FF 00 00    ???
00 01 00 00    ???
FF 00 00 00    ???
00 00 00 80    null
00 00 00 80    null
00 00 00 00    ???
48 AF 10 BF    CoordX1 = -0.565174 (It is a float, in IEEE single format)
FB 41 7E BF    CoordY1 = -0.993194
64 B8 37 BE    CoordZ1 = -0.179414
4A AF 90 3F    CoordX2 = 1.130349
DA B5 FD 3F    CoordY2 = 1.982112
57 E6 E1 3E    CoordZ2 = 0.441210
10 FE 00 00    ???
03 00 00 00    ???

Now I have another 3 offsets to read data:
0x0001FE68, 0x0001FE60 and 0x0001A058. Lets see what I find

Code: Select all

0x0001FE68
01 00 00 00    1 String
14 00 00 00    20 bytes in total, from current pointer
0C 00 00 00    12 bytes for the string   (it is not null terminated)
62 6F 74 68 72 69 6F 6C 65 70 69 73    "bothriolepis"

Got the object name

Code: Select all

0x0001FE60
48 97 FF FF    -26808      0x0001FE60 - 26808d = 0x000195A8
D8 98 FF FF    -26408      0x0001FE64 - 26408d = 0x0001973C

Another 2 offsets; I'll check it later, and now check the pending 3rd offset from last step

Code: Select all

0x0001A058
A8 5F FF FF    -41048      0x0001A058 - 41048d = 0x00010000
C4 5F FF FF    -41020      0x0001A05C - 41020d = 0x00010020
0C 00 00 00    12
01 00 00 00    1
01 00 00 00    1
00 00 00 00    0
00 00 00 00    0
0F 00 00 00    15

B8 5D 00 00    Skip      0x0001A078 + 0x5DB8 = 0x0001FE30
00 00 00 80    null
00 00 00 80    null
00 00 00 00    0
10 40 00 00    16400
42 00 00 00    66

B6 F8 00 00    Skip
E2 F8 00 00    Skip
00 00 00 80    null
00 00 00 00    0
01 00 00 00    1
00 00 00 00    0

FE F8 00 00    Skip
00 00 00 80    null
...

This seems another table, starting with 2 pointers (to the Table of Contents), and maybe with 12 items, but I will focus now just of the first record:

Code: Select all

0x0001A078
B8 5D 00 00    Skip      0x0001A078 + 0x5DB8 = 0x0001FE30
00 00 00 80    null
00 00 00 80    null
00 00 00 00    0
10 40 00 00    16400
42 00 00 00    66

So I have now another offset: 0x0001FE30. Lets check it:

Code: Select all

0x0001FE30
01 00 00 00    1 String
24 00 00 00    36 bytes in total, from current pointer
19 00 00 00    25 bytes for the string   (it is not null terminated)
61 6E 69 6D 61 6C 73 2F 69 63 65 5F 68 65 6E 6F 64 75 73 5F 61 2E 74 67 61 
"animals/ice_henodus_a.tga"
00 00 00       Padding

Got the texture filename

Let me remind myself the pending offsets: 0x000195A8 and 0x0001973C.
I guess these 2 will contain the final expected data: vertices and triangles. Going to eat, and be back later.

[BSM]

I'm back.

Before I forget, let me note that Blocks1 has 5 entries as there are 5 childs of the root node: bothriolepis, dunkleosteus, henodus, quetzalcoatlus and quetzalcoatlus_boden.
These 5 entries are of types Char, Anim, Anim, Char, Char, and that is because just dunkleosteus and henodus specifies frames (2 frames for henodus and a bunch of frames for dunkeosteus). Just in case that helps you in some way.

Now back to the point where I was.
Let me remind myself the pending offsets: 0x000195A8 and 0x0001973C.

The first one leads to a bunch of floating point numbers, I guess more coordinates for something:

Code: Select all

0x000195A8
05 00 00 00    5
1F 00 00 00    31
5B F9 F4 16    3.957769
00 00 00 00    0
5B F9 F4 16    3.957769
00 00 00 00    0

8C 74 96 B2    -1.751529
14 DE 7E 3F    0.995576
A4 A5 61 3C    0.013772
00 00 80 3F    1.0
00 00 80 3F    1.0
00 00 80 3F    1.0
9F EA FC BE    -0.493977
FF 85 01 3F    0.505950
9D EA FC 3E    0.493977
FF 85 01 3F    0.505950

01 00 00 00    1
18 00 00 00    24            0x000195E8 + 0x18 = 0x00019600
01 00 00 00    1
88 00 00 00    136            0x000195F4 + 0x88 = 0x0001967C
03 00 00 00    3
03 00 00 00    3


0x00019600
00 00 00 00    0
E3 38 C9 F9    -1.306006e35
00 00 00 00    0
8F 22 0A 3F    0.539589
4D 77 E8 35    1.732006e-6
53 12 8F B1    -4.163930e-9
00 00 80 3F    1.0
00 00 80 3F    1.0
00 00 80 3F    1.0
E2 23 40 A5    -1.666550e-16
24 73 E5 30    1.669466e-9
34 70 1C 3D    0.038199
30 D0 7F 3F    0.999270

01 00 00 00    1
08 00 00 00    8            0x00019638 + 0x08 = 0x00019640
01 00 00 00    1


From 0x00019640 to 0x0001967B (0x3C = 60 bytes)
48 B4 8B B9    -0.000266
00 00 00 00    0
25 4F 26 3F    0.649645
20 B9 67 34    2.158089e-7
0A 12 14 33    3.447534e-8
00 00 80 3F    1.0
00 00 80 3F    1.0
00 00 80 3F    1.0
3E 87 3B 26    6.506194e-16
DA C1 50 B0    -7.594543e-10
8B 54 8E BC    -0.017374
1C F6 7F 3F    0.999849
00 00 00 00    0
00 00 00 80    0/null
00 00 00 00    0

From 0x0001967C to 0x0001973B (0xC0 = 192 bytes : 68 floats)

Then I reached the point where I was trying to get (vertices and triangles), from the remaining pending offset.

Code: Select all

0x0001973C
00 00 00 80    0/null
1F 00 00 00    31
From 0x00019744 to 0x000197BF (31 x 4 bytes, surely more coordinates as floats), and we care the last 4 x 4 bytes:
0x000197B0
01 00 00 00    1
10 00 00 00    Skip 0x000197B4 + 0x10 = 0x000197C4
01 00 00 00    1
9A 08 00 00    Skip 0x000197BC + 0x89A = 0x0001A056

Code: Select all

0x000197C4
48 AF 10 BF    X1 = -0.565174
FB 41 7E BF    Y1 = -0.993194
64 B8 37 BE    Z1 = -0.179414
4C AF 10 3F    X2 = 0.565174
BA 29 7D 3F    Y2 = 0.988917
25 0A 86 3E    Z2 = 0.261796
5F 00 00 00    Vertices = 95
67 00 00 00    Triangles = 103
18 00 00 00    0x000197E4 + 0x18 = 0x000197FC (Offset to vertices)
04 06 00 00    0x000197E8 + 0x604 = 0x00019DEC (Offset to Triangles)
67 00 00 00    103 ???
10 00 00 00    16 ???
00 00 00 80    null ???
00 00 00 00    0 ???
0x000197FC

From 0x000197FC to 0x00019DEB (0x5F0 = 1520 bytes : 95 * 16) vertices: x, y, u,v?
From 0x00019DEC to 0x0001A055 (0x26A = 618 bytes-> 103 triangles)

The Triangles are easy to understand, 6 bytes each triangle by using 2 bytes (an int16) to point each vertex index on that triangle.
The Vertices seems a bit more complicated, as I expected 3 floats for each vertex (x, y, z), but instead I have found 2 floats + 2 integers.

Next time I get some spare time I will try to read that list of vertices and triangles, and write it into wavefront .obj and see what I get.

[DryFun]

I tried this with aje_buildings.gsf but what should I do with AC 59 02 00?

Code: Select all

0x00010000
00 00 00 00
4E 00 00 00    Blocks1Count = 78
29 00 00 00    Blocks2Count = 41
00 00 00 00
1C 00 00 00    SkipToBlocks1      0x00010010 + 0x1C = 0x0001002C
4E 00 00 00    Blocks3Count
AC 19 00 00    SkipToBlocks2      0x00010018 + 0x19AC = 0x000119C4
29 00 00 00    Blocks4Count
27 00 00 00    Blocks5Count
AC 59 02 00    SkipToBlocks4

Upd
Nevermind, I just thought I made a mistake
There offset to animations stuff, first model in aje_buildings.gsf doesnt have animations

[BSM]

I tried this with aje_buildings.gsf but what should I do with AC 59 02 00?

I'm a bit busy just now, but I will continue studying the GSF format.
Meanwhile, I cannot find "aje_buildings.gsf" in the files already posted, but I can give you some advances on my findings and answer to that question.

Update for the first part, the table of contents:

Code: Select all

0x00010000
00 00 00 00
4E 00 00 00    78 Graphic Containers (previously named Type 1 graphic resources). As 78 "main" nodes in the Treenode
29 00 00 00    41 Graphic Resources (previously named Type 2 resources)
00 00 00 00
1C 00 00 00    GrContainersOffset = 0x1C + 0x00010010 = 0x0001002C
4E 00 00 00    GrContainersCount = 78
AC 19 00 00    GrResourcesOffset = 0x19AC + 0x00010018 = 0x000119C4
29 00 00 00    GrResourcesCount = 41

0x00010020
27 00 00 00    TextureFilenamesCount = 39
AC 59 02 00    TextureFilenamesOffsetsPointer = 0x0259AC + 0x00010024 = 0x000359D0
E8 03 00 00    1000 entries in list (fixed value)

From 0x000359D0 to 0x0003B78F (0x5DC0 = 24000 bytes) you should find 1000 registers of 24 bytes each, being used just the first 39, and the others empty.

TextureInfoBlock structure, 24 bytes each entry (shown values are from another gsf package):

Code: Select all

0x00015038   TEXTURES_DEFINITION
00 00 00 00    This value changes, and its purpose is to be discovered.
0F 00 00 00    This value changes, and its purpose is to be discovered.
B8 5D 00 00    0x5DB8 + 0x00015040 = 0x0001ADF8      Points to TEXTURE_FILENAME
00 00 00 80    0/null
00 00 00 80    0/null
00 00 00 00    0

The chunks for texture filenames is structured as:

Code: Select all

0x0001ADF8   TEXTURE_FILENAME
01 00 00 00    1 string
14 00 00 00    20 bytes used (can be a different value, specially for long strings)
0E 00 00 00    14 chars
6D 69 73 63 2F 63 6F 6C 6F 72 2E 74 67 61    "misc/color.tga"
00 00 00 00 00 00    Padding

[BSM]

UPDATE:
Still cannot read all parts by program, but I managed to read some parts by manually specifying the right offsets, just to be able to study more parts.
Attached a few meshes from "all_special_weapons.gsf" (not all, but a bunch of meshes forming part of "cole_machete"), in wavefront .obj format:

cole_machete.zip

Surely, the uvMap is not right, as I cannot check if the coordinates are correct until you upload the needed textures:
effects/gore.tga
items/spec_weapons_a.tga
items_all_products_lod.tga
items/speedlines.tga
Mostly it will be some scale factor, and/or maybe some rotation.

The included meshes seems to be the handle and blade of the machete (in several positions), but some part of the center is missing.

I will try to put some order in my notes, then post here my findings.

[DryFun]

Cole machete ingame
https://www.dropbox.com/s/r94uirol3e3fd02/scr_20171201_170546.jpg

Maybe you extract a 2 machetes for the right and left hands, but they should be separate models
Or maybe its LoD for this machete because LoD models also in this container

Although in the file all textures have a .tga extension the game uses .dds textures

Textures.7z

[DryFun]

4 models in you .obj

2 models with 81(blade) and 52(hilt) faces
2 models with 41(blade) and 18(hilt) faces
Looks like LoD

Maybe one machete + lod for right hand and other machete + lod for left hand?

[DryFun]

So you use hex2obj program,Im right?
But how did you solve this "The Vertices seems a bit more complicated, as I expected 3 floats for each vertex (x, y, z), but instead I have found 2 floats + 2 integers."

[BSM]

4 models in you .obj

2 models with 81(blade) and 52(hilt) faces
2 models with 41(blade) and 18(hilt) faces
Looks like LoD

Maybe one machete + lod for right hand and other machete + lod for left hand?

There was several parts in the .obj file I posted, but I did not read all the needed parts for cole_machete, as it was just a test. Now I am able to get all the needed parts for some model types (will post another test). I think you are right about LOD; it was several pairs of handle + blade, for several LevelOfDetails, but for the same model (not left and right handed, as next model is named cole_machete_left_hand).

So you use hex2obj program,Im right?
But how did you solve this "The Vertices seems a bit more complicated, as I expected 3 floats for each vertex (x, y, z), but instead I have found 2 floats + 2 integers."

No, I don't even know that program. I made my own utility (in C#) to extract data and write it into .obj format.
The problem with vertices is the data is packed:
Each vertex stores 8 bytes (64 bits) in the table of vertices, but just the first 39 bits (0-12 for x, 13 to-25 for y, 26-38 for z) are used for the vertex coordinates; the other bits (30-63) I hope to be the texture vertex coordinates, but I still am unable to get the proper unpacking for that part.
I tried to get as much info as possible from the files you posted, so I investigated the source code include. That code (C++) is incomplete, so we cannot see a lot classes needed for the pack/unpack, but I got some clues for several stuff, like the vertex unpacking. The code gets values from these bits, then build a matrix and apply several transformations, to adapt the coordinates, rotations and scales to the world (that's why there are sets of 6 floats everywhere (Min and Max for x, y, z) and another more floats (for rotations and maybe scale), to setup the BoundingBox for each part of a model, for the whole model, and for the world).

I was unable to use a similar code due to the lack of some sources, but I coded on my own way to get these bits transformed into usable values. The code to read the whole pack is still plenty of garbage (previous tests) and not the best of coding practices, but for the vertex data maybe this class could illustrate how I am doing it:

Code: Select all

using System;
using System.Collections.Generic;
using System.Text;

namespace ParaWorld
{
    public class Vertex
    {
        public const float MAX_VALUE_13BITS = 8191.0f;
        public const float MAX_VALUE_12BITS = 4095.0f;


        public float x;
        public float y;
        public float z;

        public Vertex() : this(0.0f, 0.0f, 0.0f) { }

        public Vertex(float x, float y, float z)
        {
            this.x = x;
            this.y = y;
            this.z = z;
        }

        // Unpack a set of coordinates from 8 bytes read (an ulong = 64 nbits) from a GSF package
        public static Tuple<Vertex, Vertex> Unpack39bits(ulong data, Vertex min, Vertex max)
        {
            int ix, iy, iz, iu, iv;
            float x, y, z, u, v;

            ix = (int)((data >> 0) & 0x1FFF);
            iy = (int)((data >> 13) & 0x1FFF);
            iz = (int)((data >> 26) & 0x1FFF);
            iu = (int)((data >> 39) & 0x0FFF);
            iv = (int)((data >> 51) & 0x0FFF);

            x = ix * (max.x - min.x) / MAX_VALUE_13BITS + min.x;
            y = iy * (max.y - min.y) / MAX_VALUE_13BITS + min.y;
            z = iz * (max.z - min.z) / MAX_VALUE_13BITS + min.z;
            u = iu / MAX_VALUE_12BITS;
            v = iv / MAX_VALUE_12BITS;

            Vertex xyz = new Vertex(x, y, z);
            Vertex uv = new Vertex(u, v, 1.0f);

            return Tuple.Create(xyz, uv);
        }

        // Get a string usable in Wavefront .obj files, representing this set of coordinates
        public string ToWavefrontObj(bool isUv = false)
        {
            if (isUv)
            {
                return "vt " + x.ToString("R", System.Globalization.CultureInfo.InvariantCulture)
                        + " " + y.ToString("R", System.Globalization.CultureInfo.InvariantCulture)
                        + "\r\n";
            }
            else
            {
                return "v " + x.ToString("R", System.Globalization.CultureInfo.InvariantCulture)
                        + " " + y.ToString("R", System.Globalization.CultureInfo.InvariantCulture)
                        + " " + z.ToString("R", System.Globalization.CultureInfo.InvariantCulture)
                        + "\r\n";
            }
        }
    }
}

I will continue...

[BSM]

Maybe it is easier to show than to explain, so I will paste my notes from the last study I did on "all_special_weapons.gsf". I have chosen this pack because it is simpler, containing just level 1 entries in the TreeNode (I named these Type 1 resources, or Containers, or Objects and SubObjects), and the last test focused on "cole_sword_l" as it has no effects (like cole_machete), but just 2 mesh parts (handle + blade) and 1 Level of Detail.
(Note that all these stuff is typed by hand, so surely there are a bunch of errors)

Code: Select all

all_special_weapons.gsf

0x00000000      GSF_PACK_HEADER
47 53 46 00    "GSF\0"
00 00 00 00    FormatVersion = 0
00 00 01 00    ContentsTableOffset = 0x00010000

0x0000000C      TREE_NODES
From 0x0000000C to unknown (it needs to be parsed)

0x00010000      CONTENTS_TABLE
00 00 00 00    ??? = 0
5B 00 00 00    ObjectsCount = 91
00 00 00 00    SubObjectsCount = 0
00 00 00 00    ??? = 0
1C 00 00 00    ObjectsTableOffset = 0x1C + 0x00010010 = 0x0001002C
5B 00 00 00    ObjectsCount = 91
00 00 00 80    SubObjectsTableOffset = null
00 00 00 00    SubObjectsCount = 0

0x00010020      MATERIALS_HEADER
1D 00 00 00    MaterialsCount = 29
EE 2F 00 00    MaterialsTableOffset = 0x2FEE + 0x00010024 = 0x00013012
E8 03 00 00    1000

0x0001002C      OBJECTS_TABLE
From 0x0001002C to 0x0001007F (0x54 = 84 bytes)   Object 00 (cole_machete)
From 0x00010080 to 0x000100D3 (0x54 = 84 bytes)   Object 01 (cole_machete_left_hand)
From 0x000100D4 to 0x00010127 (0x54 = 84 bytes)   Object 02 (cole_shotgun)
From 0x00010128 to 0x0001017B (0x54 = 84 bytes)   Object 03 (cole_sword_r)
From 0x0001017C to 0x000101CF (0x54 = 84 bytes)   Object 04 (cole_sword_l)

0x0001017C      Object 04 (cole_sword_l)
4D 69 73 63    "Misc"
12 EA 00 00    ObjectNameOffset = 0xEA12 + 0x00010180 = 0x0001EB92
0A EA 00 00    PartDefOffsetsTableOffset = 0xEA0A + 0x00010184 = 0x0001EB8E
01 00 00 00    PartsCount = 1
E6 E9 00 00    UsedMaterialsTableOffset = 0xE9E6 + 0x0001018C = 0x0001EB72
01 00 00 00    ??? = 1
00 00 00 00    
FF FF 00 00    
00 00 00 00    ??? = 0
FF 00 00 00    
00 00 00 80    Some Offset = null
00 00 00 80    Some Offset = null
00 00 00 00    ??? = 0
D6 E5 93 BE    BoundingBoxMinX = 
0C 33 30 BF    BoundingBoxMinY = 
1F C0 5A BF    BoundingBoxMinZ = 
68 37 81 3F    BoundingBoxMaxX = 
53 33 C7 3F    BoundingBoxMaxY = 
F4 44 55 40    BoundingBoxMaxZ = 
00 00 00 80    Some Offset = null
00 00 00 00    ??? = 0

------------------------------------------

0x0001EB72      USED_MATERIALS_TABLE
8E 14 FF FF    ContentsTableOffset   = -60274 + 0x0001EB72 = 0x00010000
AA 14 FF FF    MaterialsHeaderOffset = -60246 + 0x0001EB76 = 0x00010020
0C 00 00 00    Type = 12
02 00 00 00    MaterialsCount = 2
01 00 00 00    ??? = 1
05 00 00 00    MaterialIndex = 5
06 00 00 00    MaterialIndex = 6

0x0001EB8E      PART_DEF_OFFSETS_TABLE
24 E8 FF FF    Part0Offset = -6108 + 0x0001EB8E = 0x0001D3B2

0x0001EB92      OBJECT_NAME
01 00 00 00    StringsCount = 1
14 00 00 00    BytesCount = 20
0C 00 00 00    CharsCount = 12
63 6F 6C 65 5F 73 77 6F 72 64 5F 6C         "cole_sword_l"
00 00 00 00 00 00 00 00                Padding

------------------------------------------

PART 1/1

0x0001D3B2      
172 bytes
00 00 00 00    PartType = 0
1F 00 00 00    Attributes = 0x0000001F
38 ED 99 DC    -3.466
BE 85 CE 3E    0.403
80 5E 66 BF    -0.899
FF DA 29 BE    -0.165
00 00 00 00    0.000
E6 B2 69 3F    0.912
4D 42 C4 3E    0.383
58 BF 0F 3E    0.140
00 00 00 00    0.000
4A 7E 80 BD    -0.062
8E 0A 55 BE    -0.208
F8 E1 79 3F    0.976
00 00 00 00    0.000
A0 07 B4 3E    0.351
A8 6B 61 3E    0.220
CD EA B1 BE    -0.347
00 00 80 3F    1.000
02 00 00 00    ObjectsCount = 2
08 00 00 00    ObjectsTableOffset = 0x08 + 0x0001D402 = 0x0001D40A
00 00 00 00    

0x0001D40A      OBJECTS_TABLE
65 8C 01 BF    BoundingBoxMinX = 
95 40 93 BE    BoundingBoxMinY = 
C3 50 DF BE    BoundingBoxMinZ = 
FD 66 7C 3E    BoundingBoxMaxX = 
73 DB 87 3E    BoundingBoxMaxY = 
51 4D 31 40    BoundingBoxMaxZ = 
02 00 00 00    MeshesCount = 2
10 00 00 00    MeshesTableOffset = 0x10 + 0x0001D426 = 0x0001D436
02 00 00 00    MeshesCount = 2
40 17 00 00    FramesTableOffset = 0x1740 + 0x0001D42E = 0x0001EB6E
02 00 00 00    FramesCount = 2

0x0001D436      MESHES_TABLE
46 64 B2 BD    BoundingBoxMinX = 
52 5D F5 BD    BoundingBoxMinY = 
C3 50 DF BE    BoundingBoxMinZ = 
FD 66 7C 3E    BoundingBoxMaxX = 
17 52 39 3E    BoundingBoxMaxY = 
C8 9B 0C 3F    BoundingBoxMaxZ = 
78 00 00 00    VerticesCount = 120
A4 00 00 00    TrianglesCount = 164
50 00 00 00    VerticesTableOffset = 0x50 + 0x0001D456 = 0x0001D4A6
0C 04 00 00    TrianglesTableOffset = 0x040C + 0x0001D45A = 0x0001D866
A4 00 00 00    TrianglesCount = 164
08 00 00 00    ??? = 8
00 00 00 80    SomeOffset = null
00 00 00 00    SomeCount = 0

65 8C 01 BF    BoundingBoxMinX = 
95 40 93 BE    BoundingBoxMinY = 
00 00 00 00    BoundingBoxMinZ = 
5D 96 3C 3D    BoundingBoxMaxX = 
73 DB 87 3E    BoundingBoxMaxY = 
51 4D 31 40    BoundingBoxMaxZ = 
17 01 00 00    VerticesCount = 279
14 01 00 00      TrianglesCount = 276
B0 07 00 00    VerticesTableOffset = 0x07B0 + 0x0001D48E = 0x0001DC3E
64 10 00 00    TrianglesTableOffset = 0x1064 + 0x0001D492 = 0x0001E4F6
14 01 00 00    TrianglesCount = 276
08 00 00 00    ??? = 8
00 00 00 80    SomeOffset = null
00 00 00 00    SomeCount = 0

0x0001D4A6      VERTICES_TABLE
(0x03C0 = 960 bytes = 120 * 8)

0x0001D866      TRIANGLES_TABLE
(0x03D8 = 984 bytes = 164 * 6)

0x0001DC3E      VERTICES_TABLE
(0x08B8 = 2232 bytes = 279 * 8)

0x0001E4F6      TRIANGLES_TABLE
(0x0678 = 1656 bytes = 276 * 6)

0x0001EB6E      FRAMES_TABLE
(0x04 = 4 bytes = 2 * 2)

0x0001EB72      PARTS_TABLE
8E 14 FF FF    ContentsTableOffset   = -60274 + 0x0001EB72 = 0x00010000
AA 14 FF FF    MaterialsHeaderOffset = -60246 + 0x0001EB76 = 0x00010020
0C 00 00 00    Type = 12
02 00 00 00    IndicesCount = 2
01 00 00 00    SkipParts = 1
05 00 00 00    Index = 5
06 00 00 00    Index = 6

0x0001EB8E      PART_DEF_OFFSETS_TABLE
24 E8 FF FF    Part0Offset = -6108 + 0x0001EB8E = 0x0001D3B2

0x0001EB92      OBJECT_NAME
01 00 00 00    StringsCount = 1
14 00 00 00    BytesCount = 20
0C 00 00 00    CharsCount = 12
63 6F 6C 65 5F 73 77 6F 72 64 5F 6C         "cole_sword_l"
00 00 00 00 00 00 00 00                Padding

[BSM]

I forgot another piece of my notes, about the Materials Table (also for "all_special_Weapons.gsf"):

Code: Select all

0x00013012      MATERIALS_TABLE (29 / 1000 entries)
04             TextureMapType? = 4 (effect_glow)
01 00 00 00 00 00 00    MaterialAttributes = 01
B8 5D 00 00    LightTextureFilenameOffset = 0x5DB8 + 0x0001301A = 0x00018DD2
00 00 00 80    NormalMapTextureFilenameOffset = null
00 00 00 80    AnotherTextureFilenameOffset = null
00 00 00 00    uvChannel? = 0

00 00 00 00 00 00 00 00 C0 5D 00 00 00 00 00 80 00 00 00 80 00 00 00 00 
00 40 00 00 0F 00 00 00 D8 5D 00 00 00 00 00 80 00 00 00 80 00 00 00 00 
01 00 00 00 00 00 00 00 F0 5D 00 00 00 00 00 80 00 00 00 80 00 00 00 00 
04 01 00 00 00 00 00 00 08 5E 00 00 00 00 00 80 00 00 00 80 00 00 00 00 
00 00 00 00 00 00 00 00 9C A2 00 00 00 00 00 80 00 00 00 80 00 00 00 00 
00 40 00 00 0F 00 00 00 B4 A2 00 00 00 00 00 80 00 00 00 80 00 00 00 00 
00 00 00 00 00 00 00 00 28 FE 00 00 00 00 00 80 00 00 00 80 00 00 00 00 
00 40 00 00 0F 00 00 00 30 FE 00 00 00 00 00 80 00 00 00 80 00 00 00 00 
04 01 00 00 00 00 00 00 39 01 01 00 00 00 00 80 00 00 00 80 00 00 00 00 
01 02 00 00 00 00 00 00 51 01 01 00 00 00 00 80 00 00 00 80 00 00 00 00 
04 01 00 00 00 00 00 00 04 07 01 00 00 00 00 80 00 00 00 80 00 00 00 00 
02 01 00 00 00 00 00 00 1C 07 01 00 00 00 00 80 00 00 00 80 00 00 00 00 
01 00 00 00 00 00 00 00 34 07 01 00 00 00 00 80 00 00 00 80 00 00 00 00 
... more entries


Type 0 = ambient?
0x5DC0   + 0x00013032 = 0x00018DF2   "items/spec_weapons_a.tga"      00 00 00 00 00 00 00 00 
0x5DD8   + 0x0001304A = 0x00018E22   "items/spec_weapons_a.tga"      00 40 00 00 0F 00 00 00 
0xA29C   + 0x00013092 = 0x0001D32E   "sequence/sequence_weapons.tga"      00 00 00 00 00 00 00 00 
0xA2B4   + 0x000130AA = 0x0001D35E   "sequence/sequence_weapons.tga"      00 40 00 00 0F 00 00 00 

Type 1 = diffuse?
0x5DF0   + 0x00013062 = 0x00018E52   "items/all_products_lod.tga"   01 00 00 00 00 00 00 00 
0x010151 + 0x0001310A = 0x0002325B   "items/hu_weapons_a.tga"      01 02 00 00 00 00 00 00 
0x010734 + 0x00013152 = 0x00023886   "items/ninigi_weapons_a.tga"   01 00 00 00 00 00 00 00 

Type 2 = specular?
0x01071C + 0x0001313A = 0x00023856   "effects/particle_01.tga"      02 01 00 00 00 00 00 00 

Type 4 = shininess? (glow effects)
0x5DB8   + 0x0001301A = 0x00018DD2   "effects/gore.tga"            04 01 00 00 00 00 00 00 
0x5E08   + 0x0001307A = 0x00018E82   "effects/speedlines.tga"      04 01 00 00 00 00 00 00 
0x010139 + 0x000130F2 = 0x0002322B   "effects/particle_01.tga"      04 01 00 00 00 00 00 00 
0x010704 + 0x00013122 = 0x00023826   "effects/particle_06.tga"      04 01 00 00 00 00 00 00 

I think each model is divided into several parts, so a set of vertices+triangles uses a single material. Two different materials can use the same .dds texture image, but different attributes (light color, intensity, reflectiveness...). So 2 parts of the same model could use a single .dds texture but different attributes, or use different .dds textures.

[DryFun]

I can upload more gsf files if that helps you

[BSM]

I can upload more gsf files if that helps you

Nice, I will ask for some files after a bunch more tests.

Meanwhile, I solved the uvMap
There are still 9 remaining bits, for an unknown yet purpose:
bits 0-12: x
bits 13-25: y
bits 26-38: z
bits 39-47: unknown
bits 48-55: u
bits 56-63: v

Here is an updated cole_machete in .obj format.
However, it seems the dds you posted contains transparent parts that are used for that machete, so I have converted it to .png in order to be able to show the full blade, otherwise the blade seems to be clipped.

cole_machete.zip

Next task will be to study the pending parts on the object definition, so I hope to be able to read objects from program instead of using hard-coded offsets for mesh-definitions.
I think these floats maybe a transformation matrix:

Code: Select all

PART 1/1

0x0001D3B2      
172 bytes
00 00 00 00    PartType = 0
1F 00 00 00    Attributes = 0x0000001F
38 ED 99 DC    ??? = -3.466
BE 85 CE 3E    TrasformationMatrix0A = 0.403
80 5E 66 BF    TrasformationMatrix0B = -0.899
FF DA 29 BE    TrasformationMatrix0C = -0.165
00 00 00 00    TrasformationMatrix0D = 0.000
E6 B2 69 3F    TrasformationMatrix1A = 0.912
4D 42 C4 3E    TrasformationMatrix1B = 0.383
58 BF 0F 3E    TrasformationMatrix1C = 0.140
00 00 00 00    TrasformationMatrix1D = 0.000
4A 7E 80 BD    TrasformationMatrix2A = -0.062
8E 0A 55 BE    TrasformationMatrix2B = -0.208
F8 E1 79 3F    TrasformationMatrix2C = 0.976
00 00 00 00    TrasformationMatrix2D = 0.000
A0 07 B4 3E   TrasformationMatrix3A = 0.351
A8 6B 61 3E    TrasformationMatrix3B = 0.220
CD EA B1 BE    TrasformationMatrix3C = -0.347
00 00 80 3F    TrasformationMatrix3D = 1.000
02 00 00 00    ObjectsCount = 2
08 00 00 00    ObjectsTableOffset = 0x08 + 0x0001D402 = 0x0001D40A
00 00 00 00