ParaWorld .gsf container

[BSM]

Update.
More problems found: there are different ways to store data for vertices
But at least I have clues on how to start...

Code: Select all

0x0001D436      MESHES_TABLE
46 64 B2 BD    BoundingBoxMinX = 
52 5D F5 BD    BoundingBoxMinY = 
C3 50 DF BE    BoundingBoxMinZ = 
FD 66 7C 3E    BoundingBoxMaxX = 
17 52 39 3E    BoundingBoxMaxY = 
C8 9B 0C 3F    BoundingBoxMaxZ = 
78 00 00 00    VerticesCount = 120
A4 00 00 00    TrianglesCount = 164
50 00 00 00    VerticesTableOffset = 0x50 + 0x0001D456 = 0x0001D4A6
0C 04 00 00    TrianglesTableOffset = 0x040C + 0x0001D45A = 0x0001D866
A4 00 00 00    TrianglesCount = 164
08 00 00 00    VertexDataSize = 8
00 00 00 80    SomeOffset = null
00 00 00 00    SomeCount = 0

VertexDataSize = 8, but it is 16 for ice_waste_animals.gsf (that is 128 bits instead of just 64), so I need to do some more tests to discover what bits to use for each field.

[BSM]

The vertex and texture vertex uses the same bits for 8 or 16 bytes data, so I can let these extra 8 bytes for later.
I have discovered that some objects uses two sided textures; that is, after loading a .obj into blender, I see just one 'side' of the object being textured, while the other 'side' keeps black. I need more tests to find a way to solve this.

About reading all the inner files from program, instead of providing hardcoded offsets to read meshes, I got some advances.
The whole package structure seems like this:

Code: Select all

--- GsfPack
GSF_PACK_HEADER
TREE_NODES
(Padding)
CONTENTS_TABLE
MATERIALS_HEADER
OBJECTS_TABLE
SUBOBJECTS_TABLE
DATA

The 'Data' part is:

Code: Select all

--- Data
Object00
Object01
Object02
...
Objectnn

An object is:

Code: Select all

PARTS_DEF
PARTS_TABLE
(if is first object)   SUBOBJECTS (Full, not just the childs of this object)
USED_MATERIALS_TABLE
(if is first object)   MATERIALS_TABLE (Full, not just the ones used by this object)
USED_TEXTURE_FILENAMES
PART_DEF_OFFSETS_TABLE
OBJECT_NAME

The 'PartsTable' is what needs to be parsed depending on object def, containing different chunks of data for each object type.
I will try to read a whole package and parse just the object types already implemented, so I can add later parsing for more types. For example, an 'Anim' will contain data chunks not used for a 'Char'.

All the offsets are written as relative, except the first one to give the location of the ContentsTable. So it should be possible to extract data just for an object, and include somewhere the relevant parts of the SubObjects and Materials, updating the needed offsets.

[DryFun]

If you can extract vertices and triangles data and convert it in the .obj
Can I convert vertices and triangles data from .obj to .gsf format and replace some model vertices and triangles data to new?
If the new model not bigger than the original

[BSM]

If you can extract vertices and triangles data and convert it in the .obj
Can I convert vertices and triangles data from .obj to .gsf format and replace some model vertices and triangles data to new?
If the new model not bigger than the original

Yes, it could be done in both ways, packing and unpacking, even if the new models are bigger; the problem isn't that, but to find the right properties for the new models: besides mesh data, there is some other data stored in the gsf package.

Please we really need help, we are the only team which is keeping this game alive

Are you related to this guy's team?

I will continue doing this just for fun, but just to be able to extract models; packing new models will require much more work, and I don't know if you are part of a team with coders, or just can edit/create models, and I wonder how many people could use anything that comes from this...

[DryFun]

Yes, we are from the same team
Rather it is the only one team about pw modding

This is our site: http://www.para-welt.com/ (offline/moves to https)
And discord: https://discord.gg/maauBUH

In general, we can create models right now, we simply dont have tools
And we have coders, one of them has already tried to do something with gsf but he didnt succeed
You can find his work in the Kharg's post

[DryFun]

Is there any offsets to lod?
So I can change lod2 to lod1 or just delete all lods

I also didnt see any offsets to animations

[BSM]

Is there any offsets to lod?
So I can change lod2 to lod1 or just delete all lods

I also didnt see any offsets to animations

Hi.

I don't see yet any way to guess the LOD, except the amount of vertices/triangles for a single mesh. Maybe some of these bytes still unassigned to a meaning...
As for animations, there is a chunk specific for "Anim" that I have not studied yet, and I'm pretty sure it contains info on how to show the different frames for a single object, including the timing.

I have not gave up, just need 2-3 days to recover from illness, then I will continue.

And we have coders, one of them has already tried to do something with gsf but he didnt succeed

The website keeps being offline (connection refused). If it can help the team, I will share my source code (C#), or even I could try to help a bit with C++ if you already have some guy working on that. You can send me a private message or post here a link to any repository, or I could just setup a new one.

[DryFun]

These are class names
Anim isnt animation, but animal
Char = characters (infantry)
Bldg = buildings
Deko = vegetation,trees,rocks...
Misc = weapons and other stuff
Thats names for flags

Machete with standard class (Misc)

scr_20171207_170620.jpg

And with Anim class

scr_20171207_170444.jpg

In fact, this only changes the names and the number of flags in sequences editor
New flags, which were not there before, dont affect anything
Flags effects still depend on what data are in gsf file for this model

[BSM]

In fact, this only changes the names and the number of flags in sequences editor

Nice info. Is that sequences editor part of the game, a self-made tool or what?

I got some nice advances. I completed the dissection by hand of "1 box and 1 animation.gsf" (attached my notes in .txt format), by comparing the data in the different chunks with data from the other .gsf files.

1 box and 1 animation.gsf.structure.dissection.txt

Then added some code to my test tool and I'm able now to parse all 3 sample .gsf files, just skipping some unknown parts.

For each Object or SubObject, I get several chunks of data, and these data need to be parsed depending on chunk type.
Chunk types 0x00000000 and 0x80000000 contains mesh definitions (vertices, triangles and frames).
Another types (like 0x00000005) I know how to read it, even if I don't know the meaning.
Anyway, I can now skip any chunk with unknown type and get to the mesh data. These skipped data surely contains info on how to link the different meshes into a single model (bounding box, rotation, scale...).
However, there may be more chunk types containing meshes, in another .gsf files.

I already can preview (or extract) by program any mesh included for the known chunk types. So I saw the different parts for each of the 5 animals included in "icewaste_animals.gsf", and understood something more about the TreeNodes part:
- bothriolepis has 3 childs in TreeNodes: swim, standanim and dying; I have found 3 childs in ChildsDef chunk, but just 1 of them points to a MeshesDef chunk (the other 2 are null pointers), and that MeshesChunk corresponds to the SubObject00.
- dunkleosteus has 8 childs in TreeNodes: attack_front, swim_1, swim_2, swim_3, swim_left, swim_right, standanim and dying; I have found 8 childs in ChildsDef chunk, each one pointing to a MeshesChunk, all them corresponding respectively to SubObjects 01 to 08.
- henodus has 4 childs in TreeNodes: swim, growup, standanim and dying; I have found 4 childs in ChildsDef chunk, 2 of them pointing to a MeshesDef chunk, corresponding to SubObjects 09 and 0A, and the other 2 with null pointers.
- quetzalcoatlus has 2 childs in TreeNodes: standanim and dying; I have found 2 childs in ChildsDef chunk, both them with null pointers.
- quetzalcoatlus_boden has 2 childs in TreeNodes: standanim and dying; I have found 2 childs in ChildsDef chunk, both them with null pointers.

So all data in the 11 SubObjects could be linked to the corresponding base mesh for the 5 animals.

Next task is to check again these skipped data and try to assemble the different parts in a model.

[DryFun]

Seq editor is included in the official ParaWorld SDK

[BSM]

... understood something more about the TreeNodes part:
- bothriolepis has 3 childs in TreeNodes: swim, standanim and dying; I have found 3 childs in ChildsDef chunk, but just 1 of them points to a MeshesDef chunk (the other 2 are null pointers), and that MeshesChunk corresponds to the SubObject00.

Explanations for myself about the last findings, in case I lose some notes and some questions in case you have some idea.

PackFile: icewaste_animals.gsf

ObjectsTable, first entry:

Code: Select all

0x0001002C      bothriolepis
43 68 61 72    Type = "Char"
38 FE 00 00    ObjectNameAddress =    0x00010030 + 0xFE38 = 0x0001FE68
2C FE 00 00    MeshDefAddress =       0x00010034 + 0xFE2C = 0x0001FE60
02 00 00 00    MeshDefCount = 2
1C A0 00 00    MaterialsDefsAddress =    0x0001003C + 0xA01C = 0x0001A058
01 00 00 00    MaterialsDefsCount = 1
00 00 00 00    ??? = 0
FF FF 00 00    ??? = 65535
00 01 00 00    ??? = 256
FF 00 00 00    ??? = 255
00 00 00 80    ???Address = null
00 00 00 80    ???Address = null
00 00 00 00    ??? = 0
48 AF 10 BF    MinX = -0.565
FB 41 7E BF    MinY = -0.993
64 B8 37 BE    MinZ = -0.179
4A AF 90 3F    MaxX = 1.130
DA B5 FD 3F    MaxY = 1.982
57 E6 E1 3E    MaxZ = 0.441
10 FE 00 00    ChildsDefAddress =       0x00010078 + 0xFE10 = 0x0001FE88
03 00 00 00    ChildsCount = 3

Follow the pointer to the ChildDefs:

Code: Select all

0x0001FE88
A4 01 FF FF 20 00 00 00 01 00 00 00    0x0001002C   78 04 FF FF 0A 00 00 00    0x00010324
98 01 FF FF 00 00 00 80 00 00 00 00    0x0001002C
8C 01 FF FF 00 00 00 80 00 00 00 00    0x0001002C

3 childs, but just the first one points to a chunk of data at 0x00010324

Now check the SubObjectsTable, first entry:

Code: Select all

0x000101D0      Helper01
43 68 61 72    Type = "Char"
30 01 00 00    ObjectNameAddress = 0x000101D4 + 0x0130 = 0x00010304
74 04 00 00    DefPointersTableAddress = 0x000101D8 + 0x0474 = 0x0001064C
01 00 00 00    DefPointersCount = 1
00 00 00 80    null
01 00 00 00    1
0A 00 00 00    TransformationsCount = 10

And follow the pointer to DefPointersTable:

Code: Select all

Char   0x0001064C (1)   (10)   Helper01            D8 FC FF FF    -    808 = 0x00010324

And it points to the same chunk of data at 0x00010324

However, that chunk contains no mesh, but transformations (data used to animate a mesh or set of meshes):

Code: Select all

SubObject00 for bothriolepis 1/1 (Char Helper01)

0x00010324
05 00 00 40    ChunkType = 0x40000005
00 00 00 00    
5B F9 F4 16    Guid = 0x16F4F95B
00 00 00 80    
FF FF FF FF    -1
00 00 80 BF    -1.000
00 00 01 00    65536
1C 00 00 00    DataChunkAddress = 0x00010340 + 0x001C = 0x0001035C
A8 01 00 00    NextHeaderAddress = 0x00010344 + 0x01A8 = 0x000104EC
01 00 00 00    NextHeadersCount = 1
0A 00 00 00    TransformationsCount = 10
00 00 00 00    0.000
00 00 00 00    0.000
00 00 00 00    0.000

0x0001035C      Data
40 bytes * 10 transformations = 400 bytes = 0x0190

0x000104EC      Header
01             Index = 1
02             Always 2?
01             Values found are 0, 1 and 2
00             Always 0?
0C 00 00 00    DataChunkAddress = 0x000104F0 + 0x000C = 0x000104FC
A8 00 00 00    NextHeaderAddress = 0x000104F4 + 0x00A8 = 0x0001059C
01 00 00 00    NextHeadersCount = 1

0x000104FC      Data
16 bytes * 10 transformations = 160 bytes = 0x00A0

0x0001059C      Header
02             Index = 2
02             Always 2?
00             Values found are 0, 1 and 2
00             Always 0?
0C 00 00 00    DataChunkAddress = 0x000105A0 + 0x000C = 0x000105AC
00 00 00 80    NextHeaderAddress = null
00 00 00 00    NextHeadersCount = 0

0x000105AC      Data
16 bytes * 10 transformations = 160 bytes = 0x00A0

0x0001064C      DefPointer for SubObject00
D8 FC FF FF    0x0001064C -    808 = 0x00010324

All tested chunks of this type (on different .gsf packages) uses the same structure. It seems like a way to build frames for animations. Can you check in game if that amount correspond to an amount of frames? If so, my guess from data is that it is used some amount of transformations to build the different frames:
for bothriolepis
swim - 2 transformations
standanim - I have no data
dying - I have no data
for dunkleosteus
attack_front - 41 transformations
swim_1 - 31 transformations
swim_2 - 21 transformations
swim_3 - 16 transformations
swim_left - 1 transformation
swim_right - 1 transformation
standanim - 31 transformations
dying - 41 transformations

being the amount of frames the number found in the TreeNodes. For dunkleosteus:
attack_front, index=0, childs=7 { 0, 2, 3, 4, 5, 1, 6 }
swim_1, index=1, childs=6 { 7, 10, 8, 11, 9, 12 }
swim_2, index=2, childs=6 { 7, 14, 10, 13, 15, 16 }
swim_3, index=3, childs=6 {17, 19, 20, 18, 21, 22 }
swim_left, index=4, childs=0
swim_right, index=5, childs=0
standanim, index=6, childs=29 { 23, 25, 26, 27, 28, 29, 30, 31, 32, 32,
32, 32, 32, 32, 32, 32, 32, 32, 24, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33 }
dying, index=7, childs=3 { 34, 35, 36 }

However, I have tested the corresponding chunks for several entries, and I find 2 indices for bothriolepis.swim, and none for bothriolepis.standanim and bothriolepis.dying; same weirdness for dunkleosteus, found 7 indices for attack_front (that's correct, but the order 1,5,2,3,4,6,7 seems weird), and 6 indices for swim_1 (that's correct), but found 6 indices for dying (expected just 3). I will try to check more chunks and see what I find.

About the data found, 1 block of 40*transformationsCount bytes for each animation, and several blocks of 16*transformationsCount bytes, I guess a block for each "frame". The 16 bytes maybe just a quaternion (4 floats used to store a rotation), but the 40 bytes are still of unknown purpose for me; any idea?

[DryFun]

Yes, it's animation frames

I cant say anything about these 40 bytes, you already know about the gsf more than I

[DryFun]

I know that some animations plays with sounds from header
So dying anim plays with 00_npc/npc_dino_huge1_water_die.wav (starts with anim) and 02_battle/hit_water_medium2.wav (starts after 22 frame)

Also Animals1 and Animals3 (And Animals2,Animals4) after sounds in header used for walk anim speed. swim_1 - speed 1, swim_2 - speed 2, swim_3 - speed 3, swim_4 - speed 4

I dont think that this is how fast frames changes, but how fast the model moves in world

Also there is something for effects (CAllosaurus_Sm_Scrunch_Fx, CBrachioWalking ...), I think it points to the effect class and then to effect model (SetGFX(" "); line)
So when brachio dies and at the moment when brachio falls to the ground, spawns an effect with such settings:

Code: Select all

class CBrachioDying inherit CEffect
   export constructor()
      var ^CObj pxOwner=m_xOwner.GetObj();
      SetPos(pxOwner^.GetPos());
      SetRot(pxOwner^.GetRot());
      SetGFX("Placeping_Building_Fx");
      SetAnim("anim",1,0);
      CMainController.Get().GetCameraCtrl().ShakeCameraTheGundlachStyle(1.0f,40,75.0f,GetPos());
      SetDieInterval(1.55f);
   endconstructor;

   export destructor()
   enddestructor;
endclass;

all_animals.gsf https://www.dropbox.com/s/wm7k1dtsuhw6v ... nimals.gsf

There is also a file that somehow works with speed and .gsf file, but I dont know how:

walkanimconfig.txt

And file with the links names. There arent all names, but most:

links.txt

[DryFun]

About links
Link is a point to which you can attach other models, like weapons or effects

"projectile" link is in the blue sphere center

scr_20171217_214519.jpg

"Rider" and "Levelflag"

scr_20171217_215837.jpg

About flags
All flags off

scr_20171217_220229.jpg

PartyCol and Saddle flag on

scr_20171217_220241.jpg

PartyCol and Saddle models (as well as other flags) are in gsf, but not as separate models so they dont have entries in file header

[BSM]

So much new info, interesting, specially the possibility to check a model with flags on and off.
I have found the SDK and it's free, but it doesn't install without the game (and the game is not available for sale).
I will try to get something from this.

[DryFun]

You can still download the demo version and activate all the features of the game (mostly)
Demo is a bit old game build, without campaign and multiplayer maps
http://games.softpedia.com/get/Games-De ... Demo.shtml

But still there is a lot issues
Game doesnt work on win7,8,10 without fix:
https://www.dropbox.com/s/rz74aby92x8po ... in7fix.exe
SDK: https://www.dropbox.com/s/pelzqiixffy2h ... fixsdk.exe
Path for SDK - ...\Paraworld

The order should be like this: Game - fix - SDK
Also you should add -leveled to PWClient.exe shortcut

Or tell me what you want to know

[DryFun]

If the model has some effects not from effects.gsf, then these effects should be at the model header and starts with яяяя

Like яяяяglow_01,яяяяglow_02. all_special_weapons.gsf 11E64 - 11F5F

scr_20171221_001946.png

Also found a new data, but only in one of all .gsfs so I'm not sure about its usefulness
all_characters.gsf 35DA8E - 3A3936
https://www.dropbox.com/s/rrj3aj5vb5hq0 ... acters.gsf

This is necessary for effects

Code: Select all

//character blood hit
class CBloodHit_01_01 inherit CEffect
   export constructor()
      var int iBloodAmount = 0;
      var real fBloodForce = 0.0;
      if (!CSettings.Get("Game/Gore/BloodAmount",iBloodAmount)) then
         CSettings.Set("Game/Gore/BloodAmount",iBloodAmount);
      endif;
      if (!CSettings.Get("Game/Gore/BloodForce",fBloodForce)) then
         CSettings.Set("Game/Gore/BloodForce",fBloodForce);
      endif;

      SetDieInterval(0.3);
      //SetPos(m_xOwner.GetObj()^.GetPos());
      var ^CObj pxOwner=m_xOwner.GetObj();
      var vec3 vP;var Quat qR;CFXHelper.GetLinkPosRotW(pxOwner,"Blfa",vP,qR);
      SetPos(vP);
      SetRot(qR);

      SetGFX("Blood_01");

      var CEffectInfo xFX;
      xFX.m_sEffectClass="CBloodHit_01_02";
      CEffect.AddEffect(pxOwner,xFX);

      var vec3 vOfs=vP-pxOwner^.GetPos();
      var vec3 vDir={0.0,1.0,0.0};
      qR.VecRotateU(vDir);
      xFX.m_sEffectClass="_dusttrail";
      xFX.m_xAttribs.SetValue("bloodcount",iBloodAmount);
      xFX.m_xAttribs.SetValue("force",fBloodForce);
      xFX.m_xAttribs.SetValue("offset_x",vOfs.GetX());
      xFX.m_xAttribs.SetValue("offset_y",vOfs.GetY());
      xFX.m_xAttribs.SetValue("offset_z",vOfs.GetZ());
      xFX.m_xAttribs.SetValue("direction_x",vDir.GetX());
      xFX.m_xAttribs.SetValue("direction_y",vDir.GetY());
      xFX.m_xAttribs.SetValue("direction_z",vDir.GetZ());
      CEffect.AddEffect(pxOwner,xFX);
   //m_xAttribs;
   endconstructor;

   export destructor()
   enddestructor;
endclass;

[DryFun]

Some findings about effects (яяяяglow_01)
01 isnt effect name, its the index of this effect on the model

Code: Select all

FF FF FF FF 67 6C 6F 77 5F 30 31   яяяяglow_01
00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 02 0B 00
FF 00 7F 7F   FF effect size
00 00 FF 41
00 00 48 42
00 00 00 00
00 00 00 00
00 00 00 00
00 00 00 00
00 00 04 02
00 00 00 00
00 00 00 00
00 00 80 3F
00 00 00 00
00 00 00 00
00 00 00 00
00 00 00 00
00 00 00 00
00 00 00 00
00 00 00 00
00 00 00 00
FF 00 00 00   Red strength
FF 00 00 00   Green strength
FF 00 00 00   Blue strength
00 00 00 00
00 00 00 00
00 00 00 00
00 00 80 3F

[DryFun]

After a year, most of the format specification is now clear
But I need help with bind pose. How is it stored in other formats, how to work with it, and how to check that this is exactly bind pose?

.gsf_spec.txt

[aria1996]

I have a data format file for a game and I need software that can compile and decompile.