modified the mtx script for extracting images from the Indy MTX multitex files... it didn't take into account DXTC it made them all dxt1(which is wrong), but the byte selecting it IS there, just needs to be altered. In the sub headers immediately after the group of four FF's, is a single byte that is either 01 or 05 (Dxt1 or dxt5).. modified code takes the byte, adds 0x30 and then places it correctly in the DDS header
# IJET *.mtx (multi texture format) # quickbms script by WRS, xentax.com #Modified by Medievil to add DXT compression method
idstring "MULTITEX" get SIX short get TWO short get ONE long get ZERO long
get TEXCOUNT long get TEXHEADSIZE long get TWFOUR long # head size get TEXHEADSIZE long # 128
get FNAME basename
for t = 0 < TEXCOUNT get INDEX long getdstring UNKNOWN 16 get UNKNOWN short # 256/ flags? get UNKNOWN short getdstring TEXNAME1 32 getdstring TEXNAME2 32 # not 64 because of repeated data get TWIDTH long get THEIGHT long get UNKNOWN long get BPP long get UNKNOWN long get MINIMAPS long get UNKNOWN long get DXT BYTE get UNKNOWN THREEByte get TEXPOS long get TEXTSIZE long next t # POS is a pointer to minimap info
# setup dds header math DXT += 0x30 putvarchr MEMORY_FILE 0 0x20534444 long #dwMagic (DDS ) putvarchr MEMORY_FILE 4 0x7C long #dwSize (128) putvarchr MEMORY_FILE 8 0x21007 long #dwFlags (prob. wrong)
putvarchr MEMORY_FILE 12 THEIGHT long #dwHeight putvarchr MEMORY_FILE 16 TWIDTH long #dwWidth putvarchr MEMORY_FILE 28 MINIMAPS long #dwMiniMapCount
putvarchr MEMORY_FILE 76 32 long #dwSize putvarchr MEMORY_FILE 80 4 long #dwFlags putvarchr MEMORY_FILE 84 0x00545844 long #dwFourCC putvarchr MEMORY_FILE 87 Dxt byte (DXT1 or 5 depending on byte read) putvarchr MEMORY_FILE 108 4198408 long #dwCaps1
append for i = 0 < MINIMAPS get DATASIZE long # minimap size savepos POS2 log MEMORY_FILE POS2 DATASIZE # save minimap math POS2 += DATASIZE goto POS2 next i append
hmmm I know what was wrong now....I think I have all the info to recreate the multitex file, probably be easier to work with the pak extracted mtx files rather than the dds though...they have less header to remove but still hold all the important stuff (size WxH, DXT1 or 5, number of mipmaps, etc...)....just have to figure out how to write a routine to do it....BMS scripting 101 now in session..lol
Hmm anyone know if it is possible with BMS to load a txt file with parameters in it from within the script?? Pr even better prompt for input from the user?..would make it a lot easier since I could predetermine # of textures and their file sizes so I could know how much space to give the subheader area. Obviously 2 textures means 2 sub headers then the files.... 31 textures will need a lot more space before the files start
ok, here are my thoughts on creating a script... 1) first thing I get the texture name and store in in a variable (since it will be the name of the opened file currently being read I can use basename) 2) I have to get the height/width/mipmaps and DXTC store them in variables(easy enough since I know the locations in a dds or the pak extracted single texture mtx's) 3)at this point I can start creating the file, Place MULTITEX in the first 8 bytes and finish the rest of the main header texture count will has to be increased by 1 as each file is read and the subheader added(basic set it as zero and do a math var +=01 before the next whatever in the for loop to read each file) subheader size is easy, it increases by 80 bytes for each header I can actually wait until the end to write it(Math Subhead = textcount * 80 and place it at the correct byte location) 4) start the subheader... start with placing the text count long(variable that gets increased by 1 each time) at the first subheader location, place my string of 16 FF's, 8 bytes of filler, then place the filename without extension(32 bytes to fill with it, so the remaining will be 00's, issue will be determining how many), do it again cause the name is doubled, place width, height, bitdepth always 32bit so a 0x20000000),0x08080808, mipmaps long, 0xFFFFFFFF, DXTC (depending on which I use, dds I have to sub 30, the mtx the the pak extracter made, I have to subtract 40) either 0x05000000 or 0x01000000 and next the offset for the stored location of the actual texture info (subheader total size ..number of textures* 80 + 24(the multitex header size)) it will have to be done after all the textures have been processed before the final file is written, unless there is a way to return the number of files to be processed. Each subheader will have to know and have it's texture data size (next entry below)added for the next subheader's offset... and finally the hard part..maybe??? IF asize works like I think and returns the size of the current file being read I can simply subtract 80 for dds or 30 for mtx and place it in the final subheader location, if not I will have to figure a different way... and at this point the only way I know save the texture data is to do a nested loop and save it to a memory file and keep appending the rest of the texture data to add to the main memory file at the very end after all the subheaders are made... if all thst works like I hope, rinse and repeat for all the texture files for that multitex and finally save it
ok gotta good start its now simply a matter of finishing the subheader/text data writes should finish it tomorrow evening after work...here it is so far, everything is working correctly and yes...a mess at the moment, clean up AFTER it works..lol
#create multitex header set DSIZE 00 set Ttotal ? ? #user prompt for total number of textures for THIS multitex
xMath toffset "Ttotal * 0x80" # generate offset based on 80 byte subheaders multiplied by total number of textures math Toffset += 0x24 #add multitex header size Print "%Ttotal%" #debug to make sure it was correct print "%TOffset%" #debug to make sure it was correct log memory_file 0 0 putvarchr MEMORY_FILE 35 0 #setup header Putvarchr memory_file 0 0x000000020002000658455449544c554d longlong #multitex putvarchr memory_file 16 0x00 long putvarchr memory_file 20 Ttotal long putvarchr memory_file 24 Toffset long putvarchr memory_file 28 0x00000018 long #multitex header offset putvarchr memory_file 32 0x0000 short
#get info from mtx labled single texture (can't use dds currently, once this is completed I just have to change the gets to dds format) # bpp is alway 32 for these textures as well so it wasn't needed to be obtained. getdstring Textname 32 #texture name get TWIDTH long #texture width get THEIGHT long #texture height get NULL long #not needed, usually zeros, could contain BPP flag as 01 or 08 (not sure) get NULL long #same as above get FSIZE asize long #file size math FSIZE -= 0x40 #file size minus the header to get texture data size xmath DSIZE "DSIZE + FSIZE" #as files are read need to keep up with next entry point to write next file, could also do toffset + fsize right after I write the texture data and store it in a var to elimitae a step cause I still have to take dsize and figure in the data start offset anyway get Null long #n/a get NULL Long #n/a get fourcc long #dxtc Get null long #n/a Txtpos = 0x40 #position in file where texture data begins print "%txtpos%" #debug check
print "Dsize %DSIZE%" #debug check IF fourcc = 0x10 set fourcc = 0x40 else endif math fourcc -= 0x40 # multitex needs either 01 or 05 (In this case dxt10 would need 10 dxt3 =3, etc), texture file has 41 and 45 print " 4CC %fourcc%" #debug check # Write subheaders as they are read then jump to texture offset and write text data
**EDIT** Updated cause I found a condition that had the dxt as 10 and in the indy mtx under the same name it has 00 in the dxt position so I set it to 40 so it will end up 00...will see how it goes
ok roadblock alert!!! not sure I understand something... is there any way or function for the memory file position to advance as data is added? if you add a 32 byte line will it advance 32 bytes? or can I say PutVarChr 10 + Offset data long??? trying to figure out a way to write the subheaders by using offsets cause each subheader will be 80 bytes from the previous first @ 24, second @ a4...etc..
ok forget the above, found a long way around but it works current code, up to writing the texture to the memory file now, have to read about it, not sure if I should do one byt at a time or the whole thing at once(if thats even possible)
Print "total number of textures" set Ttotal ? ? #user prompt for total number of textures for THIS multitex set textnum 0x00 set offset1 0x00 xMath toffset "Ttotal * 0x80" # generate offset based on 80 byte subheaders multiplied by total number of textures math Toffset += 0x24 #add multitex header size Print "%Ttotal%" #debug to make sure it was correct print "%TOffset%" #debug to make sure it was correct log memory_file2 0 0 #putvarchr MEMORY_FILE2 toffset 0 #setup header Putvarchr memory_file2 0 0x544c554d long Putvarchr memory_file2 4 0x58455449 long Putvarchr memory_file2 8 0x00020006 long Putvarchr memory_file2 12 0x00000002 long #multitex putvarchr memory_file2 16 0x00000000 long putvarchr memory_file2 20 Ttotal long putvarchr memory_file2 24 Toffset long putvarchr memory_file2 28 0x00000018 long #multitex header offset putvarchr memory_file2 32 0x00000000 long # 34 - 37 are the texture file number(long) in each subheader,, first is 00, second 01, etc... #get info from mtx labled single texture (can't use dds currently) # bpp is alway 32 for these textures as well so it wasn't needed to be obtained.
get Textname long #texture name
get textname2 long get textname3 long get textname4 long get Textname5 long #texture name
get textname6 long get textname7 long get textname8 long get TWIDTH long #texture width get THEIGHT long #texture height get NULL long #not needed, usually zeros, could contain BPP flag as 01 or 08 (not sure) get NULL long #same as above get FSIZE asize long #file size math FSIZE -= 0x40 #file size minus the header to get texture data size xmath DSIZE "DSIZE + FSIZE" #as files are read need to keep up with next entry point to write next file, get Null long #n/a get NULL long #n/a get fourcc long #dxtc Get null long #n/a Txtpos = 0x40 #position in file where texture data begins print "%txtpos%" #debug check
print "Dsize %DSIZE%" #debug check IF fourcc = 0x10 set fourcc = 0x40 else endif math fourcc -= 0x40 # multitex needs either 01 or 05 (In this case dxt10 would need 10 dxt3 =3, etc), texture file has 41 and 45 print " 4CC %fourcc%" #debug check
PRP1986 wrote:I’ve been following this (in awe as I struggle to understand scripting lol) - good work!!
trial and error, mostly error..lol have hit a snag,, I dunno how to process the next file without the script going back to the beginning and asking for number of textures and recreating the multitex part of the header again (as well as resetting the pointers)
hmmm...might be easier to crate a python script I can jump around in the open file (f.seek command) to get the info, place it is the output file (w.seek command (f. and w. are placeholders I can use anything I want, just easier to keep track as f and w) this simple little example is almost all I need.. just have to recreate the subheaders.... this already offsets the data and places it, creates the main header (minus the data offset cause I haven't computed it in the code yet), it gets all the mtx filenames, computes how many there are. so it is all automated (will need one user prompt though the save file name once I get it completed although I could probnaly automate that too just using one of the filenames and cutting off the (xxxx stuff at the end hardest thing here is converting to hex cause python is not a native hex program had to use a txt file with the multitex header in hex in it to avoid writing it one character at a time
import binascii import os import glob from os import listdir from os.path import isfile, join files_in_dir = [ n for n in glob.glob('y:\\test1\\*.mtx*') if isfile(join('y:\\test1\\*.mtx*',n)) ] total = len([ n for n in glob.glob('y:\\test1\\*.mtx*') if isfile(join('y:\\test1\\*.mtx*',n)) ]) print total w=open('y:\\test1.mtx','w+b') with open('y:\\header.txt') as g: for line in g: w.write( binascii.unhexlify(''.join(line.split())) )
w.seek(0) w.seek(20) w.write(chr(total))
dataoffset=(total * 128) + 36 print dataoffset w.seek(dataoffset) for file in files_in_dir: #with open( file, 'rb') as f: f=open(file,'rb') b = os.path.getsize(file) b = b - 64 f.seek(64) for i in range(b): a = f.read(1) w.write(a) #print a, f.close w.close
I am one byte (can't seem to read it as an integer) away from having a complete python solution!! I'll likely tweak it a bit, currently have to read 3 separate txt files for some static hex. Could probably place them inside as string's ..and gotta tweak the path so you'd just have to run it from the folder you put the group of mtx files in..and gotta convert it to executable so one doesn't need python installed. Python scripting is a lot easier than I always thought... reminds me of a cross between old standard basic and 8bit assembly...learned alot in 2 days!
script is done!! works perfect...all one have to do is place all the like texture files in a folder (all caustic's together for example) and the script... run it and enter the folder where the textures are (mtx format that the buffy pak extractor makes) and enter the name for the final output file(all the caustic(xxxx) should be simply caustic.mtx (,mtx not needed on script)...in the meshes folder you take for example, under scoobygang folder all the willow files, name the output willow... and in a few seconds you'll have the full mtx file in a folder inside called output. Be careful not to mix in other textures probably wouldn't hurt anything BUT you never know.. you could also use this to create all new indy textures although you have to strip thee header and create a new mtx ripped format one.. not hard bu...maybe I will work on that next Not posting yet as I am tryign to get the environment setup to turn the script into an exe
Last edited by medievil on Sat May 05, 2018 8:30 pm, edited 1 time in total.
here it is... readme included for usage removed link, found a bug...not in the program so much as the format that I missed (4 byte separator between textures) if anyone grabbed it, wait, though they look correct, the produced mtx's won't decode
still workign on it, ran across a rather perplexing obstacle... who-da thunk taking a raw 4 byte aa bb cc dd and making it dd cc bb aa would be so elusive! I need to use the data size inside the individual texure mtx's(not the total file size) and it is in reverse order (Which is fine for writing it to the main mtx, but not for calculation offset)
have fixed my tool, still not converted it to exe yet...textures extract fine from the mtx extractor that was made to extract indy textures (that 4 byte separator I found wasn't.. it was actually texture data size so had to be specifically written the textures have more than one mipmap as well, but I'll have to study how they are packed in the indy mtx (not sure they have a header as they do on the buffy extracted ones).. ANYWay.. textures work, but they still don't work in game in the level I subbed in as well as the character model I subbed in... Indy looks like buffy, but no textures..I assumed the mesh would specify the textures to use (Same for levels, like the spanish mission..no textures, fire works though (it is using the indy caustic or partical mtx that came with indy. Pretty sure they are the same on both) here is a screen shot:
new discovery that s also not helpful in the end... the indy exe can read the pak files (have to set pak in the default.cfg to 2).. after it reads the one it needs though it pops up an access violation map stalkwalker or something or other..., it specifically told me it couldn't read res_m01_srilanka_01.pak untill I placed it in the folder then it continued on, it read the res null pak first
progress... seems pc's indy res.dat is the same format and stuff as buffy's res_null.pak.. renamed it res data and placed textures in the same folder layout as buffy (not all of them yet) and.... **edit** forgert above, simply delete the res.dat, the exe will recreate it with stuff added