EA XMA multilayer audio files

Codecs, formats, encoding/decoding of game audio, video and music
AlphaTwentyThree
Posts: 909
Joined: Sat Aug 09, 2014 11:21 am

EA XMA multilayer audio files

Post by AlphaTwentyThree »

Hello there!
I'm currently taking another look at those pesky XMA files from Electronic arts, namely from Battlefield: Hardline. ea_multi_xma.exe from HCS doesn't work on these so I tried to write my own parser but strangely it doesn't work.
So here's my work-in-progress. This is an example file: http://*USE_ANOTHER_FILEHOSTING*/3408d63 ... 78/test.7z
. For testing purposes I've manually split the file into the two streams that are present and began testing with the first one. The structure of these streams is fairly simple:
(big endian)
0x00 - 4800000C long
0x04 - channel count
0x06 - frequency short
0x08 - number of samples
0x0c - END - stream structure:
short: 0x4400 - new block marker, 0x4500 stop marker
short: length of block
longlong: dummy (or something else?)
length_of_block - 0xc: stream data
To parse the blocks you just write the stream data and fill the block up to the next 0x800 multiple with zeros to get a complete XMA2 block. Afterwards it should be parsable with xma_parse. However the parsing stops at around 63kb and this is where it gets interesting: This very block has a wrong size (8 bytes short) so I've coded in a workaround that writes the missing 8 bytes into the block but to no avail: parsing stops at this block, no matter if I leave the additional bytes or discard them.
Here is the in-progress script I wrote:

Code: Select all

idstring \x48\x0\x0\xC
endian big
get UNK short
get FREQ short
get UNK long
callfunction getlayers 1

set LAYERS 1
for k = 1 <= LAYERS
   log MEMORY_FILE 0 0
   set OFFSET 0xc
   for i = 1
      goto OFFSET
      get TEST short
      if   TEST == 0x4500 # end marker
         callfunction wrapup 1
      endif
      get SIZE short
      math SIZE -= 0xc
      get DUMMY longlong
      savepos OFFSET
      set BIAS 0
      set ADJUST OFFSET
      math ADJUST += SIZE
      goto ADJUST
      get TEST short
      if TEST == 0x4400
      elif TEST == 0x4500
      else
         goto ADJUST
         do
            get TEST byte
            get DUMMY2 threebyte
            savepos SIZE2
         while TEST != 0x44
         xmath BIAS "SIZE2 - 4 - OFFSET - SIZE"
      endif
      callfunction write 1
      #set GO i
      #math GO %= LAYERS
      #if GO == k
      #   callfunction write 1
      #elif GO == 0
      #   if k == LAYERS
      #      callfunction write 1   
      #   endif
      #endif
      math OFFSET += SIZE
      math OFFSET += BIAS
   next i
next k

startfunction getlayers
   savepos MYOFF
   get DUMMY long
   set i 0
   do
      math i += 1
      get TEST short
      get DUMMY short
   while TEST == 0
   math i -= 1
   set LAYERS i
   goto MYOFF
endfunction

startfunction write
   goto OFFSET
   append
      log MEMORY_FILE OFFSET SIZE
   append
   get MSIZE asize MEMORY_FILE
   math MSIZE x= 0x800
   math MSIZE -= 1
   putVarChr MEMORY_FILE MSIZE 0 # write 0 at end of block
endfunction

startfunction wrapup
   get SIZE asize MEMORY_FILE
   get NAME basename
   string NAME += ".uxma"
   log NAME 0 SIZE MEMORY_FILE
   cleanexit
endfunction

As you can see, I'm currently treating files as single layer as I don't know how to distinguish between single- and multi-layer files (thus "set LAYERS 1"). Anyway the outcome is the same. The block-commented section ("set GO i" and so on) is for multilayer files and still work-in-progress.
Ok, this is what I got, now I'll need some help. You can parse the res files to uxma but pasing isn't possible.
Anybody has the time and patience to take a look at this? That would be wonderful!
Last edited by AlphaTwentyThree on Sun Sep 27, 2015 11:52 am, edited 2 times in total.
ili
Posts: 81
Joined: Wed Sep 17, 2014 2:28 pm

Re: EA XMA multilayer audio files

Post by ili »

old method look close to work but sure help is welcome
id-daemon
Posts: 1040
Joined: Sun Mar 22, 2015 7:09 pm

Re: EA XMA multilayer audio files

Post by id-daemon »

AlphaTwentyThree wrote:0x04 - codec byte / channel number
0x06 - frequency short
0x08 - flags / number of samples
AlphaTwentyThree
Posts: 909
Joined: Sat Aug 09, 2014 11:21 am

Re: EA XMA multilayer audio files

Post by AlphaTwentyThree »

id-daemon2 wrote:
AlphaTwentyThree wrote:0x04 - codec byte / channel number
0x06 - frequency short
0x08 - flags / number of samples

Thanks. Any idea why the files won't parse?
id-daemon
Posts: 1040
Joined: Sun Mar 22, 2015 7:09 pm

Re: EA XMA multilayer audio files

Post by id-daemon »

Can you upload the files again? zippyshare or mega

Considering parsing stops at 64k it is most possible that files were extracted wrong from CAS files. What did you use to extract them?
id-daemon
Posts: 1040
Joined: Sun Mar 22, 2015 7:09 pm

Re: EA XMA multilayer audio files

Post by id-daemon »

And to be more precise:

0x04 - codec byte
0x05 - channel number (0 = 1ch, 4 = 2ch, C = 4ch, 10 = 5ch, 14 = 6ch)
0x08 - flags
0x09 (3bytes) number of samples
AlphaTwentyThree
Posts: 909
Joined: Sat Aug 09, 2014 11:21 am

Re: EA XMA multilayer audio files

Post by AlphaTwentyThree »

id-daemon2 wrote:Can you upload the files again? zippyshare or mega

Considering parsing stops at 64k it is most possible that files were extracted wrong from CAS files. What did you use to extract them?

I've re-uploaded the sample above.
The files are from the Xbox 360 version inside the sb/toc pairs. They were extracted with Luigi's frostbyte script. Here's an example: http://*USE_ANOTHER_FILEHOSTING*/68bbd96 ... ksaudio.7z
id-daemon
Posts: 1040
Joined: Sun Mar 22, 2015 7:09 pm

Re: EA XMA multilayer audio files

Post by id-daemon »

Ok all works perfectly fine. The problem is that Frostbyte engine packs all its chunks and Luigi's script just cuts them off, without unpacking and removing segment headers.

So, first you unpack it all with Luigi's script.

Then use my tool to rebuild the data from chunks (or delete 8 bytes at each 0x10000 manually)

Then use ea_multi_xma.

Note that some files have additional info at the beginning (starting like 01100180), for these you have to manually cut it until 4800000C before using ea_multi_xma
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: EA XMA multilayer audio files

Post by aluigi »

Do you think it's a bug of the script or they are just saved in this chunked way on purpose?
As far as I know all the other files are extracted correctly.
id-daemon
Posts: 1040
Joined: Sun Mar 22, 2015 7:09 pm

Re: EA XMA multilayer audio files

Post by id-daemon »

I know that all frostbyte chunks are "chunked" on purpose. Some of them may also be compressed. I don't know how its possible that other files extracted correctly, they all must have these chunk headers every 0x10000 bytes.
id-daemon
Posts: 1040
Joined: Sun Mar 22, 2015 7:09 pm

Re: EA XMA multilayer audio files

Post by id-daemon »

By the way, why do you call these "multi-layer" ? What do you mean by layers?
AlphaTwentyThree
Posts: 909
Joined: Sat Aug 09, 2014 11:21 am

Re: EA XMA multilayer audio files

Post by AlphaTwentyThree »

In this specific case they are single layer. There are EA XMA streams that contain two streams (interleaved). In most cases these are "layers" of the same track, e.g. one orchestral track and one percussion track that only kicks in at a certain trigger inside the game. So I guess the engine buffers both layers and when the trigger is activated, fades in the second layer to mix into the first one. I'm a bit surprised you haven't heard of that mechanic.
id-daemon
Posts: 1040
Joined: Sun Mar 22, 2015 7:09 pm

Re: EA XMA multilayer audio files

Post by id-daemon »

Yes I've heard about that type of layers, but I was surprised by thread name "EA XMA multilayer audio files" when all the files in this case seem to be single layer.
AlphaTwentyThree
Posts: 909
Joined: Sat Aug 09, 2014 11:21 am

Re: EA XMA multilayer audio files

Post by AlphaTwentyThree »

If you look at the script you can see that I've already implemented multilayered files. :)
I'll get around to check this in the next few days I think.
id-daemon
Posts: 1040
Joined: Sun Mar 22, 2015 7:09 pm

Re: EA XMA multilayer audio files

Post by id-daemon »

Ok if you will need any more help with these blocks, chunks, or other EA structures, feel free to ask me.
AlphaTwentyThree
Posts: 909
Joined: Sat Aug 09, 2014 11:21 am

Re: EA XMA multilayer audio files

Post by AlphaTwentyThree »

Ok, I have another question here. I'm currently implementing the above code into my big xma transform script. I'm taking a look at the files from Need for Speed: Hot Pursuit (2010, Xbox 360) and I have to admit I have no idea how to distinguish between single and multilayer files. I thought that the block header determines the layer count but obviously I'm wrong.
Here are three multilayer/multichannel files to look at: http://www33.zippyshare.com/v/t8YtSsoM/file.html
Where can I see how many layers/channel pairs there are to process?
id-daemon
Posts: 1040
Joined: Sun Mar 22, 2015 7:09 pm

Re: EA XMA multilayer audio files

Post by id-daemon »

Ok, I'm rarely working with XMA, so I completely forgot everything about those files you uploaded before.

Anyway, these you uploaded now are 6 channel files, that's stated in the header. So because XMA files can only be mono or stereo, these must contain 3 stereo streams.
id-daemon
Posts: 1040
Joined: Sun Mar 22, 2015 7:09 pm

Re: EA XMA multilayer audio files

Post by id-daemon »

I took a closer look at the files. Now I know all the fields meaning and I'm sure there's no layer number there.

Also, I mostly work with PC, and layered files are never interlaced there, but included as separate files. It seems on consoles its otherwise. Based on 3 console games that I researched, I can say that layer number is usually not in the audio file, but in a separate srtucture, or even hardcoded in the .EXE.

Just out of interest, I'm now downloading PC version of Need for Speed: Hot Pursuit 2010 to look at the files.
AlphaTwentyThree
Posts: 909
Joined: Sat Aug 09, 2014 11:21 am

Re: EA XMA multilayer audio files

Post by AlphaTwentyThree »

id-daemon2 wrote:Anyway, these you uploaded now are 6 channel files, that's stated in the header.
Where exactly? I don't see any field that depicts that...
id-daemon
Posts: 1040
Joined: Sun Mar 22, 2015 7:09 pm

Re: EA XMA multilayer audio files

Post by id-daemon »

id-daemon2 wrote:And to be more precise:

0x04 - codec byte
0x05 - channel number (0 = 1ch, 4 = 2ch, C = 4ch, 10 = 5ch, 14 = 6ch)
0x08 - flags
0x09 (3bytes) number of samples