strange unknown audio format

[AlphaTwentyThree]

I've just encountered an audio format that I've never seen before. It's used as audio in the Mega Race 2 videos. Demultiplexing is definitely correct (see my script). Here are samples: http://*USE_ANOTHER_FILEHOSTING*/f7ed4b2 ... m_audio.7z
Can anybody identify the format? I'm quite startled right now...

[id-daemon]

What a surprise man! It was a cool riddle, and I solved it

If I open that .SD file as unsigned 8-bit sound it looks like this:



Interesting that all numbers are positive, very strange, huh? After spending the whole evening looking at these files I came to conclusion that there must be something else to it. Having only this data, it just can't be decoded. So I downloaded the original video files.
Now if I open .DD file for that video as signed 16-bit sound it looks like this:



Looks like a table or something? 256 samples?
Now, if we use bytes from .SD file as index of this table we've got this!



A sound! Indeed And it is playing perfectly as 16 bit mono 22050 Hz.

[AlphaTwentyThree]

Wow, that is quite an accomplishment!
Can you write a script that does the conversion? Or a script to demux the sound correctly?

[id-daemon]

I'm not writing scripts, but I can write a program for this. But that's not all yet. The sound has no low frequencies, this means something else is wrong.

[id-daemon]

Ah, yes! I need not to just take number from the table, but to add these numbers to the previous sample. Now everything is ok, we got that smooth bass.

[id-daemon]

This little tool will take 2 arguments: DS file, DD file. It will drop a raw 16 bit file. Works very slow, because it reads byte by byte. If you really need it, I can improve it later and make it write WAV headers.

[AlphaTwentyThree]

Thanks for the program!
If you tell me what the program does I can write a script and implement it in my demuxer.

[AlphaTwentyThree]

Wait, there's something else wrong I think. If you open the stream in a hex editor and look at the character distribution the curve should be smooth but the one from your file is jagged.

[id-daemon]

I already told. It loads .DD file as a table of 256 16-bit values. Then it reads bytes from .DS file and adds the value from the table [indexed by that byte] to the previous sample value.

[id-daemon]

Wait, there's something else wrong I think. If you open the stream in a hex editor and look at the character distribution the curve should be smooth but the one from your file is jagged.

I don't know what you mean, but the resulting sound is very good now. Just listen to it.

[aluigi]

Regarding the script, do you mean something like this?

Code: Select all

open FDDE "dd"
for i = 0 < 256
    get TMP short
    putarray 0 i TMP
next i

math LAST = 0
open FDDE "sd"
get SIZE asize

xmath TMP "SIZE * 2"
putvarchr MEMORY_FILE TMP 0
log MEMORY_FILE 0 0

for i = 0 < SIZE
    get TMP byte
    getarray TMP 0 TMP
    math LAST + TMP
    put LAST short MEMORY_FILE
next i

get SIZE asize MEMORY_FILE
get NAME basename
log NAME 0 SIZE MEMORY_FILE

I don't have the DD files and can't verify the whole thing (for example if it's ever a sum or there is a substraction if the last sample is negative). Hope it helps in making the tests on the fly.

Anyway, just a note, some people here are programmers so we appreciate and understand the source code of the tools instead of just the exe.

[id-daemon]

Anyway, just a note, some people here are programmers so we appreciate and understand the source code of the tools instead of just the exe.

Yes I know, but the code here was so simple it could be described with one phrase.

Code: Select all

        static void Main(string[] args)
        {
            short[] tabl = new short[256];
            int i;

            FileStream fs = new FileStream(args[1], FileMode.Open);
            BinaryReader br = new BinaryReader(fs);
            for (i = 0; i < 256; i++) tabl[i] = br.ReadInt16();
            br.Close();
            fs.Close();

            fs = new FileStream(args[0], FileMode.Open);
            FileStream fw = new FileStream(Path.GetFileNameWithoutExtension(args[0])+".bin", FileMode.Create);
            BinaryWriter bw = new BinaryWriter (fw);
            int ch;
            short curr=0;
            for (i=0;i<fs.Length;i++)
            {
                ch = fs.ReadByte();
                curr += tabl[ch];
                bw.Write(curr);
            }
            fs.Close();
            fw.Close();
        }

[aluigi]

Ok so the script is correct, thanks.

[AlphaTwentyThree]

First, there's one little correction to make: The stream data seems to start at offset 6 rather than 0. I don't know what the first fields are for. Sadly no stream specs. :\
Second, there's still something wrong with the attribution because of the "jaggedness" of the character distribution. Here's a picture of what I mean:

The percent values should strictly decrease till 128 and then strictly increase till 256. As you can see, neither is the case. That's why I don't think that adding the previous value alone does the whole job.
Here are some ubb samples with the demuxer for Luigi to check: http://*USE_ANOTHER_FILEHOSTING*/b96fcf1 ... samples.7z

[id-daemon]

First, there's one little correction to make: The stream data seems to start at offset 6 rather than 0. I don't know what the first fields are for.

These may be a couple of first samples, to start with. Now we just assume that we start from "0" value. And there may be some scale, to multiply the whole wave by that.

Second, there's still something wrong with the attribution because of the "jaggedness" of the character distribution.

Maybe it's caused by the compression?

[AlphaTwentyThree]

Second, there's still something wrong with the attribution because of the "jaggedness" of the character distribution.

Maybe it's caused by the compression?

Hm, of course that could be... I was just suspicious because it's so regular.
Now my final question: is this a known wave compression type that can be attributed to a whole format that already includes the table in the header? I'd like to keep the original data for archiving purposes. Of course a mock-up format would also be sufficient as long as sombeody could add vgmstream support. Maybe even another GENH type?

[id-daemon]

I think it's unknown type, their own invention.

[AlphaTwentyThree]

To close this off, here's my script for the sd/dd pairs including a header adder. I've skipped the first 6 bytes because they only produce a popping sound.

Code: Select all

open FDDE sd 0
open FDDE dd 1
for i = 0 < 256
   get VALUE short 1
   putArray i 0 VALUE
next i
get SIZE asize 0
math SIZE *= 2
putVarChr MEMORY_FILE SIZE 0
log MEMORY_FILE 0 0
get LOOPS asize 0
math LOOPS -= 6
set PREV 0
goto 6 0
for i = 0 < LOOPS
   get VAL byte 0
   getArray CORR VAL 0
   math CORR += PREV
   xmath OFFSET "i * 2"
   putVarChr MEMORY_FILE OFFSET CORR short
   set PREV CORR
next i
get SIZE asize MEMORY_FILE
set FREQ 22050
set CH 2
set CODEC 1
set BLOCKALIGN 4
set BITS 16
set PRE SIZE
math PRE += 0x2c
putVarChr MEMORY_FILE2 PRE 0
log MEMORY_FILE2 0 0
set MEMORY_FILE2 binary "\x52\x49\x46\x46\x20\xC0\xB1\x00\x57\x41\x56\x45\x66\x6D\x74\x20\x10\x00\x00\x00\x01\x00\x02\x00\x44\xAC\x00\x00\x10\xB1\x02\x00\x04\x00\x10\x00\x64\x61\x74\x61\xFC\xBF\xB1\x00"
append
get SIZE asize MEMORY_FILE
log MEMORY_FILE2 0 SIZE MEMORY_FILE
append
set RIFFSIZE SIZE
math RIFFSIZE += 36
set AVGBYTES FREQ
if CODEC != 2
   math AVGBYTES *= BLOCKALIGN
endif

putvarchr MEMORY_FILE2 0x04 RIFFSIZE long
putvarchr MEMORY_FILE2 0x14 CODEC short          # wFormatTag: Microsoft PCM Format (0x0001)
putvarchr MEMORY_FILE2 0x16 CH short   # wChannels
putvarchr MEMORY_FILE2 0x18 FREQ short   # dwSamplesPerSec
putvarchr MEMORY_FILE2 0x1c AVGBYTES long    # dwAvgBytesPerSec
putvarchr MEMORY_FILE2 0x20 BLOCKALIGN short # wBlockAlign
putvarchr MEMORY_FILE2 0x22 BITS short       # wBitsPerSample
putvarchr MEMORY_FILE2 0x28 SIZE long
get NAME basename
string NAME += ".wav"
get SIZE asize MEMORY_FILE2
log NAME 0 SIZE MEMORY_FILE2