(Non-game) Roland FA Series Keyboard Expansion and Firmware Format

Extraction and unpacking of game archives and compression, encryption, obfuscation, decoding of unknown files
jimmy
Posts: 5
Joined: Wed Nov 01, 2017 8:36 pm

(Non-game) Roland FA Series Keyboard Expansion and Firmware Format

Post by jimmy »

Hello all,

I recently bought a Roland FA-08 music keyboard, it is interesting because it has two of what Roland call "Virtual SRX" slots. These work by downloading a .zip file from Roland's website (http://axial.roland.com/category/fa-06_fa-07_fa-08/ - EXP files, not BANK), placing the .bin file inside onto a usb thumb drive, and then you flash it to the keyboard from the drive, which provides you with a selection of new sounds that are on that expansion module. The keyboard is still on the market in different sizes as the FA-08/FA-07/FA-06.

My goal is to write a program or tool that will allow people to mix and match different sounds from different expansions and combine them into customized expansions, making more efficient use of the space, perhaps even paving the way for user-created completely custom instruments eventually. You can download a sample file at:
http://axial.roland.com/downloads/FA_EXP-07.zip

Expansion format

Inside the file is the header, which is always $20 bytes long and padded with spaces if it isn't.

Code: Select all

Roland SRX  O°X§SRX-10 BgBrsEns 


Next at $40 offset is a $40/64 byte "key" or "description" of some sort

Code: Select all

1C D9 D9 1C D9 99 9C 5C 99 5C 5C D9 5C D9 D9 9C 99 9C 5C D9 5C D9 D9 9C D9 9C 9C 5C 9C 5C 5C D9 9D 5D 9D 9C 5D 9C 9C DD 9C 9D DD 9D 9D 9D 9D 1C 9C 9D DD 9D 5D 9D 9D 9C 9D 9C 9C DD 9C 5D D9 9D


Then finally at the $C0 offset we have the data which appears to be encoded and possibly encrypted.

There is another gap of 00s and description block, then another data block. This pattern repeats. The description blocks are strange because they all look very similar, which doesn't make sense for all the instrument names.

In this expansion there are new sounds for the keyboard called "TromboneSect" and "DynaBrass" (SoundList_EXP-07_je01_W.pdf inside the zip file has the patch list) and these words are shown on the keyboard screen tone list after install, but they don't show up anywhere in the raw binary file. The file contains 100 tones (patches) and 261 waveforms so these are in there somehow.

The data isn't likely compressed because it is 32MB in size exactly, and as you can see in the header they are all based on Roland's original hardware SRX modules released, but reduced a little bit, presumably to fit into the 32MB RAM in each virtual slot in the device (original SRX modules were 64MB)

I couldn't figure out much more than this, signsrch turned up nothing probably due to the encryption. It's possible the device decrypts the bin at flash-time, leaving an unencrypted ROM on the device to use, OR it is using a very fast simple encoding format quick enough for the device to realtime decode the data with little overhead so it can be played on the device.

Firmware format
To figure out how to decrypt this, I wondered if perhaps the Firmware for the device itself contained the public key. Since roland recently released a firmware update we can also get the firmware for the device on the website here:
https://static.roland.com/assets/media/ ... s_v201.zip

This started off quite easy, inside the FA_UPA.bin firmware file was the ".ustar" keyword which made me realise this was a .tar file. Renaming it to FA_UPA.tar allowed me to extract the following files:
fantom_s_up.bin
fantom_e_up.bin
fantom_c_up.bin

fantom_c_up.bin appears to be a disk image or archive, but definitely compressed. You can see some fragments of filenames inside:

Code: Select all

3228C1: %1/sd/Rýo.Pd/FA/SÿCREEN SAÿVER/%04do.BIN
199308: /TEST_¿SD.BIN


fantom_s_up.bin is harder to decode. It has a linux style reference to BOOT_IMAGE at the end and perhaps a command to cd to the etc/ folder

Code: Select all

545D94: BOOT_IMAGE
02DA70: cdetc<eW\3Xp

This could indicate it is running an embedded *nix system of some kind and that the s and c bin files are in fact disk images.

For fantom_s_up.bin signsrch found the following:

Code: Select all

  offset   num  description [bits.endian.size]
  --------------------------------------------
  001982cb 877  SHA256 Initial hash value H (0x6a09e667UL) [32.be.32&]
  001982cb 1031 SHA256 [32.be.288&]
  0019844a 1029 SHA224 [32.be.288&]

Could just be a coincidence, unless there are SHA256 codes to check the integrity in the firmware, which is possible.

The fantom_e_up.bin looked a lot different to the other files, so I wondered if this might be a low-level firmware for some hardware in the keyboard.

Thats as far as I got. I'm hoping maybe someone will help me figure this out, either with advice, tools, guidance or help so I can decompress/decrypt the expansion files and/or firmware files in an effort to help me make this program. Thanks for reading all this!
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: (Non-game) Roland FA Series Keyboard Expansion and Firmware Format

Post by aluigi »

This script for quickbms can decompress the fantom*bin files:

Code: Select all

endian big
comtype lzss0
get ZSIZE long
get SIZE long
savepos OFFSET
get NAME filename
string NAME + ".unpacked"
clog NAME OFFSET ZSIZE SIZE
jimmy
Posts: 5
Joined: Wed Nov 01, 2017 8:36 pm

Re: (Non-game) Roland FA Series Keyboard Expansion and Firmware Format

Post by jimmy »

aluigi wrote:This script for quickbms can decompress the fantom*bin files:

Code: Select all

endian big
comtype lzss0
get ZSIZE long
get SIZE long
savepos OFFSET
get NAME filename
string NAME + ".unpacked"
clog NAME OFFSET ZSIZE SIZE


Wow, nice work identifying the compression, thanks! Maybe the decompressed version will help unlock the secrets of the EXP files. Problem is the QuickBMS download link isn't working for me on http://aluigi.altervista.org/quickbms.htm - do you have a fixed download link where I can find the latest version?

EDIT: Nevermind, was a network issue on my end! Got QuickBMS now, taking a look.

Interesting, in the fantom_c_up unpacked file, all the expansions are listed out. I suspect this is for displaying on the menus in the keyboard to show which pack is installed. I'm curious how it matches this to the file installed though, as for example EXP-01 doesn't have SRX-08 in the header, makes me wonder if tdrf1/srx/SRX08_S is encoded in the EXP01.bin file somehow. EDIT: I think the tdrf1/srx/SRX08_S is the path to the EXP file installed on the file system. It actually opens the interesting possibility of putting in a bigger ROM/RAM chip and putting ALL the EXP files on there! Or modifying the firmware to read them off of a USB drive.

Code: Select all

EXP-01..Dance Trax Collection...tdrf1/srx/SRX08_S...
EXP-02..Keys Collection.tdrf1/srx/SRX07_S...
EXP-03..World Collection....tdrf1/srx/SRX09_S...
EXP-04..Concert Piano Collection....tdrf1/srx/SRX02_S...
EXP-05..Electric Piano Collection...tdrf1/srx/SRX12_S...
EXP-06..Studio Collection...tdrf1/srx/SRX03_S...
EXP-07..Brass Collection....tdrf1/srx/SRX10_S...
EXP-08..Strings Collection..tdrf1/srx/SRX04_S...
EXP-09..Complete Piano Collection...tdrf1/srx/SRX11_S...
EXP-10..Orchestra Collection....tdrf1/srx/SRX06_S...
EXP-11..Vintage Synth Collection....tdrf1/srx/SRX13_S


EDIT2: Right after this block is a list of all the waveforms inside each EXP pack. I think this area is like a dictionary for the EXP pack names and Waveform names:

Code: Select all

Phrase MENU1....Phrase MENU2....Phrase MENU3....61:Slow Grv ....75:BoomRvBel....76:TrpHpKikn....78:NinjaBrek....80:Thump Grv

These are the Waveforms in the EXP-01 expansion. I think they may just be for reference only when browsing and showing the waveforms on the keyboard, but the wave data itself is probably replacable within the EXP.

So I can probably make some sort of tool to juggle the waveforms around in the EXP, but the names inside the keyboard for those waves will be incorrect as it will always use the firmware dictionary. I guess this isn't a big deal. I suppose the issue is if the actual Tone Names are also fixed in the firmware. I can't find any of the Built in or EXP Tone names in any of the files, like "D-52 Babe" which is built in, or "PopBrass" or "DynaBrass" from EXP-07. Perhaps this is encoded in the file format somehow.

EDIT3: Found a reference to newfs, a FreeBSD-only disk formatting tool. Looks like the FA runs on FreeBSD. This leads me to believe the firmware image is probably a UFS format image. Haven't got it to open in anything yet though.

I'd appreciate your thoughts on the EXP files if you have any idea where to start there, you are much better at this than me!
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: (Non-game) Roland FA Series Keyboard Expansion and Firmware Format

Post by aluigi »

It's not possible to give you an useful opinion without reverse engineering the obfuscation method used in the fantom files, which I'm sorry but it's something on which I can't spend time.

If we consider the usage of an obfuscation (since there are no visible strings), based on my experience when you see sequences of same/similar bytes and many zeroes, it's usually an obfuscation algorithm that uses the previous/next byte as "key", something like: data[i] ^= data[i - 1]
But this is not the case since it's the first test I did with quickbms (encryption xor_prev "").
If you look at the beginning of the exp file more carefully you can notice a certain "pattern" like BAAB (0x1c 0xd9 0xd9 0x1c) but doesn't mean anything.
Just my (useless) quick thoughts, sorry.
jimmy
Posts: 5
Joined: Wed Nov 01, 2017 8:36 pm

Re: (Non-game) Roland FA Series Keyboard Expansion and Firmware Format

Post by jimmy »

aluigi wrote:It's not possible to give you an useful opinion without reverse engineering the obfuscation method used in the fantom files, which I'm sorry but it's something on which I can't spend time.

If we consider the usage of an obfuscation (since there are no visible strings), based on my experience when you see sequences of same/similar bytes and many zeroes, it's usually an obfuscation algorithm that uses the previous/next byte as "key", something like: data[i] ^= data[i - 1]
But this is not the case since it's the first test I did with quickbms (encryption xor_prev "").
If you look at the beginning of the exp file more carefully you can notice a certain "pattern" like BAAB (0x1c 0xd9 0xd9 0x1c) but doesn't mean anything.
Just my (useless) quick thoughts, sorry.


No need to apologize at all, I understand I'm basically asking for you to volunteer your time for my own project, I really appreciate your help with the decompression and your thoughts here. Just getting the firmware decompressed was a massive step, so thank you again for that!

I did notice a lot of repeating patterns in the EXP, and I did wonder if maybe it was a XOR or something like that - it would make sense since XOR can be decoded rapidly, so it's possible it literally just copies the EXP file into internal storage without any decoding, and all decoding is done at runtime. I can try and do some experimenting based on your suggestions 8-)

EDIT: I am documenting everything at http://rolandfatools.mooo.com/
jimmy
Posts: 5
Joined: Wed Nov 01, 2017 8:36 pm

Re: (Non-game) Roland FA Series Keyboard Expansion and Firmware Format

Post by jimmy »

Hi again,

I am trying to make a QuickBMS script that recompresses the firmware files. This is what I have so far:

Code: Select all

endian big
comtype LZSS0_COMPRESS
get ZSIZE ASIZE
get NAME filename
string NAME + ".packed"
clog NAME 0 ZSIZE ZSIZE


And it works, however I cannot extract it again:

Code: Select all

  offset   filesize   filename
--------------------------------------
  00000008 1108984    fantom_c_up.bin.unpacked.packed.unpacked

Error: incomplete input file 0: fantom_c_up.bin.unpacked.packed
       Can't read 514514827 bytes from offset 00612085.
       Anyway don't worry, it's possible that the BMS script has been written
       to exit in this way if it's reached the end of the archive so check it
       or contact its author or verify that all the files have been extracted.
       Please check the following coverage information to know if it's ok.

  coverage file 0   100%   6365317    6365317    . offset 00612085

Last script line before the error or that produced the error:
  8   clog NAME OFFSET ZSIZE SIZE

Press ENTER or close the window to quit

Any idea what I'm missing? Thanks!
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: (Non-game) Roland FA Series Keyboard Expansion and Firmware Format

Post by aluigi »

Code: Select all

endian big
comtype lzss0_compress
log MEMORY_FILE 0 0
put 0 long MEMORY_FILE
put 0 long MEMORY_FILE
append
get SIZE asize
clog MEMORY_FILE OFFSET SIZE SIZE
append
get ZSIZE asize MEMORY_FILE
math ZSIZE - 8
goto 0 MEMORY_FILE
put ZSIZE long MEMORY_FILE
put SIZE long MEMORY_FILE
get SIZE asize MEMORY_FILE
get NAME filename
string NAME + ".packed"
log NAME 0 SIZE MEMORY_FILE
jimmy
Posts: 5
Joined: Wed Nov 01, 2017 8:36 pm

Re: (Non-game) Roland FA Series Keyboard Expansion and Firmware Format

Post by jimmy »

Amazing - thank you!
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: (Non-game) Roland FA Series Keyboard Expansion and Firmware Format

Post by aluigi »

And this is a script for BQ_UPD.BIN of boutique_sys_121:

Code: Select all

comtype lzss0
for
    findloc OFFSET binary "PX-"
    goto OFFSET
    getdstring SIGN 0x2c
    get HEAD_SIZE long
    get ZSIZE long
    get DUMMY long
    get DUMMY long
    get SIZE long
    savepos OFFSET
    clog "" OFFSET ZSIZE SIZE
    math OFFSET + ZSIZE
    goto OFFSET
next