Beatmania - .HD

[AnonBaiter]

It's an arcade game that uses this HDD format for storing game data. Looks like an ISO file, and was featured as a game in the Konami DJ-Main arcade system.

The samples are in the attachment.

[aluigi]

This data is weird:
HDheader and *.hd have the same initial content (zero differences).
hd has no data for the first 2 megabytes and the last 2 have just 'O's
There are no offset/size fields in the first 0x200 bytes.

[AnonBaiter]

Huh.
If you wish I can send you more samples or maybe the whole file.

[aluigi]

Let's try with the first 10 megabytes of the file, you can open it with the hex editor to check if there is at least a byte different than 0x00 from offset 0x200 till its end (just to be sure).
Probably there is nothing to do but trying is free

[AnonBaiter]

Okay then.
These "O"s are probably something that could serve as the gap between the files.
The sample has the first... 112MB as you can see but maybe I'll save this offset when I'm done so that I can head up to the next sample if I think one file isn't enough:

Code: Select all

0 - 6FFFFFF
7000000 - 12000000
12000001 - 22000000
22000001 - 2FFFFFFF
30000000 - 3FFFFFFF
40000000 - 4FFFFFFF
50000000 - 46027FFF

[aluigi]

In this case I guess that the following is the best option:
http://aluigi.org/bms/beatmania_hd.bms

I use the 'O' padding to guess where the single files and headers (useless headers) start.

[AnonBaiter]

Well the dumping all went fine until this happened:

Code: Select all

  2d002000 15800320   0000002d.dat

Error: incomplete input file 0: E:\MAME ROMs\bm1stmix\753jaa11.hd
       Can't read 16 bytes from offset 56028000.
       Anyway don't worry, it's possible that the BMS script has been written
       to exit in this way if it's reached the end of the archive so check it
       or contact its author or verify that all the files have been extracted.
       Please check the following coverage information to know if it's ok.

  coverage file 0   121%   1754361424 1443004416

Last script line before the error or that produced the error:
  16  getdstring TMP 16

Is it okay for this to happen?

EDIT: Here are the rest of the .hd samples as promised(RAR5 compressed, 128MB volume, uncompressed size is more than 1GB):
https://mega.nz/#!hUUC2YyL!kfYzpRx6n0j_5xO5wFUlOp4EooKHzs0GsRJqkgOUnQg

[aluigi]

Yeah the error is ok because there is no end in the archive.
Instead 121% is very strange.

[AnonBaiter]

Actually the "archive" you mentioned is, to say the truth, a hard disk file.

Look at the start of the file:

Code: Select all

00000000  47 20 37 51 33 35 20 20 42 20 41 45 20 54 41 4D  G 7Q35  B AE TAM
00000016  49 4E 20 41 20 20 56 20 52 45 31 20 30 2E 20 30  IN A  V RE1 0. 0
00000032  C3 00 1F 0E 06 8E 8D 59 09 BA 26 82 3E 80 03 1E  Ã....Ž.Y.º&‚>€..
00000048  74 00 E8 06 09 4B 56 BA E9 83 02 CB B4 50 38 2F  t.è..KVºéƒ.Ë´P8/
00000064  5D 26 74 8D 3C 04 74 2F 3C 02 58 5C C7 C3 C6 06  ]&t.<.t/<.X\ÇÃÆ.
00000080  00 90 8A 00 5C 16 E8 00 FF 1D 33 72 64 E8 F6 04  ..Š.\.è.ÿ.3rdèö.
00000096  5D 06 02 8F 2C 75 3B B4 21 CD 13 73 9D E8 3D F6  ]...,u;´!Í.s.è=ö
00000112  00 03 1E 74 05 3D 74 00 A3 19 90 C6 8B E9 E8 00  ...t.=t.£..Æ‹éè.
00000128  06 4B 3F B0 5D BF B9 00 00 0B AA F3 C0 32 E9 C3  .K?°]¿¹...ªóÀ2éÃ
00000144  00 83 36 8B 8E B6 8A 4E FF 44 3E 80 90 E1 75 00  .ƒ6‹Ž¶ŠNÿD>€.áu.
00000160  E8 05 FF 98 64 74 CE 8B F2 8B 3B 52 74 F1 AC 13  è.ÿ˜dt΋ò‹;Rtñ¬.
00000176  CD E8 74 FC 46 03 F3 EB 81 E8 75 FF 8B EE 4A D6  ÍètüF.óë.èuÿ‹îJÖ
00000192  E9 EB F2 8B 3B 5A 74 F2 8B 56 8B CE 3B F2 74 F1  éëò‹;Ztò‹V‹Î;òtñ
00000208  C6 12 E1 06 00 90 E8 AC FC A6 F1 74 FE 46 E1 06  Æ.á...è¬ü¦ñtþFá.
00000224  EB 90 50 EA 2E B0 44 38 75 01 38 09 02 44 04 74  ë.Pê.°D8u.8..D.t
00000240  7C 80 00 02 74 58 8A 13 FF 44 3A 3C 21 74 3E 80  |€..tXŠ.ÿD:<!t>€
00000256  90 E1 75 00 E8 07 FF 34 02 75 C3 F9 04 C6 B4 00  .áu.è.ÿ4.uÃù.Æ´.
00000272  CD 3B 73 21 E8 20 F5 F4 C6 A3 0B 90 F9 F6 8A C3  Í;s!è õôÆ£..ùöŠÃ
00000288  E8 04 FF 18 75 F9 32 F7 86 DB 01 5C 3B B4 21 CD  è.ÿ.uù2÷†Û.\;´!Í
00000304  E3 72 5C 88 E8 01 05 95 89 46 5B 36 9C 8F 3E 80  ãr\ˆè..•‰F[6œ.>€
00000320  8E BE 74 FF BF 08 00 5C 02 B8 CD 29 9D 21 93 C3  Ž¾tÿ¿..\.¸Í).!“Ã
00000336  33 C3 E8 DB FE 54 06 3A 8D 5D F3 75 CB 81 80 00  3ÃèÛþT.:.]óuË.€.
00000352  E8 46 FE 46 0D 3C 00 1B 00 05 00 00 00 00 08 D2  èFþF.<.........Ò
00000368  08 D2 11 1E 16 87 16 87 16 81 13 B2 72 46 08 D2  .Ò...‡.‡...²rF.Ò
00000384  08 D2 00 36 08 D2 08 D2 11 A2 07 2B 11 32 08 D2  .Ò.6.Ò.Ò.¢.+.2.Ò
00000400  08 D2 00 02 11 34 34 36 89 00 B6 0E FE 8E C8 06  .Ò...446‰.¶.þŽÈ.
00000416  3A 42 47 5C 37 51 33 35 4F 54 5C 4F 41 50 53 53  :BG\7Q35OT\OAPSS
00000432  62 2E 6E 69 0A 00 75 00 E9 03 00 A0 A3 BF 33 85  b.ni..u.é.. £¿3…
00000448  00 40 00 00 00 14 00 30 3A BE E1 0E 75 8D 89 06  .@.....0:¾á.u.‰.
00000464  2C 30 34 36 42 2C 5C 3A 51 47 35 37 54 33 4F 4F  ,046B,\:QG57T3OO
00000480  50 5C 53 41 2E 53 69 62 0D 6E 1A 0A 00 20 11 DC  P\SA.Sib.n... .Ü
00000496  08 D2 08 D2 11 A0 16 87 13 8A 17 FE 13 B2 72 46  .Ò.Ò. .‡.Š.þ.²rF

The first 32 decimal bytes contain the production code, the title of the game, and the version the game was built on.
If you decipher all that, you get this:

Code: Select all

GQ753 BEATMANIA VER1.0( 0)

Everything beyond that contains all the data stored in a hard disk file. As such, there are filenames scattered around. Although the way they were stored doesn't make much sense.

Oh yeah and about the music itself, how can I decode those .dc5/.dat/.mp3/.fnc/.app files? From what I saw, they seem to be encoded using some kind of ADPCM codec. Here is a sample if you're interested.

[AnonBaiter]

Two posts in a day, huh? Well I'm doing this for a reason:
Here's a sample from another Konami DJ-Main arcade game. Literally, it uses the same HDD format(.hd) as beatmania, except the storing methods in this one are a bit different...

EDIT: And here are the divided bytes of the same .hd file(RAR5 method, 128MB compression volume):
https://mega.nz/#!FN1RSY4B!9ZdayQZEnmrOJSqICI7ou7pzqsWTc4BbObVXFnfbR28

[aluigi]

I have updated the script for performing the 2 bytes swapping.
That's the max I can do.

[AnonBaiter]

Okay, that`s about the update I needed. I'll check the other games to see if this script works with them, and I'll inform you about the results.

EDIT: So with popn1, it appears to freeze after the second file and then there's this "memory allocation problem" error after the third file.

Code: Select all

  offset           filesize   filename
--------------------------------------
  0000000000000000 216892928  0000000000000000.dat
  000000000ced9000 16774656   0000000000000001.dat
  000000000ded9000 2328743936 0000000000000002.dat

- error in src\xalloc.c line 617: xdbg_malloc()

Error: memory allocation problem
       Espa�o insuficiente de armazenamento para processar este comando.


press ENTER to quit

Maybe it's just my PC that doesn't handle the game, but then again I don't know if this is the only one in which it has this unstable state. Oh well...

I'm planning to post a beatmania_hd.bms compatibility list to see if the Konami DJ-Main games work with this script, but I probably don't want to flood the thread too much so let's leave it at that for now.

[aluigi]

The problem is simply related to the lack of 'O' delimiters in that file.
So it tries to dump everything from 0xded9000 till the end, probably the error is something related to a negative size (END_OFFSET - OFFSET) but you can ignore it.

[AnonBaiter]

Well, that's good to know I guess...

Anyways, I just put up an compatibility script regarding your script(it's in the attachment).

Now back to the script:

Code: Select all

  offset           filesize   filename
--------------------------------------
  0000000000000000 4327407616 0000000000000000.dat

Error: incomplete input file 0: E:\reverse.engineering.stuff\KONAMI_DJMAIN\995jaa11[bmdct].hd
       Can't read 4294967296 bytes from offset 0000000001ef0000.
       Anyway don't worry, it's possible that the BMS script has been written
       to exit in this way if it's reached the end of the archive so check it
       or contact its author or verify that all the files have been extracted.
       Please check the following coverage information to know if it's ok.

  coverage file 0   100%   4367261696 4327464960

Last script line before the error or that produced the error:
  15  log "" OFFSET SIZE

That is an error, right? I can send you a sample if you wish.

[aluigi]

Don't worry, you can ignore that error.

[AnonBaiter]

Okay so how can I ignore that error? There is a parameter in quickbms that ignores the error, so I might as well try it out...

[aluigi]

With ignore I mean that you should not worry about the termination of quickbms, what has been extracted is already good and there is nothing else to do.

[AnonBaiter]

Ah, I see...
In any case, I think I should rather take a break than kill my external HDD completely over testing these files... Whatever.