428 Shibuya Scramble PC script (.sns files)

[Sakimotor]

Thanks to the following thread : viewtopic.php?t=12952

I was able to extract the wad files and see their content. I am looking to eventually translate the game in french for the context !

Most of the UI elements can be simply either edited inside a .json file or are just image files that need to be edited, but the game's script itself seems to be stocked inside three files that work together : to.sns , to.sns64 and to.flo

The PC folder containing them is named "script", and is available on mega : https://mega.nz/#!I4UimQiY!jZFwnrpUi4zd ... GbfdF5i7qM

The PS3 version of the files are added as an attached file, if that may help (I tried replacing PS3 files for the pc version and vice versa, they don't work the same way)


Thanks for the help in advance, I would really love to get started on that little project of mine !

[aluigi]

It looks like the SNS format is composed by different sections.
This script extracts the files from the main section but don't expect anything ready to be open:

Code: Select all

set KEY binary "CSD-CSCNV:01.82"
strlen KEYSZ KEY
math KEYSZ + 1
set KEY2 string ""
for x = 0 < KEYSZ
    getvarchr TMP KEY x
    math TMP n TMP
    string KEY2 + TMP
    string KEY2 + " "
next x
filerot KEY2

math NAMESZ = 0x30

getdstring ZERO 16
get DUMMY long
get ZERO long
getdstring DUMMY 0x10
get OFFSET1 long
get OFFSET2 long
getdstring ZERO 0x30
get OFFSET3 long
getdstring ZERO 12
get OFFSET4 long
getdstring ZERO 12
getdstring ZERO 0x50
get DUMMY long
getdstring ZERO 12
getdstring NAME 0x20
get FILES long
for i = 0 < FILES
    get OFFSET long
    get SIZE long

    savepos TMP
    goto OFFSET
    getdstring NAME NAMESZ
    math OFFSET + NAMESZ
    math SIZE   - NAMESZ
    goto TMP

    log NAME OFFSET SIZE
next i

The SNS file is encrypted with the key "CSD-CSCNV:01.82" (rot+NUL byte)

[ljwlwd]

It looks like the SNS format is composed by different sections.
This script extracts the files from the main section but don't expect anything ready to be open:

Code: Select all

set KEY binary "CSD-CSCNV:01.82"
strlen KEYSZ KEY
math KEYSZ + 1
set KEY2 string ""
for x = 0 < KEYSZ
    getvarchr TMP KEY x
    math TMP n TMP
    string KEY2 + TMP
    string KEY2 + " "
next x
filerot KEY2

math NAMESZ = 0x30

getdstring ZERO 16
get DUMMY long
get ZERO long
getdstring DUMMY 0x10
get OFFSET1 long
get OFFSET2 long
getdstring ZERO 0x30
get OFFSET3 long
getdstring ZERO 12
get OFFSET4 long
getdstring ZERO 12
getdstring ZERO 0x50
get DUMMY long
getdstring ZERO 12
getdstring NAME 0x20
get FILES long
for i = 0 < FILES
    get OFFSET long
    get SIZE long

    savepos TMP
    goto OFFSET
    getdstring NAME NAMESZ
    math OFFSET + NAMESZ
    math SIZE   - NAMESZ
    goto TMP

    log NAME OFFSET SIZE
next i

The SNS file is encrypted with the key "CSD-CSCNV:01.82" (rot+NUL byte)

Sorry to reopen this thread, I'm a player who want to translate this game into Chinese,
I used your script, QuickBMS says :
"- 94 files found in 0 seconds
coverage file 0 94% 3949152 4192480 . offset 00000000000003f4".
Then I got some .XML files, it do contains English game texts, but looks like seperate marks and return marks are displayed in error.

XML_example.png

I wonder if there is a way to recover these corrupted characters.
Thank you for the instructions!

[ljwlwd]

Forgot to upload the .sns file!
This is Steam version

[aluigi]

Probably some sort of binary xml. Can't help with that.

[amas]

It looks like the SNS format is composed by different sections.
This script extracts the files from the main section but don't expect anything ready to be open:

Code: Select all

set KEY binary "CSD-CSCNV:01.82"
strlen KEYSZ KEY
math KEYSZ + 1
set KEY2 string ""
for x = 0 < KEYSZ
    getvarchr TMP KEY x
    math TMP n TMP
    string KEY2 + TMP
    string KEY2 + " "
next x
filerot KEY2

math NAMESZ = 0x30

getdstring ZERO 16
get DUMMY long
get ZERO long
getdstring DUMMY 0x10
get OFFSET1 long
get OFFSET2 long
getdstring ZERO 0x30
get OFFSET3 long
getdstring ZERO 12
get OFFSET4 long
getdstring ZERO 12
getdstring ZERO 0x50
get DUMMY long
getdstring ZERO 12
getdstring NAME 0x20
get FILES long
for i = 0 < FILES
    get OFFSET long
    get SIZE long

    savepos TMP
    goto OFFSET
    getdstring NAME NAMESZ
    math OFFSET + NAMESZ
    math SIZE   - NAMESZ
    goto TMP

    log NAME OFFSET SIZE
next i

The SNS file is encrypted with the key "CSD-CSCNV:01.82" (rot+NUL byte)

I have a file similar to the one using "CSD-CSCNV:01.82", except that the file I got has "CSD-CSCNV:01.68", for what I know, it is an archive, which contains text files... Any help???

[amas]

It looks like the SNS format is composed by different sections.
This script extracts the files from the main section but don't expect anything ready to be open:

Code: Select all

set KEY binary "CSD-CSCNV:01.82"
strlen KEYSZ KEY
math KEYSZ + 1
set KEY2 string ""
for x = 0 < KEYSZ
    getvarchr TMP KEY x
    math TMP n TMP
    string KEY2 + TMP
    string KEY2 + " "
next x
filerot KEY2

math NAMESZ = 0x30

I managed to "Encrypt" almost everything, but when the process terminates, it says that 92% of the file was extracted.Any way to make it extract it 100%?

getdstring ZERO 16
get DUMMY long
get ZERO long
getdstring DUMMY 0x10
get OFFSET1 long
get OFFSET2 long
getdstring ZERO 0x30
get OFFSET3 long
getdstring ZERO 12
get OFFSET4 long
getdstring ZERO 12
getdstring ZERO 0x50
get DUMMY long
getdstring ZERO 12
getdstring NAME 0x20
get FILES long
for i = 0 < FILES
    get OFFSET long
    get SIZE long

    savepos TMP
    goto OFFSET
    getdstring NAME NAMESZ
    math OFFSET + NAMESZ
    math SIZE   - NAMESZ
    goto TMP

    log NAME OFFSET SIZE
next i

The SNS file is encrypted with the key "CSD-CSCNV:01.82" (rot+NUL byte)

I have a file similar to the one using "CSD-CSCNV:01.82", except that the file I got has "CSD-CSCNV:01.68", for what I know, it is an archive, which contains text files... Any help???

[amas]

It looks like the SNS format is composed by different sections.
This script extracts the files from the main section but don't expect anything ready to be open:

Code: Select all

set KEY binary "CSD-CSCNV:01.82"
strlen KEYSZ KEY
math KEYSZ + 1
set KEY2 string ""
for x = 0 < KEYSZ
    getvarchr TMP KEY x
    math TMP n TMP
    string KEY2 + TMP
    string KEY2 + " "
next x
filerot KEY2

math NAMESZ = 0x30

getdstring ZERO 16
get DUMMY long
get ZERO long
getdstring DUMMY 0x10
get OFFSET1 long
get OFFSET2 long
getdstring ZERO 0x30
get OFFSET3 long
getdstring ZERO 12
get OFFSET4 long
getdstring ZERO 12
getdstring ZERO 0x50
get DUMMY long
getdstring ZERO 12
getdstring NAME 0x20
get FILES long
for i = 0 < FILES
    get OFFSET long
    get SIZE long

    savepos TMP
    goto OFFSET
    getdstring NAME NAMESZ
    math OFFSET + NAMESZ
    math SIZE   - NAMESZ
    goto TMP

    log NAME OFFSET SIZE
next i

The SNS file is encrypted with the key "CSD-CSCNV:01.82" (rot+NUL byte)

I have a file similar to the one using "CSD-CSCNV:01.82", except that the file I got has "CSD-CSCNV:01.68", for what I know, it is an archive, which contains text files... Any help???

I actually made it do extract 92% of the file, now I should just understand how to extract 100% of the file, and then repack everything back on the file.

[aluigi]

Everything about 90% is excellent.
Some formats may use padding, alignment or old copies of files that take space later counted in the coverage.
Just a general info btw.

[miegir]

Did anybody have any luck with this binary xml format? I managed to extract and detect strings in it, then changed them to translated version and repackage (and correctly update offsets and sizes). But in only works when string lengths are not changed. If the string length changed (either bigger or smaller), the game crashes or hangs when the save is loaded or the character is selected. It seems that some other changes are needed.

[amas]

Everything about 90% is excellent.
Some formats may use padding, alignment or old copies of files that take space later counted in the coverage.
Just a general info btw.

Yes, the extracting process works pretty much perfectly, but I noticed that I have problems about repacking the modified .XML files into the .sns. The files I reimport back get corrupted (Somehow), any help?

[miegir]

For now I noticed that the file contains the following structure:
1. Filename (0x30 bytes) - it is better to consider filename as part of the file for the offsets to be correct. The script that was given here cuts out filename if I understand it correctly.
2. Sparse array of 256 4-bit offsets to many but not all of the bookmark elements (I didn't figure either why this array is sparse nor why offsets of some bookmarks are not given). Offsets are given from beginning of the file. I leave this section intact.
3. 4-bit offset to the content section (from beginning of the file).
4. Bookmark (perhaps) blocks which go one after another up to the content section offset. Each bookmark consists of:
4.1. 4-bit bookmark kind (maybe two 2-bit values)
4.2. offset in the content section (from the beginning of content section) - I fix all these offsets when changing string lengths
4.3. name as zero-terminated string
4.4. 0 or more zeros (padding to 4-byte boundary)
Bookmarks go in random order different from the order of teh content section.
5. The content section that consists of several chunks each started with header that includes its name. These chunks contain strings that are to be localized. Exact content of the chunks is the mystery but it is obvious that every string starts with 01, ends with 02, and has UTF-8 encoding. I haven't found any places where the string length is specified or the length of entire chunks. It seems that chunks are simply go after each other.
6. The end of the file. File always ends with AC 00 which can be followed by zeros to pad the file size to 4-byte boundary.

But that knowledge is not sufficient. If I change strings in such a way that its total lengths are the same (i.e. one string larger but the other in the entirely different chunk smaller by the same amount of characters or simply move or swap some strings) all seems ok. But when I change the total length of the string (i.e. increase or decrease length of only one string) it sometimes works but sometimes not. I'm out of ideas for now.

UPD: new ideas now. The key is in that "tail" from .sns file that is not covered by the script. There are in fact 4 "tails" with offset to each of them specified in the file header. First "tail" is the list of relative string offsets in the 2nd "tail". The 2nd "tail" is the string table of null-terminated "global" strings. Each string is at the offset from the 1st "tail". But that offsets are relative to the 2nd "tail" so the 1st and 2nd "tails" can be leaved intact. The 4th "tail" has an unknown content and possibly can be leaved intact either. The 3rd "tail" is the key. It contains string tables for all the .xml files. But in the head of the 3rd "tail" are absolute offsets (from the beginning of the .sns) for every string table. Since these offsets are absolute and are placed after localized strings, them should be fixed too.

UPD2: now I am successfully translated all the content to Russian. No other problems found (except for one bug in the game itself). It should be noted though that the game has some limit on the .sns total file length. This is the problem for languages in which the most characters are 2-byte wide in UTF-8. It is required to transform several characters to their exactly-looking latin form in order to save space. My patcher does this automatically. But no one likes russians now, and therefore my patcher unfortunately will never be published because of that.