Finding all instances of X and trimming it and its following bytes out

[Xiron]

So I have a file, it is not a plain text file, and I want to find all string instances of "OBMGcGeneralStatic" (or "4F 42 4D 47 63 47 65 6E 65 72 61 6C 53 74 61 74 69 63" if we're going with byte arrays) within it and delete it as well as the next 238 (EE) bytes that follow its every occurrence. How would I go about doing this? .bms script welcome.

For reference I'm try to remove all instances of these:

[aluigi]

I guess this may work:

Code: Select all

get NAME basename
get EXT extension
string NAME + "_cut."
string NAME + EXT

log NAME 0 0
append
get WRL_SIZE asize
for
    savepos OFFSET
    findloc NEXT_OFF binary "OBMGcGeneralStatic" 0 ""
    if NEXT_OFF == ""
        math NEXT_OFF = WRL_SIZE
    endif
    xmath SIZE "NEXT_OFF - OFFSET"
    if SIZE != 0
        log NAME OFFSET SIZE
    endif
    if NEXT_OFF == WRL_SIZE
        break
    endif
    math NEXT_OFF + 0x100
    goto NEXT_OFF
next
append

*edit* script updated

[Xiron]

Thanks for your efforts! Although it seems to have got only 98% of them, it successfully removed 496 and there is 7 leftover.
To add to this a problem seems to have occurred where the resulting file duplicates inside itself several hundred times (once for every find?). Like the original file size is 73,878 KB and it blew up to 2,263,276 KB.

Here is the file if you want to do proper testing yourself. http://www.mediafire.com/file/nyg89ovob1jbhpj/SANDY.WRL

[aluigi]

Fixed

[Xiron]

Sweet! Thanks, it works great! The maps in the game are looking nice and clean now.


I hope I'm not being to demanding, but there was one other thing I've always wanted to mass-do in the WRL files. There are two object types in this game, one is GeneralStatic, and one GeneralMobile. Static objects do not move, while mobile can move around, either by physics or by being set on a path (the path part is controlled by a separate unrelated entry in the WRL). I was hoping that perhaps you could make a Static to Mobile converter script if it isn't too much trouble? You'd be my modding hero if you can.

Let me start by showing the differences with a comparison image:


The differences between the two is the stuff in black (and the green highlight). To make this conversion script work, a row of 16 bytes (10 hex) would need to be inserted at the 128th (80 hex) byte mark, with those bytes being exactly "03 00 00 00 00 C0 DA 45 C3 F2 18 3F C2 D1 2A 3E" - (triva: these bytes are actually just settings for the object, but they aren't too important for this, so the copy of another object's setting is just fine), at this point the object will be 272 (110 hex) bytes long, and at the end of this we need to add in those extra "00 00 00 00" to the very end.

For the cherry on top, GeneralStatic would be renamed to GeneralMobile, but I'm not too worried about this feature as due to "Static" and "Mobile" being the same length I can always search for it in my hex editor and mass replace it there myself.

Anyhow, thanks for reading my request!

[aluigi]

Code: Select all

set MOBILE_DATA binary "\x00\x00\x80\x3f\x44\xed\x36\x41\xff\xff\xef\x3f\x71\xf6\xf8\x3f"

get NAME basename
get EXT extension
string NAME + "_mobile."
string NAME + EXT

log NAME 0 0
get WRL_SIZE asize
append
for
    savepos OFFSET
    findloc NEXT_OFF binary "OBMGcGeneralStatic" 0 ""
    if NEXT_OFF == ""
        math NEXT_OFF = WRL_SIZE
    endif
    xmath SIZE "NEXT_OFF - OFFSET"
    if SIZE != 0
        log NAME OFFSET SIZE
    endif
    if NEXT_OFF == WRL_SIZE
        break
    endif

    math TMP = NEXT_OFF
    append  # disable
    log MEMORY_FILE TMP 0x80
    append  # re-enable
    putdstring "OBMGcGeneralMobile" 18 MEMORY_FILE
    goto 0x20 MEMORY_FILE
    put 0xf0 byte MEMORY_FILE
    goto 0x80 MEMORY_FILE
    putdstring MOBILE_DATA 0x10 MEMORY_FILE
    math TMP + 0x80
    log MEMORY_FILE TMP 0x80
    goto 0x110 MEMORY_FILE
    put 0 long MEMORY_FILE

    log NAME 0 0x114 MEMORY_FILE

    math NEXT_OFF + 0x100
    goto NEXT_OFF
next
append

*script updated*

[Xiron]

Oops, I missed out one vital piece! You know at the start where the byte "DC" is? (In hex, it's the 20th byte in) Well this needs to be "F0" otherwise the game crashes.

Edit: I copied some bits off your code to attempt my own quick-fix. What I did was add

Code: Select all

set MOBILE_CHECK binary "\xF0"

after your "set MOBILE_DATA" code, and then I put in

Code: Select all

goto 0x20 MEMORY_FILE
putdstring MOBILE_CHECK 0x1 MEMORY_FILE

right before the "goto 0x80 MEMORY_FILE" line.

It seems to be working okay on the test conversion I did, but I'm not sure if this was the most efficient method of doing it or not. Let me know if I did alright and how you would have gone about doing it.

[aluigi]

put 0xf0 byte MEMORY_FILE

[Xiron]

Where? It might be best if you just updated your code above to reflect your post.

[aluigi]

Done

[Xiron]

Aaah, I understand the code now. Thanks "hero"! ;P

Here, have an in-game resulting image:
http://i.imgur.com/9pCA0QB.gif

[Xiron]

Alright, going back to the first script, how would I merge two trimmer scripts together?

Because if I take these part of the code

Code: Select all

    findloc NEXT_OFF binary "OBMGcGeneralStatic" 0 ""
    ....
    math NEXT_OFF + 0x110

and make it this to work with mobile objects (or anything I want to trim)

Code: Select all

    findloc NEXT_OFF binary "OBMGcGeneralMobile" 0 ""
    ....
    math NEXT_OFF + 0x114

it totally works, but there's gotta be a way to put two and two together to make things easier, I just don't know how. :S

[aluigi]

It's not easy to do these things with quickbms but the following script should do the job:

Code: Select all

get NAME basename
get EXT extension
string NAME + "_cut."
string NAME + EXT

log NAME 0 0
append
get WRL_SIZE asize
for
    savepos OFFSET
    findloc NEXT_OFF binary "OBMGcGeneral" 0 ""
    if NEXT_OFF == ""
        math NEXT_OFF = WRL_SIZE
    endif
    xmath SIZE "NEXT_OFF - OFFSET"
    if SIZE != 0
        log NAME OFFSET SIZE
    endif
    if NEXT_OFF == WRL_SIZE
        break
    endif

    goto NEXT_OFF
    getdstring SKIP 12
    getdstring CHECK 6
    if CHECK == "Static"
        math NEXT_OFF + 0x100
    else        # "Mobile"
        math NEXT_OFF + 0x114
    endif
    goto NEXT_OFF
next
append

[Xiron]

Awesome. I really appreciate the time you take out of your day to help me and others. Such marvellous generosity. =)
Thank you.

[Xiron]

Okay going back to the simple one item type trimmer, instead of cutting it out by its length, how would we make it cut out until it hits the next instance of "OBMGc" in the file instead? The reason I'm asking this is because some things in the file, like splines, do not have a set length.

[aluigi]

An idea may be to replace "OBMGcGeneralStatic" with "OBMGc" and adding an "if" statement that uses the Log command only if the current data at OFFSET is "OBMGcGeneralStatic".

Something like this (untested):

Code: Select all

get NAME basename
get EXT extension
string NAME + "_cut."
string NAME + EXT

log NAME 0 0
append
get WRL_SIZE asize
for
    savepos OFFSET
    findloc NEXT_OFF binary "OBMGc" 0 ""
    if NEXT_OFF == ""
        math NEXT_OFF = WRL_SIZE
    endif
    xmath SIZE "NEXT_OFF - OFFSET"
    if SIZE != 0
        getdstring TMP 18
        if TMP == "OBMGcGeneralStatic"
            log NAME OFFSET SIZE
        endif
    endif
    if NEXT_OFF == WRL_SIZE
        break
    endif
    math NEXT_OFF + 0x100
    goto NEXT_OFF
next
append

[Xiron]

I got a 0 byte file from that. Anyhow, I've went and attached a Spline sample since that's what we're aiming for.
As both OBMGcSpline and OBMGcSplineLinker exist, hopefully the linker won't get caught when searching for spline (although we may as well get rid of the linker at the same time? A Linker is always 218 long. If this is to much I can always do a separate removal since we got code for that already.)

[aluigi]

Honestly I think this thread is going a bit looong and OT.
It started with a specific simple request, then become bigger, bigger and now something different (there is no "OBMGcGeneralStatic" in your sample).
If you want to extract data from offset X of size Y use quickbms, otherwise use a programming language.
Sorry, but time is limited.

[Xiron]

Well, they do come in the exact same file as Static so I figured it wouldn't be a problem, but okay.