store files in memory

[Shokoniraya]

hello sir aluigi
i have a question about binary files in QuickBMS

i want to merge two text files based on a table and append to that table, so i want to know that is there any way to keep a few files in memory and then read them based on their own id or name?
i can use MEMORY_FILE, but i cant read memory file like this: set MEMORY_FILE%id_code% binary MY_BINARY
is there any way to store files in memory with real names? i can export them to disk and then open them, but it cost so much speed

[aluigi]

If your data is just text read with operations like "get VAR string", then you can prefer to use the arrays for keeping the data in memory and ready to use with just an index:

Code: Select all

get VAR string
putarray 0 id_code VAR

Some people also use the experimental multidimensional arrays (like VAR[id_code]) but they are not that good.

In theory you can also store binary data in the arrays but honestly don't know if quickbms can really do that.

Other ideas... maybe you can just put everything in one MEMORY_FILE in append mode (sequentially) and then using 2 arrays for storing OFFSET and SIZE of that specific id_code:

Code: Select all

getdstring VAR SIZE # or just use log+append
savepos OFFSET MEMORY_FILE
putdstring VAR SIZE MEMORY_FILE
putarray 0 id_code OFFSET
putarray 1 id_code SIZE

[Shokoniraya]

thank you, but it's not just about text, i want to do it on some binary too, not just c string

but it's a good thing tool add MEMORY_FILE[NAME] for memory files, it's really useful

[aluigi]

If you need to add binary the getdstring/putdstring solution will work.
You can also use log+append.
Unfortunately something like MEMORY_FILE[NAME] would be difficult to implement, and to use too (imagine mentioning it in the manual, very messy)

[Shokoniraya]

not so messy, but getdstring can't support full binary and in QuickBMS, it will stop on \x00
for example, if we want to get a binary like \x14\x5B\x00\xAE, then it will stop on \x00 and because it's c-string and act like a null delimited string (zero terminated), so it's not a best option
and i think that's why there is a problem like unicode converting: https://zenhax.com/viewtopic.php?f=13&t=12984


i have some ideas to make QuickBMS better

1) use command line options in script
it's possible to set command line options from cmd, and QuickBMSver already have a few options, but it can be more useful when we want to do export and import at one script, so a command to switch options will solve many problems
like this: QuickBMSver "-w -r -r -r"

2) store real files in MEMORY_FILE
as i know, QuickBMS works fine with MEMORY_FILE to dump binary files. but if we want to make a script to merge two file, or store a few files with some special names or code, we need to use name or code.

get FILENUM long
set MEMORY_FILE[FILENUM] binary MY_BINARY

[FILENUM] can works with normal variables, but it can be better to support MEMORY_FILE too, as a string, not just a number

3) batch commands
one of the best options that QuickBMS can have in bms script is batch commands
something like QuickBMSver but works like a line of .bat file
like this: QuickBMSbatch "rename FILE1 MY_FILE"

[aluigi]

1) it's a security problem

2) I like the idea of having a sort of "ram disk", I will check what I can do with that MEMORY_FILE[FILENUM] idea

3) very very very dangerous

[Shokoniraya]

and just one thing left, can you make slog command to support MEMORY_FILE?

[aluigi]

Yeah I can think I can try that.