Possible next features of QuickBMS

[michalss]

Maybe I will provide some examples when 0.8.5 is out.

That would be very nice... Thx let us know pls

[aluigi]

Anyway I think the only way is releasing quickbms as a dll otherwise there is no practical way to use its functions.
And currently I don't plan to do that since it requires the release of another file (the big dll) and so on, I simply needed something that required zero effort from my side.

Exist also other possible IPC solutions that may be easier to implement and I'm interested in your feedback.

For example an option can be a TCP socket or (even better) a local webserver that can be used by any program for performing the work like

Code: Select all

POST http://localhost:12345/quickbms_compression?algo=zlib&decompressed_size=12345

input provided as binary content of the POST request and the decompressed data as content of the answer.
the decompressed size is necessary for most of the compression algorithms, "decompressed_size" is just an example of name, "size" or "output_size" would be good too (maybe all of them, since input size is implicit in POST).

That would be 100% independent by programming language, OS and even system since 127.0.0.1 can be replaced by all the interfaces.
Is it easy for you to call web API from .NET?

[michalss]

Anyway I think the only way is releasing quickbms as a dll otherwise there is no practical way to use its functions.
And currently I don't plan to do that since it requires the release of another file (the big dll) and so on, I simply needed something that required zero effort from my side.

Exist also other possible IPC solutions that may be easier to implement and I'm interested in your feedback.

For example an option can be a TCP socket or (even better) a local webserver that can be used by any program for performing the work like

Code: Select all

POST http://localhost:12345/quickbms_compression?algo=zlib&decompressed_size=12345

input provided as binary content of the POST request and the decompressed data as content of the answer.
the decompressed size is necessary for most of the compression algorithms, "decompressed_size" is just an example of name, "size" or "output_size" would be good too (maybe all of them, since input size is implicit in POST).

That would be 100% independent by programming language, OS and even system since 127.0.0.1 can be replaced by all the interfaces.
Is it easy for you to call web API from .NET?

well it is not a problem call webapi, but there will be limitation on size during the data transfer i guess.

[aluigi]

Yeah, right.
Apparently the dll is the only solution.

Using shared memory IPC, which is the only fastest solution, would be a pain for the developers to implement.

Anyway I will implement the web and named pipe IPC interfaces anyway mainly because it's easy and I already have the code, maybe it will be useful in particular situations... after all it requires no effort from my side since the code is already there

[aluigi]

Just to confirm that the web IPC is slow indeed and also prone to timeouts due to the time necessary to perform some compressions.
Anyway now it's there and it's something funny that may be useful for the future.
Currently it implements compression, encryption and execution/testing of both an input script and input file.

[michalss]

Just to confirm that the web IPC is slow indeed and also prone to timeouts due to the time necessary to perform some compressions.
Anyway now it's there and it's something funny that may be useful for the future.
Currently it implements compression, encryption and execution/testing of both an input script and input file.

Ok can you please post example of usage for c# pls?

[sergop]

Hey, I just got an idea. Can u make option force all when reimporting? It's troubling to do it 300 times.

[aluigi]

@sergop
It's troubling to do it 300 times because you are doing something wrong 300 times, even doing it one single time means corrupting the archive and quickbms warns you about it.
If it was for me I would have not even implemented that "force" choice.

[sergop]

I use it to reimport edited images, videos and text. I have no problems with the games that I used it on just that sometimes there are too many files to import and I have to do it many times.

[aluigi]

As I have already said, you are doing something wrong.

DO NOT USE, NEVER EVER, THE "FORCE" CHOICE

[sergop]

But I have always used it before and I had no problems with any game. I'll try to stop using it, but idk what I'll do without it. Thx for the tip.

[aluigi]

quickbms tells you that it's not possible to reimport the file if its size or its compressed size is bigger than the original file.

In these cases there are two options:
- find a way to make your file smaller
- find a way to reduce the "entropy" of the file, that helps when it gets compressed

As you probably know, few months ago has been introduced reimport2.bat that allows to bypass the size limits in various formats.
That's for sure a good solution if the format is compatible with this mode or it's not possible to use the normal reimport mode (for example because files are compressed with lz4 that fails in decompression).

[sergop]

Yea I think I used -w -r -r a couple of times when -w -r didn't work
Thx for the tip I wasn't exactly sure why it was the case. And most of the time the size is bigger when I reimport the things.

[aluigi]

Let's say that quickbms will have the possibility of loading the script from the clipboard, that would be useful because many people post scripts on forums and websites.
What would be the best way to allow the user to choose the clipboard instead of a real file?
Maybe a command-line option? But in that case nobody is going to use it.
Writing "clipboard" in the entry of the dialog?

[aluigi]

Currently I have opted for writing ? or a char like <>|: in the entry. They are invalid chars for a filename, perfect for forcing this "special" clipboard mode.

Ah, the IPC mode is complete, it's probably totally useless since I think I'm going to release a dll but who cares:
- named pipe in message mode \\.\pipe\quickbms
- named pipe in byte mode \\.\pipe\quickbms_byte
- mailslot (not tested) \\.\mailslot\quickbms
- web IPC

They support just compression and encryption since there is nothing else interesting to offer.

[Acewell]

Error: the script uses more array indexes (7000) than supported (7000)

could you maybe increase support to at least 10000?

[aluigi]

@Acewell
it's not a limitation of quickbms. you get that error because you have stored data in 7000 entries and you are trying to use the one after the last one.
my mistake is probably having used the word "supported" (like something referred to quickbms) instead of something else referred to the script itself.

[Acewell]

ah okay, i see the mistake now, my bad

[aluigi]

In the next version I'm going to use a lame work-around to retrieve the amount of elements in an array:
getarray ELEMENTS 0 -0x80000000

Basically a large negative index, in fact quickbms already supports negative indexes for reading elements at the end of the array, so this is the only work-around available to get the number of elements.

[aluigi]

Since I'm annoyed of Oodle changing the prototype of OodleLZ_Compress and OodleLZ_Decompress everytime (it reminds me what I did with my mydownlib library, I understand that), I have decided that from version 0.8.5 the whole operation (GetProcAddress scanning and calling) will be automatized and handled by my calling_convention library so it's no longer necessary for me to touch my code after I add the new dll