FEAR 4gb (moved post)

[Savage]

Hi Aluigi, nice to see in your new site .

I have some problems extracting the .arch05

I used the BMS from FEAR / FEAR2 / ARCH00 archives (script 0.4b), wich works great until..i want to unpack the big files (4Gb's or more), yes i use quickbms_4gb_files.exe latest version (6) and i get this error

0000000000000000 1154375 worlds\retail\blackgates/blackgates.bndlxml05

- error in src\xalloc.c line 617: xdbg_malloc()

Error: memory allocation problem
Not enough space to process this command

I have 25 gb's free...any hint?

Thanks

[aluigi]

I have moved your post because it wasn't related to Shadow of Mordor.

That error means that the file has not been parsed correctly, what exact game is it?

[Savage]

The game it's the "Middle Earth Shadow of Mordor" , they are using the same package LTAR ARch

ex: 5.337.719.732 udun.arch05

LTAR x 

gollum.bndlxml05 bundles bundles\namedcharacters gollum.embb hirgon.bndlxml05 hirgon.embb ratbag.bndlxml05 ratbag.embb marwen1loadingscreen.bndlxml05 interface interface\bundles interface\bundles\loadingscreen marwen1loadingscreen.embb vision00.bndlxml05 interface\bundles\moviebundles vision00.embb battlereport.bndlxml05 interface\bundles\tutorials battlereport.embb newregion.bndlxml05 newregion.embb vendettasunlocked.bndlxml05 vendettasunlocked.embb blackgates.bndlxml05 worlds worlds\retail worlds\retail\blackgates blackgates.embb blackgates.ss_barrowsarea.bndlxml05 blackgates.ss_barrowsarea.embb blackgates.ss_blackgates1.bndlxml05 blackgates.ss_blackgates1.embb blackgates.ss_outpostarea.bndlxml05 blackgates.ss_outpostarea.emb

[aluigi]

Interesting, it seems that the error happens at the beginning of the file.
Maybe try to upload the first 4 megabytes.

[Savage]

No problem

[Savage]

Here is, the first 4mb's of the file udun.arch05 (4,97Gb's).

[aluigi]

I guess that without the full file it's impossible to find a solution.
From what I have seen till now, it seems to work correctly and I "think" that gollum.embb should be extracted correctly.
All the fields are correct with longlong so the problem is not there.
The compression is zlib.

One thing you can try is replacing all the >= with u>= but I don't think it will change something.
I have no other ideas at the moment.

[Savage]

It Worked! Thanks!

[aluigi]

really? great I add immediately the 'u' modification to the online script

[Savage]

Same problem with another big file , son.arch05 (3,89Gb's)

0000000000000000 224898 worlds\retail\seaofnurnen/seaofnurnen.cave1_ss_01.embb

- error in src\xalloc.c line 617: xdbg_malloc()

Error: memory allocation problem

or

- error in src\cmd.c line 3043: CMD_PutVarChr_func()
Error: Not enough space

[aluigi]

You can try to remove the line "putvarchr MEMORY_FILE4 SIZE 0" but I don't think it will solve the problem, in any case it will slow the process.

[Savage]

You right, i did the change, it fails and it's extremely slow.

[aluigi]

Thanks for the file.
Apparently it's not a bug in the script and not a bug in quickbms, it's really a file that has a size of 0x589fdbae bytes and quickbms cannot allocate that size because it's a 32bit application.

I tried to compile quickbms with Gcc 64bit but there are too many errors, so I give up.

[aluigi]

Just a simple test, could you try answering 'n' when quickbms asks you to disable XDBG_ALLOC_ACTIVE?

I don't think it will work but it's worth to try because it replaces malloc() with VirtualAlloc

[Savage]

Thanks for test the file

I tried "Y" and "N" options, still crashing

But with Dragon Unpacker i can open the file and extract the data (compressed) without errors, any hint how to unzlib the files?

Thanks

http://sourceforge.net/projects/dragonunpacker/

[aluigi]

The files are compressed in chunks, that's the reason why quickbms needs to pre-allocate the memory.

To be honest there is an alternative and very simple solution.
Try the bms script I have attached.
Now it uses the output file instead of a memory file, answer 'a' when quickbms will ask you to overwrite the file.

Remember to use quickbms_4gb_files.exe.

PS: the problem with the over 2gb memory can be solved compiling quickbms with "-Wl,--large-address-aware", I will do it in the next version

[Savage]

Great, just a question, the script you attached it's asking to replace files, it's ok?

do you want to disable XDBG_ALLOC_ACTIVE? (y/N)
y
- set output folder a

offset filesize filename
--------------------------------------
0000000000000000 0 bundles/resistance.bndlxml05
0000000000003684 65536 bundles\resistance.bndlxml05

- The following output file already exists:
bundles\resistance.bndlxml05
Do you want to overwrite it?
y = overwrite (you can use also the 'o' key)
n = skip (default, just press RETURN)
a = overwrite all the files without asking
r = automatically rename the files with the same name
s = skip all the existent files without asking


(remember that you are in append mode so be sure that the output folder was
empty otherwise the new data will be appended to the existent files!) y

[aluigi]

Yes, it's the 'a' answer I told you in my post.

[Savage]

Amazing!

[AlphaTwentyThree]

Amazing work, as always, Luigi!