REIMPORT2 mode - no size limits

[valgor]

Is it possible to add the possibility to create an archive containing only the edited files?
I mean, if the process of reimporting works it's ok and all, but if someone wants to release only a small patch it becomes a bit strange if the patch is larger than the original archive.
If someone edits only a few MBs worthy text files it would be good to be able to reimport only them. There are games which natively supports patches, so it would be cool if the reimporter could simply create a new archive containing only the edited files and ignore the untouched ones. As of now, if I delete the unused files, the program will simply keep the original ones and add the others.
I don't know how hard this feature could be to implement, I'm just making a suggestion of improvement. Thanks.

[aluigi]

If you want to distribute a patch you can use those tools that create delta patches. Basically you are going to distribute only the "difference" between the old archive and the new one.
Quickbms can only reimport the files.

There is an experimental option that allows to set the content of the files in the archive to zeroes, it's an idea for various possible usages, but it's totally useless in this case because the game is going to read the references to the zeroed files.

[lostprophet]

Just use u4pak (https://github.com/panzi/u4pak).
Basically it can make a new .pak file with only the edited files in it, and the game reads from it (so if you only edit the .locres file, just pack it into a .pak file, and the game will use that).

[ISKA]

Just use u4pak (https://github.com/panzi/u4pak).
Basically it can make a new .pak file with only the edited files in it, and the game reads from it (so if you only edit the .locres file, just pack it into a .pak file, and the game will use that).

Doesn't work on Darksiders III.

[torbido]

I want some files to be deleted from .pak file, so I made files with the same filename but with zero size (0 byte), and when I use reimport2 to add them instead of the original files, it fails. How can I do that?

[moiennepe]

Hello .,
I have a suggestion.

Can we keep TOC and all asset chunks SEPARATED in real device
(when the game engine accepts uncompressed/unencrypted asset chunks),
and make a virtual file which (virtually) merges them together?
(that means, for example, copy /b TOC.0+chunk.1+chunk.2+...+chunk.999 arc.bin on REAL device, is same as that VIRTUALIZED arc.bin)

The reason is,
most game archives nowdays are made to stuff bluray disc, they're very unfriendly to PC storage.
If we keep asset chunks separated, we can hardlink all the duplicates and apply OS native compression on specific type of asset.


For example,
In Yakuza0, all archives (*.par) sums up 13.9GB,
after unpack, hardlink the duplicates, and apply "compact /C /I /S /EXE:LZX", it results 6.7GB.
Also many Unity indies have a lot duplicated large textures in their game archive.

I know this can be done with a kernel32 proxy dll, with CreateFile, ReadFile, SetFilePointer etc. hooked,
But I am looking for a more generalized solution like Dokan/Dokany (github.com/dokan-dev/dokany) , pfmap (pismotec.com/download) or CBFS (http://www.callbacktechnologies.com/cbfsconnect).


I actually made a QuickBMS-fs with Dokan before, which similar to: github.com/dorvan/chunking-filesystem,
for random accessing without extracting the whole archive (lost those codes as my old laptop hard drive broke).

Now I hope this can be supported natively. (in reverse way)









In conclusion,
I think keep separated/split chunks on real hard disk, and feed the game executable with a virtually merged archive,
can fulfill these two purposes:

1. Reduce disk space waste on PC due to the console/BD disc-affiliated archiving.
(duplicated asset across archives, both compressible and incompressible type of resource in one archive ,etc)

2. Avoid repeatedly packing/unpacking when moding, which can actually harm your hard drive.

[aluigi]

quickbms works sequentially on the archives and doesn't maintain a "status" for each entry.
In that situation quickbms should first work in "list" mode (like -l) for collecting all the files entries.
Then maintain this tree for the virtual filesystem.
When the user selects the file try to "recover" the status of the file entry (that isn't available) for reading the file.

It's a huge work on which I already tought various times years ago, last time was when I implemented the ISO and ZIP file output.

Best solution is having a PC with 32Gb RAM and creating a ramdisk of 24Gb where extracting the files, problem solved

[ferhat78]

hi, i have a problem with compressed file size.

when i use reimport2 it bypass file size problem but at the end it change original file size soo the game don't launch and stay on logo.

.bak file is the original file.

anyone can help me please ? thanks in advance.

[ferhat78]

hi, i have a problem with compressed file size.

when i use reimport2 it bypass file size problem but at the end it change original file size soo the game don't launch and stay on logo.

.bak file is the original file.

anyone can help me please ? thanks in advance.

anyone please ?

[luiscarlosapeu]

ferhat78,

You're repeating editions..? I think you will only find susceptible stages of file corruption.
Did you repack the same file using Reimport 2 more than two/ three times?
Reimport1 I think is more flexible to reimport the main file countless times (although it is restricted to the same size for compression).
But as for Reimport2. I believe. Even if i have a vague experience of version 2. Use with great caution.
Because Reimport2 despite being a luxury tool to recompose all possible, considering the possibility to force the entry of all larger files, Remport2 is still inefficient for multiple reimports of large files to the same location, try to modify all your folders smaller, making last recomposition of all of them only once to the final file.
No Make rectifications of larger compressions several times to the final file, you must resort to the error and consequently to the compromise of your base.

[luiscarlosapeu]

Depending on what you are working on, it may not work, even if it is a compression performed only once by Reimport2.
If I'm wrong, correct me. But maybe Reimport 2 does not compensate or modify just once. Because functionally, it must generate many errors due to the disproportionality of the file becoming most of the time so large, which can cause conflicts depending on the root that the original file requires to be supported.
Sorry for the English.

[aluigi]

reimport2 is not efficient for multiple reimporting of the same file (like new versions) because the structure of the archive is modified everytime the updated file is longer (longer than the last update, not just longer than the original file), since it gets appended at the end enlarging the archive each time.

[Pepita]

I try to reimport Files after modify them into the APOC Client ( Ashes )

When I do so I get the following Message for every file:

Code: Select all

KEY:
< 0000000000000000 0          Store\ItemDefinitions\Armor\Cape\APP_CMN_PLT_M_CPE_A_0.uasset

- Variable 1 ("0") at line 198 is constant but the script tries to edit it.
  Do you want to continue anyway (y/N)?

Doing so, makes the game crash on loading (He cannot read the Pack File at all)

Using the Old Method (Reimport v1) result in the following Error:

Code: Select all

EpicAccountId:

Assertion failed: [File:Unknown] [Line: 3510] Pak Decompression failed. PakFile:../../../APOC/Content/Paks/pakchunk1-WindowsClient.pak, EntryOffset:616272857, EntrySize:161, Method:Zlib, ProcessedSize:393, RawSize:161, Crc32:3952538542, BlockIndex:0, Encrypt:0, Delete:0, Output:0000023B83AAA340, Raw:0000023B80753B80, Processed:0000000000000000, Bytes:[78DA030000000001000000000000000000000000000000000000000000000000...]

[aluigi]

unreal_tournament_4.bms is not good in reimport mode.
I guess there is some tool called u4pak that users use and it's discussed on this forum too.

[GHFear]

unreal_tournament_4.bms is not good in reimport mode.
I guess there is some tool called u4pak that users use and it's discussed on this forum too.

I would not recommend using pak file reimport modding as a way of modding Unreal Engine 4 games that you purchased on steam or Epic Game Store.
New version of EAC anticheat and anti tamper hook the pak-handing function and signature-check function and make sure the signatures match the originals and modifying PAKs can get you banned on certain online games.
It also doesn't work for new versions of UE4 after 4.22 or 4.23 if I remember correctly.
If it's offline, go ahead. But for new online games using EAC, it's risky.
You can build new PAKs that you place alongside the original PAKs or sideload a mod-enabler dll that let's you run unpacked assets from the Content folder.
That way, you don't get any broken PAKs or modify any existing files and the anti cheat will not be checking signatures against your files.

[MuslimCyberGames]

Hey, I just wanna ask for the game Story of Seasons: Friends of Mineral Town.

Can you fix this? for the Script Zip

Link:https://zenhax.com/viewtopic.php?f=9&t=13854

[aluigi]

@MuslimCyberGames
Eh no, the ZIP format is a different complex thing and reimporting can't work with it (for various reasons).

[VObject]

Hi! Help me guys

This script extract .pck AKPK files fine, but when I try to import files using reimport/reimport2, the program does not process what has been changed in the exported folder. Can you please tell me how to import modified file into the archive? What am I doing wrong?



script:
https://drive.google.com/file/d/1l1NDNw ... sp=sharing

example pck file:
https://drive.google.com/file/d/1Jf5524 ... sp=sharing

Django Reinhardt and classic music for Mafia DE need your help

[aluigi]

@VObject
Try replacing this part:

Code: Select all

      putVarChr MEMORY_FILE SIZE 0
      log MEMORY_FILE 0 0
      append
      log MEMORY_FILE OFFSET SIZE
      append
      callfunction getTYPE 1
      if PC == 0
         endian big
      endif
      string NAME += EXT
      log NAME 0 SIZE MEMORY_FILE

with:

Code: Select all

      string NAME += "."
      log NAME OFFSET SIZE
      if PC == 0
         endian big
      endif

[VObject]

@VObject
Try replacing this part:

Code: Select all

      putVarChr MEMORY_FILE SIZE 0
      log MEMORY_FILE 0 0
      append
      log MEMORY_FILE OFFSET SIZE
      append
      callfunction getTYPE 1
      if PC == 0
         endian big
      endif
      string NAME += EXT
      log NAME 0 SIZE MEMORY_FILE

with:

Code: Select all

      string NAME += "."
      log NAME OFFSET SIZE
      if PC == 0
         endian big
      endif

Unfortunately, import still failed, but now it looks like this: