HELP ME,to calculate zstd compression

Programming related discussions related to game research
TUNBudi06
Posts: 52
Joined: Tue Sep 14, 2021 7:40 am

HELP ME,to calculate zstd compression

Post by TUNBudi06 »

So, I've had a lot of experience with the mini_obb.pak pubg file. extracted from obb.

so, if it was in pubg 1.5 before in the raw script. it's offset, zsize, and the size is put prefix before data compression.

well... now, pubg starts with a data compression header (ZSTD).
not offset, zsize, size, and others.


so I don't know how to determine the offset, size, and zsize.

So what I'm asking is how to calculate the ZSTD compression file. Because the average compression uses ZSTD.

and now
it's without offset,zsize,size.
and I've tried to extract the raw data.

the only way is
by

look for the hex code "\x28\xb5\x2f\xfd".
because it is a header in zstd compression (paks)mini_obb.pak

then, I cut the file by
cut from the first header to before the next header.
but the raw compression data can sometimes be detected by the tools (QUICKBMS) and sometimes there are errors with code

Code: Select all

get ZSIZE asize

comtype zstd
Clog NAME 0 ZSIZE ZSIZE
#i don't know how to calculate the zstd so i using the basic method
#which  i get it from topic that discuss about compression, decompression, offzip,and packzip.



this is information about pubg pack

magic file : 506e0406("\x06\x04\x6e\x50"

version : 10
the encrypted:
1.filename
2.offset, zsize, and size


IM sorry,im still new about compression.
But,i already learn about BMS script.
TUNBudi06
Posts: 52
Joined: Tue Sep 14, 2021 7:40 am

Re: HELP ME,to calculate zstd compression

Post by TUNBudi06 »

So, this the file


I cut 2 compression before end the file

I send it by GDrive

i am sorry, if i break the some rules
https://drive.google.com/file/d/1gHjoiM ... p=drivesdk
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: HELP ME,to calculate zstd compression

Post by aluigi »

I can't talk about PUBG because I don't know anything and it has already been explained in the UE4 topic that there is not much to do (based on the updates I saw in that topic).

While the zstd topic is very interesting.
The format of the header is described here:
https://github.com/facebook/zstd/blob/d ... _format.md

So I was able to write a zstd scanner able to find the correct position, decompressed size, and uncompressed size of each data and automatically decompressing all the zstd data contained in any file:
http://aluigi.org/bms/zstd_scan.bms
TUNBudi06
Posts: 52
Joined: Tue Sep 14, 2021 7:40 am

Re: HELP ME,to calculate zstd compression

Post by TUNBudi06 »

Ok, i don't talk about pubg. But i talk a about compression the pak of mini obb cause i have lot of experience of that file

And i already know all the compression method


It is ZSTD bcz all i know the .pak file is using lot of compression zstd


Thanks for give me the script zstd scanner.
TUNBudi06
Posts: 52
Joined: Tue Sep 14, 2021 7:40 am

Re: HELP ME,to calculate zstd compression

Post by TUNBudi06 »

Thanks so much bro.the script is work as well now i can unpack the paks pubg for raw data



One more

Thanks very much bro
TUNBudi06
Posts: 52
Joined: Tue Sep 14, 2021 7:40 am

Re: HELP ME,to calculate zstd compression

Post by TUNBudi06 »

But bro can you make it secret only


It can view by this site



Thank very much bro is work very-very well


Thanks bro
TUNBudi06
Posts: 52
Joined: Tue Sep 14, 2021 7:40 am

Re: HELP ME,to calculate zstd compression

Post by TUNBudi06 »

Hey bro,do you know about this problem

Code: Select all

quickbms.com  Homepage
                            zenhax.com  ZenHAX Forum
                     @zenhax @quickbms  Twitter & Scripts

- open input file /sdcard/mini_obb.pak
- open script /sdcard/zstd_info.bms
- set output folder /sdcard/ztes1
- the folder doesn't exist, do you want to create it (y/N)?:
  y

  offset   filesize   filename
--------------------------------------
runtimed
filecount: 1000
runtimed: 0
offset: 0
offset: 4
Dictionary ID flag: 3
offset: 5
Content Checksum flag: 0
offset: 5
Reserved bit : 0
offset: 5
Unused bit: 0
offset: 5
Single Segment flag: 1
offset: 5
Frame Content Size flag: 1
offset: 5
Frame Content Size short: 41353
Frame Content Size short + 256: 41609
Last Block: 1
Block Type: 2
Block Size: 1625074
ZIP SIZE: 1625084
  00000000 41609      file_0/00000000.dat
Info:  algorithm   478
       offset      00000000
       input size  0x0018cbfc 1625084
       output size 0x0000a289 41609
       result      0xffffffe0 -32

Error: uncompressed data (-32) bigger than allocated buffer (41609)
       It usually means that data is not compressed or uses another algorithm

Last script line before the error or that produced the error:
  111 clog name OFFSET SIZE Frame_Content_Size

- OFFSET       0x00000000
- ZSIZE        0x0018cbfc
- SIZE         0x0000a289
  coverage file 0     0%   1633286    221807307  . offset 0018cbfc
~ $


Bcs is cause error
Cause the dictionary is not zero

Bcz you code is to detect zero option

If work just like this
The output

Code: Select all

~ $ qemu-i386 $PREFIX/../usr64/share/quickbms/quickbms /sdcard/zstd_info.bms /sdcard/zak/0x79.pak /sdcard/ztes1

QuickBMS generic files extractor and reimporter 0.11.0
by Luigi Auriemma
e-mail: me@aluigi.org
web:    aluigi.org
        (Apr  5 2021 - 14:01:15)

                          quickbms.com  Homepage
                            zenhax.com  ZenHAX Forum
                     @zenhax @quickbms  Twitter & Scripts

- open input file /sdcard/zak/0x79.pak
- open script /sdcard/zstd_info.bms
- set output folder /sdcard/ztes1

  offset   filesize   filename
--------------------------------------
runtimed
filecount: 1000
runtimed: 0
offset: 302240
offset: 302244
Dictionary ID flag: 0
offset: 302245
Content Checksum flag: 0
offset: 302245
Reserved bit : 0
offset: 302245
Unused bit: 0
offset: 302245
Single Segment flag: 1
offset: 302245
Frame Content Size flag: 1                                      offset: 302245
Frame Content Size short: 2011                                  Frame Content Size short + 256: 2267
Last Block: 1
Block Type: 2
Block Size: 1024
ZIP SIZE: 1034
  00049ca0 2267       file_0/00000000.dat

- The following output file already exists:
  file_0/00000000.dat                                             Do you want to overwrite it?
    y = overwrite (you can use also the 'o' key)                    n = skip (default, just press ENTER)
    a = overwrite all the files without asking
    r = automatically rename the files with the same name
    s = skip all the existent files without asking

y
filecount: 1000
runtimed: 1
offset: 303280
offset: 303284
Dictionary ID flag: 0
offset: 303285
Content Checksum flag: 0
offset: 303285
Reserved bit : 0
offset: 303285
Unused bit: 0
offset: 303285
Single Segment flag: 1
offset: 303285
Frame Content Size flag: 1
offset: 303285
Frame Content Size short: 2263
Frame Content Size short + 256: 2519
Last Block: 1
Block Type: 2
Block Size: 486
ZIP SIZE: 496
  0004a0b0 2519       file_0/00000001.dat

- The following output file already exists:
  file_0/00000001.dat
  Do you want to overwrite it?
    y = overwrite (you can use also the 'o' key)
    n = skip (default, just press ENTER)
    a = overwrite all the files without asking
    r = automatically rename the files with the same name
    s = skip all the existent files without asking


Do You Know About Dictionary???

IM sorry,IM still new in the compression about that
TUNBudi06
Posts: 52
Joined: Tue Sep 14, 2021 7:40 am

Re: HELP ME,to calculate zstd compression

Post by TUNBudi06 »

I add that like offset,size,and other variabel


For to learn how the compression work!
TUNBudi06
Posts: 52
Joined: Tue Sep 14, 2021 7:40 am

Re: HELP ME,to calculate zstd compression

Post by TUNBudi06 »

And this for the file
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: HELP ME,to calculate zstd compression

Post by aluigi »

Script updated.
Obviously you need the correct dictionary for proceeding with the decompression.
TUNBudi06
Posts: 52
Joined: Tue Sep 14, 2021 7:40 am

Re: HELP ME,to calculate zstd compression

Post by TUNBudi06 »

Means???
TUNBudi06
Posts: 52
Joined: Tue Sep 14, 2021 7:40 am

Re: HELP ME,to calculate zstd compression

Post by TUNBudi06 »

IM just add some print code
Like this

Code: Select all

# ZSTD scanner
# https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md

get FILES asize
comtype zstd
math runtimed = 0
math filecount = 0
for I = 0 < FILES
    if runtimed == filecount
       print "runtimed"
       string namez = "file_"
       string namez + filecount
       string namez + "/"
       math filecount + 1000
    endif
    print "filecount: %filecount%"
    print "runtimed: %runtimed%"
    math runtimed + 1
   
    string name = namez
    string namesz p "%08d" i
    string name + namesz
    string name + ".dat"
    findloc OFFSET binary "\x28\xb5\x2f\xfd"
    goto OFFSET

    print "offset: %offset%"
    idstring "\x28\xb5\x2f\xfd"
    savepos b1
    print "offset: %b1%"
   
    getbits Dictionary_ID_flag 2
    savepos b1
    print "Dictionary ID flag: %Dictionary_ID_flag%"
    print "offset: %b1%"
   
    getbits Content_Checksum_flag 1
    savepos b1
    print "Content Checksum flag: %Content_Checksum_flag%"
    print "offset: %b1%"
   
    getbits Reserved_bit 1
    print "Reserved bit : %Reserved_bit%"
    savepos b1
    print "offset: %b1%"
   
    getbits Unused_bit 1
    savepos b1
    print "Unused bit: %Unused_bit%"
    print "offset: %b1%"
   
    getbits Single_Segment_flag 1
    savepos b1
    print "Single Segment flag: %Single_Segment_flag%"
    print "offset: %b1%"
   
    getbits Frame_Content_Size_flag 2
    print "Frame Content Size flag: %Frame_Content_Size_flag%"
    savepos b1
    print "offset: %b1%"

    if Single_Segment_flag == 0
        get Window_Descriptor byte
        print "Window Descriptor: %Window_Descriptor%"
        savepos b1
        print "offset: %b1%"
    endif

    if Frame_Content_Size_flag == 0
        if Single_Segment_flag == 0     # FCS_Field_Size 0
            math Frame_Content_Size = 0
            print "Frame Content Size null: %Frame_Content_Size%"
        else                            # FCS_Field_Size 1
            get Frame_Content_Size byte
            print "Frame Content Size byte: %Frame_Content_Size%"
        endif
    elif Frame_Content_Size_flag == 1   # FCS_Field_Size 2
        get Frame_Content_Size short
        print "Frame Content Size short: %Frame_Content_Size%"
        math Frame_Content_Size + 256
        print "Frame Content Size short + 256: %Frame_Content_Size%"
    elif Frame_Content_Size_flag == 2   # FCS_Field_Size 4
        get Frame_Content_Size long
        print "Frame Content Size long: %Frame_Content_Size%"
    elif Frame_Content_Size_flag == 3   # FCS_Field_Size 8
        get Frame_Content_Size longlong
        print "Frame Content Size longlong: %Frame_Content_Size%"
    endif

    do
        getbits Last_Block 1
        print "Last Block: %Last_Block%"
        getbits Block_Type 2
        print "Block Type: %Block_Type%"
        getbits Block_Size 21
        print "Block Size: %Block_Size%"
        if Block_Type == 1  # RLE_Block
            goto 1          0 SEEK_CUR
        else
            goto Block_Size 0 SEEK_CUR
        endif
        if Content_Checksum_flag != 0
            get Content_Checksum long
            print "Content Checksum: %Content_Checksum%"
        endif
    while Last_Block == 0
    savepos SIZE
    math SIZE - OFFSET
    print "ZIP SIZE: %size%"

    clog name OFFSET SIZE Frame_Content_Size
next I

TUNBudi06
Posts: 52
Joined: Tue Sep 14, 2021 7:40 am

Re: HELP ME,to calculate zstd compression

Post by TUNBudi06 »

Script
Last edited by TUNBudi06 on Tue Dec 07, 2021 8:58 am, edited 1 time in total.
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: HELP ME,to calculate zstd compression

Post by aluigi »

The -V option is a good alternative to manually putting "print" after each get command (the output is hex but it's ok).
What I meant with my previous post is that you need the exact dictionary for decompressing the data or it will fail, probably it's stored in the exe or who knows.
TUNBudi06
Posts: 52
Joined: Tue Sep 14, 2021 7:40 am

Re: HELP ME,to calculate zstd compression

Post by TUNBudi06 »

Thanks for the info of -V option bro.


So,about your post is means i must search the dictionary self.


But,i think is difficult bcz is not using exe but is .so(android library).
bcz the code of you page about to find AES KEY for paks
Is not dependent.cause if i take look (the paks format not encrypted.i mean the script(unreal_tournament) about encrypted(data byte) is not equal to 1.


And
About the .so
They seperated the file,i mean the library not put it together in one file. bcz if i look they seperated the lib.so they have many lib(i Will put it).bcz the lib seem the algorithme of compression.but they not put zstd

But,i don't know about elf.I mean how to unpack it or edit the inside code.but the lib seem not encrypted

And about seperate name
I do it bcz for the file i decompressed it almost 66000 file above!!.

And that make my phone so lag.when open the folder that contain the file

IM not using pc (IM user mobile only) so i run quickbms in termux.btw,thanks bro for make this tool support termux (armv8l).if not i using qemu emulator for run this version in arch i386


So,that some info i can get it
Ok,thanks for that and i Will try to research about dictionary.

Sorry for bad English
Btw, IM from Indonesian

Thanks
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: HELP ME,to calculate zstd compression

Post by aluigi »

Ever use a PC for doing extraction stuff and not a phone.
With a PC and a big ramdisk you don't even have to touch the disk: faster and don't limit the lifespan of the ssd/sdcard/hdd.
TUNBudi06
Posts: 52
Joined: Tue Sep 14, 2021 7:40 am

Re: HELP ME,to calculate zstd compression

Post by TUNBudi06 »

Yes,i do

But,i borrow my friend.if i need it.
TUNBudi06
Posts: 52
Joined: Tue Sep 14, 2021 7:40 am

Re: HELP ME,to calculate zstd compression

Post by TUNBudi06 »

Bro do you know how to fix it??

Code: Select all

~ $ qemu-i386 $PREFIX/../usr64/share/quickbms/quickbms -w -r /sdcard/zstd_info.bms /sdcard/mini_obb.pak /sdcard/ztes2

QuickBMS generic files extractor and reimporter 0.11.0
by Luigi Auriemma
e-mail: me@aluigi.org
web:    aluigi.org
        (Apr  5 2021 - 14:01:15)

                          quickbms.com  Homepage
                            zenhax.com  ZenHAX Forum
                     @zenhax @quickbms  Twitter & Scripts


- REIMPORT mode enabled!
  - remember to select the SAME script, file and folder you selected during
    the previous extraction
  - it's highly suggested to leave only the edited files in the folder, it's
    faster and less prone to errors with compressed files

- open input file /sdcard/mini_obb.pak
- open script /sdcard/zstd_info.bms
- set output folder /sdcard/ztes2

  offset   filesize   filename
--------------------------------------
runtimed

Error: file "file_0/00000000.dat"
       the reimport option acts as a reinjector, therefore you cannot insert a
       file if it's larger than the original for not overwriting the rest of
       the archive which cannot be loaded correctly:

         new size: 429 (1055 uncompressed)
         old size: 428 (1055 uncompressed)

- do you want to skip this file, quit or retry? (y/N/r)
  y: continue with the next file and skip the current file
  N: (default) terminate QuickBMS, maybe try later with the -r -r mode
  r: retry the reimporting so you can edit the file in the meantime
  force: corrupt the archive by writing the bigger file (NEVER use this!!!)
  y
< 000001b0 360        file_0/00000001.dat
< 00000320 397        file_0/00000002.dat
< 000004b0 478        file_0/00000003.dat

Error: file "file_0/00000004.dat"
       the reimport option acts as a reinjector, therefore you cannot insert a
       file if it's larger than the original for not overwriting the rest of
       the archive which cannot be loaded correctly:

         new size: 526 (1227 uncompressed)
         old size: 525 (1227 uncompressed)

- do you want to skip this file, quit or retry? (y/N/r)
  y: continue with the next file and skip the current file
  N: (default) terminate QuickBMS, maybe try later with the -r -r mode
  r: retry the reimporting so you can edit the file in the meantime
  force: corrupt the archive by writing the bigger file (NEVER use this!!!)
  y
< 000008a0 346        file_0/00000005.dat
< 00000a00 336        file_0/00000006.dat
< 00000b50 324        file_0/00000007.dat
< 00000ca0 327        file_0/00000008.dat

Error: file "file_0/00000009.dat"
       the reimport option acts as a reinjector, therefore you cannot insert a
       file if it's larger than the original for not overwriting the rest of
       the archive which cannot be loaded correctly:

         new size: 524 (1520 uncompressed)
         old size: 522 (1520 uncompressed)

- do you want to skip this file, quit or retry? (y/N/r)
  y: continue with the next file and skip the current file
  N: (default) terminate QuickBMS, maybe try later with the -r -r mode
  r: retry the reimporting so you can edit the file in the meantime
  force: corrupt the archive by writing the bigger file (NEVER use this!!!)
  y^C
~ $ qemu-i386 $PREFIX/../usr64/share/quickbms/quickbms -V -w -r /sdcard/zstd_info.bms /sdcard/mini_obb.pak /sdcard/ztes2

QuickBMS generic files extractor and reimporter 0.11.0
by Luigi Auriemma
e-mail: me@aluigi.org
web:    aluigi.org
        (Apr  5 2021 - 14:01:15)

                          quickbms.com  Homepage
                            zenhax.com  ZenHAX Forum
                     @zenhax @quickbms  Twitter & Scripts


- REIMPORT mode enabled!
  - remember to select the SAME script, file and folder you selected during
    the previous extraction
  - it's highly suggested to leave only the edited files in the folder, it's
    faster and less prone to errors with compressed files

- current_folder: /data/data/com.termux/files/home
- bms_folder:     /sdcard
- exe_folder:     /data/data/com.termux/files/usr/../usr64/share/quickbms
- file_folder:    /storage/emulated/0
- output_folder:  /storage/emulated/0/ztes2
- temp_folder:    /data/data/com.termux/files/usr/tm
- open input file /sdcard/mini_obb.pak
- open script /sdcard/zstd_info.bms
- set output folder /sdcard/ztes2

  offset   filesize   filename
--------------------------------------
. 00000000 get     FILES      0x25e74d68 -2
runtimed
. 00000000 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 00000000 idstr              "" 4
    28 b5 2f fd                                       (./.
. 00000004 getbits Dictionary_ID_flag 0x00000000 2
. 00000005 getbits Content_Checksum_flag 0x00000000 1
. 00000005 getbits Reserved_bit 0x00000000 1
. 00000005 getbits Unused_bit 0x00000000 1
. 00000005 getbits Single_Segment_flag 0x00000001 1
. 00000005 getbits Frame_Content_Size_flag 0x00000001 2
. 00000005 get     Frame_Content_Size 0x0000031f 2
.
. 00000007 getbits Last_Block 0x00000001 1
. 00000008 getbits Block_Type 0x00000002 2
. 00000008 getbits Block_Size 0x000001a2 21

Error: file "file_0/00000000.dat"
       the reimport option acts as a reinjector, therefore you cannot insert a
       file if it's larger than the original for not overwriting the rest of
       the archive which cannot be loaded correctly:

         new size: 429 (1055 uncompressed)
         old size: 428 (1055 uncompressed)

- do you want to skip this file, quit or retry? (y/N/r)
  y: continue with the next file and skip the current file
  N: (default) terminate QuickBMS, maybe try later with the -r -r mode
  r: retry the reimporting so you can edit the file in the meantime
  force: corrupt the archive by writing the bigger file (NEVER use this!!!)
  y
.
. 00000001
. 000001b0 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 000001b0 idstr              "" 4
    28 b5 2f fd                                       (./.
. 000001b4 getbits Dictionary_ID_flag 0x00000000 2
. 000001b5 getbits Content_Checksum_flag 0x00000000 1
. 000001b5 getbits Reserved_bit 0x00000000 1
. 000001b5 getbits Unused_bit 0x00000000 1
. 000001b5 getbits Single_Segment_flag 0x00000001 1
. 000001b5 getbits Frame_Content_Size_flag 0x00000001 2
. 000001b5 get     Frame_Content_Size 0x00000190 2
.
. 000001b7 getbits Last_Block 0x00000001 1
. 000001b8 getbits Block_Type 0x00000002 2
. 000001b8 getbits Block_Size 0x0000015e 21
< 000001b0 360        file_0/00000001.dat
.
. 00000002
. 00000320 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 00000320 idstr              "" 4
    28 b5 2f fd                                       (./.
. 00000324 getbits Dictionary_ID_flag 0x00000000 2
. 00000325 getbits Content_Checksum_flag 0x00000000 1
. 00000325 getbits Reserved_bit 0x00000000 1
. 00000325 getbits Unused_bit 0x00000000 1
. 00000325 getbits Single_Segment_flag 0x00000001 1
. 00000325 getbits Frame_Content_Size_flag 0x00000001 2
. 00000325 get     Frame_Content_Size 0x00000398 2
.
. 00000327 getbits Last_Block 0x00000001 1
. 00000328 getbits Block_Type 0x00000002 2
. 00000328 getbits Block_Size 0x00000183 21
< 00000320 397        file_0/00000002.dat
.
. 00000003
. 000004b0 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 000004b0 idstr              "" 4
    28 b5 2f fd                                       (./.
. 000004b4 getbits Dictionary_ID_flag 0x00000000 2
. 000004b5 getbits Content_Checksum_flag 0x00000000 1
. 000004b5 getbits Reserved_bit 0x00000000 1
. 000004b5 getbits Unused_bit 0x00000000 1
. 000004b5 getbits Single_Segment_flag 0x00000001 1
. 000004b5 getbits Frame_Content_Size_flag 0x00000001 2
. 000004b5 get     Frame_Content_Size 0x0000036a 2
.
. 000004b7 getbits Last_Block 0x00000001 1
. 000004b8 getbits Block_Type 0x00000002 2
. 000004b8 getbits Block_Size 0x000001d4 21
< 000004b0 478        file_0/00000003.dat
.
. 00000004
. 00000690 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 00000690 idstr              "" 4
    28 b5 2f fd                                       (./.
. 00000694 getbits Dictionary_ID_flag 0x00000000 2
. 00000695 getbits Content_Checksum_flag 0x00000000 1
. 00000695 getbits Reserved_bit 0x00000000 1
. 00000695 getbits Unused_bit 0x00000000 1
. 00000695 getbits Single_Segment_flag 0x00000001 1
. 00000695 getbits Frame_Content_Size_flag 0x00000001 2
. 00000695 get     Frame_Content_Size 0x000003cb 2
.
. 00000697 getbits Last_Block 0x00000001 1
. 00000698 getbits Block_Type 0x00000002 2
. 00000698 getbits Block_Size 0x00000203 21

Error: file "file_0/00000004.dat"
       the reimport option acts as a reinjector, therefore you cannot insert a
       file if it's larger than the original for not overwriting the rest of
       the archive which cannot be loaded correctly:

         new size: 526 (1227 uncompressed)
         old size: 525 (1227 uncompressed)

- do you want to skip this file, quit or retry? (y/N/r)
  y: continue with the next file and skip the current file
  N: (default) terminate QuickBMS, maybe try later with the -r -r mode
  r: retry the reimporting so you can edit the file in the meantime
  force: corrupt the archive by writing the bigger file (NEVER use this!!!)
  y
.
. 00000005
. 000008a0 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 000008a0 idstr              "" 4
    28 b5 2f fd                                       (./.
. 000008a4 getbits Dictionary_ID_flag 0x00000000 2
. 000008a5 getbits Content_Checksum_flag 0x00000000 1
. 000008a5 getbits Reserved_bit 0x00000000 1
. 000008a5 getbits Unused_bit 0x00000000 1
. 000008a5 getbits Single_Segment_flag 0x00000001 1
. 000008a5 getbits Frame_Content_Size_flag 0x00000001 2
. 000008a5 get     Frame_Content_Size 0x000001aa 2
.
. 000008a7 getbits Last_Block 0x00000001 1
. 000008a8 getbits Block_Type 0x00000002 2
. 000008a8 getbits Block_Size 0x00000150 21
< 000008a0 346        file_0/00000005.dat
.
. 00000006
. 00000a00 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 00000a00 idstr              "" 4
    28 b5 2f fd                                       (./.
. 00000a04 getbits Dictionary_ID_flag 0x00000000 2
. 00000a05 getbits Content_Checksum_flag 0x00000000 1
. 00000a05 getbits Reserved_bit 0x00000000 1
. 00000a05 getbits Unused_bit 0x00000000 1
. 00000a05 getbits Single_Segment_flag 0x00000001 1
. 00000a05 getbits Frame_Content_Size_flag 0x00000001 2
. 00000a05 get     Frame_Content_Size 0x00000148 2
.
. 00000a07 getbits Last_Block 0x00000001 1
. 00000a08 getbits Block_Type 0x00000002 2
. 00000a08 getbits Block_Size 0x00000146 21
< 00000a00 336        file_0/00000006.dat
.
. 00000007
. 00000b50 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 00000b50 idstr              "" 4
    28 b5 2f fd                                       (./.
. 00000b54 getbits Dictionary_ID_flag 0x00000000 2
. 00000b55 getbits Content_Checksum_flag 0x00000000 1
. 00000b55 getbits Reserved_bit 0x00000000 1
. 00000b55 getbits Unused_bit 0x00000000 1
. 00000b55 getbits Single_Segment_flag 0x00000001 1
. 00000b55 getbits Frame_Content_Size_flag 0x00000001 2
. 00000b55 get     Frame_Content_Size 0x0000013b 2
.
. 00000b57 getbits Last_Block 0x00000001 1
. 00000b58 getbits Block_Type 0x00000002 2
. 00000b58 getbits Block_Size 0x0000013a 21
< 00000b50 324        file_0/00000007.dat
.
. 00000008
. 00000ca0 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 00000ca0 idstr              "" 4
    28 b5 2f fd                                       (./.
. 00000ca4 getbits Dictionary_ID_flag 0x00000000 2
. 00000ca5 getbits Content_Checksum_flag 0x00000000 1
. 00000ca5 getbits Reserved_bit 0x00000000 1
. 00000ca5 getbits Unused_bit 0x00000000 1
. 00000ca5 getbits Single_Segment_flag 0x00000001 1
. 00000ca5 getbits Frame_Content_Size_flag 0x00000001 2
. 00000ca5 get     Frame_Content_Size 0x0000014a 2
.
. 00000ca7 getbits Last_Block 0x00000001 1
. 00000ca8 getbits Block_Type 0x00000002 2
. 00000ca8 getbits Block_Size 0x0000013d 21
< 00000ca0 327        file_0/00000008.dat
.
. 00000009
. 00000df0 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 00000df0 idstr              "" 4
    28 b5 2f fd                                       (./.
. 00000df4 getbits Dictionary_ID_flag 0x00000000 2
. 00000df5 getbits Content_Checksum_flag 0x00000000 1
. 00000df5 getbits Reserved_bit 0x00000000 1
. 00000df5 getbits Unused_bit 0x00000000 1
. 00000df5 getbits Single_Segment_flag 0x00000001 1
. 00000df5 getbits Frame_Content_Size_flag 0x00000001 2
. 00000df5 get     Frame_Content_Size 0x000004f0 2
.
. 00000df7 getbits Last_Block 0x00000001 1
. 00000df8 getbits Block_Type 0x00000002 2
. 00000df8 getbits Block_Size 0x00000200 21

Error: file "file_0/00000009.dat"
       the reimport option acts as a reinjector, therefore you cannot insert a
       file if it's larger than the original for not overwriting the rest of
       the archive which cannot be loaded correctly:

         new size: 524 (1520 uncompressed)
         old size: 522 (1520 uncompressed)

- do you want to skip this file, quit or retry? (y/N/r)
  y: continue with the next file and skip the current file
  N: (default) terminate QuickBMS, maybe try later with the -r -r mode
  r: retry the reimporting so you can edit the file in the meantime
  force: corrupt the archive by writing the bigger file (NEVER use this!!!)
  y
.
. 0000000a
. 00001000 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 00001000 idstr              "" 4
    28 b5 2f fd                                       (./.
. 00001004 getbits Dictionary_ID_flag 0x00000000 2
. 00001005 getbits Content_Checksum_flag 0x00000000 1
. 00001005 getbits Reserved_bit 0x00000000 1
. 00001005 getbits Unused_bit 0x00000000 1
. 00001005 getbits Single_Segment_flag 0x00000001 1
. 00001005 getbits Frame_Content_Size_flag 0x00000001 2
. 00001005 get     Frame_Content_Size 0x0000015d 2
.
. 00001007 getbits Last_Block 0x00000001 1
. 00001008 getbits Block_Type 0x00000002 2
. 00001008 getbits Block_Size 0x0000014d 21

Error: file "file_0/00000010.dat"
       the reimport option acts as a reinjector, therefore you cannot insert a
       file if it's larger than the original for not overwriting the rest of
       the archive which cannot be loaded correctly:

         new size: 344 (605 uncompressed)
         old size: 343 (605 uncompressed)

- do you want to skip this file, quit or retry? (y/N/r)
  y: continue with the next file and skip the current file
  N: (default) terminate QuickBMS, maybe try later with the -r -r mode
  r: retry the reimporting so you can edit the file in the meantime
  force: corrupt the archive by writing the bigger file (NEVER use this!!!)
  y
.
. 0000000b
. 00001160 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 00001160 idstr              "" 4
    28 b5 2f fd                                       (./.
. 00001164 getbits Dictionary_ID_flag 0x00000000 2
. 00001165 getbits Content_Checksum_flag 0x00000000 1
. 00001165 getbits Reserved_bit 0x00000000 1
. 00001165 getbits Unused_bit 0x00000000 1
. 00001165 getbits Single_Segment_flag 0x00000001 1
. 00001165 getbits Frame_Content_Size_flag 0x00000001 2
. 00001165 get     Frame_Content_Size 0x00000168 2
.
. 00001167 getbits Last_Block 0x00000001 1
. 00001168 getbits Block_Type 0x00000002 2
. 00001168 getbits Block_Size 0x00000129 21
< 00001160 307        file_0/00000011.dat
.
. 0000000c
. 000012a0 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 000012a0 idstr              "" 4
    28 b5 2f fd                                       (./.
. 000012a4 getbits Dictionary_ID_flag 0x00000000 2
. 000012a5 getbits Content_Checksum_flag 0x00000000 1
. 000012a5 getbits Reserved_bit 0x00000000 1
. 000012a5 getbits Unused_bit 0x00000000 1
. 000012a5 getbits Single_Segment_flag 0x00000001 1
. 000012a5 getbits Frame_Content_Size_flag 0x00000001 2
. 000012a5 get     Frame_Content_Size 0x00000257 2
.
. 000012a7 getbits Last_Block 0x00000001 1
. 000012a8 getbits Block_Type 0x00000002 2
. 000012a8 getbits Block_Size 0x00000184 21
< 000012a0 398        file_0/00000012.dat
.
. 0000000d
. 00001430 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 00001430 idstr              "" 4
    28 b5 2f fd                                       (./.
. 00001434 getbits Dictionary_ID_flag 0x00000000 2
. 00001435 getbits Content_Checksum_flag 0x00000000 1
. 00001435 getbits Reserved_bit 0x00000000 1
. 00001435 getbits Unused_bit 0x00000000 1
. 00001435 getbits Single_Segment_flag 0x00000001 1
. 00001435 getbits Frame_Content_Size_flag 0x00000001 2
. 00001435 get     Frame_Content_Size 0x000001b3 2
.
. 00001437 getbits Last_Block 0x00000001 1
. 00001438 getbits Block_Type 0x00000002 2
. 00001438 getbits Block_Size 0x00000130 21

Error: file "file_0/00000013.dat"
       the reimport option acts as a reinjector, therefore you cannot insert a
       file if it's larger than the original for not overwriting the rest of
       the archive which cannot be loaded correctly:

         new size: 315 (691 uncompressed)
         old size: 314 (691 uncompressed)

- do you want to skip this file, quit or retry? (y/N/r)
  y: continue with the next file and skip the current file
  N: (default) terminate QuickBMS, maybe try later with the -r -r mode
  r: retry the reimporting so you can edit the file in the meantime
  force: corrupt the archive by writing the bigger file (NEVER use this!!!)
  y
.
. 0000000e
. 00001570 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 00001570 idstr              "" 4
    28 b5 2f fd                                       (./.
. 00001574 getbits Dictionary_ID_flag 0x00000000 2
. 00001575 getbits Content_Checksum_flag 0x00000000 1
. 00001575 getbits Reserved_bit 0x00000000 1
. 00001575 getbits Unused_bit 0x00000000 1
. 00001575 getbits Single_Segment_flag 0x00000001 1
. 00001575 getbits Frame_Content_Size_flag 0x00000001 2
. 00001575 get     Frame_Content_Size 0x00000252 2
.
. 00001577 getbits Last_Block 0x00000001 1
. 00001578 getbits Block_Type 0x00000002 2
. 00001578 getbits Block_Size 0x00000185 21
< 00001570 399        file_0/00000014.dat
.
. 0000000f
. 00001700 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 00001700 idstr              "" 4
    28 b5 2f fd                                       (./.
. 00001704 getbits Dictionary_ID_flag 0x00000000 2
. 00001705 getbits Content_Checksum_flag 0x00000000 1
. 00001705 getbits Reserved_bit 0x00000000 1
. 00001705 getbits Unused_bit 0x00000000 1
. 00001705 getbits Single_Segment_flag 0x00000001 1
. 00001705 getbits Frame_Content_Size_flag 0x00000001 2
. 00001705 get     Frame_Content_Size 0x000001a0 2
.
. 00001707 getbits Last_Block 0x00000001 1
. 00001708 getbits Block_Type 0x00000002 2
. 00001708 getbits Block_Size 0x0000015e 21
< 00001700 360        file_0/00000015.dat
.
. 00000010
. 00001870 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 00001870 idstr              "" 4
    28 b5 2f fd                                       (./.
. 00001874 getbits Dictionary_ID_flag 0x00000000 2
. 00001875 getbits Content_Checksum_flag 0x00000000 1
. 00001875 getbits Reserved_bit 0x00000000 1
. 00001875 getbits Unused_bit 0x00000000 1
. 00001875 getbits Single_Segment_flag 0x00000001 1
. 00001875 getbits Frame_Content_Size_flag 0x00000001 2
. 00001875 get     Frame_Content_Size 0x00000128 2
.
. 00001877 getbits Last_Block 0x00000001 1
. 00001878 getbits Block_Type 0x00000002 2
. 00001878 getbits Block_Size 0x0000011d 21
< 00001870 295        file_0/00000016.dat
.
. 00000011
. 000019a0 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 000019a0 idstr              "" 4
    28 b5 2f fd                                       (./.
. 000019a4 getbits Dictionary_ID_flag 0x00000000 2
. 000019a5 getbits Content_Checksum_flag 0x00000000 1
. 000019a5 getbits Reserved_bit 0x00000000 1
. 000019a5 getbits Unused_bit 0x00000000 1
. 000019a5 getbits Single_Segment_flag 0x00000001 1
. 000019a5 getbits Frame_Content_Size_flag 0x00000001 2
. 000019a5 get     Frame_Content_Size 0x000001af 2
.
. 000019a7 getbits Last_Block 0x00000001 1
. 000019a8 getbits Block_Type 0x00000002 2
. 000019a8 getbits Block_Size 0x00000159 21
< 000019a0 355        file_0/00000017.dat
.
. 00000012
. 00001b10 getdstr skippedheader "" 5
    28 b5 2f fd 60                                    (./.`
.
.
. 00001b10 idstr              "" 4
    28 b5 2f fd                                       (./.
. 00001b14 getbits Dictionary_ID_flag 0x00000000 2
. 00001b15 getbits Content_Checksum_flag 0x00000000 1
. 00001b15 getbits Reserved_bit 0x00000000 1
. 00001b15 getbits Unused_bit 0x00000000 1
. 00001b15 getbits Single_Segment_flag 0x00000001 1
. 00001b15 getbits Frame_Content_Size_flag 0x00000001 2
. 00001b15 get     Frame_Content_Size 0x0000012a 2
.
. 00001b17 getbits Last_Block 0x00000001 1
. 00001b18 getbits Block_Type 0x00000002 2
. 00001b18 getbits Block_Size 0x00000127 21

Error: file "file_0/00000018.dat"
       the reimport option acts as a reinjector, therefore you cannot insert a
       file if it's larger than the original for not overwriting the rest of
       the archive which cannot be loaded correctly:

         new size: 306 (554 uncompressed)
         old size: 305 (554 uncompressed)

- do you want to skip this file, quit or retry? (y/N/r)
  y: continue with the next file and skip the current file
  N: (default) terminate QuickBMS, maybe try later with the -r -r mode
  r: retry the reimporting so you can edit the file in the meantime
  force: corrupt the archive by writing the bigger file (NEVER use this!!!)
  ^C
~ $
TUNBudi06
Posts: 52
Joined: Tue Sep 14, 2021 7:40 am

Re: HELP ME,to calculate zstd compression

Post by TUNBudi06 »

Bcz,i want to repack the size not same as old size
Its seem like the size + 2
aluigi
Site Admin
Posts: 12984
Joined: Wed Jul 30, 2014 9:32 pm

Re: HELP ME,to calculate zstd compression

Post by aluigi »

Long story short, you can't reimport the files back.

ZSTD, just like Oodle and few other algorithms, doesn't work if the compressed and decompressed size don't match the expected one.
Even if your compressed file is smaller then the zstd function in the game will return an error.
Indeed the zstd used in quickbms has two modifications that do not return the error and therefore it can be used to decompress whatever data even if ZSIZE/SIZE is wrong (for example bigger than the real one).

I don't have any solution or alternative.