[PC] Lost Ark : .Upk / .lpk

[brekkez]

Here is script for unpacking lpk files from steam (default) and ru versions of the game. Credits to Brouznouf for algorithm.

Update: Script is updated to properly decrypt databases from lpk files, but they're additionally encrypted on the inner level regardless.

Does this work on the eu steam version?

[spiritovod]

@brekkez: There is only one version of the game in steam, as far as I can see at steamdb. It's not the first time people are asking about eu client specifically, so I'm curious, why it's like that? Both korean and ru versions are shipped through their own launchers outside steam.

[brekkez]

@brekkez: There is only one version of the game in steam, as far as I can see at steamdb. It's not the first time people are asking about eu client specifically, so I'm curious, why it's like that? Both korean and ru versions are shipped through their own launchers outside steam.

Maybe it’s just the steam version? It worked with umodel though.

[spiritovod]

@brekkez: To simplify things, there are only 3 versions of the game at the moment: steam (regardless of the region), korean and russian. And most tools/scripts are divided accordingly. If you mean the script above, it's compatible with both steam (by default) and ru versions - to switch between them you simply need to edit bms script in any text editor according to included comments.

[brekkez]

@brekkez: To simplify things, there are only 3 versions of the game at the moment: steam (regardless of the region), korean and russian. And most tools/scripts are divided accordingly. If you mean the script above, it's compatible with both steam (by default) and ru versions - to switch between them you simply need to edit bms script in any text editor according to included comments.

It worked!. It’s so hard to find the model I’m looking for lmao. I managed to find a costume and a head. I’m still trying to find the missing textures .

[spiritovod]

Meanwhile I've solved issue with databases, latest script update should decrypt them properly. Not sure about their format though, but that's out of my scope.

[a652311]

Meanwhile I've solved issue with databases, latest script update should decrypt them properly. Not sure about their format though, but that's out of my scope.

Hi,spiritovod how can i decrypt for .db files?

[daranah]

@brekkez: To simplify things, there are only 3 versions of the game at the moment: steam (regardless of the region), korean and russian. And most tools/scripts are divided accordingly. If you mean the script above, it's compatible with both steam (by default) and ru versions - to switch between them you simply need to edit bms script in any text editor according to included comments.

It worked!. It’s so hard to find the model I’m looking for lmao. I managed to find a costume and a head. I’m still trying to find the missing textures .

That's so cool! :O
Can you please do a step by step on how?
I downloaded even the old versions but i can't get anything to work, umodel crashes, quickbms won't do anything (memory error??) I have no idea what i'm doing wrong Help?

[zerozero]

Meanwhile I've solved issue with databases, latest script update should decrypt them properly. Not sure about their format though, but that's out of my scope.

Yo, first of all thanks for making this script.
Im getting [unrecognized token: ""] when trying to open .db files , are you sure that they are decrypted?

[spiritovod]

@daranah: You can start with reading this topic.

@zerozero: You can check it by opening extracted database in any hex editor, if it's somehow readable - it's decrypted. Though, like already said above, they may use custom format for it.

[brekkez]

@brekkez: To simplify things, there are only 3 versions of the game at the moment: steam (regardless of the region), korean and russian. And most tools/scripts are divided accordingly. If you mean the script above, it's compatible with both steam (by default) and ru versions - to switch between them you simply need to edit bms script in any text editor according to included comments.

It worked!. It’s so hard to find the model I’m looking for lmao. I managed to find a costume and a head. I’m still trying to find the missing textures .

That's so cool! :O
Can you please do a step by step on how?
I downloaded even the old versions but i can't get anything to work, umodel crashes, quickbms won't do anything (memory error??) I have no idea what i'm doing wrong Help?

Use umodel dms

[korribanscion]

It worked!. It’s so hard to find the model I’m looking for lmao. I managed to find a costume and a head. I’m still trying to find the missing textures .

That's so cool! :O
Can you please do a step by step on how?
I downloaded even the old versions but i can't get anything to work, umodel crashes, quickbms won't do anything (memory error??) I have no idea what i'm doing wrong Help?

Use umodel dms

So, when I attempt to run the steam version updated exe of umodel, I get this error when trying to open a upk file:
lzo_decompress(4969,38731) returned -6
appDecompress: CompSize=4969 UncompSize=38731 Flags=0x44 Bytes=A4A5 <- DecompressBlock: block=AD+1369 <- FUE3ArchiveReader::PrepareBuffer <- FUE3ArchiveReader::Serialize <- FArchive::ByteOrderSerialize <- FString<< <- Name: 0 <- UnPackage::LoadNameTable3 <- UnPackage::UnPackage: ZHJ4TZKZ4EDB4E224EWXXSBH2V.upk, ver=868/16, game=ue3 <- UnPackage::LoadPackage(info): ZHJ4TZKZ4EDB4E224EWXXSBH2V.upk <- CUmodelApp::ShowPackageUI <- Main: umodel_build=1544 based

Any idea why this is? If I need to decrypt all these upk package files first, can someone point me in the direction of a decrypter that actually works for the Steam version of Lost Ark?

EDIT: Honestly, all I need is 1 male and 1 female 3d mesh to use for proportion comparison in Maya, not all of them. If anyone wants to just pm or upload a pair somewhere, that'd work too

[daranah]

yeah, same for me. the decoder didn't work for any version for me either, it just says invalid folder no matter what i try...

[MonkeyFlops]

So, while everyone is trying to be able to open the LPK files, I'm looking for a script to decrypt the IPK files.

Does anyone have anything for this already?
Not sure how I would go about into creating this.

[brekkez]

That's so cool! :O
Can you please do a step by step on how?
I downloaded even the old versions but i can't get anything to work, umodel crashes, quickbms won't do anything (memory error??) I have no idea what i'm doing wrong Help?

Use umodel dms

So, when I attempt to run the steam version updated exe of umodel, I get this error when trying to open a upk file:
lzo_decompress(4969,38731) returned -6
appDecompress: CompSize=4969 UncompSize=38731 Flags=0x44 Bytes=A4A5 <- DecompressBlock: block=AD+1369 <- FUE3ArchiveReader::PrepareBuffer <- FUE3ArchiveReader::Serialize <- FArchive::ByteOrderSerialize <- FString<< <- Name: 0 <- UnPackage::LoadNameTable3 <- UnPackage::UnPackage: ZHJ4TZKZ4EDB4E224EWXXSBH2V.upk, ver=868/16, game=ue3 <- UnPackage::LoadPackage(info): ZHJ4TZKZ4EDB4E224EWXXSBH2V.upk <- CUmodelApp::ShowPackageUI <- Main: umodel_build=1544 based

Any idea why this is? If I need to decrypt all these upk package files first, can someone point me in the direction of a decrypter that actually works for the Steam version of Lost Ark?

EDIT: Honestly, all I need is 1 male and 1 female 3d mesh to use for proportion comparison in Maya, not all of them. If anyone wants to just pm or upload a pair somewhere, that'd work too

Use the lost ark one worked for me

[elegos]

Hello!

I'm new to game client hacking and I'm a little confused.

What I want to achieve: extract bitmaps (skills, maps etc), data info (skill names and props, item names and props, mob names etc) from the Lost Ark game
What I've done so far: I've downloaded and unarchived QuickBMS, downloaded the bms file on page 6 and tried the following command:

Code: Select all

.\quickbms.exe .\scripts\lpk_extract_v2_fix.bms 'C:\Program Files (x86)\Steam\steamapps\common\Lost Ark\EFGame\ReleasePC\Packages' "C:\LostArk_packages"

The result:

Code: Select all

QuickBMS generic files extractor and reimporter 0.11.0
by Luigi Auriemma
e-mail: me@aluigi.org
web:    aluigi.org
        (Apr  5 2021 - 13:56:34)

                          quickbms.com  Homepage
                            zenhax.com  ZenHAX Forum
                     @zenhax @quickbms  Twitter & Scripts

- start the scanning of the input folder: C:\Program Files (x86)\Steam\steamapps\common\Lost Ark\EFGame\ReleasePC\Packages
- open input file C:\Program Files (x86)\Steam\steamapps\common\Lost Ark\EFGame\ReleasePC\Packages\.\07F8Q0K8X0JS8DCXQ6WK8XX0CM5JYDYE40.upk
- open script .\scripts\lpk_extract_v2_fix.bms
- set output folder C:\LostArk_packages
- the folder doesn't exist, do you want to create it (y/N)?:
  y

  offset   filesize   filename
--------------------------------------

Error: incomplete input file 0: C:\Program Files (x86)\Steam\steamapps\common\Lost Ark\EFGame\ReleasePC\Packages\07F8Q0K8X0JS8DCXQ6WK8XX0CM5JYDYE40.upk
       Can't read 934261218 bytes from offset 00000c32.
       Anyway don't worry, it's possible that the BMS script has been written
       to exit in this way if it's reached the end of the archive so check it
       or contact its author or verify that all the files have been extracted.
       Please check the following coverage information to know if it's ok.

  coverage file 0   100%   3122       3122       . offset 00000c32

Last script line before the error or that produced the error:
  15  log MEMORY_FILE 4 SIZE

- OFFSET       0x00000004
- SIZE         0x37afbe10
  coverage file 0   100%   3122       3122       . offset 00000c32

What am I doing wrong? I saw someone suggesting to use umodel, but isn't it a tool to view 3d models?

Thanks

EDIT: actually I just manually selected all the lpk files and it worked fine. Now the question is: how to read the db files? According to the header it's a standard SQLite3 file, but with 16 reserved bytes at each page, which means they're using some kind of extension (not excluded an encrypting one). Also there are no bitmaps, but "loa" and "epf" files. Do you have any clue on what they are?

[tresTaq]

I unpacked a couple of large lpk files like data.lpk but didn't seem to find anything interesting there. can anyone point to where body\head or costumes meshes can be? or how to get umodel to work as it looks like it doesn't support this game yet. any help, please?

[elegos]

I've created a simple python script which attempts to create a sqlite3 connection to each EFGame_Extra/ClientData/TableData/*.db files, and it seems there are some valid and some invalid files.

For now the analysis only tries to analyze the readability of files, not their contents.

There seem to be 365 readable and 142 unreadable files.

This is the (simple enough) code used (to put in the root of the unpackaged client):

Code: Select all

from pathlib import Path
import sqlite3
from typing import List

tables_path = Path(__file__).parent.joinpath('EFGame_Extra', 'ClientData', 'TableData')

valid_table_files: List[str] = []
invalid_table_files: List[str] = []
for path in tables_path.glob('*.db'):
    try:
        con = sqlite3.connect(path.as_posix())
        cursor = con.cursor()

        cursor.execute("SELECT name FROM sqlite_master WHERE type='table';")
        result_tuples = cursor.fetchall()
        # print('\n'.join([f'- {tpl[0]}' for tpl in result_tuples]))

        cursor.close()
        con.close()

        valid_table_files.append(path.name)
    except:
        invalid_table_files.append(path.name)

print(f'Valid table files ({len(valid_table_files)}):')
# print('\n'.join([f'- {file}' for file in valid_table_files]))
print(f'Invalid table files ({len(invalid_table_files)}):')
# print('\n'.join([f'- {file}' for file in invalid_table_files]))

EDIT: and it seems the rows data is being generally encrypted. All the openable db files are empty tables!

Next step seems to understand how the client decrypts the database table data. Where should we start from? I'm not a reverse engineer and unfortunately I don't know the tools used to sniff memory at runtime.

[Htto]

I've created a simple python script which attempts to create a sqlite3 connection to each EFGame_Extra/ClientData/TableData/*.db files, and it seems there are some valid and some invalid files.

For now the analysis only tries to analyze the readability of files, not their contents.

There seem to be 365 readable and 142 unreadable files.

This is the (simple enough) code used (to put in the root of the unpackaged client):

Code: Select all

EDIT: and it seems the rows data is being generally encrypted. All the openable db files are empty tables!

Next step seems to understand how the client decrypts the database table data. Where should we start from? I'm not a reverse engineer and unfortunately I don't know the tools used to sniff memory at runtime.

I'm also working on this. I know that LaoTranslation.exe is capable of decrypting those .db files and I'm trying to see what that does
Does someone have a RU/JP/KR data.lpk file from a game client between October 2019 - September 2020? Would help a lot with this

[elegos]

I've downloaded the tool, and it seems it's using the standard SQLite Qt driver, which means it's indeed a standard SQLite3 database with encrypted data. This restricts the area of research, meaning that probably LostArk is using a standard SQLite3 plugin for encrypting data. Now we need to know (1) which plugin it is and (2) which keys are being used, hoping they're the same of the steam/western version.

EDIT: reading it in a hex editor gave me the idea that it downloads the patch, rather than patching the files (which makes sense). This means we're back to the game client again to know which keys are being used.