arkparse 0.4.4__tar.gz → 0.5.1__tar.gz

{arkparse-0.4.4 → arkparse-0.5.1}/.gitignore

@@ -185,3 +185,8 @@ temp
 **/exports/**
 **/**.ark.gz
 /src/.idea
+.venv**
+arkparse-fast-core/**
+/arkparse-fast-core
+
+**.zip

{arkparse-0.4.4 → arkparse-0.5.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: arkparse
-Version: 0.4.4
+Version: 0.5.1
 Summary: A package to parse and modify ark save files
 Project-URL: Homepage, https://github.com/VincentHenauGithub/ark-save-parser
 Author-email: Vincent Henau <vincent.henau.github@gmail.com>
@@ -11,6 +11,8 @@ Requires-Dist: matplotlib>=3.10.3
 Requires-Dist: numpy>=2.3.2
 Requires-Dist: pandas>=2.3.1
 Requires-Dist: rcon>=2.4.9
+Provides-Extra: fast
+Requires-Dist: arkparse-fast-core; extra == 'fast'
 Description-Content-Type: text/markdown
 
 # ArkParse: A Python Library for Reading and Modifying ARK Save Files
@@ -33,7 +35,6 @@ I mainly use this package for server management tasks. Some highlights include:
  - Sending random stats of the server to the chat
  - Monitoring player activity like when people log off and on and such
  - Randomly spawning bases with random loot for my friends to raid; probably my favorite feature (and the most complicated)
-If you're curious or want to explore the features yourself, you can find the project here: [ark-server-manager](https://github.com/VincentHenauGithub/ark-server-manager).
 
 Hope you find it useful or inspiring or both! 😊
 
@@ -63,17 +64,49 @@ I just hope it's usefull for someone!
 ---
 ## Installation
 
+### Standard Installation
+
 Install via pip:
 
 ```bash
 pip install arkparse
 ```
 
-Or install locally on your PC so you can add modifications
-First clone the repo and then run the following in repo directory:
+For faster parsing, install with the optional Rust accelerator (requires pre-built wheels to be available on PyPI):
+
+Current available wheels: MacOS, Windows and Linux; Python 3.9, 3.10, 3.11, 3.12, 3.13, 3.14 (3.13t and 3.14t)
 ```bash
+pip install arkparse[fast]
+```
+
+### Recommended python version: Free-Threaded Python (~40% speedup)
+
+ArkParse supports Python 3.13+ free-threaded builds (no-GIL), which provides additional performance benefits through parallel workloads. In my benchmarks this provides over **40% increased performance**, so I would highly recommend it!
+
+ArkParse automatically detects if you are using free-threaded Python and parallelizes intensive tasks accordingly — no code changes required on your end.
+
+Since free-threaded python is experimental I would advise using a virtual environment.
+
+### Development Installation
+
+For local development, clone and install in editable mode:
+
+```bash
+git clone https://github.com/VincentHenauGithub/ark-save-parser.git
+cd ark-save-parser
 pip install -e .
 ```
+
+**Optional: Rust Accelerator (up to ~20% increased performance single threaded mode and ~5% in free threaded mode)**
+
+The Rust accelerator is something I started and is still very much not finished, it currently provides some decent performance benefits but I would like to see this become a much larger benefit. I will update that package in the future.
+
+Requires [Rust](https://rustup.rs/) and [maturin](https://www.maturin.rs/):
+```bash
+git clone https://github.com/VincentHenauGithub/arkparse-fast-core.git
+cd arkparse-fast-core && pip install maturin && maturin develop --release
+```
+
 ---
 
 ### 4. **Quickstart**
@@ -233,7 +266,7 @@ This project is licensed under the MIT License. See the [LICENSE](LICENSE) file
 ## Feedback & Support
 
 - **Issues or Feature Requests**: Open an issue on the repo!
-- **Help**: If you need help for something specific, you can always messageme, I will try to help you out
+- **Help**: If you need help for something specific, you can always message me, I will try to help you out
 
 ## Donation
 

{arkparse-0.4.4 → arkparse-0.5.1}/README.md

@@ -18,7 +18,6 @@ I mainly use this package for server management tasks. Some highlights include:
  - Sending random stats of the server to the chat
  - Monitoring player activity like when people log off and on and such
  - Randomly spawning bases with random loot for my friends to raid; probably my favorite feature (and the most complicated)
-If you're curious or want to explore the features yourself, you can find the project here: [ark-server-manager](https://github.com/VincentHenauGithub/ark-server-manager).
 
 Hope you find it useful or inspiring or both! 😊
 
@@ -48,17 +47,49 @@ I just hope it's usefull for someone!
 ---
 ## Installation
 
+### Standard Installation
+
 Install via pip:
 
 ```bash
 pip install arkparse
 ```
 
-Or install locally on your PC so you can add modifications
-First clone the repo and then run the following in repo directory:
+For faster parsing, install with the optional Rust accelerator (requires pre-built wheels to be available on PyPI):
+
+Current available wheels: MacOS, Windows and Linux; Python 3.9, 3.10, 3.11, 3.12, 3.13, 3.14 (3.13t and 3.14t)
 ```bash
+pip install arkparse[fast]
+```
+
+### Recommended python version: Free-Threaded Python (~40% speedup)
+
+ArkParse supports Python 3.13+ free-threaded builds (no-GIL), which provides additional performance benefits through parallel workloads. In my benchmarks this provides over **40% increased performance**, so I would highly recommend it!
+
+ArkParse automatically detects if you are using free-threaded Python and parallelizes intensive tasks accordingly — no code changes required on your end.
+
+Since free-threaded python is experimental I would advise using a virtual environment.
+
+### Development Installation
+
+For local development, clone and install in editable mode:
+
+```bash
+git clone https://github.com/VincentHenauGithub/ark-save-parser.git
+cd ark-save-parser
 pip install -e .
 ```
+
+**Optional: Rust Accelerator (up to ~20% increased performance single threaded mode and ~5% in free threaded mode)**
+
+The Rust accelerator is something I started and is still very much not finished, it currently provides some decent performance benefits but I would like to see this become a much larger benefit. I will update that package in the future.
+
+Requires [Rust](https://rustup.rs/) and [maturin](https://www.maturin.rs/):
+```bash
+git clone https://github.com/VincentHenauGithub/arkparse-fast-core.git
+cd arkparse-fast-core && pip install maturin && maturin develop --release
+```
+
 ---
 
 ### 4. **Quickstart**
@@ -218,7 +249,7 @@ This project is licensed under the MIT License. See the [LICENSE](LICENSE) file
 ## Feedback & Support
 
 - **Issues or Feature Requests**: Open an issue on the repo!
-- **Help**: If you need help for something specific, you can always messageme, I will try to help you out
+- **Help**: If you need help for something specific, you can always message me, I will try to help you out
 
 ## Donation
 

{arkparse-0.4.4 → arkparse-0.5.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "arkparse"
-version = "0.4.4"
+version = "0.5.1"
 dependencies = [
     "rcon>=2.4.9",
     "numpy>=2.3.2",
@@ -22,6 +22,9 @@ classifiers = [
     # Add additional classifiers as needed
 ]
 
+[project.optional-dependencies]
+fast = ["arkparse-fast-core"]
+
 [project.urls]
 Homepage = "https://github.com/VincentHenauGithub/ark-save-parser"
 

arkparse-0.5.1/src/arkparse/__init__.py

@@ -0,0 +1,3 @@
+from .saves.asa_save import AsaSave
+from .classes import Classes
+from .parsing.struct.actor_transform import MapCoords, ActorTransform

{arkparse-0.4.4 → arkparse-0.5.1}/src/arkparse/api/base_api.py

@@ -18,9 +18,9 @@ from arkparse.utils import ImportFile
 from arkparse.logging import ArkSaveLogger
 
 class BaseApi(StructureApi):
-    def __init__(self, save, map: ArkMap):
+    def __init__(self, save, map: ArkMap = None):
         super().__init__(save)
-        self.map = map
+        self.map = map if map is not None else self.save.map
 
     def __get_closest_to(self, structures: Dict[UUID, Structure], coords: MapCoords):
         closest = None

{arkparse-0.4.4 → arkparse-0.5.1}/src/arkparse/api/dino_api.py

@@ -1,7 +1,10 @@
-from typing import Dict, List, Optional
+from typing import Dict, List, Optional, Tuple
 from uuid import UUID, uuid4
 from pathlib import Path
 import os
+import sys
+import threading
+from concurrent.futures import ThreadPoolExecutor
 
 from arkparse.object_model.cryopods.cryopod import Cryopod
 from arkparse.object_model.dinos.dino import Dino, DinoStats, DinoId
@@ -23,13 +26,26 @@ from arkparse.classes.dinos import Dinos
 from arkparse.object_model.misc.inventory import Inventory
 from arkparse.object_model.misc.inventory_item import InventoryItem
 from arkparse.object_model.dinos.pedigree import Pedigree
+from arkparse.logging.ark_save_logger import mark_as_worker_thread
+
+
+def _is_parallel_enabled() -> bool:
+    """Check if parallel parsing should be enabled (GIL is disabled)."""
+    if hasattr(sys, '_is_gil_enabled'):
+        return not sys._is_gil_enabled()
+    return False
+
+
+_PARALLEL_ENABLED = _is_parallel_enabled()
+
 
 class DinoApi:
     _DEFAULT_CONFIG = GameObjectReaderConfiguration(
         blueprint_name_filter=lambda name: \
             name is not None and \
-                (("Dinos/" in name and "_Character_" in name) or \
-                ("PrimalItem_WeaponEmptyCryopod" in name or "PrimalItem_SCSCryopod" in name)))
+                (((("/Creatures/" in name) or ("/Dinos/" in name) or ("/SDinoVariants/" in name)) and "_Character_" in name) or \
+                ("DinoCharacterStatusComponent" in name) or \
+                ("PrimalItem_WeaponEmptyCryopod" in name or "PrimalItem_SCSCryopod" in name or "ItemDinoball.ItemDinoball_C" in name)))
 
     def __init__(self, save: AsaSave):
         self.save = save
@@ -37,6 +53,13 @@ class DinoApi:
         self.parsed_dinos: Dict[UUID, Dino] = {}
         self.parsed_cryopods: Dict[UUID, Cryopod] = {}
 
+    @staticmethod
+    def _create_dino(uuid: UUID, save: AsaSave, is_tamed: bool, is_baby: bool, bypass_inventory: bool = True) -> Dino:
+        if is_tamed:
+            return TamedBaby(uuid, save=save) if is_baby else TamedDino(uuid, save=save, bypass_inventory=bypass_inventory)
+        else:
+            return Baby(uuid, save=save) if is_baby else Dino(uuid, save=save)
+
     @staticmethod
     def is_applicable_bp(blueprint: str) -> bool:
         return DinoApi._DEFAULT_CONFIG.blueprint_name_filter(blueprint)
@@ -66,99 +89,159 @@ class DinoApi:
         
         dino = None
         if "Dinos/" in object.blueprint and "_Character_" in object.blueprint:
-            is_tamed = object.get_property_value("TamedTimeStamp") is not None
-
             if uuid in self.parsed_dinos:
                 dino = self.parsed_dinos[uuid]
             else:
-                if is_tamed:
-                    dino = TamedDino(uuid, self.save)
-                else:
-                    dino = Dino(uuid, self.save)
+                is_tamed = object.get_property_value("TamedTimeStamp") is not None
+                is_baby = object.get_property_value("bIsBaby", False)
+                dino = DinoApi._create_dino(uuid, self.save, is_tamed, is_baby)
                 self.parsed_dinos[uuid] = dino
 
         return dino
 
-    def get_all(self, config = None, include_cryos: bool = True, include_wild: bool = True, include_tamed: bool = True, include_babies: bool = True, only_cryopodded: bool = False) -> Dict[UUID, Dino]:
+    def get_all(self, config = None, include_cryos: bool = True, include_wild: bool = True, include_tamed: bool = True, include_babies: bool = True, only_cryopodded: bool = False, max_workers: int = 6, bypass_inventory: bool = True) -> Dict[UUID, Dino]:
         ArkSaveLogger.api_log("Retrieving all dinos from save...")
 
         objects = self.get_all_objects(config)
+        
         dinos = {}
 
         if self.all_objects and len(objects) != len(self.all_objects):
             ArkSaveLogger.api_log(f"Found {len(objects)} dinos, parsing them... (and retrieving inventories)")
 
+        # Classify objects into categories for parallel processing
+        dino_objects_to_parse: List[Tuple[UUID, ArkGameObject, bool, bool]] = []  # (uuid, obj, is_tamed, is_baby)
+        cryopod_objects_to_parse: List[Tuple[UUID, ArkGameObject]] = []  # (uuid, obj)
+        
         for key, obj in objects.items():
-            try:
-                dino = None
-                if not only_cryopodded and "Dinos/" in obj.blueprint and "_Character_" in obj.blueprint:
-                    is_tamed = obj.get_property_value("TamedTimeStamp") is not None
-                    is_baby = obj.get_property_value("bIsBaby", False)
-
-                    if obj.uuid in self.parsed_dinos:
-                        if is_tamed and include_tamed:
-                            if is_baby and include_babies:
-                                dino = self.parsed_dinos[obj.uuid]
-                            else:
-                                dino = self.parsed_dinos[obj.uuid]
-                        elif not is_tamed and include_wild:
-                            if is_baby and include_babies:
-                                dino = self.parsed_dinos[obj.uuid]
-                            else:
-                                dino = self.parsed_dinos[obj.uuid]
-                    elif is_tamed and include_tamed:
-                        if is_baby and include_babies:
-                            dino = TamedBaby(obj.uuid, save=self.save)
-                        else:
-                            dino = TamedDino(obj.uuid, save=self.save)
-                        self.parsed_dinos[obj.uuid] = dino
-                    elif include_wild and not is_tamed:
-                        if is_baby and include_babies:
-                            dino = Baby(obj.uuid, save=self.save)
-                        else:
-                            dino = Dino(obj.uuid, save=self.save)
-                        self.parsed_dinos[obj.uuid] = dino
-
-                elif ("PrimalItem_SCSCryopod" in obj.blueprint or "PrimalItem_WeaponEmptyCryopod" in obj.blueprint) and include_cryos and include_tamed:
-                    if not obj.get_property_value("bIsEngram", default=False) and obj.get_property_value("CustomItemDatas") is not None:
-                        if obj.uuid in self.parsed_cryopods:
-                            is_baby = self.parsed_cryopods[obj.uuid].dino is not None and isinstance(self.parsed_cryopods[obj.uuid].dino, TamedBaby)
-                            if is_baby and include_babies:
-                                dino = self.parsed_cryopods[obj.uuid].dino
-                            else:
-                                dino = self.parsed_cryopods[obj.uuid].dino
-                            if dino is not None:
-                                self.parsed_dinos[dino.uuid] = dino
-                        else:
-                            try:
-                                cryopod = Cryopod(obj.uuid, save=self.save)
-                                self.parsed_cryopods[obj.uuid] = cryopod
-                                if cryopod.dino is not None:
-                                    dino = cryopod.dino
-                                    dino.is_cryopodded = True
-                            except Exception as e:
-                                if "Unsupported embedded data version" in str(e):
-                                    ArkSaveLogger.warning_log(f"Skipping cryopod {obj.uuid} due to unsupported embedded data version (pre Unreal 5.5)")
-                                    continue
-                                ArkSaveLogger.set_log_level(ArkSaveLogger.LogTypes.PARSER, True)
-                                cryopod = Cryopod(obj.uuid, save=self.save)
-                                ArkSaveLogger.set_log_level(ArkSaveLogger.LogTypes.PARSER, False)
-                                ArkSaveLogger.error_log(f"Error parsing cryopod {obj.uuid}: {e}")
-                                raise e
-                            finally:
-                                ArkSaveLogger.set_log_level(ArkSaveLogger.LogTypes.PARSER, False)
-                if dino is not None:
-                    dinos[key] = dino
-            except Exception as e:
-                if ArkSaveLogger._allow_invalid_objects:
-                    ArkSaveLogger.error_log(f"Failed to parse dino {obj.uuid}: {e}")
-                else:
-                    raise e
+            if not only_cryopodded and "Dinos/" in obj.blueprint and "_Character_" in obj.blueprint:
+                is_tamed = obj.get_property_value("TamedTimeStamp") is not None
+                is_baby = obj.get_property_value("bIsBaby", False)
+                
+                if obj.uuid in self.parsed_dinos:
+                    if (is_tamed and include_tamed) or (not is_tamed and include_wild):
+                        dinos[key] = self.parsed_dinos[obj.uuid]
+                elif (is_tamed and include_tamed) or (not is_tamed and include_wild):
+                    if not is_baby or include_babies:
+                        dino_objects_to_parse.append((key, obj, is_tamed, is_baby))
+            
+            elif ("PrimalItem_SCSCryopod" in obj.blueprint or "PrimalItem_WeaponEmptyCryopod" in obj.blueprint or "ItemDinoball.ItemDinoball_C" in obj.blueprint) and include_cryos and include_tamed:
+                if not obj.get_property_value("bIsEngram", default=False) and obj.get_property_value("CustomItemDatas") is not None:
+                    if obj.uuid in self.parsed_cryopods:
+                        self._collect_cached_cryopod(self.parsed_cryopods[obj.uuid], key, dinos, include_babies)
+                    else:
+                        cryopod_objects_to_parse.append((key, obj))
+        
+        # Parse dinos - parallel when GIL is disabled
+        if dino_objects_to_parse:
+            self._parse_dinos_batch(dino_objects_to_parse, dinos, max_workers, bypass_inventory)
+        
+        # Parse cryopods - parallel when GIL is disabled
+        if cryopod_objects_to_parse:
+            self._parse_cryopods_batch(cryopod_objects_to_parse, dinos, include_babies, max_workers)
 
         ArkSaveLogger.api_log(f"Parsed {len(dinos)} dinos")
 
         return dinos
-    
+
+    def _collect_cached_cryopod(self, cryopod: Cryopod, key: UUID, dinos: Dict[UUID, Dino], include_babies: bool):
+        if cryopod.dino is not None:
+            is_baby = isinstance(cryopod.dino, TamedBaby)
+            if include_babies or not is_baby:
+                dinos[key] = cryopod.dino
+                self.parsed_dinos[cryopod.dino.uuid] = cryopod.dino
+
+    def _collect_parsed_cryopod(self, cryopod: Cryopod, key: UUID, dinos: Dict[UUID, Dino], include_babies: bool):
+        self.parsed_cryopods[cryopod.uuid] = cryopod
+        if cryopod.dino is not None:
+            cryopod.dino.is_cryopodded = True
+            is_baby = isinstance(cryopod.dino, TamedBaby)
+            if include_babies or not is_baby:
+                dinos[key] = cryopod.dino
+                self.parsed_dinos[cryopod.dino.uuid] = cryopod.dino
+
+    def _parse_dinos_batch(self, dino_objects_to_parse: List[Tuple[UUID, ArkGameObject, bool, bool]], dinos: Dict[UUID, Dino], max_workers: int, bypass_inventory: bool = True):
+        if _PARALLEL_ENABLED and max_workers > 1:
+            ArkSaveLogger.api_log(f"Parsing {len(dino_objects_to_parse)} dinos with {max_workers} workers...")
+            save = self.save
+            _worker_initialized = threading.local()
+            error_count = [0]
+
+            def parse_single_dino(item):
+                if not getattr(_worker_initialized, 'done', False):
+                    mark_as_worker_thread()
+                    _worker_initialized.done = True
+                key, obj, is_tamed, is_baby = item
+                try:
+                    return (key, DinoApi._create_dino(obj.uuid, save, is_tamed, is_baby, bypass_inventory))
+                except Exception as e:
+                    error_count[0] += 1
+                    if error_count[0] <= 3:
+                        ArkSaveLogger.error_log(f"Parallel dino parse error #{error_count[0]}: {type(e).__name__}: {e}")
+                    return (key, None)
+
+            with ThreadPoolExecutor(max_workers=max_workers) as executor:
+                results = list(executor.map(parse_single_dino, dino_objects_to_parse))
+
+            for key, dino in results:
+                if dino is not None:
+                    dinos[key] = dino
+                    self.parsed_dinos[dino.uuid] = dino
+        else:
+            for key, obj, is_tamed, is_baby in dino_objects_to_parse:
+                try:
+                    dino = DinoApi._create_dino(obj.uuid, self.save, is_tamed, is_baby, bypass_inventory)
+                    dinos[key] = dino
+                    self.parsed_dinos[dino.uuid] = dino
+                except Exception as e:
+                    if ArkSaveLogger._allow_invalid_objects:
+                        ArkSaveLogger.error_log(f"Failed to parse dino {obj.uuid}: {e}")
+                    else:
+                        raise e
+
+    def _parse_cryopods_batch(self, cryopod_objects_to_parse: List[Tuple[UUID, ArkGameObject]], dinos: Dict[UUID, Dino], include_babies: bool, max_workers: int):
+        if _PARALLEL_ENABLED and max_workers > 1:
+            ArkSaveLogger.api_log(f"Parsing {len(cryopod_objects_to_parse)} cryopods with {max_workers} workers...")
+            save = self.save
+            _worker_initialized = threading.local()
+
+            def parse_single_cryopod(item):
+                if not getattr(_worker_initialized, 'done', False):
+                    mark_as_worker_thread()
+                    _worker_initialized.done = True
+                key, obj = item
+                try:
+                    return (key, Cryopod(obj.uuid, save=save), None)
+                except Exception as e:
+                    if "Unsupported embedded data version" in str(e):
+                        return (key, None, "unsupported_version")
+                    return (key, None, str(e))
+
+            with ThreadPoolExecutor(max_workers=max_workers) as executor:
+                cryo_results = list(executor.map(parse_single_cryopod, cryopod_objects_to_parse))
+
+            for key, cryopod, error in cryo_results:
+                if cryopod is not None:
+                    self._collect_parsed_cryopod(cryopod, key, dinos, include_babies)
+                elif error == "unsupported_version":
+                    ArkSaveLogger.warning_log(f"Skipping cryopod due to unsupported embedded data version (pre Unreal 5.5)")
+                elif error is not None:
+                    ArkSaveLogger.error_log(f"Error parsing cryopod: {error}")
+                    if not ArkSaveLogger._allow_invalid_objects:
+                        raise RuntimeError(f"Cryopod parse error: {error}")
+        else:
+            for key, obj in cryopod_objects_to_parse:
+                try:
+                    cryopod = Cryopod(obj.uuid, save=self.save)
+                    self._collect_parsed_cryopod(cryopod, key, dinos, include_babies)
+                except Exception as e:
+                    if "Unsupported embedded data version" in str(e):
+                        ArkSaveLogger.warning_log(f"Skipping cryopod {obj.uuid} due to unsupported embedded data version (pre Unreal 5.5)")
+                        continue
+                    ArkSaveLogger.error_log(f"Error parsing cryopod {obj.uuid}: {e}")
+                    if not ArkSaveLogger._allow_invalid_objects:
+                        raise e
+
     def get_at_location(self, map: ArkMap, coords: MapCoords, radius: float = 0.3, tamed: bool = True, untamed: bool = True) -> Dict[UUID, Dino]:
         dinos = self.get_all()
 

{arkparse-0.4.4 → arkparse-0.5.1}/src/arkparse/api/equipment_api.py

@@ -82,20 +82,23 @@ class EquipmentApi(GeneralApi):
         else:
             return None
     
-    def get_all(self, cls: "EquipmentApi.Classes", config: GameObjectReaderConfiguration = None) -> Dict[UUID, Equipment]:
+    def get_all(self, cls: "EquipmentApi.Classes", config: GameObjectReaderConfiguration = None, max_workers: int = 6) -> Dict[UUID, Equipment]:
         def is_valid(obj: ArkGameObject):
             is_engram = obj.get_property_value("bIsEngram")
             return not is_engram
         
+        allowed = set(self.__get_cls_filter(cls))
         _config = GameObjectReaderConfiguration(
-            blueprint_name_filter=lambda name: (True if config is None else config.blueprint_name_filter(name)) and name in self.__get_cls_filter(cls)
+            blueprint_name_filter=lambda name: (True if config is None else config.blueprint_name_filter(name)) and name in allowed
         )
 
-        return super().get_all(cls, valid_filter=is_valid, config=_config)
+        return super().get_all(cls, valid_filter=is_valid, config=_config, max_workers=max_workers)
     
     def get_by_class(self, cls: "EquipmentApi.Classes", classes: List[str]) -> Dict[UUID, Equipment]:
+        allowed = set(self.__get_cls_filter(cls))
+        classes_set = set(classes)
         config = GameObjectReaderConfiguration(
-            blueprint_name_filter=lambda name: name is not None and name in classes and name in self.__get_cls_filter(cls)
+            blueprint_name_filter=lambda name: name is not None and name in classes_set and name in allowed
         )
 
         return self.get_all(cls, config)

arkparse-0.5.1/src/arkparse/api/general_api.py

@@ -0,0 +1,106 @@
+from typing import Dict, List, Tuple
+from uuid import UUID
+import sys
+import threading
+from concurrent.futures import ThreadPoolExecutor
+
+from arkparse.logging.ark_save_logger import ArkSaveLogger, mark_as_worker_thread
+from arkparse.object_model.ark_game_object import ArkGameObject
+from arkparse.parsing import ArkBinaryParser
+from arkparse.saves.asa_save import AsaSave
+from arkparse.parsing import GameObjectReaderConfiguration
+from arkparse.object_model.misc.__parsed_object_base import ParsedObjectBase
+
+
+def _is_parallel_enabled() -> bool:
+    if hasattr(sys, '_is_gil_enabled'):
+        return not sys._is_gil_enabled()
+    return False
+
+
+_PARALLEL_ENABLED = _is_parallel_enabled()
+
+
+class GeneralApi:
+    def __init__(self, save: AsaSave, config: GameObjectReaderConfiguration= GameObjectReaderConfiguration()):
+        self.save = save
+        self.config = config
+        self.all_objects = None
+        self.parsed_objects: Dict[UUID, ParsedObjectBase] = {}
+
+    def get_all_objects(self, config: GameObjectReaderConfiguration = None) -> Dict[UUID, ArkGameObject]:
+        reuse = False
+        if config is None:
+            reuse = True
+            if self.all_objects is not None:
+                return self.all_objects
+
+            config = self.config
+
+        objects = self.save.get_game_objects(config)
+
+        if reuse:
+            self.all_objects = objects
+
+        return objects
+    
+    def get_all(self, constructor, valid_filter = None, config = None, max_workers: int = 4) -> Dict[UUID, object]:
+        objects = self.get_all_objects(config)
+
+        parsed = {}
+        to_parse: List[Tuple[UUID, ArkGameObject]] = []
+
+        for key, obj in objects.items():
+            if valid_filter and not valid_filter(obj):
+                continue
+
+            if key in self.parsed_objects:
+                parsed[key] = self.parsed_objects[key]
+            else:
+                to_parse.append((key, obj))
+
+        self._parse_batch(to_parse, parsed, constructor, max_workers)
+
+        return parsed
+
+    def _parse_batch(self, to_parse: List[Tuple[UUID, ArkGameObject]], parsed: Dict, constructor, max_workers: int):
+        if _PARALLEL_ENABLED and max_workers > 1 and len(to_parse) > 0:
+            ArkSaveLogger.api_log(f"Parsing {len(to_parse)} objects with {max_workers} workers...")
+            save = self.save
+            _worker_initialized = threading.local()
+            error_count = [0]
+
+            def parse_single(item):
+                if not getattr(_worker_initialized, 'done', False):
+                    mark_as_worker_thread()
+                    _worker_initialized.done = True
+                key, obj = item
+                try:
+                    return (key, constructor(obj.uuid, save), None)
+                except Exception as e:
+                    error_count[0] += 1
+                    if error_count[0] <= 3:
+                        ArkSaveLogger.error_log(f"Parallel parse error #{error_count[0]}: {type(e).__name__}: {e}")
+                    return (key, None, str(e))
+
+            with ThreadPoolExecutor(max_workers=max_workers) as executor:
+                results = list(executor.map(parse_single, to_parse))
+
+            for key, obj, error in results:
+                if obj is not None:
+                    parsed[key] = obj
+                    self.parsed_objects[key] = obj
+                elif error is not None:
+                    if not ArkSaveLogger._allow_invalid_objects:
+                        raise RuntimeError(f"Parse error for {key}: {error}")
+        else:
+            for key, obj in to_parse:
+                try:
+                    parsed[key] = constructor(obj.uuid, self.save)
+                    self.parsed_objects[key] = parsed[key]
+                except Exception as e:
+                    if ArkSaveLogger._allow_invalid_objects:
+                        ArkSaveLogger.error_log(f"Failed to parse object {obj.uuid}: {e}")
+                    else:
+                        raise e
+    

{arkparse-0.4.4 → arkparse-0.5.1}/src/arkparse/api/stackable_api.py

@@ -25,13 +25,13 @@ class StackableApi(GeneralApi):
         config = StackableApi._DEFAULT_CONFIG
         super().__init__(save, config)
     
-    def get_all(self, cls: "StackableApi.Classes", config = None) -> Dict[UUID, InventoryItem]:
+    def get_all(self, cls: "StackableApi.Classes", config = None, max_workers: int = 4) -> Dict[UUID, InventoryItem]:
         def is_valid(obj: ArkGameObject):
             is_bp = obj.get_property_value("bIsBlueprint")
             is_engram = obj.get_property_value("bIsEngram")
             return not (is_bp or is_engram)
 
-        return super().get_all(cls, valid_filter=is_valid, config=config)
+        return super().get_all(cls, valid_filter=is_valid, config=config, max_workers=max_workers)
     
     def get_by_class(self, cls: "StackableApi.Classes", classes: List[str]) -> Dict[UUID, InventoryItem]:
         config = GameObjectReaderConfiguration(