django-fast-treenode 2.1.2__py3-none-any.whl → 2.1.4__py3-none-any.whl

{django_fast_treenode-2.1.2.dist-info → django_fast_treenode-2.1.4.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: django-fast-treenode
-Version: 2.1.2
+Version: 2.1.4
 Summary: Application for supporting tree (hierarchical) data structure in Django projects
 Home-page: https://django-fast-treenode.readthedocs.io/
 Author: Timur Kady
@@ -55,19 +55,25 @@ License-File: LICENSE
 Requires-Dist: Django>=4.0
 Requires-Dist: pympler>=1.0
 Requires-Dist: django-widget-tweaks>=1.5
+Requires-Dist: msgpack>=1.1
 Provides-Extra: import-export
 Requires-Dist: openpyxl; extra == "import-export"
 Requires-Dist: pyyaml; extra == "import-export"
 Requires-Dist: xlsxwriter; extra == "import-export"
 
 # Django-fast-treenode 
-**Combining Adjacency List and Closure Table for Optimal Performance**
+**Hybrid Tree Storage**
 
+[![Tests](https://github.com/TimurKady/django-fast-treenode/actions/workflows/test.yaml/badge.svg?branch=main)](https://github.com/TimurKady/django-fast-treenode/actions/workflows/test.yaml)
+[![Docs](https://readthedocs.org/projects/django-fast-treenode/badge/?version=latest)](https://django-fast-treenode.readthedocs.io/)
+[![PyPI](https://img.shields.io/pypi/v/django-fast-treenode.svg)](https://pypi.org/project/django-fast-treenode/)
+[![Published on Django Packages](https://img.shields.io/badge/Published%20on-Django%20Packages-0c3c26)](https://djangopackages.org/packages/p/django-fast-treenode/)
+[![Sponsor](https://img.shields.io/github/sponsors/TimurKady)](https://github.com/sponsors/TimurKady)
 
-**Django Fast TreeNode** is a high-performance Django application for working with tree structures, combining **Adjacency List** and **Closure Table** models. Each **TreeNodeModel** instance maintains two synchronized tables, enabling most operations to be performed with a single database query.
+**Django Fast TreeNode** is a high-performance Django application for working with tree structures. 
 
 ## Features
-- **Hybrid storage model**: Combines Adjacency List and Closure Table for optimal performance.
+- **Hybrid storage model**: Combines Adjacency List and Materialized Path (versions 2.2 and above) Closure Table (versions 2.1 and earlier) for optimal performance.
 - **Custom caching system**: A built-in caching mechanism, specifically designed for this package, significantly boosts execution speed.
 - **Efficient queries**: Retrieve ancestors, descendants, breadcrumbs, and tree depth with only one SQL queriy.
 - **Bulk operations**: Supports fast insertion, movement, and deletion of nodes.
@@ -75,6 +81,8 @@ Requires-Dist: xlsxwriter; extra == "import-export"
 - **Admin panel integration**: Full compatibility with Django's admin panel, allowing intuitive management of tree structures.
 - **Import & Export functionality**: Built-in support for importing and exporting tree structures in multiple formats (CSV, JSON, XLSX, YAML, TSV), including integration with the Django admin panel.
 
+It seems that django-fast-treenode is currently the most balanced and performant solution for most tasks, especially those related to dynamic hierarchical data structures. Check out the results of (comparison tests)[#] with other Django packages.
+
 ## Use Cases
 Django Fast TreeNode is suitable for a wide range of applications, from simple directories to complex systems with deep hierarchical structures:
 - **Categories and taxonomies**: Manage product categories, tags, and classification systems.
@@ -155,4 +163,4 @@ Released under [MIT License](https://github.com/TimurKady/django-fast-treenode/b
 ## Credits
 Thanks to everyone who contributed to the development and testing of this package, as well as the Django community for their inspiration and support. 
 
-Special thanks to [Fabio Caccamo](https://github.com/fabiocaccamo) for the idea behind creating a fast Django application for handling hierarchies and [Mathieu Leplatre](https://github.com/leplatrem) for the advice used in writing this application.
+Special thanks to [Fabio Caccamo](https://github.com/fabiocaccamo) for the idea behind creating a fast Django application for handling hierarchies.

{django_fast_treenode-2.1.2.dist-info → django_fast_treenode-2.1.4.dist-info}/RECORD

@@ -1,10 +1,10 @@
-treenode/__init__.py,sha256=CZ-0uZyhi8OZJP8xmqbTqXrZUjcQ4SNAXhKFgM0qw2M,99
+treenode/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 treenode/apps.py,sha256=a7UasXiZZudPccjmHEudP79TkhR_53Mvnb-dBXLHRRQ,862
-treenode/cache.py,sha256=utTmMJ87fjbutaKbOcSp8bHqIDbI_Yr2nPXJoZLAqlQ,7213
+treenode/cache.py,sha256=GoN2J-ypEQWIK05WSw9LYo7boKHGPXNFxqHorFPUqX8,12481
 treenode/forms.py,sha256=Mjrpuyd1CPsitcElDVagE3k-p2kU4xIlRuy1f5Zgt3c,3800
 treenode/signals.py,sha256=ERrlKjGqhYaPYVKKRk1JBBlPFOmJKpJ6bXsJavcTlo0,518
 treenode/urls.py,sha256=CsgX0hRyDVrMS8YnRlr_CxmDlgGIhDpqZ9ldoMYZCac,866
-treenode/version.py,sha256=Q1ilorcSAKP3-SntEXcfmFMj0ClySSAldpayaO22S8E,222
+treenode/version.py,sha256=QArF1TMhRPActA4pib2dl6jP5b51jCmD6-Wh2ROUD78,222
 treenode/views.py,sha256=rEZEgdbEA3AJDHrvtrAm-t60QTJcJ4JEhNsNMR1Y_I4,5549
 treenode/widgets.py,sha256=Mi0F-AK_UcmU6C50ENK9vv6xGQNuDtrtzXSnXSOXhLM,4760
 treenode/admin/__init__.py,sha256=TdlPIyRW8i9qTVqGLmLWiBw4DyoGHUYZErE6rCyGOPE,119
@@ -12,23 +12,23 @@ treenode/admin/admin.py,sha256=6H3N2Dg6l-MrFwIcyKR5YENg0cEo-I4uKCP9MuhHkqo,10580
 treenode/admin/changelist.py,sha256=YZm3zNniX75CgLjnbHpVr0OIP91halDEBHmrcS8m5Og,2128
 treenode/admin/mixins.py,sha256=-dVZwEjKsfRzMkBe87dkI0SZ9MH45YE_o39SIhSJWy4,11194
 treenode/managers/__init__.py,sha256=EG_tj9P1Hama3kaqMfHck4lfzUWoPaJJVOXe3qaKMUo,585
-treenode/managers/adjacency.py,sha256=NVN8dq5z7gIh90yqW2uxV7MokmUfXTOT7crqMDyMaH0,7889
+treenode/managers/adjacency.py,sha256=OOjHCSTo0aAcSxOOwz7OsQTGdTRkM1mAxSN7jlzRpho,7896
 treenode/managers/closure.py,sha256=PcScdJJUnLcKe8Y1wqROYPsRtAnBMUO4xn5sILk9AIM,10638
 treenode/models/__init__.py,sha256=pBiMlEpC_Thh7asraNzA7W_7BKu2oAHtcn-K6_sdJe8,112
-treenode/models/adjacency.py,sha256=ijStfIQDSd48L3nA8OnLD1nHGYo5YsnokqUVfzDt68w,12422
+treenode/models/adjacency.py,sha256=QWGOidd4tH3afqVedPNQqeh-W-zUTNs1m-iAhCAXub4,12396
 treenode/models/classproperty.py,sha256=J4W6snsfsEUSHKHkIlM9yOJYQ_FSrp3P3oEYMKJengg,571
-treenode/models/closure.py,sha256=NEC8pi9QIxNtUnctEi0lPHBHEPX2K3V1oeSs9JrDGA0,4588
+treenode/models/closure.py,sha256=eZtLbnCOR1xYWhgbo1Pml_K0Pd0MM2DjiZl3SWMVe2A,3712
 treenode/models/factory.py,sha256=10FEGGC5PGWaR58qErs0oOrCS0KeI8x9H-SknZAAWqw,2291
 treenode/models/mixins/__init__.py,sha256=gTdMZFh1slNHMvxrnu-hGl46xqnWd4W7TOEFWTVJq40,757
-treenode/models/mixins/ancestors.py,sha256=_nF8V99SBjT-G88BmcRtiTC0DgSEIgHrI0g02jqTnss,2075
-treenode/models/mixins/children.py,sha256=OchaH6m6pOr6uuiZRRBHoZXoCSWM-ENTNWu1iLHXaBM,2564
-treenode/models/mixins/descendants.py,sha256=2mhnIhC8VJomTqntzzAwFFW_CcMiwujzQoD5_mfMsK0,2208
+treenode/models/mixins/ancestors.py,sha256=QZywMcIVZK82j13QsgevVN2ZhRLa86DfRIt2BsiM2to,1526
+treenode/models/mixins/children.py,sha256=xgenQFyZBG7_S33QQlznSmNhXEdeo9DeLyi7dKmvFhw,2637
+treenode/models/mixins/descendants.py,sha256=PYYfd7oqlv3Gnfahm0u9ACHjpWSDNM6Z8oJaJXPVQ8w,1910
 treenode/models/mixins/family.py,sha256=h2IRRADkQxve97QqBHKv0evVz4cFQtcNR8CbPi9Ri_w,1645
 treenode/models/mixins/logical.py,sha256=jlhBSq3AfCYNyNjqyKM9siyioS3SYcGD-aG2b4MV2RM,2169
-treenode/models/mixins/node.py,sha256=E-eUgZoqKlyaCaDk0H7VQg6tDnfeufyGHn7hjTxIDqM,7694
+treenode/models/mixins/node.py,sha256=VpLiFI1olvj5Gp2yV4n-aG4z4mZ7vOS6ytloWwO5s6w,7149
 treenode/models/mixins/properties.py,sha256=pfv80KLXcPeGx00IFCBcst1_cf0AmzhjshFjq1XQWMY,3876
 treenode/models/mixins/roots.py,sha256=MoFQq1fph70awc26UMUbfeTpt0ToUOvMz1c7LlDyIP8,2956
-treenode/models/mixins/siblings.py,sha256=fh0ZrlFXKxOQ4Qrp6sElTMvRhU5PyRRykLHDcbH-3Rk,3113
+treenode/models/mixins/siblings.py,sha256=JTQjaxnDH9t-AVMCQFiuc0nHLdIsE4v5vJ5z6LcUZLY,3236
 treenode/models/mixins/tree.py,sha256=CsO0ynwcwkrWgQbTzvF4yws-y7n1GGM2zImJH0hgV00,13042
 treenode/static/.gitkeep,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 treenode/static/treenode/.gitkeep,sha256=frcCV1k9oG9oKj3dpUqdJg1PxRT2RSN_XKdLCPjaYaY,2
@@ -48,7 +48,6 @@ treenode/templates/admin/tree_node_import_report.html,sha256=azHJ8JFrSRu60lF1Uh2
 treenode/templates/admin/treenode_ajax_rows.html,sha256=zFyPaTbSyxRjOqQ85SMv__qTIYDjEna6chYODBypDZA,224
 treenode/templates/widgets/tree_widget.css,sha256=2bEaxu1x7QJZ7erbs2SLMaxeaiMkjQXadfcDEW8wfok,551
 treenode/templates/widgets/tree_widget.html,sha256=GKcCU-B2FkkJ2BSOuXOw9e_PdYTtADcvyITEXqOlZ9Y,723
-treenode/tests/tests.py,sha256=9Bd2BhvwtVhYBp5DEtkzKPpAP1iFo4asMsydzuIRASM,19316
 treenode/utils/__init__.py,sha256=B4bv96ivtHELPv0_DllJa5z-k1QMo7z-MKuvj-3NdtI,356
 treenode/utils/aid.py,sha256=o8Jgc1vDRtQpx4XYdv0qR5Lqvens55Jfbdca1nr-EOA,1013
 treenode/utils/base16.py,sha256=U1PMit2aZOpYusG_u1c7eVpXO-cFrFPyVyk9zdHrehg,817
@@ -57,8 +56,8 @@ treenode/utils/db.py,sha256=36q4OckKmEd6uHTbMTxdKpV9nOIZ55DAantRWR9bxWg,4297
 treenode/utils/exporter.py,sha256=mV6Gch7XfW8f_1x3WqWgtV0qekMLdo-_n9gz6GJjXjw,7259
 treenode/utils/importer.py,sha256=Hvirbd6NyZ2MHa56_jOrUF3kYFeby1DbSLR3mhHy-9s,12891
 treenode/utils/radix.py,sha256=zHpOuDxsebiv9Gza6snNhAtBKiex6CDrAVRtB6esaWo,1642
-django_fast_treenode-2.1.2.dist-info/LICENSE,sha256=T0evsb7y-63fg18ovdNSx3wwWWRwyluQvN9J4zFSvfE,1093
-django_fast_treenode-2.1.2.dist-info/METADATA,sha256=68M56fkOFsBldq0sybbmXF0mG0m3HDHvsDs6Ofk9cDU,7558
-django_fast_treenode-2.1.2.dist-info/WHEEL,sha256=EaM1zKIUYa7rQnxGiOCGhzJABRwy4WO57rWMR3_tj4I,91
-django_fast_treenode-2.1.2.dist-info/top_level.txt,sha256=fmgxHbXyx1O2MPi_9kjx8aL9L-8TmV0gre4Go8XgqFk,9
-django_fast_treenode-2.1.2.dist-info/RECORD,,
+django_fast_treenode-2.1.4.dist-info/LICENSE,sha256=T0evsb7y-63fg18ovdNSx3wwWWRwyluQvN9J4zFSvfE,1093
+django_fast_treenode-2.1.4.dist-info/METADATA,sha256=UoyzLdrnOMH4yEGblkK2SI8IK-mYu_lHqW5GDqOM-0U,8273
+django_fast_treenode-2.1.4.dist-info/WHEEL,sha256=beeZ86-EfXScwlR_HKu4SllMC9wUEj_8Z_4FJ3egI2w,91
+django_fast_treenode-2.1.4.dist-info/top_level.txt,sha256=fmgxHbXyx1O2MPi_9kjx8aL9L-8TmV0gre4Go8XgqFk,9
+django_fast_treenode-2.1.4.dist-info/RECORD,,

{django_fast_treenode-2.1.2.dist-info → django_fast_treenode-2.1.4.dist-info}/WHEEL

@@ -1,5 +1,5 @@
 Wheel-Version: 1.0
-Generator: setuptools (75.9.1)
+Generator: setuptools (76.1.0)
 Root-Is-Purelib: true
 Tag: py3-none-any
 

treenode/__init__.py

@@ -1,5 +0,0 @@
-"""
-Django Fast TreeNode.
-
-📖 Documentation: https://django-fast-treenode.readthedocs.io/
-"""

treenode/cache.py

@@ -12,218 +12,339 @@ Features:
 - Automatic cache eviction when memory limits are exceeded.
 - Decorator `@cached_method` for caching method results.
 
-Version: 2.0.0
+Version: 2.2.0
 Author: Timur Kady
 Email: timurkady@yandex.com
 """
 
-
-from django.core.cache import caches
-from django.conf import settings
-import threading
 import hashlib
-import json
-import logging
-from pympler import asizeof
+import msgpack
+import sys
+import threading
+from collections import deque, defaultdict, OrderedDict
+from django.conf import settings
+from django.core.cache import caches
+from functools import lru_cache
+from functools import wraps
 
-from .utils.base36 import to_base36
 
-logger = logging.getLogger(__name__)
+# ---------------------------------------------------
+# Utilities
+# ---------------------------------------------------
+
+_DIGITS = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+_CLEARINT_THESHOLD = 0.8
+_EVICT_INTERVAL = 50
+
+
+@lru_cache(maxsize=1000)
+def to_base36(num):
+    """
+    Convert an integer to a base36 string.
+
+    For example: 10 -> 'A', 35 -> 'Z', 36 -> '10', etc.
+    """
+    if num == 0:
+        return '0'
+    sign = '-' if num < 0 else ''
+    num = abs(num)
+    result = []
+    while num:
+        num, rem = divmod(num, 36)
+        result.append(_DIGITS[rem])
+    return sign + ''.join(reversed(result))
 
 
 # ---------------------------------------------------
 # Caching
 # ---------------------------------------------------
 
-class TreeNodeCache:
-    """Singleton-класс для управления кэшем TreeNode."""
+class TreeCache:
+    """Singleton class for managing the TreeNode cache."""
 
     _instance = None
-    _lock = threading.Lock()
-    _keys = dict()
-    _total_size = 0
-    _cache_limit = 0
-
-    def __new__(cls):
-        """Create only one instance of the class (Singleton)."""
-        with cls._lock:
+    _instance_lock = threading.Lock()
+
+    def __new__(cls, *args, **kwargs):
+        """Singleton new."""
+        with cls._instance_lock:
             if cls._instance is None:
-                cls._instance = super(TreeNodeCache, cls).__new__(cls)
-                cls._instance._initialize()
+                cls._instance = super(TreeCache, cls).__new__(cls)
         return cls._instance
 
-    def _initialize(self):
-        """Initialize cache."""
-        self.cache_timeout = None
-        limit = getattr(settings, 'TREENODE_CACHE_LIMIT', 100)*1024*1024
-        self._cache_limit = limit
-        self.cache_timeout = None
-        cache_name = 'treenode' if 'treenode' in settings.CACHES else 'default'
-        self.cache = caches[cache_name]
-        self._total_size = 0
-        self.cache.clear()
+    def __init__(self, cache_limit=100 * 1024 * 1024):
+        """
+        Initialize the cache.
+
+        If the 'treenode' key is present in settings.CACHES, the corresponding
+        backend is used.
+        Otherwise, the custom dictionary is used.
+        The cache size (in bytes) is taken from
+        settings.TREENODE_CACHE_LIMIT (MB), by default 100 MB.
+        """
+        if hasattr(self, '_initialized') and self._initialized:
+            return
+
+        # Get the cache limit (MB), then convert to bytes.
+        cache_limit_mb = getattr(settings, 'TREENODE_CACHE_LIMIT', 100)
+        self.cache_limit = cache_limit_mb * 1024 * 1024
+
+        # Select backend: if there is 'treenode' in settings.CACHES, use it.
+        # Otherwise, use our own dictionary.
+        if hasattr(settings, 'CACHES') and 'treenode' in settings.CACHES:
+            self.cache = caches['treenode']
+        else:
+            # We use our dictionary as a backend.
+            self.cache = OrderedDict()
+
+        self.order = deque()            # Queue for FIFO implementation.
+        self.total_size = 0             # Current cache size in bytes.
+        self.lock = threading.Lock()    # Lock for thread safety.
+
+        # Additional index for fast search of keys by prefix
+        # Format: {prefix: {key1, key2, ...}}
+        self.prefix_index = defaultdict(set)
+        # Dictionary to store the sizes of each key (key -> size in bytes)
+        self.sizes = {}
+        # Dictionary to store the prefix for each key to avoid repeated
+        # splitting
+        self.key_prefix = {}
+
+        # Counter for number of set operations for periodic eviction
+        self._set_counter = 0
+        # Evict cache every _evict_interval set operations when using external
+        # backend
+        self._evict_interval = _EVICT_INTERVAL
+
+        self._initialized = True
 
     def generate_cache_key(self, label, func_name, unique_id, *args, **kwargs):
         """
-        Generate Cache Key.
-
-        Generates a cache key of the form:
-            <model_name>_<func_name>_<id>_<hash>,
-        where <hash> is calculated from the function parameters
-        (args and kwargs).
-        If the parameters can be serialized via JSON, use this, otherwise we
-        use repr to generate the string.
+        Generate a cache key.
+
+        <label>_<func_name>_<unique_id>_<hash>
         """
-        try:
-            # Sort dictionary keys to ensure determinism.
-            params_repr = json.dumps(
-                (args, kwargs),
-                sort_keys=True,
-                default=str
-            )
-        except (TypeError, ValueError) as e:
-            # If JSON serialization fails, use repr.
-            params_repr = repr((args, kwargs))
-            logger.warning(f"Failed to serialize cache key params: {e}")
-
-        # Calculate the MD5 hash from the received string.
-        hash_value = hashlib.sha256(params_repr.encode("utf-8")).hexdigest()
-
-        # Forming the final key.
-        cache_key = f"{label}_{func_name}_{unique_id}_{hash_value}"
-
-        return cache_key
+        # If using custom dict backend, use simple key generation without
+        # serialization.
+        if isinstance(self.cache, dict):
+            sorted_kwargs = sorted(kwargs.items())
+            return f"{label}_{func_name}_{unique_id}_{args}_{sorted_kwargs}"
+        else:
+            try:
+                # Using msgpack for fast binary representation of arguments
+                sorted_kwargs = sorted(kwargs.items())
+                params_bytes = msgpack.packb(
+                    (args, sorted_kwargs), use_bin_type=True)
+            except Exception:
+                params_bytes = repr((args, kwargs)).encode('utf-8')
+            # Using MD5 for speed (no cryptographic strength)
+            hash_value = hashlib.md5(params_bytes).hexdigest()
+            return f"{label}_{func_name}_{unique_id}_{hash_value}"
 
     def get_obj_size(self, value):
-        """Determine the size of the object in bytes."""
-        try:
-            return len(json.dumps(value).encode("utf-8"))
-        except (TypeError, ValueError):
-            return asizeof.asizeof(value)
-
-    def cache_size(self):
-        """Return the total size of the cache in bytes."""
-        return self._total_size
-
-    def set(self, cache_key, value):
-        """Push to cache."""
-        size = self.get_obj_size(value)
-        self.cache.set(cache_key, value, timeout=self.cache_timeout)
-
-        # Update cache size
-        if cache_key in self._keys:
-            self._total_size -= self._keys[cache_key]
-        self._keys[cache_key] = size
-        self._total_size += size
-
-        # Check if the limit has been exceeded
-        self._evict_cache()
-
-    def get(self, cache_key):
-        """Get from cache."""
-        return self.cache.get(cache_key)
-
-    def invalidate(self, label):
-        """Clear cache for a specific model only."""
-        prefix = f"{label}_"
-        keys_to_remove = [key for key in self._keys if key.startswith(prefix)]
-        for key in keys_to_remove:
-            self.cache.delete(key)
-            self._total_size -= self._keys.pop(key, 0)
-            if self._total_size < 0:
-                self._total_size = 0
+        """
+        Determine the size of the object in bytes.
 
-    def clear(self):
-        """Full cache clearing."""
-        self.cache.clear()
-        self._keys.clear()
-        self._total_size = 0
+        If the value is already in bytes or bytearray, simply returns its
+        length. Otherwise, uses sys.getsizeof for an approximate estimate.
+        """
+        if isinstance(value, (bytes, bytearray)):
+            return len(value)
+        return sys.getsizeof(value)
 
-    def _evict_cache(self):
-        """Delete old entries if the cache has exceeded the limit."""
-        if self._total_size <= self._cache_limit:
-            # If the size is within the limit, do nothing
-            return
+    def set(self, key, value):
+        """
+        Store the value in the cache.
+
+        Stores the value in the cache, updates the FIFO queue, prefix index,
+        size dictionary, and total cache size.
+        """
+        # Idea 1: Store raw object if using custom dict backend, otherwise
+        # serialize using msgpack.
+        if isinstance(self.cache, dict):
+            stored_value = value
+        else:
+            try:
+                stored_value = msgpack.packb(value, use_bin_type=True)
+            except Exception:
+                stored_value = value
+
+        # Calculate the size of the stored value
+        if isinstance(stored_value, (bytes, bytearray)):
+            size = len(stored_value)
+        else:
+            size = sys.getsizeof(stored_value)
 
-        if not self._keys:
-            self.clear()
+        # Store the value in the cache backend
+        if isinstance(self.cache, dict):
+            self.cache[key] = stored_value
+        else:
+            self.cache.set(key, stored_value)
+
+        # Update internal structures under lock
+        with self.lock:
+            if key in self.sizes:
+                # If the key already exists, adjust the total size
+                old_size = self.sizes[key]
+                self.total_size -= old_size
+            else:
+                # New key: add to FIFO queue
+                self.order.append(key)
+                # Compute prefix once and store it in key_prefix
+                if "_" in key:
+                    prefix = key.split('_', 1)[0] + "_"
+                else:
+                    prefix = key
+                self.key_prefix[key] = prefix
+                self.prefix_index[prefix].add(key)
+            # Save the size for this key and update total_size
+            self.sizes[key] = size
+            self.total_size += size
+
+            # Increment the set counter for periodic eviction
+            self._set_counter += 1
+
+        # Idea 3: If using external backend, evict cache every _evict_interval
+        # sets. Otherwise, always evict immediately.
+        if self._set_counter >= self._evict_interval:
+            with self.lock:
+                self._set_counter = 0
+            self._evict_cache()
+
+    def get(self, key):
+        """
+        Get a value from the cache by key.
 
-        logger.warning(f"Cache limit exceeded! Current size: \
-{self._total_size}, Limit: {self._cache_limit}")
+        Quickly retrieves a value from the cache by key.
+        Here we simply request a value from the backend (either a dictionary or
+        Django cache-backend) and return it without any additional operations.
+        """
+        if isinstance(self.cache, dict):
+            return self.cache.get(key)
+        else:
+            packed_value = self.cache.get(key)
+            if packed_value is None:
+                return None
+            try:
+                return msgpack.unpackb(packed_value, raw=False)
+            except Exception:
+                # If unpacking fails, return what we got
+                return packed_value
+
+    def invalidate(self, prefix):
+        """
+        Invalidate model cache.
+
+        Quickly removes all items from the cache whose keys start with prefix.
+        Uses prefix_index for instant access to keys.
+        When removing, each key's size is retrieved from self.sizes,
+        and total_size is reduced by the corresponding amount.
+        """
+        prefix += '_'
+        with self.lock:
+            keys_to_remove = self.prefix_index.get(prefix, set())
+            if not keys_to_remove:
+                return
+
+            # Remove keys from main cache and update total_size via sizes
+            # dictionary
+            if isinstance(self.cache, dict):
+                for key in keys_to_remove:
+                    self.cache.pop(key, None)
+                    size = self.sizes.pop(key, 0)
+                    self.total_size -= size
+                    # Remove key from key_prefix as well
+                    self.key_prefix.pop(key, None)
+            else:
+                # If using Django backend
+                self.cache.delete_many(list(keys_to_remove))
+                for key in keys_to_remove:
+                    size = self.sizes.pop(key, 0)
+                    self.total_size -= size
+                    self.key_prefix.pop(key, None)
+
+            # Remove prefix from index and update FIFO queue
+            del self.prefix_index[prefix]
+            self.order = deque(k for k in self.order if k not in keys_to_remove)
 
-        # Sort keys by insertion order (FIFO)
-        keys_sorted = list(self._keys.keys())
+    def clear(self):
+        """Clear cache completely."""
+        with self.lock:
+            if isinstance(self.cache, dict):
+                self.cache.clear()
+            else:
+                self.cache.clear()
+            self.order.clear()
+            self.prefix_index.clear()
+            self.sizes.clear()
+            self.key_prefix.clear()
+            self.total_size = 0
 
-        keys_to_delete = []
-        freed_size = 0
+    def _evict_cache(self):
+        """
+        Perform FIFO cache evacuation.
 
-        # Delete old keys until we reach the limit
-        for key in keys_sorted:
-            freed_size += self._keys[key]
-            keys_to_delete.append(key)
-            if self._total_size - freed_size <= self._cache_limit:
-                break
+        Removes old items until the total cache size is less than
+        _CLEARINT_THESHOLD of the limit.
+        """
+        with self.lock:
+            # Evict until total_size is below 80% of cache_limit
+            target_size = _CLEARINT_THESHOLD * self.cache_limit
+            while self.total_size > target_size and self.order:
+                # Extract the oldest key from the queue (FIFO)
+                key = self.order.popleft()
 
-        # Delete keys in batches (delete_many)
-        self.cache.delete_many(keys_to_delete)
+                # Delete entry from backend cache
+                if isinstance(self.cache, dict):
+                    self.cache.pop(key, None)
+                else:
+                    self.cache.delete(key)
 
-        # Update data in `_keys` and `_total_size`
-        for key in keys_to_delete:
-            self._total_size -= self._keys.pop(key, 0)
+                # Extract the size of the entry to be deleted and reduce
+                # the overall cache size
+                size = self.sizes.pop(key, 0)
+                self.total_size -= size
 
-        logger.info(f"Evicted {len(keys_to_delete)} keys from cache, \
-freed {freed_size} bytes.")
+                # Retrieve prefix from key_prefix without splitting
+                prefix = self.key_prefix.pop(key, None)
+                if prefix is not None:
+                    self.prefix_index[prefix].discard(key)
+                    if not self.prefix_index[prefix]:
+                        del self.prefix_index[prefix]
 
 
-# Create a global cache object (there is only one for the entire system)
-treenode_cache = TreeNodeCache()
+# Global cache object (unique for the system)
+treenode_cache = TreeCache()
 
 
 # ---------------------------------------------------
 # Decorator
 # ---------------------------------------------------
 
-
 def cached_method(func):
-    """
-    Decorate instance methods for caching.
-
-    The decorator caches the results of the decorated class or instance method.
-    If the cache is cleared or invalidated, the cached results will be
-    recalculated.
-
-    Usage:
-        @cached_tree_method
-        def model_method(self):
-            # Tree method logic
-    """
-
+    """Decorate instance or class methods."""
+    @wraps(func)
     def wrapper(self, *args, **kwargs):
-        # Generate a cache key.
+        cache = treenode_cache
+
         if isinstance(self, type):
-            # Если self — класс, используем его имя
             unique_id = to_base36(id(self))
             label = getattr(self._meta, 'label', self.__name__)
         else:
-            unique_id = getattr(self, "pk", id(self))
+            unique_id = getattr(self, "pk", None) or to_base36(id(self))
             label = self._meta.label
 
-        cache_key = treenode_cache.generate_cache_key(
+        cache_key = cache.generate_cache_key(
             label,
             func.__name__,
             unique_id,
             *args,
             **kwargs
         )
-
-        # Retrieving from cache
-        value = treenode_cache.get(cache_key)
-
+        value = cache.get(cache_key)
         if value is None:
             value = func(self, *args, **kwargs)
-
-            # Push to cache
-            treenode_cache.set(cache_key, value)
+            cache.set(cache_key, value)
         return value
     return wrapper
 

treenode/managers/adjacency.py

@@ -14,6 +14,7 @@ Email: timurkady@yandex.com
 from collections import deque, defaultdict
 from django.db import models, transaction
 from django.db import connection
+from django.db.models import F
 
 
 class TreeNodeQuerySet(models.QuerySet):
@@ -137,10 +138,11 @@ class TreeNodeModelManager(models.Manager):
 
     def get_queryset(self):
         """Return a sorted QuerySet."""
-        queryset = TreeNodeQuerySet(self.model, using=self._db)\
-            .annotate(_depth_db=models.Max("parents_set__depth"))\
-            .order_by("_depth_db", "tn_parent", "tn_priority")
-        return queryset
+        return TreeNodeQuerySet(self.model, using=self._db)\
+            .order_by(
+                # F('tn_parent').asc(nulls_first=True),
+                'tn_parent', 'tn_priority'
+        )
 
     # Service methods -------------------