django-fast-treenode 2.1.3__py3-none-any.whl → 2.1.5__py3-none-any.whl

{django_fast_treenode-2.1.3.dist-info → django_fast_treenode-2.1.5.dist-info}/METADATA

@@ -1,6 +1,6 @@
-Metadata-Version: 2.2
+Metadata-Version: 2.4
 Name: django-fast-treenode
-Version: 2.1.3
+Version: 2.1.5
 Summary: Application for supporting tree (hierarchical) data structure in Django projects
 Home-page: https://django-fast-treenode.readthedocs.io/
 Author: Timur Kady
@@ -53,22 +53,24 @@ Requires-Python: >=3.9
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: Django>=4.0
-Requires-Dist: pympler>=1.0
 Requires-Dist: django-widget-tweaks>=1.5
+Requires-Dist: msgpack>=1.1
 Provides-Extra: import-export
 Requires-Dist: openpyxl; extra == "import-export"
 Requires-Dist: pyyaml; extra == "import-export"
 Requires-Dist: xlsxwriter; extra == "import-export"
+Dynamic: license-file
 
 # Django-fast-treenode 
-**Combining Adjacency List and Closure Table for Optimal Performance**
+**Hybrid Tree Storage**
 
 [![Tests](https://github.com/TimurKady/django-fast-treenode/actions/workflows/test.yaml/badge.svg?branch=main)](https://github.com/TimurKady/django-fast-treenode/actions/workflows/test.yaml)
 [![Docs](https://readthedocs.org/projects/django-fast-treenode/badge/?version=latest)](https://django-fast-treenode.readthedocs.io/)
 [![PyPI](https://img.shields.io/pypi/v/django-fast-treenode.svg)](https://pypi.org/project/django-fast-treenode/)
 [![Published on Django Packages](https://img.shields.io/badge/Published%20on-Django%20Packages-0c3c26)](https://djangopackages.org/packages/p/django-fast-treenode/)
+[![Sponsor](https://img.shields.io/github/sponsors/TimurKady)](https://github.com/sponsors/TimurKady)
 
-**Django Fast TreeNode** is a high-performance Django application for working with tree structures, combining **Adjacency List** and **Closure Table** models. Each **TreeNodeModel** instance maintains two synchronized tables, enabling most operations to be performed with a single database query.
+**Django Fast TreeNode** is a high-performance Django application for working with tree structures. 
 
 ## Features
 - **Hybrid storage model**: Combines Adjacency List and Closure Table for optimal performance.
@@ -79,6 +81,8 @@ Requires-Dist: xlsxwriter; extra == "import-export"
 - **Admin panel integration**: Full compatibility with Django's admin panel, allowing intuitive management of tree structures.
 - **Import & Export functionality**: Built-in support for importing and exporting tree structures in multiple formats (CSV, JSON, XLSX, YAML, TSV), including integration with the Django admin panel.
 
+It seems that django-fast-treenode is currently the most balanced and performant solution for most tasks, especially those related to dynamic hierarchical data structures. Check out the results of (comparison tests)[#] with other Django packages.
+
 ## Use Cases
 Django Fast TreeNode is suitable for a wide range of applications, from simple directories to complex systems with deep hierarchical structures:
 - **Categories and taxonomies**: Manage product categories, tags, and classification systems.
@@ -87,7 +91,6 @@ Django Fast TreeNode is suitable for a wide range of applications, from simple d
 - **Geographical data**: Represent administrative divisions, regions, and areas of influence.
 - **Organizational and Business Structures**: Model company hierarchies, business processes, employees and departments.
 
-In all applications, `django-fast-treenode` models show excellent performance and stability.
 
 ## Quick start
 1. Run `pip install django-fast-treenode`.
@@ -159,4 +162,4 @@ Released under [MIT License](https://github.com/TimurKady/django-fast-treenode/b
 ## Credits
 Thanks to everyone who contributed to the development and testing of this package, as well as the Django community for their inspiration and support. 
 
-Special thanks to [Fabio Caccamo](https://github.com/fabiocaccamo) for the idea behind creating a fast Django application for handling hierarchies and [Mathieu Leplatre](https://github.com/leplatrem) for the advice used in writing this application.
+Special thanks to [Fabio Caccamo](https://github.com/fabiocaccamo) for the idea behind creating a fast Django application for handling hierarchies.

{django_fast_treenode-2.1.3.dist-info → django_fast_treenode-2.1.5.dist-info}/RECORD

@@ -1,34 +1,35 @@
-treenode/__init__.py,sha256=CZ-0uZyhi8OZJP8xmqbTqXrZUjcQ4SNAXhKFgM0qw2M,99
+django_fast_treenode-2.1.5.dist-info/licenses/LICENSE,sha256=T0evsb7y-63fg18ovdNSx3wwWWRwyluQvN9J4zFSvfE,1093
+treenode/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 treenode/apps.py,sha256=a7UasXiZZudPccjmHEudP79TkhR_53Mvnb-dBXLHRRQ,862
-treenode/cache.py,sha256=JLbI0EWq1XmH24SF46glC6e7sdizmfObbFA5vQJvMiY,7213
+treenode/cache.py,sha256=GoN2J-ypEQWIK05WSw9LYo7boKHGPXNFxqHorFPUqX8,12481
 treenode/forms.py,sha256=Mjrpuyd1CPsitcElDVagE3k-p2kU4xIlRuy1f5Zgt3c,3800
 treenode/signals.py,sha256=ERrlKjGqhYaPYVKKRk1JBBlPFOmJKpJ6bXsJavcTlo0,518
 treenode/urls.py,sha256=CsgX0hRyDVrMS8YnRlr_CxmDlgGIhDpqZ9ldoMYZCac,866
-treenode/version.py,sha256=PBeSQ_6jPFoS_aURwvCBCsE-VlTS5vbGFlvnO41XfMY,222
+treenode/version.py,sha256=mv001KtDXkO8dnqphEI_VEhvLuHmmmhI515DitY0q2U,222
 treenode/views.py,sha256=rEZEgdbEA3AJDHrvtrAm-t60QTJcJ4JEhNsNMR1Y_I4,5549
 treenode/widgets.py,sha256=Mi0F-AK_UcmU6C50ENK9vv6xGQNuDtrtzXSnXSOXhLM,4760
-treenode/admin/__init__.py,sha256=TdlPIyRW8i9qTVqGLmLWiBw4DyoGHUYZErE6rCyGOPE,119
-treenode/admin/admin.py,sha256=6H3N2Dg6l-MrFwIcyKR5YENg0cEo-I4uKCP9MuhHkqo,10580
+treenode/admin/__init__.py,sha256=K5GgagrfLwzF8GvOYfwXpJYLCexM8DbEoK1bhsqIBvc,119
+treenode/admin/admin.py,sha256=iVi8s8mPVVDlbbJFqCcuXzDbE29KPj6XntFDDRECkmY,10580
 treenode/admin/changelist.py,sha256=YZm3zNniX75CgLjnbHpVr0OIP91halDEBHmrcS8m5Og,2128
 treenode/admin/mixins.py,sha256=-dVZwEjKsfRzMkBe87dkI0SZ9MH45YE_o39SIhSJWy4,11194
 treenode/managers/__init__.py,sha256=EG_tj9P1Hama3kaqMfHck4lfzUWoPaJJVOXe3qaKMUo,585
-treenode/managers/adjacency.py,sha256=NVN8dq5z7gIh90yqW2uxV7MokmUfXTOT7crqMDyMaH0,7889
+treenode/managers/adjacency.py,sha256=OOjHCSTo0aAcSxOOwz7OsQTGdTRkM1mAxSN7jlzRpho,7896
 treenode/managers/closure.py,sha256=PcScdJJUnLcKe8Y1wqROYPsRtAnBMUO4xn5sILk9AIM,10638
 treenode/models/__init__.py,sha256=pBiMlEpC_Thh7asraNzA7W_7BKu2oAHtcn-K6_sdJe8,112
-treenode/models/adjacency.py,sha256=ijStfIQDSd48L3nA8OnLD1nHGYo5YsnokqUVfzDt68w,12422
+treenode/models/adjacency.py,sha256=QWGOidd4tH3afqVedPNQqeh-W-zUTNs1m-iAhCAXub4,12396
 treenode/models/classproperty.py,sha256=J4W6snsfsEUSHKHkIlM9yOJYQ_FSrp3P3oEYMKJengg,571
-treenode/models/closure.py,sha256=NEC8pi9QIxNtUnctEi0lPHBHEPX2K3V1oeSs9JrDGA0,4588
+treenode/models/closure.py,sha256=eZtLbnCOR1xYWhgbo1Pml_K0Pd0MM2DjiZl3SWMVe2A,3712
 treenode/models/factory.py,sha256=10FEGGC5PGWaR58qErs0oOrCS0KeI8x9H-SknZAAWqw,2291
 treenode/models/mixins/__init__.py,sha256=gTdMZFh1slNHMvxrnu-hGl46xqnWd4W7TOEFWTVJq40,757
-treenode/models/mixins/ancestors.py,sha256=_nF8V99SBjT-G88BmcRtiTC0DgSEIgHrI0g02jqTnss,2075
-treenode/models/mixins/children.py,sha256=OchaH6m6pOr6uuiZRRBHoZXoCSWM-ENTNWu1iLHXaBM,2564
-treenode/models/mixins/descendants.py,sha256=2mhnIhC8VJomTqntzzAwFFW_CcMiwujzQoD5_mfMsK0,2208
+treenode/models/mixins/ancestors.py,sha256=QZywMcIVZK82j13QsgevVN2ZhRLa86DfRIt2BsiM2to,1526
+treenode/models/mixins/children.py,sha256=xgenQFyZBG7_S33QQlznSmNhXEdeo9DeLyi7dKmvFhw,2637
+treenode/models/mixins/descendants.py,sha256=PYYfd7oqlv3Gnfahm0u9ACHjpWSDNM6Z8oJaJXPVQ8w,1910
 treenode/models/mixins/family.py,sha256=h2IRRADkQxve97QqBHKv0evVz4cFQtcNR8CbPi9Ri_w,1645
 treenode/models/mixins/logical.py,sha256=jlhBSq3AfCYNyNjqyKM9siyioS3SYcGD-aG2b4MV2RM,2169
-treenode/models/mixins/node.py,sha256=wgLbFKA99QuLV4l32GR3JZ5gYIVKUbfDzM8iDw4C8Bs,7694
+treenode/models/mixins/node.py,sha256=VpLiFI1olvj5Gp2yV4n-aG4z4mZ7vOS6ytloWwO5s6w,7149
 treenode/models/mixins/properties.py,sha256=pfv80KLXcPeGx00IFCBcst1_cf0AmzhjshFjq1XQWMY,3876
 treenode/models/mixins/roots.py,sha256=MoFQq1fph70awc26UMUbfeTpt0ToUOvMz1c7LlDyIP8,2956
-treenode/models/mixins/siblings.py,sha256=fh0ZrlFXKxOQ4Qrp6sElTMvRhU5PyRRykLHDcbH-3Rk,3113
+treenode/models/mixins/siblings.py,sha256=JTQjaxnDH9t-AVMCQFiuc0nHLdIsE4v5vJ5z6LcUZLY,3236
 treenode/models/mixins/tree.py,sha256=CsO0ynwcwkrWgQbTzvF4yws-y7n1GGM2zImJH0hgV00,13042
 treenode/static/.gitkeep,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 treenode/static/treenode/.gitkeep,sha256=frcCV1k9oG9oKj3dpUqdJg1PxRT2RSN_XKdLCPjaYaY,2
@@ -53,11 +54,10 @@ treenode/utils/aid.py,sha256=o8Jgc1vDRtQpx4XYdv0qR5Lqvens55Jfbdca1nr-EOA,1013
 treenode/utils/base16.py,sha256=U1PMit2aZOpYusG_u1c7eVpXO-cFrFPyVyk9zdHrehg,817
 treenode/utils/base36.py,sha256=yICmyPE-yyPNO9T2oALOt-b6uYf37ahFfx0R4tXn3X0,847
 treenode/utils/db.py,sha256=36q4OckKmEd6uHTbMTxdKpV9nOIZ55DAantRWR9bxWg,4297
-treenode/utils/exporter.py,sha256=mV6Gch7XfW8f_1x3WqWgtV0qekMLdo-_n9gz6GJjXjw,7259
+treenode/utils/exporter.py,sha256=LGC5VfJj7wMFp7BkaWjmfrImgCVRpJ8gjkDpn4IDTEs,7258
 treenode/utils/importer.py,sha256=Hvirbd6NyZ2MHa56_jOrUF3kYFeby1DbSLR3mhHy-9s,12891
 treenode/utils/radix.py,sha256=zHpOuDxsebiv9Gza6snNhAtBKiex6CDrAVRtB6esaWo,1642
-django_fast_treenode-2.1.3.dist-info/LICENSE,sha256=T0evsb7y-63fg18ovdNSx3wwWWRwyluQvN9J4zFSvfE,1093
-django_fast_treenode-2.1.3.dist-info/METADATA,sha256=glnnVS6RVwKFZkbQOLVkcow74o-MKFS_AsPTY_smBd4,8165
-django_fast_treenode-2.1.3.dist-info/WHEEL,sha256=52BFRY2Up02UkjOa29eZOS2VxUrpPORXg1pkohGGUS8,91
-django_fast_treenode-2.1.3.dist-info/top_level.txt,sha256=fmgxHbXyx1O2MPi_9kjx8aL9L-8TmV0gre4Go8XgqFk,9
-django_fast_treenode-2.1.3.dist-info/RECORD,,
+django_fast_treenode-2.1.5.dist-info/METADATA,sha256=KpOVLmk1TDKx5_iXE1geP9_GlfmVIvJK-IhuSh7Lu8w,8103
+django_fast_treenode-2.1.5.dist-info/WHEEL,sha256=DK49LOLCYiurdXXOXwGJm6U4DkHkg4lcxjhqwRa0CP4,91
+django_fast_treenode-2.1.5.dist-info/top_level.txt,sha256=fmgxHbXyx1O2MPi_9kjx8aL9L-8TmV0gre4Go8XgqFk,9
+django_fast_treenode-2.1.5.dist-info/RECORD,,

{django_fast_treenode-2.1.3.dist-info → django_fast_treenode-2.1.5.dist-info}/WHEEL

@@ -1,5 +1,5 @@
 Wheel-Version: 1.0
-Generator: setuptools (76.0.0)
+Generator: setuptools (78.0.2)
 Root-Is-Purelib: true
 Tag: py3-none-any
 

treenode/__init__.py

@@ -1,5 +0,0 @@
-"""
-Django Fast TreeNode.
-
-📖 Documentation: https://django-fast-treenode.readthedocs.io/
-"""

treenode/admin/__init__.py

@@ -1,9 +1,9 @@
 # -*- coding: utf-8 -*-
 
-from .admin import TreeNodeAdminModel
+from .admin import TreeNodeModelAdmin
 
 
-__all__ = ["TreeNodeAdminModel"]
+__all__ = ["TreeNodeModelAdmin"]
 
 
 # The end

treenode/admin/admin.py

@@ -29,7 +29,7 @@ import logging
 logger = logging.getLogger(__name__)
 
 
-class TreeNodeAdminModel(AdminMixin, admin.ModelAdmin):
+class TreeNodeModelAdmin(AdminMixin, admin.ModelAdmin):
     """
     TreeNodeAdmin class.
 

treenode/cache.py

@@ -12,218 +12,339 @@ Features:
 - Automatic cache eviction when memory limits are exceeded.
 - Decorator `@cached_method` for caching method results.
 
-Version: 2.1.0
+Version: 2.2.0
 Author: Timur Kady
 Email: timurkady@yandex.com
 """
 
-
-from django.core.cache import caches
-from django.conf import settings
-import threading
 import hashlib
-import json
-import logging
-from pympler import asizeof
+import msgpack
+import sys
+import threading
+from collections import deque, defaultdict, OrderedDict
+from django.conf import settings
+from django.core.cache import caches
+from functools import lru_cache
+from functools import wraps
 
-from .utils.base36 import to_base36
 
-logger = logging.getLogger(__name__)
+# ---------------------------------------------------
+# Utilities
+# ---------------------------------------------------
+
+_DIGITS = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+_CLEARINT_THESHOLD = 0.8
+_EVICT_INTERVAL = 50
+
+
+@lru_cache(maxsize=1000)
+def to_base36(num):
+    """
+    Convert an integer to a base36 string.
+
+    For example: 10 -> 'A', 35 -> 'Z', 36 -> '10', etc.
+    """
+    if num == 0:
+        return '0'
+    sign = '-' if num < 0 else ''
+    num = abs(num)
+    result = []
+    while num:
+        num, rem = divmod(num, 36)
+        result.append(_DIGITS[rem])
+    return sign + ''.join(reversed(result))
 
 
 # ---------------------------------------------------
 # Caching
 # ---------------------------------------------------
 
-class TreeNodeCache:
-    """Singleton-класс для управления кэшем TreeNode."""
+class TreeCache:
+    """Singleton class for managing the TreeNode cache."""
 
     _instance = None
-    _lock = threading.Lock()
-    _keys = dict()
-    _total_size = 0
-    _cache_limit = 0
-
-    def __new__(cls):
-        """Create only one instance of the class (Singleton)."""
-        with cls._lock:
+    _instance_lock = threading.Lock()
+
+    def __new__(cls, *args, **kwargs):
+        """Singleton new."""
+        with cls._instance_lock:
             if cls._instance is None:
-                cls._instance = super(TreeNodeCache, cls).__new__(cls)
-                cls._instance._initialize()
+                cls._instance = super(TreeCache, cls).__new__(cls)
         return cls._instance
 
-    def _initialize(self):
-        """Initialize cache."""
-        self.cache_timeout = None
-        limit = getattr(settings, 'TREENODE_CACHE_LIMIT', 100)*1024*1024
-        self._cache_limit = limit
-        self.cache_timeout = None
-        cache_name = 'treenode' if 'treenode' in settings.CACHES else 'default'
-        self.cache = caches[cache_name]
-        self._total_size = 0
-        self.cache.clear()
+    def __init__(self, cache_limit=100 * 1024 * 1024):
+        """
+        Initialize the cache.
+
+        If the 'treenode' key is present in settings.CACHES, the corresponding
+        backend is used.
+        Otherwise, the custom dictionary is used.
+        The cache size (in bytes) is taken from
+        settings.TREENODE_CACHE_LIMIT (MB), by default 100 MB.
+        """
+        if hasattr(self, '_initialized') and self._initialized:
+            return
+
+        # Get the cache limit (MB), then convert to bytes.
+        cache_limit_mb = getattr(settings, 'TREENODE_CACHE_LIMIT', 100)
+        self.cache_limit = cache_limit_mb * 1024 * 1024
+
+        # Select backend: if there is 'treenode' in settings.CACHES, use it.
+        # Otherwise, use our own dictionary.
+        if hasattr(settings, 'CACHES') and 'treenode' in settings.CACHES:
+            self.cache = caches['treenode']
+        else:
+            # We use our dictionary as a backend.
+            self.cache = OrderedDict()
+
+        self.order = deque()            # Queue for FIFO implementation.
+        self.total_size = 0             # Current cache size in bytes.
+        self.lock = threading.Lock()    # Lock for thread safety.
+
+        # Additional index for fast search of keys by prefix
+        # Format: {prefix: {key1, key2, ...}}
+        self.prefix_index = defaultdict(set)
+        # Dictionary to store the sizes of each key (key -> size in bytes)
+        self.sizes = {}
+        # Dictionary to store the prefix for each key to avoid repeated
+        # splitting
+        self.key_prefix = {}
+
+        # Counter for number of set operations for periodic eviction
+        self._set_counter = 0
+        # Evict cache every _evict_interval set operations when using external
+        # backend
+        self._evict_interval = _EVICT_INTERVAL
+
+        self._initialized = True
 
     def generate_cache_key(self, label, func_name, unique_id, *args, **kwargs):
         """
-        Generate Cache Key.
-
-        Generates a cache key of the form:
-            <model_name>_<func_name>_<id>_<hash>,
-        where <hash> is calculated from the function parameters
-        (args and kwargs).
-        If the parameters can be serialized via JSON, use this, otherwise we
-        use repr to generate the string.
+        Generate a cache key.
+
+        <label>_<func_name>_<unique_id>_<hash>
         """
-        try:
-            # Sort dictionary keys to ensure determinism.
-            params_repr = json.dumps(
-                (args, kwargs),
-                sort_keys=True,
-                default=str
-            )
-        except (TypeError, ValueError) as e:
-            # If JSON serialization fails, use repr.
-            params_repr = repr((args, kwargs))
-            logger.warning(f"Failed to serialize cache key params: {e}")
-
-        # Calculate the MD5 hash from the received string.
-        hash_value = hashlib.sha256(params_repr.encode("utf-8")).hexdigest()
-
-        # Forming the final key.
-        cache_key = f"{label}_{func_name}_{unique_id}_{hash_value}"
-
-        return cache_key
+        # If using custom dict backend, use simple key generation without
+        # serialization.
+        if isinstance(self.cache, dict):
+            sorted_kwargs = sorted(kwargs.items())
+            return f"{label}_{func_name}_{unique_id}_{args}_{sorted_kwargs}"
+        else:
+            try:
+                # Using msgpack for fast binary representation of arguments
+                sorted_kwargs = sorted(kwargs.items())
+                params_bytes = msgpack.packb(
+                    (args, sorted_kwargs), use_bin_type=True)
+            except Exception:
+                params_bytes = repr((args, kwargs)).encode('utf-8')
+            # Using MD5 for speed (no cryptographic strength)
+            hash_value = hashlib.md5(params_bytes).hexdigest()
+            return f"{label}_{func_name}_{unique_id}_{hash_value}"
 
     def get_obj_size(self, value):
-        """Determine the size of the object in bytes."""
-        try:
-            return len(json.dumps(value).encode("utf-8"))
-        except (TypeError, ValueError):
-            return asizeof.asizeof(value)
-
-    def cache_size(self):
-        """Return the total size of the cache in bytes."""
-        return self._total_size
-
-    def set(self, cache_key, value):
-        """Push to cache."""
-        size = self.get_obj_size(value)
-        self.cache.set(cache_key, value, timeout=self.cache_timeout)
-
-        # Update cache size
-        if cache_key in self._keys:
-            self._total_size -= self._keys[cache_key]
-        self._keys[cache_key] = size
-        self._total_size += size
-
-        # Check if the limit has been exceeded
-        self._evict_cache()
-
-    def get(self, cache_key):
-        """Get from cache."""
-        return self.cache.get(cache_key)
-
-    def invalidate(self, label):
-        """Clear cache for a specific model only."""
-        prefix = f"{label}_"
-        keys_to_remove = [key for key in self._keys if key.startswith(prefix)]
-        for key in keys_to_remove:
-            self.cache.delete(key)
-            self._total_size -= self._keys.pop(key, 0)
-            if self._total_size < 0:
-                self._total_size = 0
+        """
+        Determine the size of the object in bytes.
 
-    def clear(self):
-        """Full cache clearing."""
-        self.cache.clear()
-        self._keys.clear()
-        self._total_size = 0
+        If the value is already in bytes or bytearray, simply returns its
+        length. Otherwise, uses sys.getsizeof for an approximate estimate.
+        """
+        if isinstance(value, (bytes, bytearray)):
+            return len(value)
+        return sys.getsizeof(value)
 
-    def _evict_cache(self):
-        """Delete old entries if the cache has exceeded the limit."""
-        if self._total_size <= self._cache_limit:
-            # If the size is within the limit, do nothing
-            return
+    def set(self, key, value):
+        """
+        Store the value in the cache.
+
+        Stores the value in the cache, updates the FIFO queue, prefix index,
+        size dictionary, and total cache size.
+        """
+        # Idea 1: Store raw object if using custom dict backend, otherwise
+        # serialize using msgpack.
+        if isinstance(self.cache, dict):
+            stored_value = value
+        else:
+            try:
+                stored_value = msgpack.packb(value, use_bin_type=True)
+            except Exception:
+                stored_value = value
+
+        # Calculate the size of the stored value
+        if isinstance(stored_value, (bytes, bytearray)):
+            size = len(stored_value)
+        else:
+            size = sys.getsizeof(stored_value)
 
-        if not self._keys:
-            self.clear()
+        # Store the value in the cache backend
+        if isinstance(self.cache, dict):
+            self.cache[key] = stored_value
+        else:
+            self.cache.set(key, stored_value)
+
+        # Update internal structures under lock
+        with self.lock:
+            if key in self.sizes:
+                # If the key already exists, adjust the total size
+                old_size = self.sizes[key]
+                self.total_size -= old_size
+            else:
+                # New key: add to FIFO queue
+                self.order.append(key)
+                # Compute prefix once and store it in key_prefix
+                if "_" in key:
+                    prefix = key.split('_', 1)[0] + "_"
+                else:
+                    prefix = key
+                self.key_prefix[key] = prefix
+                self.prefix_index[prefix].add(key)
+            # Save the size for this key and update total_size
+            self.sizes[key] = size
+            self.total_size += size
+
+            # Increment the set counter for periodic eviction
+            self._set_counter += 1
+
+        # Idea 3: If using external backend, evict cache every _evict_interval
+        # sets. Otherwise, always evict immediately.
+        if self._set_counter >= self._evict_interval:
+            with self.lock:
+                self._set_counter = 0
+            self._evict_cache()
+
+    def get(self, key):
+        """
+        Get a value from the cache by key.
 
-        logger.warning(f"Cache limit exceeded! Current size: \
-{self._total_size}, Limit: {self._cache_limit}")
+        Quickly retrieves a value from the cache by key.
+        Here we simply request a value from the backend (either a dictionary or
+        Django cache-backend) and return it without any additional operations.
+        """
+        if isinstance(self.cache, dict):
+            return self.cache.get(key)
+        else:
+            packed_value = self.cache.get(key)
+            if packed_value is None:
+                return None
+            try:
+                return msgpack.unpackb(packed_value, raw=False)
+            except Exception:
+                # If unpacking fails, return what we got
+                return packed_value
+
+    def invalidate(self, prefix):
+        """
+        Invalidate model cache.
+
+        Quickly removes all items from the cache whose keys start with prefix.
+        Uses prefix_index for instant access to keys.
+        When removing, each key's size is retrieved from self.sizes,
+        and total_size is reduced by the corresponding amount.
+        """
+        prefix += '_'
+        with self.lock:
+            keys_to_remove = self.prefix_index.get(prefix, set())
+            if not keys_to_remove:
+                return
+
+            # Remove keys from main cache and update total_size via sizes
+            # dictionary
+            if isinstance(self.cache, dict):
+                for key in keys_to_remove:
+                    self.cache.pop(key, None)
+                    size = self.sizes.pop(key, 0)
+                    self.total_size -= size
+                    # Remove key from key_prefix as well
+                    self.key_prefix.pop(key, None)
+            else:
+                # If using Django backend
+                self.cache.delete_many(list(keys_to_remove))
+                for key in keys_to_remove:
+                    size = self.sizes.pop(key, 0)
+                    self.total_size -= size
+                    self.key_prefix.pop(key, None)
+
+            # Remove prefix from index and update FIFO queue
+            del self.prefix_index[prefix]
+            self.order = deque(k for k in self.order if k not in keys_to_remove)
 
-        # Sort keys by insertion order (FIFO)
-        keys_sorted = list(self._keys.keys())
+    def clear(self):
+        """Clear cache completely."""
+        with self.lock:
+            if isinstance(self.cache, dict):
+                self.cache.clear()
+            else:
+                self.cache.clear()
+            self.order.clear()
+            self.prefix_index.clear()
+            self.sizes.clear()
+            self.key_prefix.clear()
+            self.total_size = 0
 
-        keys_to_delete = []
-        freed_size = 0
+    def _evict_cache(self):
+        """
+        Perform FIFO cache evacuation.
 
-        # Delete old keys until we reach the limit
-        for key in keys_sorted:
-            freed_size += self._keys[key]
-            keys_to_delete.append(key)
-            if self._total_size - freed_size <= self._cache_limit:
-                break
+        Removes old items until the total cache size is less than
+        _CLEARINT_THESHOLD of the limit.
+        """
+        with self.lock:
+            # Evict until total_size is below 80% of cache_limit
+            target_size = _CLEARINT_THESHOLD * self.cache_limit
+            while self.total_size > target_size and self.order:
+                # Extract the oldest key from the queue (FIFO)
+                key = self.order.popleft()
 
-        # Delete keys in batches (delete_many)
-        self.cache.delete_many(keys_to_delete)
+                # Delete entry from backend cache
+                if isinstance(self.cache, dict):
+                    self.cache.pop(key, None)
+                else:
+                    self.cache.delete(key)
 
-        # Update data in `_keys` and `_total_size`
-        for key in keys_to_delete:
-            self._total_size -= self._keys.pop(key, 0)
+                # Extract the size of the entry to be deleted and reduce
+                # the overall cache size
+                size = self.sizes.pop(key, 0)
+                self.total_size -= size
 
-        logger.info(f"Evicted {len(keys_to_delete)} keys from cache, \
-freed {freed_size} bytes.")
+                # Retrieve prefix from key_prefix without splitting
+                prefix = self.key_prefix.pop(key, None)
+                if prefix is not None:
+                    self.prefix_index[prefix].discard(key)
+                    if not self.prefix_index[prefix]:
+                        del self.prefix_index[prefix]
 
 
-# Create a global cache object (there is only one for the entire system)
-treenode_cache = TreeNodeCache()
+# Global cache object (unique for the system)
+treenode_cache = TreeCache()
 
 
 # ---------------------------------------------------
 # Decorator
 # ---------------------------------------------------
 
-
 def cached_method(func):
-    """
-    Decorate instance methods for caching.
-
-    The decorator caches the results of the decorated class or instance method.
-    If the cache is cleared or invalidated, the cached results will be
-    recalculated.
-
-    Usage:
-        @cached_tree_method
-        def model_method(self):
-            # Tree method logic
-    """
-
+    """Decorate instance or class methods."""
+    @wraps(func)
     def wrapper(self, *args, **kwargs):
-        # Generate a cache key.
+        cache = treenode_cache
+
         if isinstance(self, type):
-            # Если self — класс, используем его имя
             unique_id = to_base36(id(self))
             label = getattr(self._meta, 'label', self.__name__)
         else:
-            unique_id = getattr(self, "pk", id(self))
+            unique_id = getattr(self, "pk", None) or to_base36(id(self))
             label = self._meta.label
 
-        cache_key = treenode_cache.generate_cache_key(
+        cache_key = cache.generate_cache_key(
             label,
             func.__name__,
             unique_id,
             *args,
             **kwargs
         )
-
-        # Retrieving from cache
-        value = treenode_cache.get(cache_key)
-
+        value = cache.get(cache_key)
         if value is None:
             value = func(self, *args, **kwargs)
-
-            # Push to cache
-            treenode_cache.set(cache_key, value)
+            cache.set(cache_key, value)
         return value
     return wrapper
 

treenode/managers/adjacency.py

@@ -14,6 +14,7 @@ Email: timurkady@yandex.com
 from collections import deque, defaultdict
 from django.db import models, transaction
 from django.db import connection
+from django.db.models import F
 
 
 class TreeNodeQuerySet(models.QuerySet):
@@ -137,10 +138,11 @@ class TreeNodeModelManager(models.Manager):
 
     def get_queryset(self):
         """Return a sorted QuerySet."""
-        queryset = TreeNodeQuerySet(self.model, using=self._db)\
-            .annotate(_depth_db=models.Max("parents_set__depth"))\
-            .order_by("_depth_db", "tn_parent", "tn_priority")
-        return queryset
+        return TreeNodeQuerySet(self.model, using=self._db)\
+            .order_by(
+                # F('tn_parent').asc(nulls_first=True),
+                'tn_parent', 'tn_priority'
+        )
 
     # Service methods -------------------
 

treenode/models/adjacency.py

@@ -74,8 +74,6 @@ class TreeNodeModel(
 
         abstract = True
         indexes = [
-            models.Index(fields=["tn_parent"]),
-            models.Index(fields=["tn_parent", "id"]),
             models.Index(fields=["tn_parent", "tn_priority"]),
         ]
 
@@ -151,43 +149,45 @@ class TreeNodeModel(
             using=self._state.db,
             update_fields=kwargs.get("update_fields", None)
         )
-
-        # If the object already exists, get the old parent and priority values
-        is_new = self.pk is None
-        if not is_new:
-            old_parent, old_priority = model.objects\
-                .filter(pk=self.pk)\
-                .values_list('tn_parent', 'tn_priority')\
-                .first()
-            is_move = (old_priority != self.tn_priority)
-        else:
-            force_insert = True
-            is_move = False
-            old_parent = None
-
-        # Check if we are trying to move a node to a child
-        if old_parent and old_parent != self.tn_parent and self.tn_parent:
-            # Get pk of children via values_list to avoid creating full
-            # set of objects
-            if self.tn_parent.pk in self.get_descendants_pks():
-                raise ValueError("You cannot move a node into its own child.")
-
-        # Save the object and synchronize with the closing table
-        # Disable signals
-        with (disable_signals(pre_save, model),
-              disable_signals(post_save, model)):
-
-            if is_new or is_move:
-                self._update_priority()
-            super().save(force_insert=force_insert, *args, **kwargs)
-            # Run synchronize
-            if is_new:
-                self.closure_model.insert_node(self)
-            elif is_move:
-                subtree_nodes = self.get_descendants(include_self=True)
-                self.closure_model.move_node(subtree_nodes)
-            # Update priorities among neighbors or clear cache if there was
-            # no movement
+        with transaction.atomic():
+            # If the object already exists, get the old parent and priority
+            # values
+            is_new = self.pk is None
+            if not is_new:
+                old_parent, old_priority = model.objects\
+                    .filter(pk=self.pk)\
+                    .values_list('tn_parent', 'tn_priority')\
+                    .first()
+                is_move = (old_priority != self.tn_priority)
+            else:
+                force_insert = True
+                is_move = False
+                old_parent = None
+
+            descendants = self.get_descendants(include_self=True)
+
+            # Check if we are trying to move a node to a child
+            if old_parent and old_parent != self.tn_parent and self.tn_parent:
+                # Get pk of children via values_list to avoid creating full
+                # set of objects
+                if self.tn_parent in descendants:
+                    raise ValueError(
+                        "You cannot move a node into its own child."
+                    )
+
+            # Save the object and synchronize with the closing table
+            # Disable signals
+            with (disable_signals(pre_save, model),
+                  disable_signals(post_save, model)):
+
+                if is_new or is_move:
+                    self._update_priority()
+                super().save(force_insert=force_insert, *args, **kwargs)
+                # Run synchronize
+                if is_new:
+                    self.closure_model.insert_node(self)
+                elif is_move:
+                    self.closure_model.move_node(descendants)
 
         # Clear model cache
         model.clear_cache()
@@ -203,7 +203,7 @@ class TreeNodeModel(
 
     def _update_priority(self):
         """Update tn_priority field for siblings."""
-        siblings = self.get_siblings()
+        siblings = self.get_siblings(include_self=False)
         siblings = sorted(siblings, key=lambda x: x.tn_priority)
         insert_pos = min(self.tn_priority, len(siblings))
         siblings.insert(insert_pos, self)
@@ -214,7 +214,6 @@ class TreeNodeModel(
         # Save changes
         model = self._meta.model
         model.objects.bulk_update(siblings, ['tn_priority'])
-        model.clear_cache()
 
     @classmethod
     def _get_place(cls, target, position=0):

treenode/models/closure.py

@@ -62,7 +62,8 @@ class ClosureModel(models.Model):
         unique_together = (("parent", "child"),)
         indexes = [
             models.Index(fields=["parent", "child"]),
-            models.Index(fields=["child", "parent"]),
+            models.Index(fields=["parent", "depth"]),
+            models.Index(fields=["child", "depth"]),
             models.Index(fields=["parent", "child", "depth"]),
         ]
 
@@ -72,28 +73,6 @@ class ClosureModel(models.Model):
 
     # ----------- Methods of working with tree structure ----------- #
 
-    @classmethod
-    def get_ancestors_pks(cls, node, include_self=True, depth=None):
-        """Get the ancestors pks list."""
-        options = dict(child_id=node.pk, depth__gte=0 if include_self else 1)
-        if depth:
-            options["depth__lte"] = depth
-        queryset = cls.objects.filter(**options)\
-            .order_by('depth')\
-            .values_list('parent_id', flat=True)
-        return list(queryset.values_list("parent_id", flat=True))
-
-    @classmethod
-    def get_descendants_pks(cls, node, include_self=False, depth=None):
-        """Get a list containing all descendants."""
-        options = dict(parent_id=node.pk, depth__gte=0 if include_self else 1)
-        if depth:
-            options.update({'depth__lte': depth})
-        queryset = cls.objects.filter(**options)\
-            .order_by('depth')\
-            .values_list('child_id', flat=True)
-        return queryset
-
     @classmethod
     def get_root(cls, node):
         """Get the root node pk for the current node."""

treenode/models/mixins/ancestors.py

@@ -2,12 +2,13 @@
 """
 TreeNode Ancestors Mixin
 
-Version: 2.1.0
+Version: 2.1.4
 Author: Timur Kady
 Email: timurkady@yandex.com
 """
 
 from django.db import models
+from django.db.models import OuterRef, Subquery, IntegerField, Case, When, Value
 from ...cache import treenode_cache, cached_method
 
 
@@ -22,41 +23,23 @@ class TreeNodeAncestorsMixin(models.Model):
     @cached_method
     def get_ancestors_queryset(self, include_self=True, depth=None):
         """Get the ancestors queryset (ordered from root to parent)."""
-        qs = self._meta.model.objects.filter(tn_closure__child=self.pk)
+        options = dict(child_id=self.pk, depth__gte=0 if include_self else 1)
+        if depth:
+            options.update({'depth__lte': depth})
 
-        if depth is not None:
-            qs = qs.filter(tn_closure__depth__lte=depth)
-
-        if include_self:
-            qs = qs | self._meta.model.objects.filter(pk=self.pk)
-
-        return qs.distinct().order_by("tn_closure__depth")
+        return self.closure_model.objects\
+            .filter(**options)\
+            .order_by('-depth')
 
     @cached_method
     def get_ancestors_pks(self, include_self=True, depth=None):
         """Get the ancestors pks list."""
-        cache_key = treenode_cache.generate_cache_key(
-            label=self._meta.label,
-            func_name=getattr(self, "get_ancestors_queryset").__name__,
-            unique_id=self.pk,
-            arg={
-                "include_self": include_self,
-                "depth": depth
-            }
-        )
-        queryset = treenode_cache.get(cache_key)
-        if queryset is not None:
-            return list(queryset.values_list("id", flat=True))
-        elif hasattr(self, "closure_model"):
-            return self.closure_model.get_ancestors_pks(
-                self, include_self, depth
-            )
-        return []
+        return self.get_ancestors_queryset(include_self, depth)\
+            .values_list('id', flat=True)
 
     def get_ancestors(self, include_self=True, depth=None):
         """Get a list with all ancestors (ordered from root to self/parent)."""
-        queryset = self.get_ancestors_queryset(include_self, depth)
-        return list(queryset)
+        return list(self.get_ancestors_queryset(include_self, depth))
 
     def get_ancestors_count(self, include_self=True, depth=None):
         """Get the ancestors count."""

treenode/models/mixins/children.py

@@ -60,7 +60,8 @@ class TreeNodeChildrenMixin(models.Model):
     @cached_method
     def get_children_queryset(self):
         """Get the children queryset with prefetch."""
-        return self.tn_children.prefetch_related('tn_children')
+        # return self.tn_children.prefetch_related('tn_children')
+        return self._meta.model.objects.filter(tn_parent__pk=self.id)
 
     def get_children(self):
         """Get a list containing all children."""

treenode/models/mixins/descendants.py

@@ -8,6 +8,8 @@ Email: timurkady@yandex.com
 """
 
 from django.db import models
+from django.db.models import OuterRef, Subquery, Min
+
 from treenode.cache import treenode_cache, cached_method
 
 
@@ -22,37 +24,29 @@ class TreeNodeDescendantsMixin(models.Model):
     @cached_method
     def get_descendants_queryset(self, include_self=False, depth=None):
         """Get the descendants queryset."""
-        queryset = self._meta.model.objects\
-            .annotate(min_depth=models.Min("parents_set__depth"))\
-            .filter(parents_set__parent=self.pk)
+        Closure = self.closure_model
+        desc_qs = Closure.objects.filter(child=OuterRef('pk'), parent=self.pk)
+        desc_qs = desc_qs.values('child').annotate(
+            mdepth=Min('depth')).values('mdepth')[:1]
+
+        queryset = self._meta.model.objects.annotate(
+            min_depth=Subquery(desc_qs)
+        ).filter(min_depth__isnull=False)
 
         if depth is not None:
             queryset = queryset.filter(min_depth__lte=depth)
-        if include_self and not queryset.filter(pk=self.pk).exists():
+
+        # add self if needed
+        if include_self:
             queryset = queryset | self._meta.model.objects.filter(pk=self.pk)
 
-        return queryset.order_by("min_depth", "tn_priority")
+        return queryset.order_by('min_depth', 'tn_priority')
 
     @cached_method
     def get_descendants_pks(self, include_self=False, depth=None):
         """Get the descendants pks list."""
-        cache_key = treenode_cache.generate_cache_key(
-            label=self._meta.label,
-            func_name=getattr(self, "get_descendants_queryset").__name__,
-            unique_id=self.pk,
-            arg={
-                "include_self": include_self,
-                "depth": depth
-            }
-        )
-        queryset = treenode_cache.get(cache_key)
-        if queryset is not None:
-            return list(queryset.values_list("id", flat=True))
-        elif hasattr(self, "closure_model"):
-            return self.closure_model.get_descendants_pks(
-                self, include_self, depth
-            )
-        return []
+        return self.get_descendants_queryset(include_self, depth)\
+            .values_list("id", flat=True)
 
     def get_descendants(self, include_self=False, depth=None):
         """Get a list containing all descendants."""

treenode/models/mixins/node.py

@@ -2,7 +2,7 @@
 """
 TreeNode Node Mixin
 
-Version: 2.1.0
+Version: 2.1.3
 Author: Timur Kady
 Email: timurkady@yandex.com
 """
@@ -25,30 +25,14 @@ class TreeNodeNodeMixin(models.Model):
     @cached_method
     def get_breadcrumbs(self, attr='id'):
         """Optimized breadcrumbs retrieval with direct cache check."""
+
         try:
             self._meta.get_field(attr)
         except FieldDoesNotExist:
             raise ValueError(f"Invalid attribute name: {attr}")
 
-        # Easy logics for roots
-        if self.tn_parent is None:
-            return [getattr(self, attr)]
-
-        # Generate parents cache key
-        cache_key = treenode_cache.generate_cache_key(
-            self._meta.label,
-            self.get_breadcrumbs.__name__,
-            self.tn_parent.pk,
-            attr
-        )
-
-        # Try get value from cache
-        breadcrumbs = treenode_cache.get(cache_key)
-        if breadcrumbs is not None:
-            return breadcrumbs + [getattr(self, attr)]
-
-        queryset = self.get_ancestors_queryset(include_self=True).only(attr)
-        return [getattr(item, attr) for item in queryset]
+        ancestors = self.get_ancestors(include_self=True)
+        return [getattr(node, attr) for node in ancestors]
 
     @cached_method
     def get_depth(self):

treenode/models/mixins/siblings.py

@@ -45,25 +45,25 @@ class TreeNodeSiblingsMixin(models.Model):
         return instance
 
     @cached_method
-    def get_siblings_queryset(self):
+    def get_siblings_queryset(self, include_self=True):
         """Get the siblings queryset with prefetch."""
         if self.tn_parent:
-            qs = self.tn_parent.tn_children.prefetch_related('tn_children')
+            qs = self._meta.model.objects.filter(tn_parent=self.tn_parent)
         else:
             qs = self._meta.model.objects.filter(tn_parent__isnull=True)
-        return qs.exclude(pk=self.pk)
+        return qs if include_self else qs.exclude(pk=self.pk)
 
-    def get_siblings(self):
+    def get_siblings(self, include_self=True):
         """Get a list with all the siblings."""
         return list(self.get_siblings_queryset())
 
-    def get_siblings_count(self):
+    def get_siblings_count(self, include_self=True):
         """Get the siblings count."""
-        return self.get_siblings_queryset().count()
+        return self.get_siblings_queryset(include_self).count()
 
-    def get_siblings_pks(self):
+    def get_siblings_pks(self, include_self=True):
         """Get the siblings pks list."""
-        return [item.pk for item in self.get_siblings_queryset()]
+        return [item.pk for item in self.get_siblings_queryset(include_self)]
 
     def get_first_sibling(self):
         """

treenode/utils/exporter.py

@@ -12,7 +12,7 @@ Features:
 - Provides optimized data extraction for QuerySets.
 - Generates downloadable files with appropriate HTTP responses.
 
-Version: 2.0.11
+Version: 3.0.0
 Author: Timur Kady
 Email: timurkady@yandex.com
 """

treenode/version.py

@@ -4,10 +4,10 @@ TreeNode Version Module
 
 This module defines the current version of the TreeNode package.
 
-Version: 2.1.2
+Version: 2.1.5
 Author: Timur Kady
 Email: timurkady@yandex.com
 """
 
 
-__version__ = '2.1.3'
+__version__ = '2.1.5'