ic-python-db 0.11.0__tar.gz → 0.13.0__tar.gz

{ic_python_db-0.11.0/ic_python_db.egg-info → ic_python_db-0.13.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ic-python-db
-Version: 0.11.0
+Version: 0.13.0
 Summary: A lightweight key-value database written in Python, intended for use on the Internet Computer (IC)
 Home-page: https://github.com/smart-social-contracts/ic-python-db
 Author: Smart Social Contracts
@@ -60,6 +60,7 @@ A lightweight key-value database with entity relationships and audit logging cap
 ## Features
 
 - **Persistent Storage**: Works with StableBTreeMap stable structure for persistent storage on your canister's stable memory so your data persists automatically across canister upgrades.
+- **Range-based paging**: `page`, `load_some` and `rebuild_field_index` read a page of entities with a single ordered `range` call on the stable map (Basilisk `StableBTreeMap.range`), so cost tracks the rows returned rather than the highest id ever issued, and deleted ids are free. Backends without `range` (older CDK builds, plain dict test doubles) transparently fall back to one read per id. `instances()` and `find()` load a whole type: they are deprecated and raise `TooLargeError` past 1,000 entities (use `count()`, `find_by()` or `page()`).
 - **Entity-Relational Database**: Create, read and write entities with OneToOne, OneToMany, ManyToOne, and ManyToMany relationships.
 - **Schema Versioning & Upgrade Safety**: Automatic schema introspection, compatibility checking, and auto-migration for safe changes. Breaking changes without a `migrate()` method are rejected, preventing data corruption on canister upgrades.
 - **Entity Hooks**: Intercept and control entity lifecycle events (create, modify, delete) with `on_event` hooks.
@@ -162,11 +163,12 @@ Then use the defined entities to store objects:
 
     assert Person.count() == 3
     assert Person.max_id() == 4
-    assert Person.instances() == [john, alice, eva]
 
-    # Cursor-based pagination
-    assert Person.load_some(0, 2) == [john, alice]
-    assert Person.load_some(2, 2) == [eva]
+    # Cursor-based pagination: (page, next_from_id), None on the last page
+    page, cursor = Person.page(1, 2)
+    assert page == [john, alice]
+    page, cursor = Person.page(cursor, 2)
+    assert page == [eva] and cursor is None
 
     # Retrieve database contents in JSON format
     print(Database.get_instance().dump_json(pretty=True))
@@ -228,7 +230,29 @@ while cursor is not None:
 
 The rebuild is idempotent. The `indexed` flag is not part of the schema descriptor, so toggling it never triggers upgrade-compatibility errors.
 
-Note: each distinct value's ID list is stored in a single storage entry, so extremely hot values are bounded by the backing store's `max_value_size` — the same constraint as relationship reverse indexes.
+Each matching id is its own storage entry, so a value can match any number of entities without reaching the backing store's `max_value_size`.
+
+## Large Relations
+
+`OneToMany` and `ManyToMany` getters return a `RelationView`, not a list. Questions about the relation and single pages never load every member, so they stay cheap however large it grows:
+
+```python
+if section.stands: ...                       # one read
+len(section.stands)                          # one read
+stand in section.stands                      # two reads
+stands, cursor = section.stands.page(1, 50)  # one range read + 50 loads
+group.members.add(user)                      # index writes only
+```
+
+Iterating, indexing and comparing still work, but they load every member, so past `constants.MAX_UNPAGED` (1,000) they raise `TooLargeError` instead of running into the IC instruction limit. Walk a large relation with `page()`, across calls if needed:
+
+```python
+cursor = 1
+while cursor is not None:
+    stands, cursor = section.stands.page(cursor, 50)
+```
+
+`list(view)` gives a plain list. Deleting a parent is O(1): its index entries are purged in bounded steps by later writes, or by `Database.get_instance().purge_step()` from a timer.
 
 ## Namespaces
 

{ic_python_db-0.11.0 → ic_python_db-0.13.0}/README.md

@@ -11,6 +11,7 @@ A lightweight key-value database with entity relationships and audit logging cap
 ## Features
 
 - **Persistent Storage**: Works with StableBTreeMap stable structure for persistent storage on your canister's stable memory so your data persists automatically across canister upgrades.
+- **Range-based paging**: `page`, `load_some` and `rebuild_field_index` read a page of entities with a single ordered `range` call on the stable map (Basilisk `StableBTreeMap.range`), so cost tracks the rows returned rather than the highest id ever issued, and deleted ids are free. Backends without `range` (older CDK builds, plain dict test doubles) transparently fall back to one read per id. `instances()` and `find()` load a whole type: they are deprecated and raise `TooLargeError` past 1,000 entities (use `count()`, `find_by()` or `page()`).
 - **Entity-Relational Database**: Create, read and write entities with OneToOne, OneToMany, ManyToOne, and ManyToMany relationships.
 - **Schema Versioning & Upgrade Safety**: Automatic schema introspection, compatibility checking, and auto-migration for safe changes. Breaking changes without a `migrate()` method are rejected, preventing data corruption on canister upgrades.
 - **Entity Hooks**: Intercept and control entity lifecycle events (create, modify, delete) with `on_event` hooks.
@@ -113,11 +114,12 @@ Then use the defined entities to store objects:
 
     assert Person.count() == 3
     assert Person.max_id() == 4
-    assert Person.instances() == [john, alice, eva]
 
-    # Cursor-based pagination
-    assert Person.load_some(0, 2) == [john, alice]
-    assert Person.load_some(2, 2) == [eva]
+    # Cursor-based pagination: (page, next_from_id), None on the last page
+    page, cursor = Person.page(1, 2)
+    assert page == [john, alice]
+    page, cursor = Person.page(cursor, 2)
+    assert page == [eva] and cursor is None
 
     # Retrieve database contents in JSON format
     print(Database.get_instance().dump_json(pretty=True))
@@ -179,7 +181,29 @@ while cursor is not None:
 
 The rebuild is idempotent. The `indexed` flag is not part of the schema descriptor, so toggling it never triggers upgrade-compatibility errors.
 
-Note: each distinct value's ID list is stored in a single storage entry, so extremely hot values are bounded by the backing store's `max_value_size` — the same constraint as relationship reverse indexes.
+Each matching id is its own storage entry, so a value can match any number of entities without reaching the backing store's `max_value_size`.
+
+## Large Relations
+
+`OneToMany` and `ManyToMany` getters return a `RelationView`, not a list. Questions about the relation and single pages never load every member, so they stay cheap however large it grows:
+
+```python
+if section.stands: ...                       # one read
+len(section.stands)                          # one read
+stand in section.stands                      # two reads
+stands, cursor = section.stands.page(1, 50)  # one range read + 50 loads
+group.members.add(user)                      # index writes only
+```
+
+Iterating, indexing and comparing still work, but they load every member, so past `constants.MAX_UNPAGED` (1,000) they raise `TooLargeError` instead of running into the IC instruction limit. Walk a large relation with `page()`, across calls if needed:
+
+```python
+cursor = 1
+while cursor is not None:
+    stands, cursor = section.stands.page(cursor, 50)
+```
+
+`list(view)` gives a plain list. Deleting a parent is O(1): its index entries are purged in bounded steps by later writes, or by `Database.get_instance().purge_step()` from a timer.
 
 ## Namespaces
 

{ic_python_db-0.11.0 → ic_python_db-0.13.0}/ic_python_db/__init__.py

@@ -5,6 +5,7 @@ IC Python DB - A lightweight key-value database with entity relationships and au
 from .constants import ACTION_CREATE, ACTION_DELETE, ACTION_MODIFY
 from .db_engine import Database
 from .entity import Entity
+from .errors import TooLargeError
 from .mixins import TimestampedMixin
 from .properties import (
     Boolean,
@@ -14,13 +15,14 @@ from .properties import (
     ManyToOne,
     OneToMany,
     OneToOne,
+    RelationView,
     String,
 )
 from .schema import SchemaIncompatibleError, build_schema, diff_schemas, schema_hash
 from .storage import MemoryStorage, Storage
 from .system_time import SystemTime
 
-__version__ = "0.11.0"
+__version__ = "0.13.0"
 __all__ = [
     "Database",
     "Entity",
@@ -40,6 +42,8 @@ __all__ = [
     "ACTION_MODIFY",
     "ACTION_DELETE",
     "SchemaIncompatibleError",
+    "TooLargeError",
+    "RelationView",
     "build_schema",
     "diff_schemas",
     "schema_hash",

ic_python_db-0.13.0/ic_python_db/constants.py

@@ -0,0 +1,11 @@
+LEVEL_MAX_DEFAULT = 3
+
+# Hook action types
+ACTION_CREATE = "create"
+ACTION_MODIFY = "modify"
+ACTION_DELETE = "delete"
+
+# Largest collection an unpaged operation (iterating a relation, instances())
+# loads in one call; past it they raise TooLargeError. At about 1M
+# instructions per load this stays well inside a query's 5B budget.
+MAX_UNPAGED = 1000