vention-storage 0.5.1__tar.gz → 0.6.5__tar.gz

vention_storage-0.6.5/PKG-INFO

@@ -0,0 +1,494 @@
+Metadata-Version: 2.4
+Name: vention-storage
+Version: 0.6.5
+Summary: A framework for storing and managing component and application data for machine apps.
+License: Proprietary
+Author: VentionCo
+Requires-Python: >=3.10,<3.11
+Classifier: License :: Other/Proprietary License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Requires-Dist: annotated-doc (==0.0.4) ; python_version == "3.10"
+Requires-Dist: annotated-types (==0.7.0) ; python_version == "3.10"
+Requires-Dist: anyio (==4.12.1) ; python_version == "3.10"
+Requires-Dist: click (==8.3.1) ; python_version == "3.10"
+Requires-Dist: colorama (==0.4.6) ; python_version == "3.10" and platform_system == "Windows"
+Requires-Dist: exceptiongroup (==1.3.1) ; python_version == "3.10"
+Requires-Dist: fastapi (==0.121.1) ; python_version == "3.10"
+Requires-Dist: greenlet (==3.3.1) ; python_version == "3.10" and (platform_machine == "aarch64" or platform_machine == "ppc64le" or platform_machine == "x86_64" or platform_machine == "amd64" or platform_machine == "AMD64" or platform_machine == "win32" or platform_machine == "WIN32")
+Requires-Dist: h11 (==0.16.0) ; python_version == "3.10"
+Requires-Dist: idna (==3.11) ; python_version == "3.10"
+Requires-Dist: pydantic (==2.12.5) ; python_version == "3.10"
+Requires-Dist: pydantic-core (==2.41.5) ; python_version == "3.10"
+Requires-Dist: python-multipart (==0.0.20) ; python_version == "3.10"
+Requires-Dist: sqlalchemy (==2.0.46) ; python_version == "3.10"
+Requires-Dist: sqlmodel (==0.0.27) ; python_version == "3.10"
+Requires-Dist: starlette (==0.49.3) ; python_version == "3.10"
+Requires-Dist: typing-extensions (==4.15.0) ; python_version == "3.10"
+Requires-Dist: typing-inspection (==0.4.2) ; python_version == "3.10"
+Requires-Dist: uvicorn (==0.35.0) ; python_version == "3.10"
+Requires-Dist: vention-communication (==0.3.0) ; python_version == "3.10"
+Description-Content-Type: text/markdown
+
+# Vention Storage
+
+A framework for storing and managing component and application data with persistence, validation, and audit trails for machine applications.
+
+## Table of Contents
+
+- [✨ Features](#-features)
+- [🧠 Concepts & Overview](#-concepts--overview)
+- [⚙️ Installation & Setup](#️-installation--setup)
+- [🚀 Quickstart Tutorial](#-quickstart-tutorial)
+- [🛠 How-to Guides](#-how-to-guides)
+- [📖 API Reference](#-api-reference)
+- [🔍 Troubleshooting & FAQ](#-troubleshooting--faq)
+
+## ✨ Features
+
+- Persistent storage with SQLite
+- Automatic audit trails (who, when, what changed)
+- Strong typing & validation via SQLModel
+- Lifecycle hooks before/after insert, update, delete
+- Soft delete with `deleted_at` fields
+- ConnectRPC bundle generation with Create, Read, Update, Delete + audit
+- Health & monitoring actions (audit log, schema diagram)
+- Batch operations for insert/delete
+- Session management with smart reuse & transactions
+- Bootstrap system for one-command setup
+- CSV export/import for backups and migration
+- Database backup/restore with integrity checking
+
+## 🧠 Concepts & Overview
+
+Vention Storage is a component-based persistence layer for machine apps:
+
+- **Database** → SQLite database with managed sessions and transactions
+- **ModelAccessor** → Strongly-typed Create, Read, Update, Delete interface for your SQLModel classes
+- **Hooks** → Functions that run before/after Create, Read, Update, Delete operations
+- **AuditLog** → Automatically records all data mutations
+- **RpcBundle** → Auto-generated ConnectRPC bundle with Create, Read, Update, Delete + database management actions
+
+## ⚙️ Installation & Setup
+
+```bash
+pip install vention-storage
+```
+
+**Optional dependencies:**
+- sqlalchemy-schemadisplay and Graphviz → enable database schema visualization
+
+MacOS:
+```bash
+brew install graphviz
+pip install sqlalchemy-schemadisplay
+```
+
+Linux (Debian/Ubuntu)
+```bash
+sudo apt-get install graphviz
+pip install sqlalchemy-schemadisplay
+```
+
+## 🚀 Quickstart Tutorial
+
+Define a model, bootstrap storage, and get full Create, Read, Update, Delete RPC actions in minutes:
+
+```python
+from datetime import datetime
+from typing import Optional
+from sqlmodel import Field, SQLModel
+from communication.app import VentionApp
+from storage.bootstrap import bootstrap
+from storage.accessor import ModelAccessor
+from storage.vention_communication import build_storage_bundle
+
+class User(SQLModel, table=True):
+    id: Optional[int] = Field(default=None, primary_key=True)
+    name: str
+    email: str
+    deleted_at: Optional[datetime] = Field(default=None, index=True)
+
+# Initialize database
+bootstrap(
+    database_url="sqlite:///./my_app.db",
+    create_tables=True
+)
+
+# Create accessor
+user_accessor = ModelAccessor(User, "users")
+
+# Build RPC bundle and add to app
+app = VentionApp(name="my-app")
+storage_bundle = build_storage_bundle(
+    accessors=[user_accessor],
+    max_records_per_model=100,
+    enable_db_actions=True
+)
+app.add_bundle(storage_bundle)
+app.finalize()
+```
+
+➡️ You now have Create, Read, Update, Delete, audit, backup, and CSV actions available via ConnectRPC.
+
+## 🛠 How-to Guides
+
+### Bootstrap Multiple Models
+
+```python
+# user_accessor was created earlier in the Quickstart example
+# Reuse it here to bootstrap multiple models at once
+
+product_accessor = ModelAccessor(Product, "products")
+
+# Build bundle with multiple accessors
+storage_bundle = build_storage_bundle(
+    accessors=[user_accessor, product_accessor],
+    max_records_per_model=100,
+    enable_db_actions=True
+)
+app.add_bundle(storage_bundle)
+```
+
+### Export to CSV
+
+```python
+# Using ConnectRPC client
+from communication.client import ConnectClient
+
+client = ConnectClient("http://localhost:8000")
+response = await client.call("Database_ExportZip", {})
+with open("backup.zip", "wb") as f:
+    f.write(response.data)
+```
+
+### Backup & Restore
+
+```python
+# Backup
+backup_response = await client.call("Database_BackupSqlite", {})
+with open(backup_response.filename, "wb") as f:
+    f.write(backup_response.data)
+
+# Restore
+with open("backup.sqlite", "rb") as f:
+    restore_response = await client.call(
+        "Database_RestoreSqlite",
+        {
+            "bytes": f.read(),
+            "filename": "backup.sqlite",
+            "integrity_check": True,
+            "dry_run": False
+        }
+    )
+```
+
+### Use Lifecycle Hooks
+
+```python
+@user_accessor.before_insert()
+def validate_email(session, instance):
+    if "@" not in instance.email:
+        raise ValueError("Invalid email")
+
+@user_accessor.after_insert()
+def log_creation(session, instance):
+    print(f"User created: {instance.name}")
+```
+
+### Query Audit Logs
+
+```python
+from storage.auditor import AuditLog
+from sqlmodel import select
+
+with database.transaction() as session:
+    logs = session.exec(select(AuditLog).where(AuditLog.component == "users")).all()
+```
+
+### Using the model accessors
+
+```python
+# Create
+user = user_accessor.insert(User(name="Alice", email="alice@example.com"), actor="admin")
+
+# Read
+user = user_accessor.get(user.id)
+
+# Update
+user.name = "Alice Smith"
+user_accessor.save(user, actor="admin")
+
+# Delete
+user_accessor.delete(user.id, actor="admin")
+
+# Restore (for soft-deleted models)
+user_accessor.restore(user.id, actor="admin")
+
+# Find users by exact match
+users = user_accessor.find(user_accessor.where.email == "alice@example.com")
+
+# Multiple conditions (AND logic)
+users = user_accessor.find(
+    user_accessor.where.name == "Alice",
+    user_accessor.where.email == "alice@example.com"
+)
+
+# Comparison operators
+adults = user_accessor.find(user_accessor.where.age >= 18)
+recent = user_accessor.find(user_accessor.where.created_at > cutoff_date)
+
+# String operations
+smiths = user_accessor.find(user_accessor.where.name.contains("Smith"))
+gmail_users = user_accessor.find(user_accessor.where.email.endswith("@gmail.com"))
+search = user_accessor.find(user_accessor.where.name.ilike("%alice%"))  # case-insensitive
+
+# Collection check
+admins = user_accessor.find(user_accessor.where.role.in_(["admin", "superadmin"]))
+
+# Null checks
+unverified = user_accessor.find(user_accessor.where.verified_at.is_(None))
+verified = user_accessor.find(user_accessor.where.verified_at.isnot(None))
+
+# With pagination and sorting
+page = user_accessor.find(
+    user_accessor.where.status == "active",
+    limit=10,
+    offset=20,
+    order_by="created_at",
+    order_desc=True
+)
+
+# Include soft-deleted records
+all_users = user_accessor.find(
+    user_accessor.where.role == "admin",
+    include_deleted=True
+)
+```
+
+### Using ConnectRPC Client
+
+Once the bundle is added to your `VentionApp`, each `ModelAccessor` automatically exposes full CRUD actions via ConnectRPC.
+
+Example: interacting with the `Users` RPC actions.
+
+```typescript
+import { createPromiseClient } from "@connectrpc/connect";
+import { createConnectTransport } from "@connectrpc/connect-web";
+
+const transport = createConnectTransport({
+  baseUrl: "http://localhost:8000",
+});
+
+const client = createPromiseClient(YourServiceClient, transport);
+
+// Create
+export async function createUser(name: string, email: string) {
+  const res = await client.usersCreateRecord({
+    record: { name, email },
+    actor: "operator"
+  });
+  return res.record;
+}
+
+// Read
+export async function getUser(id: number) {
+  const res = await client.usersGetRecord({
+    recordId: id,
+    includeDeleted: false
+  });
+  return res.record;
+}
+
+// Update
+export async function updateUser(id: number, name: string) {
+  const res = await client.usersUpdateRecord({
+    recordId: id,
+    record: { name },
+    actor: "operator"
+  });
+  return res.record;
+}
+
+// Delete (soft delete if model supports deleted_at)
+export async function deleteUser(id: number) {
+  await client.usersDeleteRecord({
+    recordId: id,
+    actor: "operator"
+  });
+}
+
+// Restore
+export async function restoreUser(id: number) {
+  const res = await client.usersRestoreRecord({
+    recordId: id,
+    actor: "operator"
+  });
+  return res.record;
+}
+
+// List
+export async function listUsers() {
+  const res = await client.usersListRecords({
+    includeDeleted: false
+  });
+  return res.records;
+}
+
+// Find by exact match
+export async function findUserByEmail(email: string) {
+  const res = await client.usersFindRecords({
+    filters: [
+      { field: "email", operation: "eq", value: email }
+    ]
+  });
+  return res.records;
+}
+
+// Find with multiple conditions (AND logic)
+export async function findActiveAdmins() {
+  const res = await client.usersFindRecords({
+    filters: [
+      { field: "role", operation: "eq", value: "admin" },
+      { field: "age", operation: "gte", value: "18" }
+    ]
+  });
+  return res.records;
+}
+
+// Find with null checks
+export async function findUnverifiedUsers() {
+  const res = await client.usersFindRecords({
+    filters: [
+      { field: "verified_at", operation: "is_null" }
+    ]
+  });
+  return res.records;
+}
+
+// Complex query example
+export async function findRecentPremiumUsers(cutoffDate: string) {
+  const res = await client.usersFindRecords({
+    filters: [
+      { field: "subscription", operation: "in", value: ["premium", "enterprise"] },
+      { field: "created_at", operation: "gte", value: cutoffDate },
+      { field: "email_verified", operation: "is_not_null" }
+    ],
+    limit: 50,
+    orderBy: "created_at",
+    orderDesc: true
+  });
+  return res.records;
+}
+```
+
+### Filter Operations Reference
+
+| Operation      | Description                          |
+|----------------|--------------------------------------|
+| `eq`           | Exact match                          |
+| `ne`           | Not equal to value                   |
+| `gt`           | Greater than value                   |
+| `gte`          | Greater than or equal                |
+| `lt`           | Less than value                      |
+| `lte`          | Less than or equal                   |
+| `in`           | Value in array                       |
+| `not_in`       | Value not in array                   |
+| `contains`     | Field contains substring             |
+| `starts_with`  | Field starts with prefix             |
+| `ends_with`    | Field ends with suffix               |
+| `like`         | Case-insensitive pattern match       |
+| `is_null`      | Field is null (no value needed)      |
+| `is_not_null`  | Field is not null (no value needed)  |
+
+
+## 📖 API Reference
+
+### bootstrap
+
+```python
+def bootstrap(
+    *,
+    database_url: Optional[str] = None,
+    create_tables: bool = True,
+) -> None
+```
+
+Initialize the database engine and optionally create tables. This function performs environment setup only.
+
+### build_storage_bundle
+
+```python
+def build_storage_bundle(
+    *,
+    accessors: Sequence[ModelAccessor[Any]],
+    max_records_per_model: Optional[int] = 5,
+    enable_db_actions: bool = True,
+) -> RpcBundle
+```
+
+Build a ConnectRPC RpcBundle exposing CRUD and database utilities. Returns an `RpcBundle` that can be added to a `VentionApp` using `app.add_bundle()`.
+
+### ModelAccessor
+
+```python
+ModelAccessor(
+    model: Type[ModelType],
+    component_name: str,
+    *,
+    enable_auditing: bool = True,
+)
+```
+
+**Read**
+- `get(id, include_deleted=False) -> Optional[ModelType]`
+- `all(include_deleted=False) -> List[ModelType]`
+
+**Write**
+- `insert(obj, actor="internal") -> ModelType`
+- `save(obj, actor="internal") -> ModelType`
+- `delete(id, actor="internal") -> bool`
+- `restore(id, actor="internal") -> bool`
+
+**Batch**
+- `insert_many(objs, actor="internal") -> List[ModelType]`
+- `delete_many(ids, actor="internal") -> int`
+
+**Hooks**
+- `@accessor.before_insert()`
+- `@accessor.after_insert()`
+- `@accessor.before_update()`
+- `@accessor.after_update()`
+- `@accessor.before_delete()`
+- `@accessor.after_delete()`
+
+**Parameters**
+- `enable_auditing`: If `False`, disables audit logging for this accessor. Useful for models that shouldn't be audited (e.g., audit logs themselves). Defaults to `True`.
+
+### Database Helpers
+
+- `database.set_database_url(url: str) -> None`
+- `database.get_engine() -> Engine`
+- `database.transaction() -> Iterator[Session]`
+- `database.use_session(session: Optional[Session] = None) -> Iterator[Session]`
+
+### AuditLog model
+
+```python
+class AuditLog(SQLModel, table=True):
+    id: int
+    timestamp: datetime
+    component: str
+    record_id: int
+    operation: str
+    actor: str
+    before: Optional[Dict[str, Any]]
+    after: Optional[Dict[str, Any]]
+```
+
+## 🔍 Troubleshooting & FAQ
+
+- **Diagram endpoint fails** → Ensure Graphviz + sqlalchemy-schemadisplay are installed.
+- **No audit actor shown** → Provide X-User header in API requests.
+- **Soft delete not working** → Your model must have a `deleted_at` field.
+- **Restore fails** → Ensure `integrity_check=True` passes when restoring backups.