vention-storage 0.5.1__py3-none-any.whl → 0.5.4__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,318 @@
1
+ Metadata-Version: 2.1
2
+ Name: vention-storage
3
+ Version: 0.5.4
4
+ Summary: A framework for storing and managing component and application data for machine apps.
5
+ License: Proprietary
6
+ Author: VentionCo
7
+ Requires-Python: >=3.10,<3.11
8
+ Classifier: License :: Other/Proprietary License
9
+ Classifier: Programming Language :: Python :: 3
10
+ Classifier: Programming Language :: Python :: 3.10
11
+ Requires-Dist: fastapi (==0.121.1)
12
+ Requires-Dist: python-multipart (>=0.0.20,<0.0.21)
13
+ Requires-Dist: sqlmodel (==0.0.27)
14
+ Requires-Dist: uvicorn (>=0.35.0,<0.36.0)
15
+ Description-Content-Type: text/markdown
16
+
17
+ # Vention Storage
18
+
19
+ A framework for storing and managing component and application data with persistence, validation, and audit trails for machine applications.
20
+
21
+ ## Table of Contents
22
+
23
+ - [✨ Features](#-features)
24
+ - [🧠 Concepts & Overview](#-concepts--overview)
25
+ - [⚙️ Installation & Setup](#️-installation--setup)
26
+ - [🚀 Quickstart Tutorial](#-quickstart-tutorial)
27
+ - [🛠 How-to Guides](#-how-to-guides)
28
+ - [📖 API Reference](#-api-reference)
29
+ - [🔍 Troubleshooting & FAQ](#-troubleshooting--faq)
30
+
31
+ ## ✨ Features
32
+
33
+ - Persistent storage with SQLite
34
+ - Automatic audit trails (who, when, what changed)
35
+ - Strong typing & validation via SQLModel
36
+ - Lifecycle hooks before/after insert, update, delete
37
+ - Soft delete with `deleted_at` fields
38
+ - REST API generation with Create, Read, Update, Delete + audit
39
+ - Health & monitoring endpoints (audit log, schema diagram)
40
+ - Batch operations for insert/delete
41
+ - Session management with smart reuse & transactions
42
+ - Bootstrap system for one-command setup
43
+ - CSV export/import for backups and migration
44
+ - Database backup/restore with integrity checking
45
+
46
+ ## 🧠 Concepts & Overview
47
+
48
+ Vention Storage is a component-based persistence layer for machine apps:
49
+
50
+ - **Database** → SQLite database with managed sessions and transactions
51
+ - **ModelAccessor** → Strongly-typed Create, Read, Update, Delete interface for your SQLModel classes
52
+ - **Hooks** → Functions that run before/after Create, Read, Update, Delete operations
53
+ - **AuditLog** → Automatically records all data mutations
54
+ - **Routers** → Auto-generated FastAPI endpoints for Create, Read, Update, Delete + database management
55
+
56
+ ## ⚙️ Installation & Setup
57
+
58
+ ```bash
59
+ pip install vention-storage
60
+ ```
61
+
62
+ **Optional dependencies:**
63
+ - sqlalchemy-schemadisplay and Graphviz → enable database schema visualization
64
+
65
+ MacOS:
66
+ ```bash
67
+ brew install graphviz
68
+ pip install sqlalchemy-schemadisplay
69
+ ```
70
+
71
+ Linux (Debian/Ubuntu)
72
+ ```bash
73
+ sudo apt-get install graphviz
74
+ pip install sqlalchemy-schemadisplay
75
+ ```
76
+
77
+ ## 🚀 Quickstart Tutorial
78
+
79
+ Define a model, bootstrap storage, and get full Create, Read, Update, Delete endpoints in minutes:
80
+
81
+ ```python
82
+ from datetime import datetime
83
+ from typing import Optional
84
+ from sqlmodel import Field, SQLModel
85
+ from fastapi import FastAPI
86
+ from storage.bootstrap import bootstrap
87
+ from storage.accessor import ModelAccessor
88
+
89
+ class User(SQLModel, table=True):
90
+ id: Optional[int] = Field(default=None, primary_key=True)
91
+ name: str
92
+ email: str
93
+ deleted_at: Optional[datetime] = Field(default=None, index=True)
94
+
95
+ app = FastAPI()
96
+ user_accessor = ModelAccessor(User, "users")
97
+
98
+ bootstrap(
99
+ app,
100
+ accessors=[user_accessor],
101
+ database_url="sqlite:///./my_app.db",
102
+ create_tables=True
103
+ )
104
+ ```
105
+
106
+ ➡️ You now have Create, Read, Update, Delete, audit, backup, and CSV endpoints at `/users` and `/db`.
107
+
108
+ ## 🛠 How-to Guides
109
+
110
+ ### Bootstrap Multiple Models
111
+
112
+ ```python
113
+ # user_accessor was created earlier in the Quickstart example
114
+ # Reuse it here to bootstrap multiple models at once
115
+
116
+ product_accessor = ModelAccessor(Product, "products")
117
+
118
+ bootstrap(
119
+ app,
120
+ accessors=[user_accessor, product_accessor],
121
+ database_url="sqlite:///./my_app.db",
122
+ create_tables=True,
123
+ max_records_per_model=100,
124
+ enable_db_router=True
125
+ )
126
+ ```
127
+
128
+ ### Export to CSV
129
+
130
+ ```python
131
+ response = requests.get("http://localhost:8000/db/export.zip")
132
+ with open("backup.zip", "wb") as f:
133
+ f.write(response.content)
134
+ ```
135
+
136
+ ### Backup & Restore
137
+
138
+ ```python
139
+ # Backup
140
+ r = requests.get("http://localhost:8000/db/backup.sqlite")
141
+ with open("backup.sqlite", "wb") as f: f.write(r.content)
142
+
143
+ # Restore
144
+ with open("backup.sqlite", "rb") as f:
145
+ files = {"file": ("backup.sqlite", f, "application/x-sqlite3")}
146
+ requests.post("http://localhost:8000/db/restore", files=files)
147
+ ```
148
+
149
+ ### Use Lifecycle Hooks
150
+
151
+ ```python
152
+ @user_accessor.before_insert()
153
+ def validate_email(session, instance):
154
+ if "@" not in instance.email:
155
+ raise ValueError("Invalid email")
156
+
157
+ @user_accessor.after_insert()
158
+ def log_creation(session, instance):
159
+ print(f"User created: {instance.name}")
160
+ ```
161
+
162
+ ### Query Audit Logs
163
+
164
+ ```python
165
+ from storage.auditor import AuditLog
166
+ from sqlmodel import select
167
+
168
+ with database.transaction() as session:
169
+ logs = session.exec(select(AuditLog).where(AuditLog.component == "users")).all()
170
+ ```
171
+
172
+ ### Using the model accessors
173
+
174
+ ```python
175
+ # Create
176
+ user = user_accessor.insert(User(name="Alice", email="alice@example.com"), actor="admin")
177
+
178
+ # Read
179
+ user = user_accessor.get(user.id)
180
+
181
+ # Update
182
+ user.name = "Alice Smith"
183
+ user_accessor.save(user, actor="admin")
184
+
185
+ # Delete
186
+ user_accessor.delete(user.id, actor="admin")
187
+
188
+ # Restore (for soft-deleted models)
189
+ user_accessor.restore(user.id, actor="admin")
190
+ ```
191
+
192
+
193
+ ### Using the REST API
194
+
195
+ Once bootstrapped, each `ModelAccessor` automatically exposes full CRUD endpoints.
196
+
197
+ Example: interacting with the `/users` API.
198
+
199
+ ```typescript
200
+ import axios from "axios";
201
+
202
+ const api = axios.create({
203
+ baseURL: "http://localhost:8000", // your backend url
204
+ headers: { "X-User": "operator" }, // used in audit logs
205
+ });
206
+
207
+ // Create
208
+ export async function createUser(name: string, email: string) {
209
+ const res = await api.post("/users/", { name, email });
210
+ return res.data;
211
+ }
212
+
213
+ // Read
214
+ export async function getUser(id: number) {
215
+ const res = await api.get(`/users/${id}`);
216
+ return res.data;
217
+ }
218
+
219
+ // Update
220
+ export async function updateUser(id: number, name: string) {
221
+ const res = await api.put(`/users/${id}`, { name });
222
+ return res.data;
223
+ }
224
+
225
+ // Delete (soft delete if model supports deleted_at)
226
+ export async function deleteUser(id: number) {
227
+ await api.delete(`/users/${id}`);
228
+ }
229
+
230
+ // Restore
231
+ export async function restoreUser(id: number) {
232
+ await api.post(`/users/${id}/restore`);
233
+ }
234
+
235
+ // List
236
+ export async function listUsers() {
237
+ const res = await api.get("/users/");
238
+ return res.data;
239
+ }
240
+ ```
241
+
242
+
243
+ ## 📖 API Reference
244
+
245
+ ### bootstrap
246
+
247
+ ```python
248
+ def bootstrap(
249
+ app: FastAPI,
250
+ *,
251
+ accessors: Iterable[ModelAccessor[Any]],
252
+ database_url: Optional[str] = None,
253
+ create_tables: bool = True,
254
+ max_records_per_model: Optional[int] = 5,
255
+ enable_db_router: bool = True,
256
+ ) -> None
257
+ ```
258
+
259
+ ### ModelAccessor
260
+
261
+ ```python
262
+ ModelAccessor(model: Type[ModelType], component_name: str)
263
+ ```
264
+
265
+ **Read**
266
+ - `get(id, include_deleted=False) -> Optional[ModelType]`
267
+ - `all(include_deleted=False) -> List[ModelType]`
268
+
269
+ **Write**
270
+ - `insert(obj, actor="internal") -> ModelType`
271
+ - `save(obj, actor="internal") -> ModelType`
272
+ - `delete(id, actor="internal") -> bool`
273
+ - `restore(id, actor="internal") -> bool`
274
+
275
+ **Batch**
276
+ - `insert_many(objs, actor="internal") -> List[ModelType]`
277
+ - `delete_many(ids, actor="internal") -> int`
278
+
279
+ **Hooks**
280
+ - `@accessor.before_insert()`
281
+ - `@accessor.after_insert()`
282
+ - `@accessor.before_update()`
283
+ - `@accessor.after_update()`
284
+ - `@accessor.before_delete()`
285
+ - `@accessor.after_delete()`
286
+
287
+ ### Routers
288
+
289
+ - `build_crud_router(accessor, max_records=100) -> APIRouter`
290
+ - `build_db_router(audit_default_limit=100, audit_max_limit=1000) -> APIRouter`
291
+
292
+ ### Database Helpers
293
+
294
+ - `database.set_database_url(url: str) -> None`
295
+ - `database.get_engine() -> Engine`
296
+ - `database.transaction() -> Iterator[Session]`
297
+ - `database.use_session(session: Optional[Session] = None) -> Iterator[Session]`
298
+
299
+ ### AuditLog model
300
+
301
+ ```python
302
+ class AuditLog(SQLModel, table=True):
303
+ id: int
304
+ timestamp: datetime
305
+ component: str
306
+ record_id: int
307
+ operation: str
308
+ actor: str
309
+ before: Optional[Dict[str, Any]]
310
+ after: Optional[Dict[str, Any]]
311
+ ```
312
+
313
+ ## 🔍 Troubleshooting & FAQ
314
+
315
+ - **Diagram endpoint fails** → Ensure Graphviz + sqlalchemy-schemadisplay are installed.
316
+ - **No audit actor shown** → Provide X-User header in API requests.
317
+ - **Soft delete not working** → Your model must have a `deleted_at` field.
318
+ - **Restore fails** → Ensure `integrity_check=True` passes when restoring backups.
@@ -8,6 +8,6 @@ storage/io_helpers.py,sha256=c9Lzqc1kKwMquleCriGHYIIWVgDUmJRakIB1TNMpOjs,5061
8
8
  storage/router_database.py,sha256=UcsU_OBXVynNAGW6cHYdHvFdmfTtzBcO6x28mXlbsR4,10177
9
9
  storage/router_model.py,sha256=uLxnn1zzeWTorEOEH5ExEsc3lHWCxjZwROSWaYbOXuU,8316
10
10
  storage/utils.py,sha256=uK30rKwwb1lqxfP3qQzsqH53umW-dCLh1jOec1r2p1k,588
11
- vention_storage-0.5.1.dist-info/METADATA,sha256=bfwnKrU2lBjo3W5TD-kPBhWNN7b_jFLRQPWMdiwkZvM,14952
12
- vention_storage-0.5.1.dist-info/WHEEL,sha256=sP946D7jFCHeNz5Iq4fL4Lu-PrWrFsgfLXbbkciIZwg,88
13
- vention_storage-0.5.1.dist-info/RECORD,,
11
+ vention_storage-0.5.4.dist-info/METADATA,sha256=y7xsutKsOl6jkroktn2T8Qkm4vhnVQRnuk7UkQoTwOc,8396
12
+ vention_storage-0.5.4.dist-info/WHEEL,sha256=Nq82e9rUAnEjt98J6MlVmMCZb-t9cYE2Ir1kpBmnWfs,88
13
+ vention_storage-0.5.4.dist-info/RECORD,,
@@ -1,4 +1,4 @@
1
1
  Wheel-Version: 1.0
2
- Generator: poetry-core 1.9.0
2
+ Generator: poetry-core 1.9.1
3
3
  Root-Is-Purelib: true
4
4
  Tag: py3-none-any
@@ -1,509 +0,0 @@
1
- Metadata-Version: 2.1
2
- Name: vention-storage
3
- Version: 0.5.1
4
- Summary: A framework for storing and managing component and application data for machine apps.
5
- License: Proprietary
6
- Author: VentionCo
7
- Requires-Python: >=3.9,<3.11
8
- Classifier: License :: Other/Proprietary License
9
- Classifier: Programming Language :: Python :: 3
10
- Classifier: Programming Language :: Python :: 3.9
11
- Classifier: Programming Language :: Python :: 3.10
12
- Requires-Dist: fastapi (>=0.116.1,<0.117.0)
13
- Requires-Dist: poetry-dynamic-versioning (>=1.9.1,<2.0.0)
14
- Requires-Dist: python-multipart (>=0.0.20,<0.0.21)
15
- Requires-Dist: sqlmodel (>=0.0.24,<0.0.25)
16
- Requires-Dist: uvicorn (>=0.35.0,<0.36.0)
17
- Description-Content-Type: text/markdown
18
-
19
- # Vention Storage
20
-
21
- A framework for storing and managing component and application data with persistence, validation, and audit trails for machine applications.
22
-
23
- ## 🎯 Overview
24
-
25
- Vention Storage provides a modular, component-based storage system that offers:
26
-
27
- - **🔒 Persistent Data Storage** - Data survives reboots with SQLite
28
- - **🔄 Automatic Audit Trails** - Track who made changes and when
29
- - **🛡️ Strong Type Safety** - Full type hints and validation
30
- - **⚡ Lifecycle Hooks** - Before/after insert/update/delete operations
31
- - **🗑️ Soft Delete Support** - Optional soft deletion with `deleted_at` fields
32
- - **🌐 REST API Endpoints** - Automatic CRUD API generation with audit trails
33
- - **📊 Database Health & Monitoring** - Health checks and database schema visualization
34
- - **Batch Operations** - Efficient bulk insert/delete operations
35
- - **Session Management** - Smart session reuse and transaction handling
36
- - **🚀 Bootstrap System** - One-command setup for entire storage system
37
- - **📊 CSV Export/Import** - Easy data backup and migration
38
- - **💾 Database Backup/Restore** - Full SQLite backup and restore functionality
39
-
40
- ### Basic Usage
41
-
42
- ```python
43
- from datetime import datetime
44
- from typing import Optional
45
- from sqlmodel import Field, SQLModel
46
- from storage import database
47
- from storage.accessor import ModelAccessor
48
- from storage.router_model import build_crud_router
49
- from storage.router_database import build_db_router
50
- from storage.bootstrap import bootstrap
51
- from fastapi import FastAPI
52
-
53
- # 1. Define your models
54
- class User(SQLModel, table=True):
55
- id: Optional[int] = Field(default=None, primary_key=True)
56
- name: str
57
- email: str
58
- deleted_at: Optional[datetime] = Field(default=None, index=True) # Optional soft delete
59
-
60
- # 2. Quick setup with bootstrap
61
- app = FastAPI()
62
- user_accessor = ModelAccessor(User, "users")
63
-
64
- bootstrap(
65
- app,
66
- accessors=[user_accessor],
67
- database_url="sqlite:///./my_app.db",
68
- create_tables=True
69
- )
70
-
71
- # Now you have full CRUD API at /users with audit trails, backup/restore, and CSV export!
72
- ```
73
-
74
- ## 🚀 Bootstrap System
75
-
76
- The `bootstrap` function provides a convenient way to set up the entire storage system for a FastAPI application:
77
-
78
- ```python
79
- from fastapi import FastAPI
80
- from storage.bootstrap import bootstrap
81
- from storage.accessor import ModelAccessor
82
-
83
- app = FastAPI()
84
-
85
- # Define your accessors
86
- user_accessor = ModelAccessor(User, "users")
87
- product_accessor = ModelAccessor(Product, "products")
88
-
89
- # Bootstrap the entire system
90
- bootstrap(
91
- app,
92
- accessors=[user_accessor, product_accessor],
93
- database_url="sqlite:///./my_app.db",
94
- create_tables=True,
95
- max_records_per_model=100,
96
- enable_db_router=True
97
- )
98
- ```
99
-
100
- ### Bootstrap Features
101
-
102
- - **Automatic database setup** with optional table creation
103
- - **CRUD router generation** for all registered accessors
104
- - **Database monitoring endpoints** (health, audit, diagram)
105
- - **Configurable record limits** to prevent abuse
106
- - **Optional database URL override**
107
-
108
- ## 📊 CSV Export
109
-
110
- Export your entire database as CSV files for backup and migration:
111
-
112
- ```python
113
- # Export all tables as CSV
114
- response = requests.get("http://localhost:8000/db/export.zip")
115
- with open("backup.zip", "wb") as f:
116
- f.write(response.content)
117
- ```
118
-
119
- The export creates a ZIP file containing one CSV file per table, with proper handling of datetime fields and data types.
120
-
121
- ## 💾 Backup & Restore
122
-
123
- Full database backup and restore functionality using SQLite's native backup API:
124
-
125
- ### Backup
126
- ```python
127
- # Create a complete database backup
128
- response = requests.get("http://localhost:8000/db/backup.sqlite")
129
- with open("backup.sqlite", "wb") as f:
130
- f.write(response.content)
131
- ```
132
-
133
- ### Restore
134
- ```python
135
- # Restore from a backup file
136
- with open("backup.sqlite", "rb") as f:
137
- files = {"file": ("backup.sqlite", f, "application/x-sqlite3")}
138
- response = requests.post(
139
- "http://localhost:8000/db/restore",
140
- files=files,
141
- params={"integrity_check": True, "dry_run": False}
142
- )
143
- ```
144
-
145
- ### Backup/Restore Features
146
-
147
- - **Atomic operations** - Database replacement is atomic to prevent corruption
148
- - **Integrity checking** - Optional PRAGMA integrity_check before restore
149
- - **Dry run mode** - Validate backup files without actually restoring
150
- - **Consistent backups** - Uses SQLite's backup API for data consistency
151
- - **Automatic engine disposal** - Properly handles database connections during restore
152
-
153
- ## 🔧 Core Components
154
-
155
- ### Database Management
156
-
157
- ```python
158
- from storage import database
159
-
160
- # Configure database URL (must be called before first use)
161
- database.set_database_url("sqlite:///./my_app.db")
162
-
163
- # Get the engine
164
- engine = database.get_engine()
165
-
166
- # Use transactions for atomic operations
167
- with database.transaction() as session:
168
- # All operations in this block are atomic
169
- user1 = user_accessor.insert(User(name="Alice"), actor="system")
170
- user2 = user_accessor.insert(User(name="Bob"), actor="system")
171
- # If any operation fails, both are rolled back
172
- ```
173
-
174
- ### Model Accessors
175
-
176
- The `ModelAccessor` provides a strongly-typed interface for CRUD operations:
177
-
178
- ```python
179
- # Create accessor for your model
180
- accessor = ModelAccessor(MyModel, "component_name")
181
-
182
- # Basic CRUD operations
183
- obj = accessor.insert(MyModel(...), actor="user")
184
- obj = accessor.get(123)
185
- obj = accessor.save(updated_obj, actor="user")
186
- success = accessor.delete(123, actor="user")
187
-
188
- # Batch operations
189
- objects = accessor.insert_many([MyModel(...), MyModel(...)], actor="user")
190
- deleted_count = accessor.delete_many([123, 456], actor="user")
191
-
192
- # Soft delete operations
193
- success = accessor.restore(123, actor="user") # Only for models with deleted_at
194
- ```
195
-
196
- ### Lifecycle Hooks
197
-
198
- Register hooks to run before/after operations:
199
-
200
- ```python
201
- @accessor.before_insert()
202
- def validate_email(session, instance):
203
- """Enforce schema-level validation (safe here)."""
204
- if not instance.email or "@" not in instance.email:
205
- raise ValueError("Invalid email")
206
-
207
- @accessor.after_insert()
208
- def log_creation(session, instance):
209
- """Lightweight side-effect: write to a log or metrics system."""
210
- print(f"Row created in {session.bind.url}: {instance}")
211
-
212
- # Hooks run in the same transaction as the operation
213
- # If a hook fails, the entire operation is rolled back
214
- ```
215
-
216
- ### Audit Trails
217
-
218
- All operations are automatically audited:
219
-
220
- ```python
221
- from storage.auditor import AuditLog
222
- from sqlmodel import select
223
-
224
- # Audit logs are automatically created for all operations
225
- with database.transaction() as session:
226
- # This operation will be audited
227
- user = user_accessor.insert(User(name="Alice"), actor="admin")
228
-
229
- # Query audit logs
230
- logs = session.exec(select(AuditLog).where(AuditLog.component == "users")).all()
231
- for log in logs:
232
- print(f"{log.timestamp}: {log.operation} by {log.actor}")
233
- print(f" Before: {log.before}")
234
- print(f" After: {log.after}")
235
- ```
236
-
237
- ## 🌐 REST API Generation
238
-
239
- ### Automatic CRUD Endpoints
240
-
241
- The `build_crud_router` function automatically generates full CRUD endpoints for any model:
242
-
243
- ```python
244
- from storage.router_model import build_crud_router
245
- from fastapi import FastAPI
246
-
247
- app = FastAPI()
248
-
249
- # Create router for User model
250
- user_router = build_crud_router(user_accessor)
251
- app.include_router(user_router)
252
-
253
- # Automatically generates these endpoints:
254
- # GET /users/ - List all users
255
- # GET /users/{id} - Get specific user
256
- # POST /users/ - Create new user
257
- # PUT /users/{id} - Update user
258
- # DELETE /users/{id} - Delete user
259
- # POST /users/{id}/restore - Restore soft-deleted user
260
- ```
261
-
262
- ### API Features
263
-
264
- - **Automatic validation** using your SQLModel schemas
265
- - **Audit trails** for all operations (requires `X-User` header, which is used to identify users in the audit trail)
266
- - **Soft delete support** for models with `deleted_at` fields
267
- - **Configurable record limits** to prevent abuse
268
- - **Proper HTTP status codes** and error handling
269
- - **OpenAPI documentation** automatically generated
270
-
271
- ### Usage Example
272
-
273
- ```python
274
- import requests
275
-
276
- # Create user, X-User is the user who will be blamed in the audit log: Operator, Supervisor, Admin, etc.
277
- response = requests.post(
278
- "http://localhost:8000/users/",
279
- json={"name": "Alice", "email": "alice@example.com"},
280
- headers={"X-User": "admin"}
281
- )
282
- user = response.json()
283
-
284
- # Update user
285
- response = requests.put(
286
- f"http://localhost:8000/users/{user['id']}",
287
- json={"name": "Alice Smith"},
288
- headers={"X-User": "admin"}
289
- )
290
-
291
- # List users
292
- response = requests.get("http://localhost:8000/users/")
293
- users = response.json()
294
-
295
- # Soft delete user
296
- requests.delete(f"http://localhost:8000/users/{user['id']}", headers={"X-User": "admin"})
297
-
298
- # Restore user
299
- requests.post(f"http://localhost:8000/users/{user['id']}/restore", headers={"X-User": "admin"})
300
- ```
301
-
302
- ### Database Health & Monitoring
303
-
304
- The `build_db_router` function provides database health and monitoring endpoints:
305
-
306
- ```python
307
- from storage.router_database import build_db_router
308
-
309
- app.include_router(build_db_router())
310
-
311
- # Available endpoints:
312
- # GET /db/health - Database health check
313
- # GET /db/audit - Query audit logs with filters
314
- # GET /db/diagram.svg - Database schema visualization
315
- # GET /db/export.zip - Export all tables as CSV
316
- # GET /db/backup.sqlite - Full database backup
317
- # POST /db/restore - Restore from backup file
318
- ```
319
-
320
- ### Audit Log Querying
321
-
322
- ```python
323
- # Query audit logs with filters
324
- response = requests.get("http://localhost:8000/db/audit", params={
325
- "component": "users",
326
- "operation": "create",
327
- "actor": "admin",
328
- "since": "2023-01-01T00:00:00Z",
329
- "limit": 100,
330
- "offset": 0
331
- })
332
-
333
- audit_logs = response.json()
334
- ```
335
-
336
- ### Database Schema Visualization
337
-
338
- ```python
339
- # Get database schema as SVG diagram
340
- response = requests.get("http://localhost:8000/db/diagram.svg")
341
- # Returns SVG image of your database schema
342
- # Requires sqlalchemy-schemadisplay and Graphviz
343
- ```
344
-
345
- ## API Reference
346
-
347
- ### Bootstrap Function
348
-
349
- ```python
350
- def bootstrap(
351
- app: FastAPI,
352
- *,
353
- accessors: Iterable[ModelAccessor[Any]],
354
- database_url: Optional[str] = None,
355
- create_tables: bool = True,
356
- max_records_per_model: Optional[int] = 5,
357
- enable_db_router: bool = True,
358
- ) -> None:
359
- """Bootstrap the storage system for a FastAPI app."""
360
- ```
361
-
362
- ### ModelAccessor
363
-
364
- #### Constructor
365
- ```python
366
- ModelAccessor(model: Type[ModelType], component_name: str)
367
- ```
368
-
369
- #### Read Operations
370
- ```python
371
- # Get single record
372
- accessor.get(id: int, *, include_deleted: bool = False) -> Optional[ModelType]
373
-
374
- # Get all records
375
- accessor.all(*, include_deleted: bool = False) -> List[ModelType]
376
- ```
377
-
378
- #### Write Operations
379
- ```python
380
- # Insert new record
381
- accessor.insert(obj: ModelType, *, actor: str = "internal") -> ModelType
382
-
383
- # Save record (insert if new, update if exists)
384
- accessor.save(obj: ModelType, *, actor: str = "internal") -> ModelType
385
-
386
- # Delete record
387
- accessor.delete(id: int, *, actor: str = "internal") -> bool
388
-
389
- # Restore soft-deleted record
390
- accessor.restore(id: int, *, actor: str = "internal") -> bool
391
- ```
392
-
393
- #### Batch Operations
394
- ```python
395
- # Insert multiple records
396
- accessor.insert_many(objs: Sequence[ModelType], *, actor: str = "internal") -> List[ModelType]
397
-
398
- # Delete multiple records
399
- accessor.delete_many(ids: Sequence[int], *, actor: str = "internal") -> int
400
- ```
401
-
402
- #### Hook Decorators
403
- ```python
404
- @accessor.before_insert()
405
- @accessor.after_insert()
406
- @accessor.before_update()
407
- @accessor.after_update()
408
- @accessor.before_delete()
409
- @accessor.after_delete()
410
- ```
411
-
412
- ### Router Functions
413
-
414
- #### build_crud_router
415
- ```python
416
- def build_crud_router(
417
- accessor: ModelAccessor[ModelType],
418
- *,
419
- max_records: Optional[int] = 100
420
- ) -> APIRouter:
421
- """Generate CRUD router for a model with audit trails."""
422
- ```
423
-
424
- #### build_db_router
425
- ```python
426
- def build_db_router(
427
- *,
428
- audit_default_limit: int = 100,
429
- audit_max_limit: int = 1000
430
- ) -> APIRouter:
431
- """Generate database health and monitoring router."""
432
- ```
433
-
434
- ### Database Management
435
-
436
- ```python
437
- # Configure database URL
438
- database.set_database_url(url: str) -> None
439
-
440
- # Get database engine
441
- database.get_engine() -> Engine
442
-
443
- # Transaction context manager
444
- database.transaction() -> Iterator[Session]
445
-
446
- # Session context manager
447
- database.use_session(session: Optional[Session] = None) -> Iterator[Session]
448
- ```
449
-
450
- ## 🔍 Audit System
451
-
452
- ### AuditLog Model
453
-
454
- ```python
455
- class AuditLog(SQLModel, table=True):
456
- id: Optional[int] = Field(default=None, primary_key=True)
457
- timestamp: datetime = Field(index=True)
458
- component: str = Field(index=True)
459
- record_id: int = Field(index=True)
460
- operation: str
461
- actor: str
462
- before: Optional[Dict[str, Any]] = Field(default=None, sa_column=Column(JSON))
463
- after: Optional[Dict[str, Any]] = Field(default=None, sa_column=Column(JSON))
464
- ```
465
-
466
- ### Querying Audit Logs
467
-
468
- ```python
469
- from storage.auditor import AuditLog
470
- from sqlmodel import select
471
-
472
- # Get all audit logs for a component
473
- logs = session.exec(select(AuditLog).where(AuditLog.component == "users")).all()
474
-
475
- # Get audit logs for a specific record
476
- logs = session.exec(select(AuditLog).where(AuditLog.record_id == 123)).all()
477
-
478
- # Get audit logs by operation type
479
- logs = session.exec(select(AuditLog).where(AuditLog.operation == "create")).all()
480
-
481
- # Get audit logs by actor
482
- logs = session.exec(select(AuditLog).where(AuditLog.actor == "admin")).all()
483
- ```
484
-
485
- ### Audit Log API Endpoints
486
-
487
- ```python
488
- # Query audit logs via REST API
489
- GET /db/audit?component=users&operation=create&actor=admin&limit=100
490
-
491
- # Query parameters:
492
- # - component: Filter by component name
493
- # - record_id: Filter by specific record ID
494
- # - actor: Filter by user who made the change
495
- # - operation: Filter by operation type (create, update, delete, etc.)
496
- # - since: Filter records since this timestamp
497
- # - until: Filter records until this timestamp
498
- # - limit: Maximum number of records to return (1-1000)
499
- # - offset: Number of records to skip for pagination
500
- ```
501
-
502
- ## Dependencies
503
-
504
- - **SQLModel**: Modern SQL database toolkit for Python
505
- - **SQLAlchemy**: SQL toolkit and Object-Relational Mapping library
506
- - **FastAPI**: Modern web framework for building APIs
507
- - **sqlalchemy-schemadisplay**: Database schema visualization (optional)
508
- - **Graphviz**: Graph visualization software (optional, for schema diagrams)
509
- - **Python 3.9+**: Required for type hints and modern Python features