dataman-engine 0.2.0__tar.gz → 0.3.0__tar.gz

dataman_engine-0.3.0/PKG-INFO

@@ -0,0 +1,268 @@
+Metadata-Version: 2.4
+Name: dataman-engine
+Version: 0.3.0
+Summary: A dynamic, headless data layer framework built on Django and DRF for instant CRUD APIs and custom business logic.
+Keywords: django,rest,api,crud,headless,generator,scaffold
+Author: Vikash G
+Author-email: Vikash G <vikashgraja@gmail.com>
+License-Expression: MIT
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Environment :: Web Environment
+Classifier: Framework :: Django
+Classifier: Framework :: Django :: 6.0
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Internet :: WWW/HTTP
+Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
+Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
+Requires-Dist: django>=6.0.2
+Requires-Dist: djangorestframework>=3.15.0
+Requires-Dist: click>=8.1.7
+Requires-Dist: python-dotenv>=1.0.1
+Requires-Dist: django-filter>=25.1
+Requires-Dist: requests>=2.32.3
+Requires-Dist: pydantic>=2.10.5
+Requires-Dist: drf-spectacular>=0.28.0
+Requires-Dist: dj-database-url>=2.3.0
+Requires-Dist: inflection>=0.5.1
+Requires-Dist: uvicorn>=0.34.0
+Requires-Dist: cryptography>=42.0.0
+Requires-Dist: psycopg2-binary>=2.9.13
+Requires-Python: >=3.12
+Project-URL: Homepage, https://github.com/vikashgraja/dataman
+Project-URL: Repository, https://github.com/vikashgraja/dataman.git
+Description-Content-Type: text/markdown
+
+# DataMan (Data MiddleMan)
+
+[![Quickstart Guide](https://img.shields.io/badge/Docs-Quickstart_Guide-blue.svg)](https://github.com/vikashgraja/DataMan/blob/main/docs/QUICKSTART.md)
+[![Databases](https://img.shields.io/badge/Databases-PostgreSQL%20%7C%20MySQL%20%7C%20SQLite-success.svg)](#supported-databases)
+[![ASGI Engine](https://img.shields.io/badge/ASGI-Uvicorn%20Ready-purple.svg)](#asgi-production-mode)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/vikashgraja/DataMan/blob/main/LICENSE)
+
+**DataMan** is a dynamic, headless backend engine built on top of Django and Django REST Framework. It eliminates the boilerplate of writing standard CRUD APIs, routing, and serializers by allowing you to scaffold endpoints instantly from the command line while preserving your ability to inject custom business logic and strict validation whenever you need it.
+
+Looking to build an API in 5 minutes? Check out the [Developer Quickstart Tutorial](https://github.com/vikashgraja/DataMan/blob/main/docs/QUICKSTART.md).
+
+---
+
+## 30-Second Demo: Instant REST API
+
+Get a fully functional, paginated, and documented REST API running in 4 commands:
+
+```bash
+# 1. Install DataMan
+pip install dataman-engine
+
+# 2. Initialize project in current directory
+dataman init
+
+# 3. Scaffold an Employee table with full CRUD operations
+dataman create table Employee -o crud
+
+# 4. Apply migrations and start the server
+dataman makemigration && dataman migrate
+dataman server start
+```
+
+### What You Get Instantly:
+* **`GET    /api/default/employee/`** — Paginated list with multi-column filtering, search, and ordering.
+* **`POST   /api/default/employee/`** — Create records with validation and lifecycle hooks.
+* **`GET    /api/default/employee/{id}/`** — Point lookup with automatic relational joins (`?depth=1`).
+* **`PUT    /api/default/employee/{id}/`** — Full update.
+* **`PATCH  /api/default/employee/{id}/`** — Partial update.
+* **`DELETE /api/default/employee/{id}/`** — Delete record.
+* **Interactive Swagger UI**: [`http://127.0.0.1:8000/api/docs/`](http://127.0.0.1:8000/api/docs/)
+* **OpenAPI 3.0 Schema**: [`http://127.0.0.1:8000/api/schema/`](http://127.0.0.1:8000/api/schema/)
+
+---
+
+## Why DataMan?
+
+Building standard CRUD backends with existing frameworks requires boilerplate code:
+
+```text
+Without DataMan:
+Model ---> Serializer ---> ViewSet ---> Router ---> FilterSet ---> Permissions
+
+With DataMan:
+Model ---> Done! (Instant REST Endpoints + Swagger Docs + Auth)
+```
+
+### Framework Comparison
+
+| Framework | CRUD API Setup Requirements |
+| :--- | :--- |
+| **Django REST Framework (DRF)** | Model + Serializer + ViewSet + Router + FilterSet + Permissions |
+| **FastAPI** | SQLAlchemy Model + Pydantic Schema + Router + Endpoints + Dependency Injection |
+| **DataMan** | **Django Model Only (Instant Endpoints, Swagger, Auth & Hooks)** |
+
+---
+
+## Out-of-the-Box API Features
+
+Every scaffolded endpoint automatically delivers production response envelopes:
+
+```json
+// GET /api/default/employee/?is_active=true&search=Engineering&ordering=-created_at
+{
+  "count": 48,
+  "next": "http://127.0.0.1:8000/api/default/employee/?page=2",
+  "previous": null,
+  "results": [
+    {
+      "id": 101,
+      "name": "Jane Doe",
+      "department": "Engineering",
+      "email": "jane.doe@example.com",
+      "is_active": true,
+      "created_at": "2026-09-19T18:00:00Z"
+    }
+  ]
+}
+```
+
+* **Filtering**: Filter by query params (e.g., `?is_active=true&salary__gte=80000`).
+* **Fast B-Tree Prefix Search**: Full-text prefix seeks on indexed columns (e.g., `?search=Jane`).
+* **Multi-Column Ordering**: Sort descending or ascending (e.g., `?ordering=-created_at,salary`).
+* **Fast Pagination**: Sliced pagination without expensive full-table `COUNT(*)` overhead on million-row tables.
+* **Relational Expansion**: Expand foreign keys automatically (e.g., `?depth=1`).
+
+---
+
+## Supported Databases
+
+DataMan provides native connection pooling, health probes, and migration routing for:
+
+| Database | Support Level | Recommended Use |
+| :--- | :--- | :--- |
+| **PostgreSQL** | Primary / Production | 1M+ scale workloads, multi-database routing, high-concurrency ASGI |
+| **MySQL / MariaDB** | Production | Enterprise relational storage with connection lifecycle pooling |
+| **SQLite** | Prototyping | Zero-configuration local development and rapid test suites |
+
+---
+
+## Enterprise Adoption & Target Use Cases
+
+DataMan is designed for data-intensive enterprise architectures:
+* Continuous Control Monitoring (CCM) & internal audit platforms.
+* SAP & ERP Data Middleware: Expose transactional SQL stores as structured REST APIs.
+* Master Data Management (MDM): Instant CRUD data maintenance portals.
+* Approval Workflows & Vendor Portals: Scoped, token-authenticated transactional APIs.
+
+---
+
+## Advanced Features
+
+Every table generated under `tables/<TableName>/` gives you modular files for fine-grained control:
+
+### 1. `config.py` (API Settings)
+```python
+# tables/Customer/config.py
+ALLOWED_OPERATIONS = ["C", "R"]  # Only allow Create (POST) and Read (GET)
+REQUIRE_AUTH = True  # Lock down this endpoint
+DEPTH = 1  # Automatically serialize nested Foreign Key relationships on read (GET)
+
+# Advanced Filtering, Search & Ordering
+FILTER_FIELDS = {
+    "price": ["gte", "lte", "exact"],
+    "name": ["icontains", "exact"],
+    "is_active": ["exact"],
+}
+SEARCH_FIELDS = ["^name", "^email"]  # Prefix seek for optimal B-Tree index utilization
+ORDERING_FIELDS = ["created_at", "price"]
+```
+
+### 2. Authentication & Scoped Access Tokens
+Generate restricted, fine-grained access tokens directly from the CLI:
+```bash
+dataman token create "Frontend Service" --scopes "customer:read,order:create"
+```
+Use the token with standard `Bearer` or `Token` headers:
+```http
+Authorization: Bearer <your_generated_token_key>
+```
+
+### 3. `validation.py` (Data Validation)
+Validate incoming JSON payloads before database execution. Raise `ValidationError` to immediately return `400 Bad Request`.
+```python
+# tables/Customer/validation.py
+from rest_framework.exceptions import ValidationError
+
+
+def validate_customer(data):
+    if "admin" in data.get("name", "").lower():
+        raise ValidationError({"name": "Reserved keyword used."})
+    data["name"] = data["name"].strip().title()
+    return data
+```
+
+### 4. `service.py` (Pre/Post Lifecycle Hooks)
+Run transactional business logic before or after database commits (`before_create`, `after_create`, `before_update`, `after_update`, `before_destroy`, `after_destroy`):
+```python
+# tables/Order/service.py
+from rest_framework.exceptions import ValidationError
+from tables.Product.models import Product
+
+
+def before_create(data):
+    product = Product.objects.get(id=data["product"])
+    quantity = int(data.get("quantity", 1))
+    if product.stock_quantity < quantity:
+        raise ValidationError({"quantity": "Insufficient inventory available."})
+    product.stock_quantity -= quantity
+    product.save()
+    return data
+```
+
+### 5. Multi-Database Architecture
+Organize large projects across isolated physical databases:
+```bash
+# 1. Create a database namespace
+dataman create database analytics_db
+
+# 2. Scaffold a table bound to that database
+dataman create table events --database analytics_db
+
+# 3. Run migrations across all databases (or target a single database)
+dataman migrate --database analytics_db
+```
+Endpoints are automatically registered at `api/<database_name>/<table_name>/` (e.g. `api/default/employee/` or `api/analytics_db/events/`).
+
+---
+
+## Production Health & Monitoring
+
+DataMan comes with built-in health check endpoints for Kubernetes, Docker, and AWS ECS:
+* `GET /health/live/`: Process liveness probe returning `200 OK`.
+* `GET /health/ready/`: Sanitized readiness probe validating database connectivity and migration synchronization (returns `200 OK` or `503 Service Unavailable`).
+* `GET /health/`: Unified health status with database latency metrics.
+
+---
+
+## ASGI Production Mode
+
+Start high-concurrency production server powered by Uvicorn:
+```bash
+dataman server start --asgi --host 0.0.0.0 --port 8000 --workers 4
+```
+
+---
+
+## Testing
+
+DataMan is tested with **95%+ branch coverage**, verifying authentication edge cases, multi-database routing, and dynamic lifecycle hooks.
+
+```bash
+uv run pytest tests/ -v
+```
+
+---
+
+## License
+MIT License

dataman_engine-0.3.0/README.md

@@ -0,0 +1,228 @@
+# DataMan (Data MiddleMan)
+
+[![Quickstart Guide](https://img.shields.io/badge/Docs-Quickstart_Guide-blue.svg)](https://github.com/vikashgraja/DataMan/blob/main/docs/QUICKSTART.md)
+[![Databases](https://img.shields.io/badge/Databases-PostgreSQL%20%7C%20MySQL%20%7C%20SQLite-success.svg)](#supported-databases)
+[![ASGI Engine](https://img.shields.io/badge/ASGI-Uvicorn%20Ready-purple.svg)](#asgi-production-mode)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/vikashgraja/DataMan/blob/main/LICENSE)
+
+**DataMan** is a dynamic, headless backend engine built on top of Django and Django REST Framework. It eliminates the boilerplate of writing standard CRUD APIs, routing, and serializers by allowing you to scaffold endpoints instantly from the command line while preserving your ability to inject custom business logic and strict validation whenever you need it.
+
+Looking to build an API in 5 minutes? Check out the [Developer Quickstart Tutorial](https://github.com/vikashgraja/DataMan/blob/main/docs/QUICKSTART.md).
+
+---
+
+## 30-Second Demo: Instant REST API
+
+Get a fully functional, paginated, and documented REST API running in 4 commands:
+
+```bash
+# 1. Install DataMan
+pip install dataman-engine
+
+# 2. Initialize project in current directory
+dataman init
+
+# 3. Scaffold an Employee table with full CRUD operations
+dataman create table Employee -o crud
+
+# 4. Apply migrations and start the server
+dataman makemigration && dataman migrate
+dataman server start
+```
+
+### What You Get Instantly:
+* **`GET    /api/default/employee/`** — Paginated list with multi-column filtering, search, and ordering.
+* **`POST   /api/default/employee/`** — Create records with validation and lifecycle hooks.
+* **`GET    /api/default/employee/{id}/`** — Point lookup with automatic relational joins (`?depth=1`).
+* **`PUT    /api/default/employee/{id}/`** — Full update.
+* **`PATCH  /api/default/employee/{id}/`** — Partial update.
+* **`DELETE /api/default/employee/{id}/`** — Delete record.
+* **Interactive Swagger UI**: [`http://127.0.0.1:8000/api/docs/`](http://127.0.0.1:8000/api/docs/)
+* **OpenAPI 3.0 Schema**: [`http://127.0.0.1:8000/api/schema/`](http://127.0.0.1:8000/api/schema/)
+
+---
+
+## Why DataMan?
+
+Building standard CRUD backends with existing frameworks requires boilerplate code:
+
+```text
+Without DataMan:
+Model ---> Serializer ---> ViewSet ---> Router ---> FilterSet ---> Permissions
+
+With DataMan:
+Model ---> Done! (Instant REST Endpoints + Swagger Docs + Auth)
+```
+
+### Framework Comparison
+
+| Framework | CRUD API Setup Requirements |
+| :--- | :--- |
+| **Django REST Framework (DRF)** | Model + Serializer + ViewSet + Router + FilterSet + Permissions |
+| **FastAPI** | SQLAlchemy Model + Pydantic Schema + Router + Endpoints + Dependency Injection |
+| **DataMan** | **Django Model Only (Instant Endpoints, Swagger, Auth & Hooks)** |
+
+---
+
+## Out-of-the-Box API Features
+
+Every scaffolded endpoint automatically delivers production response envelopes:
+
+```json
+// GET /api/default/employee/?is_active=true&search=Engineering&ordering=-created_at
+{
+  "count": 48,
+  "next": "http://127.0.0.1:8000/api/default/employee/?page=2",
+  "previous": null,
+  "results": [
+    {
+      "id": 101,
+      "name": "Jane Doe",
+      "department": "Engineering",
+      "email": "jane.doe@example.com",
+      "is_active": true,
+      "created_at": "2026-09-19T18:00:00Z"
+    }
+  ]
+}
+```
+
+* **Filtering**: Filter by query params (e.g., `?is_active=true&salary__gte=80000`).
+* **Fast B-Tree Prefix Search**: Full-text prefix seeks on indexed columns (e.g., `?search=Jane`).
+* **Multi-Column Ordering**: Sort descending or ascending (e.g., `?ordering=-created_at,salary`).
+* **Fast Pagination**: Sliced pagination without expensive full-table `COUNT(*)` overhead on million-row tables.
+* **Relational Expansion**: Expand foreign keys automatically (e.g., `?depth=1`).
+
+---
+
+## Supported Databases
+
+DataMan provides native connection pooling, health probes, and migration routing for:
+
+| Database | Support Level | Recommended Use |
+| :--- | :--- | :--- |
+| **PostgreSQL** | Primary / Production | 1M+ scale workloads, multi-database routing, high-concurrency ASGI |
+| **MySQL / MariaDB** | Production | Enterprise relational storage with connection lifecycle pooling |
+| **SQLite** | Prototyping | Zero-configuration local development and rapid test suites |
+
+---
+
+## Enterprise Adoption & Target Use Cases
+
+DataMan is designed for data-intensive enterprise architectures:
+* Continuous Control Monitoring (CCM) & internal audit platforms.
+* SAP & ERP Data Middleware: Expose transactional SQL stores as structured REST APIs.
+* Master Data Management (MDM): Instant CRUD data maintenance portals.
+* Approval Workflows & Vendor Portals: Scoped, token-authenticated transactional APIs.
+
+---
+
+## Advanced Features
+
+Every table generated under `tables/<TableName>/` gives you modular files for fine-grained control:
+
+### 1. `config.py` (API Settings)
+```python
+# tables/Customer/config.py
+ALLOWED_OPERATIONS = ["C", "R"]  # Only allow Create (POST) and Read (GET)
+REQUIRE_AUTH = True  # Lock down this endpoint
+DEPTH = 1  # Automatically serialize nested Foreign Key relationships on read (GET)
+
+# Advanced Filtering, Search & Ordering
+FILTER_FIELDS = {
+    "price": ["gte", "lte", "exact"],
+    "name": ["icontains", "exact"],
+    "is_active": ["exact"],
+}
+SEARCH_FIELDS = ["^name", "^email"]  # Prefix seek for optimal B-Tree index utilization
+ORDERING_FIELDS = ["created_at", "price"]
+```
+
+### 2. Authentication & Scoped Access Tokens
+Generate restricted, fine-grained access tokens directly from the CLI:
+```bash
+dataman token create "Frontend Service" --scopes "customer:read,order:create"
+```
+Use the token with standard `Bearer` or `Token` headers:
+```http
+Authorization: Bearer <your_generated_token_key>
+```
+
+### 3. `validation.py` (Data Validation)
+Validate incoming JSON payloads before database execution. Raise `ValidationError` to immediately return `400 Bad Request`.
+```python
+# tables/Customer/validation.py
+from rest_framework.exceptions import ValidationError
+
+
+def validate_customer(data):
+    if "admin" in data.get("name", "").lower():
+        raise ValidationError({"name": "Reserved keyword used."})
+    data["name"] = data["name"].strip().title()
+    return data
+```
+
+### 4. `service.py` (Pre/Post Lifecycle Hooks)
+Run transactional business logic before or after database commits (`before_create`, `after_create`, `before_update`, `after_update`, `before_destroy`, `after_destroy`):
+```python
+# tables/Order/service.py
+from rest_framework.exceptions import ValidationError
+from tables.Product.models import Product
+
+
+def before_create(data):
+    product = Product.objects.get(id=data["product"])
+    quantity = int(data.get("quantity", 1))
+    if product.stock_quantity < quantity:
+        raise ValidationError({"quantity": "Insufficient inventory available."})
+    product.stock_quantity -= quantity
+    product.save()
+    return data
+```
+
+### 5. Multi-Database Architecture
+Organize large projects across isolated physical databases:
+```bash
+# 1. Create a database namespace
+dataman create database analytics_db
+
+# 2. Scaffold a table bound to that database
+dataman create table events --database analytics_db
+
+# 3. Run migrations across all databases (or target a single database)
+dataman migrate --database analytics_db
+```
+Endpoints are automatically registered at `api/<database_name>/<table_name>/` (e.g. `api/default/employee/` or `api/analytics_db/events/`).
+
+---
+
+## Production Health & Monitoring
+
+DataMan comes with built-in health check endpoints for Kubernetes, Docker, and AWS ECS:
+* `GET /health/live/`: Process liveness probe returning `200 OK`.
+* `GET /health/ready/`: Sanitized readiness probe validating database connectivity and migration synchronization (returns `200 OK` or `503 Service Unavailable`).
+* `GET /health/`: Unified health status with database latency metrics.
+
+---
+
+## ASGI Production Mode
+
+Start high-concurrency production server powered by Uvicorn:
+```bash
+dataman server start --asgi --host 0.0.0.0 --port 8000 --workers 4
+```
+
+---
+
+## Testing
+
+DataMan is tested with **95%+ branch coverage**, verifying authentication edge cases, multi-database routing, and dynamic lifecycle hooks.
+
+```bash
+uv run pytest tests/ -v
+```
+
+---
+
+## License
+MIT License

{dataman_engine-0.2.0 → dataman_engine-0.3.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "dataman-engine"
-version = "0.2.0"
+version = "0.3.0"
 description = "A dynamic, headless data layer framework built on Django and DRF for instant CRUD APIs and custom business logic."
 readme = "README.md"
 license = "MIT"

{dataman_engine-0.2.0 → dataman_engine-0.3.0}/pyproject.toml.orig

@@ -1,6 +1,6 @@
 [project]
 name = "dataman-engine"
-version = "0.2.0"
+version = "0.3.0"
 description = "A dynamic, headless data layer framework built on Django and DRF for instant CRUD APIs and custom business logic."
 readme = "README.md"
 license = "MIT"

{dataman_engine-0.2.0 → dataman_engine-0.3.0}/src/dataman/__init__.py

@@ -9,7 +9,7 @@ from dataman.core.fields import (
     EncryptedTextField,
 )
 
-__version__ = "0.2.0"
+__version__ = "0.3.0"
 __all__ = [
     "EncryptedCharField",
     "EncryptedTextField",

{dataman_engine-0.2.0 → dataman_engine-0.3.0}/src/dataman/cli.py

@@ -102,7 +102,10 @@ def init():
             "MAX_PAGE_SIZE = 500\n\n"
             "# Audit & Telemetry\n"
             "ENABLE_AUDIT_LOGGING = True\n"
-            "ENABLE_TELEMETRY = True\n\n"
+            "ENABLE_TELEMETRY = True\n"
+            'TELEMETRY_BACKEND = "db"  # Options: "db" (default), "stdout", "none"\n\n'
+            "# Webhooks (pluggable queue or worker dispatcher)\n"
+            "# WEBHOOK_DISPATCHER = 'my_app.tasks.emit_webhook'\n\n"
             "# Custom Installed Apps (e.g. third-party Django apps)\n"
             "EXTRA_INSTALLED_APPS = []\n\n"
             "# Custom Middleware (appended to request/response pipeline)\n"
@@ -245,11 +248,18 @@ def create_table(name, database, operations):
         click.echo(click.style(f"Table '{table_name}' already exists.", fg="yellow"))
         raise click.Abort()
 
-    table_dir.mkdir(parents=True)
-    (table_dir / "__init__.py").touch()
+    cleaned_ops = operations.lower().strip()
+    if not cleaned_ops or any(char not in "crud" for char in cleaned_ops):
+        raise click.BadParameter(
+            f"Invalid operations '{operations}'. Allowed characters are 'c', 'r', 'u', 'd' (e.g. -o crud or -o cr)."
+        )
 
     ops_map = {"c": "C", "r": "R", "u": "U", "d": "D"}
-    ops_list = [ops_map[char] for char in operations.lower() if char in ops_map]
+    seen = set()
+    ops_list = [ops_map[c] for c in cleaned_ops if not (c in seen or seen.add(c))]
+
+    table_dir.mkdir(parents=True)
+    (table_dir / "__init__.py").touch()
 
     # Write config.py
     with open(table_dir / "config.py", "w") as f:

{dataman_engine-0.2.0 → dataman_engine-0.3.0}/src/dataman/core/auth.py

@@ -25,7 +25,7 @@ class ServiceTokenAuthentication(authentication.BaseAuthentication):
     def authenticate(self, request):
         auth = request.headers.get("Authorization", "").split()
 
-        if not auth or auth[0].lower() != "token":
+        if not auth or auth[0].lower() not in ("token", "bearer"):
             return None
 
         if len(auth) == 1 or len(auth) > 2:

{dataman_engine-0.2.0 → dataman_engine-0.3.0}/src/dataman/core/crypto.py

@@ -62,20 +62,33 @@ class AESGCMCryptoProvider(BaseCryptoProvider):
         elif isinstance(key, bytes) and key:
             secret_source = key
         else:
-            # Check environment variables or Django settings
+            # 1. Check explicit DATAMAN_ENCRYPTION_KEY from environment
             env_key = os.getenv("DATAMAN_ENCRYPTION_KEY")
             if env_key:
                 secret_source = env_key.encode("utf-8")
             else:
-                secret_key = os.getenv(
-                    "DATAMAN_SECRET_KEY", "default-insecure-key-change-me"
-                )
-                with contextlib.suppress(Exception):
-                    from django.conf import settings
-
-                    if settings.configured and getattr(settings, "SECRET_KEY", None):
-                        secret_key = settings.SECRET_KEY
-                secret_source = secret_key.encode("utf-8")
+                # 2. Check DATAMAN_SECRET_KEY from environment
+                secret_key = os.getenv("DATAMAN_SECRET_KEY")
+                # 3. Check Django SECRET_KEY if configured
+                if not secret_key:
+                    with contextlib.suppress(Exception):
+                        from django.conf import settings
+
+                        if settings.configured and getattr(
+                            settings, "SECRET_KEY", None
+                        ):
+                            secret_key = settings.SECRET_KEY
+
+                if secret_key:
+                    secret_source = secret_key.encode("utf-8")
+
+        if not secret_source:
+            from django.core.exceptions import ImproperlyConfigured
+
+            raise ImproperlyConfigured(
+                "DataMan Encryption Key is not configured. Please set DATAMAN_ENCRYPTION_KEY "
+                "or DATAMAN_SECRET_KEY environment variable, or configure Django SECRET_KEY."
+            )
 
         # Derive a cryptographically strong 256-bit key using HKDF-SHA256
         hkdf = HKDF(

{dataman_engine-0.2.0 → dataman_engine-0.3.0}/src/dataman/core/middleware.py

@@ -1,9 +1,11 @@
 import atexit
+import json
 import logging
 import queue
 import threading
 import time
 
+from django.conf import settings
 from django.db import close_old_connections
 
 from .models import APILog
@@ -85,6 +87,15 @@ class APILoggingMiddleware:
         self.get_response = get_response
 
     def __call__(self, request):
+        backend = getattr(settings, "TELEMETRY_BACKEND", None)
+        if backend is None:
+            enable_telemetry = getattr(settings, "ENABLE_TELEMETRY", True)
+            backend = "db" if enable_telemetry else "none"
+        backend = str(backend).lower()
+
+        if backend in ("none", "false", "off", "disabled"):
+            return self.get_response(request)
+
         path = request.path
         # Only log requests to actual data endpoints,
         # exclude internal/dashboard/docs/health routes.
@@ -111,13 +122,24 @@ class APILoggingMiddleware:
             else request.META.get("REMOTE_ADDR")
         )
 
-        try:
-            _log_queue.put_nowait(
-                (request.method, path, response.status_code, duration_ms, ip)
-            )
-        except queue.Full:
-            logger.warning(
-                "Telemetry log queue full, dropping record under extreme load."
-            )
+        if backend == "stdout":
+            record = {
+                "event": "api_request",
+                "method": request.method,
+                "path": path,
+                "status_code": response.status_code,
+                "duration_ms": duration_ms,
+                "ip_address": ip,
+            }
+            logger.info(json.dumps(record))
+        else:
+            try:
+                _log_queue.put_nowait(
+                    (request.method, path, response.status_code, duration_ms, ip)
+                )
+            except queue.Full:
+                logger.warning(
+                    "Telemetry log queue full, dropping record under extreme load."
+                )
 
         return response