cello-framework 1.2.3__tar.gz → 1.3.0__tar.gz

{cello_framework-1.2.3 → cello_framework-1.3.0}/CLAUDE.md

@@ -2,9 +2,9 @@
 
 ## Project Overview
 
-**Cello** is an ultra-fast, Rust-powered Python async web framework designed to achieve C-level performance on the hot path while maintaining Python's developer experience. It's the successor to frameworks like FastAPI, Robyn, and Litestar, combining their best features with pure Rust implementation for maximum performance.
+**Cello** is an ultra-fast, Rust-powered Python async web framework designed to achieve C-level performance on the hot path while maintaining Python's developer experience. It combines a modern feature set with a pure Rust implementation for maximum performance.
 
-**Version:** 1.2.3
+**Version:** 1.3.0
 **License:** MIT
 **Python Requirement:** 3.12+
 **Author:** Jagadeesh Katla
@@ -78,7 +78,7 @@ Request → Rust HTTP Engine → Python Handler → Rust Response
 │   │   └── redis.rs               # Redis integration (v0.8.0)
 │   ├── routing/                   # Route constraints
 │   ├── server/                    # Server modes (cluster, TLS)
-│   ├── blueprint.rs               # Flask-like route grouping
+│   ├── blueprint.rs               # Modular route grouping
 │   ├── websocket.rs               # WebSocket support
 │   ├── sse.rs                     # Server-Sent Events
 │   ├── json.rs                    # SIMD JSON parsing
@@ -346,6 +346,41 @@ def handle_value_error(request, exc):
 
 ## Version History
 
+- **v1.3.0**: The Speed/Simplicity/Security release — async rework (below), native data layer + ORM, a full `enable_*` plugin audit, and three-pillar upgrades. Native data layer + ORM resolves issue #5 ("how do I use the database pool and Redis client?"); the previous DB/Redis layer was **mock scaffolding** (methods returned `[]`/`None`, never connected) and is now real.
+  - **Native Postgres pool** (`src/db/postgres.rs`, `PyDatabase`): backed by `deadpool-postgres` + `tokio-postgres`. `request.database` / `app.database` expose `fetch` (→`list[dict]`), `fetchrow` (→`dict|None`), `fetchval`, `execute` (→rows affected), and `transaction()`. Positional `$1` params via a `SqlParam`/`ToSql` bridge (`src/db/value.rs`) that dispatches on the target column type; rows decode common pg types incl. `jsonb`→nested Python.
+  - **Native Redis client** (`src/db/redis_client.rs`, `PyRedis`): `redis` crate `aio::ConnectionManager`. `request.redis` / `app.redis` expose get/set/del/expire/incr/hget/hset/lpush/lrange/sadd/publish/eval/evalsha/script_load/ping/… (real, verified against a live server).
+  - **Transactions**: `async with request.database.transaction() as tx` (explicit BEGIN/COMMIT/ROLLBACK on a held pooled connection), plus a rewritten `@transactional` decorator (async-only; injects a `tx` argument; auto commit/rollback).
+  - **Built-in ORM** (`python/cello/orm.py`): `Model` + typed fields (`AutoField/IntegerField/CharField/TextField/BooleanField/FloatField/JSONField/DateTimeField/ForeignKey`), a chainable async `QuerySet` (`filter/exclude/order_by/limit/offset` with field lookups; `get/first/all/count/exists/values/create/update/delete`), `create_table`/`drop_table`. Intentionally lightweight (no migration diffing, lazy reverse relations, `select_related`, signals/admin) — documented as such.
+  - **Async bridge**: all methods return `pyo3_asyncio::tokio::future_into_py` awaitables (same proven pattern as `AsyncClient`); verified to resolve on the persistent asyncio loop.
+  - **Wiring**: `lib.rs` `enable_database`/`enable_redis` build real pools; `app.state` namespace added; `request.database`/`request.db`/`request.redis` injected per request. Stub `Database`/`Redis`/`Transaction` in `python/cello/database.py` removed — native classes exported from `cello`.
+  - **Deps**: `deadpool-postgres`/`tokio-postgres` made non-optional (with `with-chrono-0_4`/`with-uuid-1`/`with-serde_json-1`); added `redis = 0.25`. `postgres` feature kept as an empty alias.
+  - **Tests**: `tests/test_native_db.py` — live-server integration for raw queries/types, transaction commit+rollback, Redis commands, ORM CRUD/filters/FK (auto-skips when PG/Redis unreachable). Obsolete mock-based unit tests removed. Docs: `docs/data-layer.md`, `docs/issue-5-answer.md`, `examples/database_orm_demo.py`.
+  - **Middleware verification + Prometheus fix**: verified the 7 core `enable_*` middleware end-to-end (cors, logging, compression, caching, rate_limit, circuit_breaker, prometheus). Six worked; **`enable_prometheus` was broken** — `/metrics` returned 404 because `handle_request` (`src/server/mod.rs`) does "route match FIRST, fast-return 404" before the prometheus middleware runs, and `/metrics` isn't a registered route. Fixed via `PrometheusMiddleware::try_serve(path)` (`src/middleware/prometheus.rs`), called in the routing-miss branch before returning 404. Tests: `tests/test_middleware.py`; docs: `docs/middleware.md`; example: `examples/middleware_full_demo.py`. Known middleware notes (documented): CORS preflight `OPTIONS` to an *unrouted* path still 404s (same fast-404 cause; register an `OPTIONS` route); a cache HIT serves the stored body uncompressed (bypasses compression); rate-limit + circuit-breaker are global.
+  - **Full `enable_*` plugin audit (all 27)**: verified every plugin end-to-end over live HTTP (extends the 7-core pass above). Bugs found & fixed:
+    - **Health checks & GraphQL returned 404** — identical root cause to the prometheus bug (fast-404 runs before the middleware chain; `/health*` and `/graphql` aren't registered routes). Fixed generally: added a `serves_unrouted(method, path)` hook to the `Middleware`/`AsyncMiddleware` traits (default `false`; overridden in `src/middleware/health.rs` and `graphql.rs`), plus `MiddlewareChain::has_unrouted` / `execute_before_unrouted` / `execute_before_async_unrouted` (`src/middleware/mod.rs`) and a `serve_unrouted()` helper in the router-miss branch (`src/server/mod.rs`). **Only** path-owning middleware run on a miss, so unknown paths under auth still 404 (no 404→401 route-existence leak).
+    - **BasicAuth 401 lacked `WWW-Authenticate`** — `before` returned `Err`, so the server short-circuited and the `after` hook that set the header never ran. Fixed by returning `Stop(challenge())` with the header set inline (`src/middleware/auth.rs`).
+    - **JWT rejected tokens without `iat`** — a standard `sub`+`exp` token failed with "missing field iat". Fixed with `#[serde(default)]` on `JwtClaims.iat`. `exp` stays required (secure default).
+    - **`enable_security_headers` only accepted a `bool`** while docs/examples pass a `SecurityHeadersConfig` (TypeError). Now accepts `None` | `bool` | `SecurityHeadersConfig` via `Option<&PyAny>` + `build_security_headers_mw()` (`src/lib.rs`).
+    - **`set_timeouts` panicked** — hyper 1.10 requires `builder.timer(TokioTimer::new())` when `header_read_timeout` is set ("timeout set, but no timer set"); was missing. Fixed in `src/server/mod.rs` (unblocks the 3 timeout/limit tests in `tests/test_v130_fixes.py`).
+    - **Announce-only plugins** (documented, not "broken"): `enable_grpc`, `enable_messaging`, `enable_rabbitmq`, `enable_sqs`, `enable_event_sourcing`, `enable_cqrs`, `enable_saga` print config but do not wire runtime behaviour into the HTTP app — real functionality is in `cello.grpc/messaging/cqrs/saga/eventsourcing`.
+    - Tests: `tests/test_plugins.py` (20 live-HTTP checks); example: `examples/plugins_demo.py`; docs: `docs/reference/api/plugins.md` (+ mkdocs nav). Full suite: **433 pass** with `pytest --asyncio-mode=auto` (`pytest-asyncio` IS required for `test_cello.py`'s `async def` tests).
+  - **Three-pillar upgrades (Speed / Simplicity / Security)**:
+    - **Security — full headers**: `SecurityHeadersConfig` now exposes `csp` (a `CSP` builder), `permissions_policy` (`{feature: [origins]}`), and `coep`/`coop`/`corp` (string values); `build_security_headers_mw` in `src/lib.rs` bridges them onto the Rust `SecurityHeadersMiddleware` (which already emitted them). `SecurityHeadersConfig.secure()` now includes cross-origin isolation (`require-corp` / `same-origin`).
+    - **Security — Redis TLS**: added rustls features (`tls-rustls`, `tls-rustls-webpki-roots`, `tokio-rustls-comp`) to the `redis` crate; `rediss://` URLs now work (`src/db/redis_client.rs` unchanged — `Client::open` handles the scheme). Resolves the v1.4.0 "TLS for Redis" follow-up.
+    - **Speed — compressed cache**: a cache HIT short-circuits the pipeline so the compression middleware never ran on it (was served uncompressed). `CacheMiddleware` now gzips the HIT inline for `Accept-Encoding: gzip` clients (`compress`/`compress_min_size` on `CacheConfig`, default on, min 1 KB), sets `Vary: Accept-Encoding`, and serves identity to non-gzip clients. `enable_caching(..., compress=True)` toggles it. Cache stores identity because async-`after` (store) runs before sync-`after` (compress).
+    - **Simplicity — honest stubs**: the 7 announce-only `enable_*` methods (`grpc`, `messaging`, `rabbitmq`, `sqs`, `event_sourcing`, `cqrs`, `saga`) no longer print a misleading "enabled" banner — they emit a clear "records config only — use the `cello.X` module" note (Rust + Python docstrings). No behavior change; just honest.
+    - **DX + Security — request validation**: `@app.{get,post,...}(path, body=DTO)` (App + Blueprint) parses/validates the JSON body and returns **400 `{"detail": [...]}`** before the handler runs, injecting the validated instance (`wrap_handler_with_body` in `python/cello/validation.py`). Works with Pydantic, dataclasses, and plain classes. Complements the pre-existing type-hint validation (returns 422).
+    - Tests: `tests/test_upgrades.py` (11 checks). Example: `examples/three_pillars_demo.py`. Docs: `docs/reference/api/plugins.md`, `docs/features/advanced/dto-validation.md`. Full suite: **464 pass** (`pytest --asyncio-mode=auto`).
+  - **Known follow-up**: `numeric`/`timestamptz` params need an explicit `$1::type` cast; `cargo test --lib` can't link libpython (pre-existing pyo3 `extension-module` limitation) — Python integration tests are the verification path.
+  - **Async runtime rework, security hardening & DoS protection** (the async-rework portion of v1.3.0):
+  - **Async runtime rework** (`src/async_loop.rs`, `handler.rs`, `lib.rs`): async `def` handlers now run on a single **persistent asyncio loop** (dedicated daemon thread) via `run_coroutine_threadsafe` instead of a fresh `asyncio.run()` per request. Loop-bound resources (aiohttp/asyncpg pools, `asyncio.Lock`/`Queue`) survive across requests, and the GIL is released while a coroutine awaits I/O (async handlers no longer serialize on the GIL). Async startup/shutdown hooks now actually run (previously the `pyo3_asyncio::into_future` path failed silently).
+  - **Security**: CSRF Origin/Referer validation rewritten to exact-authority matching (`example.com.evil.com` no longer bypasses); all middleware skip/exclude paths use `path_matches_skip` (no prefix bypass); CORS adds `Vary: Origin` on reflected responses; BasicAuth uses non-short-circuit comparison (no username-timing oracle); SSE `id`/`event` strip CR/LF (no stream injection).
+  - **DoS hardening**: `max_body_size` enforced (413, default 100 MB, `Limited`-capped streaming) via `App.set_limits()`; `App.set_timeouts()` wires header/body read timeouts (Slowloris) and a handler timeout (504). Previously `LimitsConfig`/`TimeoutConfig` were inert.
+  - **Correctness**: large Python ints (`> i64::MAX`, up to `u64::MAX`) serialize exactly instead of becoming lossy floats; `Range` header no longer underflows on 0-byte files; DELETE/OPTIONS request bodies are read; query `+` decodes to space in keys as well as values.
+  - **Python API**: `CsrfConfig` passed to `app.use()` is now honored (`cookie_name`/`header_name`/`allowed_origins`); `options()`/`head()`/`route()` apply the same validation + redis-injection wrapping as other verbs.
+  - **Tests**: new Rust `#[cfg(test)]` units (CSRF authority, `path_matches_skip`, Range, SSE) + `tests/test_v130_fixes.py` integration suite.
+  - **Known follow-up**: per-process Tokio runtime remains current-thread (multicore via the existing multi-process model); `AsyncClient` still uses `pyo3_asyncio::future_into_py`.
+- **v1.2.4**: Critical fix — async handlers broken since v1.2.1; `pyo3_asyncio::tokio::into_future` failed silently after server startup switched to `py.allow_threads + block_on` (v1.2.1), causing all `async def` handlers to return 500; fixed by driving coroutines via `tokio::task::spawn_blocking + asyncio.run()` (`handler.rs`)
 - **v1.2.3**: Full middleware Python API — `cello.middleware` module with `JwtAuth`, `BasicAuth`, `ApiKeyAuth`, `CsrfConfig`, `AdaptiveRateLimitConfig`; `app.use()` dispatcher; 6 new `enable_*` methods on App (`enable_jwt`, `enable_session`, `enable_security_headers`, `enable_csrf`, `enable_basic_auth`, `enable_api_key`); all docs import paths corrected (`from cello import RoleGuard` not `cello.guards`)
 - **v1.2.2**: Security & bug fixes — CSRF `HttpOnly` on double-submit cookie (critical, broke all AJAX CSRF); all middleware `skip_path` prefix bypass fixed via `path_matches_skip()` helper; `FixedWindowStore` window_start never updated after reset; unused `mut` cleaned in minijinja tests
 - **v1.2.1**: Bug fixes — server port never bound (`pyo3_asyncio` replaced with native `py.allow_threads` + `tokio::block_on`); `ProblemDetails` was missing from Python module export; `And`/`Or` guards now accept both `*args` and list styles; CSRF `HttpOnly` removed from double-submit cookie (JS must read it); `FixedWindowStore` window_start never updated after reset; all middleware `skip_path` used raw `starts_with` allowing prefix bypass; doc corrections (`type_uri` not `type_url`)