fastapi-calf 0.1.2__tar.gz → 0.2.0__tar.gz

fastapi_calf-0.2.0/PKG-INFO

@@ -0,0 +1,196 @@
+Metadata-Version: 2.4
+Name: fastapi-calf
+Version: 0.2.0
+Summary: A lightweight real-time terminal monitor for FastAPI servers
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: fastapi
+Requires-Dist: rich
+Requires-Dist: psutil
+Requires-Dist: uvicorn
+
+# 🐮 fastapi-calf
+
+**A lightweight, real-time observability for FastAPI.**
+
+`fastapi-calf` watches decorated API routes and presents request activity,
+latency, throughput, failures, CPU, memory, and worker information in a compact
+terminal UI. Monitoring is intentionally best-effort: if the dashboard is not
+available, your API continues serving requests normally.
+
+## Highlights
+
+- ⚡ Live requests-per-second and latency sparklines
+- 🧭 Per-route request, failure, status, and response-time statistics
+- 🖥️ CPU and memory usage across multiple application workers
+- 🎮 `CUDA_VISIBLE_DEVICES` reporting for GPU-aware deployments
+- 🛡️ Short-timeout event delivery that never takes down the application
+
+## Quick start
+
+Install the runtime dependencies from the repository root:
+
+```powershell
+python -m pip install fastapi uvicorn rich psutil
+```
+
+Start the included example API with four workers:
+
+```powershell
+python -m uvicorn server.app:app --port 8000 --workers 4
+```
+
+`Calf.listen()` detects whether its collector is running on port `8765`. If it
+is missing, the first application worker starts it automatically; the other
+workers reuse it. The auto-started collector exits after the application
+workers stop.
+
+The `port` passed to `Calf.listen()` is the Calf collector/dashboard port. The
+FastAPI application port remains whatever you pass to Uvicorn (`8000` above).
+
+Visit `http://127.0.0.1:8000/docs` to explore the API while the terminal
+dashboard updates in real time.
+
+Open `http://127.0.0.1:8000/calf` to redirect to the aggregated dashboard at
+`http://127.0.0.1:8765/`. It refreshes once per second and combines route
+traffic, failures, latency, event-loop stalls, and lifecycle timing from every
+Uvicorn worker.
+
+The automatically started collector serves the web dashboard without a terminal
+UI. To use the terminal view, start the collector explicitly before Uvicorn:
+
+```powershell
+python -m fastapi_calf.daemon --port 8765
+```
+
+The web dashboard is enabled by default and can be moved or disabled:
+
+```python
+Calf.listen(app, port=8765, server="jobs_api", dashboard_path="/observability")
+# Or: Calf.listen(..., dashboard=False)
+# Or manage the collector yourself: Calf.listen(..., auto_start_collector=False)
+```
+
+Ignore health checks, metrics collectors, or other noisy paths with exact paths
+or shell-style wildcards. Calf ignores `/favicon.ico` and its own dashboard
+routes automatically.
+
+```python
+Calf.listen(
+    app,
+    port=8005,
+    server="jobs_api",
+    ignore_paths=("/health", "/ready", "/metrics", "/internal/*"),
+)
+```
+
+Workers send best-effort HTTP events to the collector's `/api/events` endpoint.
+The collector exposes aggregate JSON at `/api/stats`; both the web and terminal
+views read the same in-memory process-wide aggregation.
+
+## Instrumenting an API
+
+Configure the monitor once, then decorate the routes you want to observe:
+
+```python
+from fastapi import FastAPI, Request
+from fastapi_calf import Calf, lookout
+
+app = FastAPI()
+Calf.listen(app, port=8005, server="jobs_api")
+
+
+@app.get("/health")
+@lookout
+def health(request: Request):
+    return {"status": "healthy"}
+
+
+@app.post("/jobs")
+@lookout
+async def create_job(request: Request):
+    payload = await request.json()
+    return {"accepted": True, "job": payload}
+```
+
+The `Request` parameter allows `@lookout` to capture the method, path, query
+parameters, request size, status, and timing without changing the response.
+Metrics are kept in memory by the live dashboard and reset when it stops.
+
+### Event-loop blocking detection
+
+Calf watches each worker's asyncio event loop and warns when it cannot run for
+more than 100 ms. This catches blocking calls such as `time.sleep()`, synchronous
+HTTP clients, filesystem operations, and CPU-heavy work used inside `async def`
+endpoints. Overlapping requests are shown as affected rather than definitively
+identified as the cause.
+
+Tune or disable the detector when configuring the application:
+
+```python
+Calf.listen(
+    app,
+    port=8005,
+    server="jobs_api",
+    detect_blocking=True,
+    block_threshold=100,
+    block_check_interval=50,
+)
+```
+
+Both values are milliseconds. `block_check_interval` controls how often the
+watchdog asks the event loop to run. If it wakes at least `block_threshold`
+milliseconds later than expected, Calf records a blocking event. A smaller
+check interval detects stalls more precisely but wakes the watchdog more often.
+
+The example server exposes `/block-event-loop` as an intentional blocking case
+and `/cooperative-sleep` as its non-blocking comparison.
+
+### Request lifecycle timing
+
+Every HTTP event includes a `lifecycle` breakdown in milliseconds for request
+body parsing, dependency injection and validation, endpoint execution, response
+serialization, middleware/framework overhead, and background tasks. The live
+dashboard shows the most recent breakdown for each route.
+
+`middleware` is the residual request time after the directly measured phases;
+it includes routing, middleware work, response construction, and response
+transport. Streaming generator work is also included there because it occurs
+after the endpoint has returned its iterator.
+
+Try all lifecycle phases with the example endpoint:
+
+```powershell
+curl -X POST http://127.0.0.1:8000/lifecycle `
+  -H "Content-Type: application/json" `
+  -d '{"message":"hello"}'
+```
+
+## Tests and traffic simulation
+
+Install the testing tools and run the correctness suite:
+
+```powershell
+python -m pip install -r requirements-test.txt
+python -m pytest tests/test_api.py -q
+```
+
+The load runner starts four Uvicorn workers, verifies every sample endpoint,
+and pushes **100 concurrent API calls** in a short burst:
+
+```powershell
+python -m tests.load_test
+```
+
+Customize a run with `--requests`, `--concurrency`, and `--workers`. See
+[TESTING.md](TESTING.md) for additional examples.
+
+## Project layout
+
+```text
+fastapi_calf/   monitoring library and terminal dashboard
+server/         instrumented example FastAPI application
+tests/          endpoint and concurrent load tests
+```
+
+> **Status:** This is an early-stage project intended for experimentations with ML pipelines and POCs.

fastapi_calf-0.2.0/README.md

@@ -0,0 +1,185 @@
+# 🐮 fastapi-calf
+
+**A lightweight, real-time observability for FastAPI.**
+
+`fastapi-calf` watches decorated API routes and presents request activity,
+latency, throughput, failures, CPU, memory, and worker information in a compact
+terminal UI. Monitoring is intentionally best-effort: if the dashboard is not
+available, your API continues serving requests normally.
+
+## Highlights
+
+- ⚡ Live requests-per-second and latency sparklines
+- 🧭 Per-route request, failure, status, and response-time statistics
+- 🖥️ CPU and memory usage across multiple application workers
+- 🎮 `CUDA_VISIBLE_DEVICES` reporting for GPU-aware deployments
+- 🛡️ Short-timeout event delivery that never takes down the application
+
+## Quick start
+
+Install the runtime dependencies from the repository root:
+
+```powershell
+python -m pip install fastapi uvicorn rich psutil
+```
+
+Start the included example API with four workers:
+
+```powershell
+python -m uvicorn server.app:app --port 8000 --workers 4
+```
+
+`Calf.listen()` detects whether its collector is running on port `8765`. If it
+is missing, the first application worker starts it automatically; the other
+workers reuse it. The auto-started collector exits after the application
+workers stop.
+
+The `port` passed to `Calf.listen()` is the Calf collector/dashboard port. The
+FastAPI application port remains whatever you pass to Uvicorn (`8000` above).
+
+Visit `http://127.0.0.1:8000/docs` to explore the API while the terminal
+dashboard updates in real time.
+
+Open `http://127.0.0.1:8000/calf` to redirect to the aggregated dashboard at
+`http://127.0.0.1:8765/`. It refreshes once per second and combines route
+traffic, failures, latency, event-loop stalls, and lifecycle timing from every
+Uvicorn worker.
+
+The automatically started collector serves the web dashboard without a terminal
+UI. To use the terminal view, start the collector explicitly before Uvicorn:
+
+```powershell
+python -m fastapi_calf.daemon --port 8765
+```
+
+The web dashboard is enabled by default and can be moved or disabled:
+
+```python
+Calf.listen(app, port=8765, server="jobs_api", dashboard_path="/observability")
+# Or: Calf.listen(..., dashboard=False)
+# Or manage the collector yourself: Calf.listen(..., auto_start_collector=False)
+```
+
+Ignore health checks, metrics collectors, or other noisy paths with exact paths
+or shell-style wildcards. Calf ignores `/favicon.ico` and its own dashboard
+routes automatically.
+
+```python
+Calf.listen(
+    app,
+    port=8005,
+    server="jobs_api",
+    ignore_paths=("/health", "/ready", "/metrics", "/internal/*"),
+)
+```
+
+Workers send best-effort HTTP events to the collector's `/api/events` endpoint.
+The collector exposes aggregate JSON at `/api/stats`; both the web and terminal
+views read the same in-memory process-wide aggregation.
+
+## Instrumenting an API
+
+Configure the monitor once, then decorate the routes you want to observe:
+
+```python
+from fastapi import FastAPI, Request
+from fastapi_calf import Calf, lookout
+
+app = FastAPI()
+Calf.listen(app, port=8005, server="jobs_api")
+
+
+@app.get("/health")
+@lookout
+def health(request: Request):
+    return {"status": "healthy"}
+
+
+@app.post("/jobs")
+@lookout
+async def create_job(request: Request):
+    payload = await request.json()
+    return {"accepted": True, "job": payload}
+```
+
+The `Request` parameter allows `@lookout` to capture the method, path, query
+parameters, request size, status, and timing without changing the response.
+Metrics are kept in memory by the live dashboard and reset when it stops.
+
+### Event-loop blocking detection
+
+Calf watches each worker's asyncio event loop and warns when it cannot run for
+more than 100 ms. This catches blocking calls such as `time.sleep()`, synchronous
+HTTP clients, filesystem operations, and CPU-heavy work used inside `async def`
+endpoints. Overlapping requests are shown as affected rather than definitively
+identified as the cause.
+
+Tune or disable the detector when configuring the application:
+
+```python
+Calf.listen(
+    app,
+    port=8005,
+    server="jobs_api",
+    detect_blocking=True,
+    block_threshold=100,
+    block_check_interval=50,
+)
+```
+
+Both values are milliseconds. `block_check_interval` controls how often the
+watchdog asks the event loop to run. If it wakes at least `block_threshold`
+milliseconds later than expected, Calf records a blocking event. A smaller
+check interval detects stalls more precisely but wakes the watchdog more often.
+
+The example server exposes `/block-event-loop` as an intentional blocking case
+and `/cooperative-sleep` as its non-blocking comparison.
+
+### Request lifecycle timing
+
+Every HTTP event includes a `lifecycle` breakdown in milliseconds for request
+body parsing, dependency injection and validation, endpoint execution, response
+serialization, middleware/framework overhead, and background tasks. The live
+dashboard shows the most recent breakdown for each route.
+
+`middleware` is the residual request time after the directly measured phases;
+it includes routing, middleware work, response construction, and response
+transport. Streaming generator work is also included there because it occurs
+after the endpoint has returned its iterator.
+
+Try all lifecycle phases with the example endpoint:
+
+```powershell
+curl -X POST http://127.0.0.1:8000/lifecycle `
+  -H "Content-Type: application/json" `
+  -d '{"message":"hello"}'
+```
+
+## Tests and traffic simulation
+
+Install the testing tools and run the correctness suite:
+
+```powershell
+python -m pip install -r requirements-test.txt
+python -m pytest tests/test_api.py -q
+```
+
+The load runner starts four Uvicorn workers, verifies every sample endpoint,
+and pushes **100 concurrent API calls** in a short burst:
+
+```powershell
+python -m tests.load_test
+```
+
+Customize a run with `--requests`, `--concurrency`, and `--workers`. See
+[TESTING.md](TESTING.md) for additional examples.
+
+## Project layout
+
+```text
+fastapi_calf/   monitoring library and terminal dashboard
+server/         instrumented example FastAPI application
+tests/          endpoint and concurrent load tests
+```
+
+> **Status:** This is an early-stage project intended for experimentations with ML pipelines and POCs.

fastapi_calf-0.2.0/fastapi_calf/autostart.py

@@ -0,0 +1,101 @@
+from __future__ import annotations
+
+import http.client
+import os
+import subprocess
+import sys
+import tempfile
+import time
+from pathlib import Path
+
+
+LOCAL_HOSTS = {"127.0.0.1", "localhost", "::1"}
+
+
+def collector_is_running(host: str, port: int, timeout: float = 0.15) -> bool:
+    connection = None
+    try:
+        connection = http.client.HTTPConnection(host, port, timeout=timeout)
+        connection.request("GET", "/api/health")
+        response = connection.getresponse()
+        response.read()
+        return response.status == 200
+    except (OSError, TimeoutError, http.client.HTTPException):
+        return False
+    finally:
+        if connection is not None:
+            connection.close()
+
+
+def _wait_for_collector(host: str, port: int, timeout: float = 4.0) -> bool:
+    deadline = time.monotonic() + timeout
+    while time.monotonic() < deadline:
+        if collector_is_running(host, port):
+            return True
+        time.sleep(0.05)
+    return False
+
+
+def _spawn_collector(host: str, port: int) -> None:
+    command = [
+        sys.executable,
+        "-m",
+        "fastapi_calf.daemon",
+        "--host",
+        host,
+        "--port",
+        str(port),
+        "--no-terminal",
+        "--auto-started",
+    ]
+    options = {
+        "stdin": subprocess.DEVNULL,
+        "stdout": subprocess.DEVNULL,
+        "stderr": subprocess.DEVNULL,
+        "close_fds": True,
+    }
+    if os.name == "nt":
+        options["creationflags"] = (
+            subprocess.CREATE_NEW_PROCESS_GROUP | subprocess.CREATE_NO_WINDOW
+        )
+    else:
+        options["start_new_session"] = True
+    subprocess.Popen(command, **options)
+
+
+def ensure_collector(host: str, port: int) -> bool:
+    """Start one local collector when none is already listening."""
+    if collector_is_running(host, port):
+        return True
+    if host not in LOCAL_HOSTS:
+        return False
+
+    lock_path = Path(tempfile.gettempdir()) / f"fastapi-calf-{port}.starting"
+    owns_lock = False
+    try:
+        try:
+            lock_path.mkdir()
+            owns_lock = True
+        except FileExistsError:
+            # Another Uvicorn worker is starting the collector.
+            if _wait_for_collector(host, port):
+                return True
+            # Recover a lock left behind by a worker that was force-killed.
+            try:
+                if time.time() - lock_path.stat().st_mtime > 4:
+                    lock_path.rmdir()
+                    return ensure_collector(host, port)
+            except OSError:
+                pass
+            return False
+
+        if collector_is_running(host, port):
+            return True
+        _spawn_collector(host, port)
+        return _wait_for_collector(host, port)
+    finally:
+        if owns_lock:
+            try:
+                lock_path.rmdir()
+            except OSError:
+                pass

fastapi_calf-0.2.0/fastapi_calf/blocking.py

@@ -0,0 +1,126 @@
+from __future__ import annotations
+
+import asyncio
+import os
+import time
+import uuid
+from collections import deque
+from dataclasses import asdict, dataclass
+from typing import Awaitable, Callable
+
+
+@dataclass
+class ActiveRequest:
+    request_id: str
+    method: str
+    path: str
+    started_at: float
+    task_name: str | None
+    ended_at: float | None = None
+
+
+class BlockingDetector:
+    """Detect periods during which an asyncio event loop cannot run.
+
+    Event-loop lag proves that the worker was stalled, but when requests overlap
+    it cannot prove which request caused the stall. Events therefore contain all
+    requests that overlapped the blocked interval as ``affected_requests``.
+    """
+
+    def __init__(
+        self,
+        emit: Callable[[dict], Awaitable[None]],
+        *,
+        interval: float = 0.05,
+        threshold: float = 0.10,
+    ) -> None:
+        if interval <= 0:
+            raise ValueError("interval must be greater than zero")
+        if threshold <= 0:
+            raise ValueError("threshold must be greater than zero")
+
+        self.emit = emit
+        self.interval = interval
+        self.threshold = threshold
+        self.active: dict[str, ActiveRequest] = {}
+        self.recently_finished: deque[ActiveRequest] = deque()
+        self._task: asyncio.Task | None = None
+
+    def register(self, method: str, path: str) -> str:
+        request_id = uuid.uuid4().hex
+        task = asyncio.current_task()
+        self.active[request_id] = ActiveRequest(
+            request_id=request_id,
+            method=method,
+            path=path,
+            started_at=time.perf_counter(),
+            task_name=task.get_name() if task else None,
+        )
+        return request_id
+
+    def unregister(self, request_id: str) -> None:
+        request = self.active.pop(request_id, None)
+        if request is not None:
+            request.ended_at = time.perf_counter()
+            self.recently_finished.append(request)
+
+    def start(self) -> None:
+        if self._task is None or self._task.done():
+            self._task = asyncio.create_task(
+                self._watch(), name="fastapi-calf-loop-watchdog"
+            )
+
+    async def stop(self) -> None:
+        if self._task is None:
+            return
+
+        self._task.cancel()
+        try:
+            await self._task
+        except asyncio.CancelledError:
+            pass
+        self._task = None
+
+    def _requests_overlapping(self, blocked_since: float) -> list[dict]:
+        while (
+            self.recently_finished
+            and self.recently_finished[0].ended_at is not None
+            and self.recently_finished[0].ended_at < blocked_since
+        ):
+            self.recently_finished.popleft()
+
+        requests = [
+            request
+            for request in (*self.active.values(), *self.recently_finished)
+            if request.started_at <= time.perf_counter()
+            and (request.ended_at is None or request.ended_at >= blocked_since)
+        ]
+        return [asdict(request) for request in requests]
+
+    async def _watch(self) -> None:
+        loop = asyncio.get_running_loop()
+        expected = loop.time() + self.interval
+
+        while True:
+            await asyncio.sleep(self.interval)
+            now = loop.time()
+            lag = max(0.0, now - expected)
+            blocked_since = time.perf_counter() - lag
+            expected = now + self.interval
+
+            if lag < self.threshold:
+                self._requests_overlapping(time.perf_counter())
+                continue
+
+            await self.emit(
+                {
+                    "type": "loop_blocked",
+                    "pid": os.getpid(),
+                    "blocked_ms": lag * 1000,
+                    "affected_requests": self._requests_overlapping(blocked_since),
+                    "timestamp": time.time(),
+                }
+            )
+            # Event delivery is deliberately outside the measured interval. A
+            # slow or unavailable dashboard must not look like application lag.
+            expected = loop.time() + self.interval

fastapi_calf-0.2.0/fastapi_calf/calf.py

@@ -0,0 +1,70 @@
+import asyncio
+import os
+from .config import config
+from .blocking import BlockingDetector
+from .emitter import emit_event_async
+from .lifecycle import CalfLifecycleMiddleware, install_lifecycle_hooks
+from .dashboard import install_dashboard_redirect
+from .autostart import ensure_collector
+
+
+class Calf:
+    @staticmethod
+    def listen(
+        app,
+        port: int,
+        server: str,
+        host: str = "127.0.0.1",
+        *,
+        detect_blocking: bool = True,
+        block_threshold: float = 100,
+        block_check_interval: float = 50,
+        dashboard: bool = True,
+        dashboard_path: str = "/calf",
+        ignore_paths: tuple[str, ...] | list[str] = (),
+        auto_start_collector: bool = True,
+    ):
+        config.host = host
+        config.port = port
+        config.server = server
+
+        cuda_devices = os.getenv("CUDA_VISIBLE_DEVICES")
+
+        detector = None
+        if detect_blocking:
+            detector = BlockingDetector(
+                emit_event_async,
+                interval=block_check_interval / 1000,
+                threshold=block_threshold / 1000,
+            )
+
+            @app.on_event("shutdown")
+            async def stop_blocking_detector():
+                await detector.stop()
+
+        @app.on_event("startup")
+        async def start_calf():
+            if auto_start_collector:
+                await asyncio.to_thread(ensure_collector, host, port)
+            await emit_event_async({
+                "pid": os.getpid(),
+                "cuda_visible_devices": cuda_devices,
+            })
+            if detector is not None:
+                detector.start()
+
+        install_lifecycle_hooks()
+        if dashboard:
+            install_dashboard_redirect(
+                app,
+                dashboard_path,
+                f"http://{host}:{port}/",
+            )
+        app.add_middleware(
+            CalfLifecycleMiddleware,
+            emit=emit_event_async,
+            detector=detector,
+            cuda_devices=cuda_devices,
+            dashboard_path=dashboard_path if dashboard else None,
+            ignore_paths=("/favicon.ico", *ignore_paths),
+        )

{fastapi_calf-0.1.2 → fastapi_calf-0.2.0}/fastapi_calf/config.py

@@ -1,6 +1,7 @@
-class CalfConfig:
-    host = "127.0.0.1"
-    port = 8765
+class CalfConfig:
+    host = "127.0.0.1"
+    port = 8765
+    server = None
 
 
 config = CalfConfig()