cctv-stack 0.3.0__tar.gz → 0.3.1__tar.gz

{cctv_stack-0.3.0 → cctv_stack-0.3.1}/.gitignore

@@ -14,6 +14,13 @@ venv/
 .ruff_cache/
 .mypy_cache/
 
+# Editor state is per-machine, not project configuration
+.vscode/
+
+# Personal agent memory and local agent settings. This repository is public; nothing here is
+# project configuration and nothing here belongs in its history.
+.claude/
+
 # uv.lock is deliberately COMMITTED — do not ignore it (ADR-0018).
 .uv-cache/
 
@@ -51,6 +58,5 @@ config/frigate.generated.yml
 # deliberately ignored and is deleted once its VERIFY items are measurements.
 spike/
 
-# dev-synthetic/ is **committed**. It is the camera-free harness M2 requires and the
-# stack the e2e CI job runs. Its own .gitignore keeps the generated fixture, runtime
-# data and Frigate's state out of git while leaving the harness itself reviewable.
+# Internal planning and notes — not for public consumption.
+.private/

{cctv_stack-0.3.0 → cctv_stack-0.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: cctv-stack
-Version: 0.3.0
+Version: 0.3.1
 Summary: A packaging-first self-hosted CCTV stack built on Frigate.
 Project-URL: Homepage, https://github.com/aravindmyd/cctv-stack
 Project-URL: Repository, https://github.com/aravindmyd/cctv-stack
@@ -222,6 +222,8 @@ Requires-Python: >=3.12
 Requires-Dist: pyyaml>=6.0
 Requires-Dist: tzdata>=2024.1
 Provides-Extra: dev
+Requires-Dist: mkdocs-material>=9.6; extra == 'dev'
+Requires-Dist: mkdocs<2,>=1.6; extra == 'dev'
 Requires-Dist: pytest>=8.0; extra == 'dev'
 Requires-Dist: ruff>=0.6; extra == 'dev'
 Provides-Extra: mqtt
@@ -232,179 +234,252 @@ Description-Content-Type: text/markdown
 
 # cctv-stack
 
+[![PyPI](https://img.shields.io/pypi/v/cctv-stack.svg)](https://pypi.org/project/cctv-stack/)
+[![Python versions](https://img.shields.io/pypi/pyversions/cctv-stack.svg)](https://pypi.org/project/cctv-stack/)
 [![CI](https://github.com/aravindmyd/cctv-stack/actions/workflows/ci.yml/badge.svg)](https://github.com/aravindmyd/cctv-stack/actions/workflows/ci.yml)
-[![Publish](https://github.com/aravindmyd/cctv-stack/actions/workflows/publish.yml/badge.svg)](https://github.com/aravindmyd/cctv-stack/actions/workflows/publish.yml)
 
-Self-hosted CCTV built on **Frigate**, adding the four things Frigate deliberately leaves
-to the operator:
+`cctv-stack` is a Python toolkit and container image for a self-hosted CCTV pipeline built
+on **[Frigate](https://github.com/blakeblackshear/frigate)**.
 
-1. **Canonical clips** — every review becomes one file at a deterministic path.
-2. **Durable cloud backup** — clips are uploaded to Backblaze B2 exactly once.
-3. **Independent retention** — local cleanup that never deletes an un-uploaded clip.
-4. **Health surfaces** — each service reports whether it is actually working.
+Frigate does camera ingest, detection, recording and review segmentation. `cctv-stack`
+adds the four pieces it leaves to the operator:
 
-Every installation is independent: no central hub, no cross-site data, no inbound
-connectivity requirement.
+1. **Canonical clips** — one deterministic file per review item.
+2. **Durable backup** — clips are uploaded to Backblaze B2 exactly once.
+3. **Upload-aware retention** — local cleanup never deletes a clip that is not safely
+   uploaded.
+4. **Health surfaces** — the uploader and cleanup expose `/health` and `/metrics`.
+
+It is not a replacement for Frigate. It is the missing layer around it.
+
+**Current release: `0.3.0`.** Clips, B2 upload, local retention and health are
+implemented. Cloud retention, an aggregate status page and the one-command installer are
+not — see [`CONTRIBUTING.md`](CONTRIBUTING.md) for the exact state and roadmap.
 
 ---
 
-## Status
+## Install
 
-| Capability | State |
-|---|---|
-| Detect → canonical clip | ✅ |
-| Upload to Backblaze B2, exactly once | ✅ |
-| Local retention (upload-aware) | ✅ |
-| Cloud retention and reconciliation | ⬜ |
-| Aggregate status page and alerting | ⬜ |
-| One-command client installer and signed bundle | ⬜ |
-| Backup, restore, upgrade | ⬜ |
+```bash
+pip install cctv-stack
+```
 
-v0.3 is usable for a local-only installation or a B2-backed one. The deferred work, and
-why it is deferred, is in [`docs/plans/backlog.md`](docs/plans/backlog.md). The client-site
-delivery plan is [`docs/plans/packaging.md`](docs/plans/packaging.md).
+Requires **Python 3.12+**. The distribution is `cctv-stack`; the import package is `cctv`.
 
-## How it fits together
+Optional extras:
 
-```text
- cameras ──RTSP──> Frigate ──review──> clip-worker ──> data/clips/... ──> uploader ──> B2
-                      │                                    │
-                      └── recordings (recovery window)      └── cleanup (only after upload)
+| Extra | Adds | Needed by |
+|---|---|---|
+| `mqtt` | `paho-mqtt` | `cctv-clip-worker` when subscribing to `frigate/reviews` |
+| `storage` | `boto3` | `cctv-uploader` for B2/S3 |
+| `dev` | pytest, ruff | Contributors |
+
+```bash
+pip install "cctv-stack[mqtt,storage]"
+```
+
+### Container image
+
+```bash
+docker pull ghcr.io/aravindmyd/cctv-stack:0.3.0
 ```
 
-Frigate does camera ingest, detection, recording and review segmentation. The
-`clip-worker` cuts **one canonical clip per review item**. The uploader pushes it once,
-and cleanup deletes a local clip only after its upload is terminal-success (INV-1).
+Tags published per release: `:0.3.0`, `:0.3`, and the commit tag `:sha-<short>`.
+A moving `:edge` tag follows `master`. If the registry package is private, run
+`docker login ghcr.io` first. The image includes `ffmpeg`/`ffprobe`.
 
-MQTT is the low-latency path. The Frigate review sweep is the guarantee: if the broker or
-the worker was down, the next sweep still materialises the clip. A review arriving twice
-produces one clip, not two.
+## Commands
+
+| Command | What it does |
+|---|---|
+| `cctv-config validate --cameras cameras.yml` | Validate the camera configuration. Exits 78 (`EX_CONFIG`) on an invalid config |
+| `cctv-config generate --cameras cameras.yml --out frigate.yml` | Generate Frigate's configuration from `cameras.yml` |
+| `cctv-clip-worker` | Turn completed Frigate reviews into canonical clips |
+| `cctv-uploader` | Upload clips to B2 through the persisted retry state machine |
+| `cctv-cleanup` | Enforce local retention after a successful upload |
+
+The three services are long-running and configuration-driven; they take no command-line
+arguments. Run them under systemd, Docker, or Compose.
 
 ## Quick start
 
-**Requirements:** `git`, Linux x86_64 or arm64 (macOS is supported for development),
-Docker Engine with the Compose v2 plugin, and disk space for Frigate's recordings. The
-first real run pulls the Frigate image and builds the CCTV image, so expect several
-minutes.
+The only required input is a `cameras.yml`. This minimal file is valid; `${VAR}` values
+are read from the environment and an unset variable is a hard error.
 
-```bash
-git clone https://github.com/aravindmyd/cctv-stack.git
-cd cctv-stack
-./install.sh
+```yaml
+site:
+  location: home
+  timezone: UTC
+
+cameras:
+  front-door:
+    display_name: Front Door
+    rtsp:
+      main: ${CAMERA_FRONT_DOOR_RTSP}
+      sub: ${CAMERA_FRONT_DOOR_SUB}
 ```
 
-The first run creates the two files that only you can fill in, then exits:
+Validate it and generate Frigate's config:
 
-| File | Contains | Committed? |
-|---|---|---|
-| `.env` | Camera RTSP URLs, timezone, optional B2 credentials | **No** — gitignored |
-| `config/cameras.yml` | Camera names, clip padding, retention policy | **No** — gitignored |
+```bash
+export CAMERA_FRONT_DOOR_RTSP='rtsp://user:pass@192.168.1.10:554/stream1'
+export CAMERA_FRONT_DOOR_SUB='rtsp://user:pass@192.168.1.10:554/stream2'
 
-Edit both, then run `./install.sh` again. It validates the configuration, generates
-Frigate's config, builds the image, starts the stack, and waits until Frigate answers.
+cctv-config validate --cameras cameras.yml
+cctv-config generate --cameras cameras.yml --out frigate.yml
+```
 
-Frigate's UI is HTTPS on `https://127.0.0.1:8971` (loopback only). The certificate is
-self-signed, so the browser warns once. Port 5000 and MQTT are never published.
+`validate` checks the camera model and the environment substitution. `generate` writes a
+Frigate config with a `GENERATED FILE — DO NOT EDIT` header.
 
-Clips land at:
+### Running the services
 
-```text
-data/clips/<camera>/YYYY/MM/DD/HH-MM-SS.mp4
-data/clips/<camera>/YYYY/MM/DD/HH-MM-SS.mp4.json
-```
+```bash
+export CCTV_CONFIG_DIR="$PWD"
+export CCTV_DATA_DIR="$PWD/data"
+export CCTV_FRIGATE_URL="http://127.0.0.1:5000"
 
-The sidecar records the review, event ids, requested boundaries, coverage, size and
-SHA-256. The clips tree uses the **site timezone**; Frigate's own recordings tree uses
-**UTC** — both are deliberate ([ADR-0017](docs/adr/0017-time-handling.md)).
+cctv-clip-worker
+```
 
-A clip appears **after** the visit ends, not during it: Frigate keeps the review open
-until the alert cutoff passes. There is no partial clip by design.
+Run each service in its own process. `cctv-clip-worker` works without MQTT (sweep-only);
+set `CCTV_MQTT_HOST` to receive reviews immediately. `cctv-uploader` reports `DISABLED`
+and uploads nothing when `CCTV_STORAGE_BUCKET` is empty; a bucket with no key exits `78`.
 
-## What runs
+> `ffprobe` must be on `PATH` for the clip worker. The container image includes it; a
+> `pip install` does not install ffmpeg.
 
-| Service | Role |
-|---|---|
-| **Frigate** | Camera ingest, detection, recording and review segmentation |
-| **clip-worker** | Materialises completed reviews at the canonical path, with a sidecar |
-| **uploader** | Uploads to B2 through a storage adapter, with a persisted retry state machine. Reports `DISABLED` when no bucket is configured and exits `78` on a half-configuration |
-| **cleanup** | Enforces local retention; refuses to delete a clip whose upload is not terminal-success |
-| **config** | One-shot generator: `cameras.yml` → Frigate's config (INV-3) |
-| **mosquitto** | MQTT broker, internal only |
-| **status-collector** | ⬜ not built yet |
+## Configuration
 
-### Health surfaces
+There are two kinds of configuration, and they must not be mixed:
 
-| Surface | Address | Notes |
+| File | Holds | Secret? |
 |---|---|---|
-| Frigate UI | `https://127.0.0.1:8971` | HTTPS, self-signed, loopback only |
-| Uploader | `http://127.0.0.1:8082/health` | `DISABLED` without a B2 bucket |
-| Cleanup | `http://127.0.0.1:8083/health` | Local retention state |
-| Frigate API, MQTT | not published | Reachable only on the compose network |
+| `cameras.yml` | Site name, timezone, cameras, clip policy, retention | No — but `${VAR}` values come from the environment |
+| environment / `.env` | RTSP URLs, B2 keys, paths, ports, intervals | **Yes** |
 
-## Configuration
+The full template is [`.env.example`](.env.example). The settings most consumers need:
 
-The split is deliberate and load-bearing:
+| Variable | Default | Meaning |
+|---|---|---|
+| `CCTV_CONFIG_DIR` | `/config` | Directory containing `cameras.yml` |
+| `CCTV_DATA_DIR` | `/data` | Root for `clips/` and the state database |
+| `CCTV_DB` | `$CCTV_DATA_DIR/cctv.db` | SQLite state store |
+| `CCTV_FRIGATE_URL` | `http://frigate:5000` | Frigate API base URL. Port 5000 must stay on a private network |
+| `CCTV_MQTT_HOST` | unset | Unset means sweep-only; no broker required |
+| `CCTV_SWEEP_INTERVAL_S` | `60` | Worst-case delay before a missed review is noticed |
+| `CCTV_STORAGE_BUCKET` | empty | Empty means cloud backup `DISABLED` |
+| `CCTV_STORAGE_KEY_ID` | empty | B2 application key ID |
+| `CCTV_STORAGE_APPLICATION_KEY` | empty | B2 application key |
+| `CCTV_STORAGE_REGION` | `us-west-004` | B2 region |
+| `CCTV_UPLOADER_HEALTH_PORT` | `8082` | Uploader `/health` and `/metrics` |
+| `CCTV_CLEANUP_HEALTH_PORT` | `8083` | Cleanup `/health` and `/metrics` |
+| `INTERNAL_API_TOKEN` | empty | Enables the uploader admin endpoints when set |
+
+Keep `.env` out of version control. RTSP URLs and B2 keys are credentials.
+
+Camera names must be lowercase kebab-case (`front-door`): they become directory names and
+cloud object keys.
+
+## What it produces
 
-- **`config/cameras.yml`** is the single source of truth for cameras (INV-3). Frigate's
-  config is generated from it and must never be hand-edited.
-- **`.env`** holds location values and secrets: RTSP URLs, B2 keys, ports, data path.
+```text
+$CCTV_DATA_DIR/
+├── clips/
+│   └── <camera>/YYYY/MM/DD/HH-MM-SS.mp4
+│       <camera>/YYYY/MM/DD/HH-MM-SS.mp4.json
+└── cctv.db
+```
 
-Both real files are gitignored. `.env.example` and `config/cameras.example.yml` are the
-templates. Never commit the filled-in versions, and never paste an RTSP URL or B2 key into
-an issue. If a credential is exposed, follow
-[`docs/runbooks/rotate-credentials.md`](docs/runbooks/rotate-credentials.md).
+The clips tree is rendered in the **site timezone**. The sidecar records the review id,
+event ids, requested boundaries, measured coverage, codecs, size and SHA-256. The cloud
+object key is `<location>/clips/<path-inside-clips>`.
 
-Camera names are lowercase kebab-case because they become filesystem paths and cloud
-object keys.
+A clip appears **after** the visit ends: Frigate keeps the review open until its alert
+cutoff passes. There is no partial clip by design.
 
-## Published artefacts
+### Health endpoints
 
-| Artefact | Where | Published when |
+| Service | Endpoint | Notes |
 |---|---|---|
-| Branch image | `ghcr.io/aravindmyd/cctv-stack:edge` | Every merge to `main`/`master` |
-| Commit image | `ghcr.io/aravindmyd/cctv-stack:sha-<short>` | Every merge |
-| Version image | `ghcr.io/aravindmyd/cctv-stack:<version>` | A `v*` tag |
-| Python package | `pypi.org/project/cctv-stack` | A `v*` tag |
-| Wheel and sdist | Workflow artefact, and GitHub Release on tags | Every merge / tag |
+| `cctv-uploader` | `http://127.0.0.1:8082/health`, `/metrics` | `DISABLED` without a bucket |
+| `cctv-cleanup` | `http://127.0.0.1:8083/health`, `/metrics` | Local retention state |
+| Uploader admin | `/admin/queue`, `/admin/retry` | Mounted only when `INTERNAL_API_TOKEN` is set |
 
-```bash
-docker pull ghcr.io/aravindmyd/cctv-stack:edge
-pip install cctv-stack
+Bind these to loopback. Nothing in the package needs an inbound port from the LAN.
+
+## Python API
+
+The supported contract is the CLI, the environment variables, and the HTTP health
+endpoints. The importable modules are tested, but pre-1.0 names may change between minor
+versions.
+
+```python
+from cctv.config import ConfigError, load_cameras_config
+
+try:
+    config = load_cameras_config("cameras.yml")
+except ConfigError as exc:
+    raise SystemExit(f"invalid camera config: {exc}")
+
+print(config.site.location)
+for name, camera in config.enabled_cameras().items():
+    print(name, camera.display_name, camera.record.mode)
 ```
 
-The image is the runtime. `pip install cctv-stack` installs the four CLI entry points
-(`cctv-clip-worker`, `cctv-config`, `cctv-uploader`, `cctv-cleanup`) but **not** Frigate,
-Compose, or site configuration. The complete client-site install is the signed release
-bundle in [`docs/plans/packaging.md`](docs/plans/packaging.md).
+Storage is available behind the `storage` extra:
 
-## Development
+```python
+import os
+from cctv.storage import S3StorageAdapter
 
-```bash
-uv sync
-uv run pytest                    # unit suite; live tests skip without Docker/S3
-uv run ruff check .
-uv run ruff format --check .
+adapter = S3StorageAdapter(
+    bucket=os.environ["CCTV_STORAGE_BUCKET"],
+    access_key_id=os.environ["CCTV_STORAGE_KEY_ID"],
+    secret_access_key=os.environ["CCTV_STORAGE_APPLICATION_KEY"],
+    region=os.environ.get("CCTV_STORAGE_REGION", "us-west-004"),
+)
+```
+
+`cctv.__version__` reports the installed package version.
+
+## Complete installation
 
-./tests/integration/run.sh       # live S3 suite against RustFS; needs Docker
-./tests/e2e/run.sh               # synthetic camera → real clip; needs Docker + Frigate
+For a full CCTV site — Frigate, MQTT, the worker, uploader and cleanup — use the Compose
+deployment in this repository:
+
+```bash
+git clone https://github.com/aravindmyd/cctv-stack.git
+cd cctv-stack
+./install.sh
 ```
 
-The unit suite is **717 passed, 19 skipped** without a configured S3 endpoint or a running
-Docker daemon. CI runs pre-commit, unit tests on Python 3.12 and 3.13, the pip-install
-fallback, live integration, and the synthetic e2e pipeline. The publish workflow builds
-the multi-arch image and the Python distributions.
+The first run creates `.env` and `config/cameras.yml`; edit both and run it again. See
+[`docs/install.md`](docs/install.md) for the operator procedure and
+[`CONTRIBUTING.md`](CONTRIBUTING.md) for how releases are built and delivered.
+
+## Compatibility
+
+| | |
+|---|---|
+| Python | 3.12 or newer |
+| Frigate | 0.18 API and MQTT contract (verified) |
+| Platforms | Linux x86_64/arm64; macOS for development |
+| Cloud backup | Backblaze B2 through its S3-compatible endpoint; optional |
+| Inbound network | None required |
 
 ## Documentation
 
-- [`docs/index.md`](docs/index.md) — documentation map and reading order
-- [`docs/architecture/hld.md`](docs/architecture/hld.md) — how the system fits together
-- [`docs/architecture/invariants.md`](docs/architecture/invariants.md) — rules that must never break
-- [`docs/runbooks/install.md`](docs/runbooks/install.md) — the target operator runbook (M6; the current installer is the two-run flow above)
-- [`docs/plans/roadmap.md`](docs/plans/roadmap.md) — what gets built, in what order
-- [`docs/plans/backlog.md`](docs/plans/backlog.md) — deliberately deferred work
-- [`docs/plans/packaging.md`](docs/plans/packaging.md) — client-site delivery and release artefacts
+Full documentation: **https://aravindmyd.github.io/cctv-stack/**
+
+- [Install](docs/install.md) — Docker Compose from the published image, or pip
+- [Configure](docs/configure.md) — `cameras.yml`, `.env`, adding a camera
+- [Frigate UI login](docs/frigate-ui.md)
+- [Health](docs/health.md) — endpoints and what to check first
+- [Recover failed uploads](docs/recover-uploads.md)
+- [Glossary](docs/glossary.md) — the vocabulary these docs use
+- [`CONTRIBUTING.md`](CONTRIBUTING.md) — development, CI and release process
 
 ## License
 
-Apache-2.0 — see [`LICENSE`](LICENSE). The reasoning is in
-[`docs/plans/opensourcing.md`](docs/plans/opensourcing.md).
+Apache-2.0 — see [`LICENSE`](LICENSE).