cemi-cli 0.1.1__tar.gz

cemi_cli-0.1.1/PKG-INFO

@@ -0,0 +1,362 @@
+Metadata-Version: 2.4
+Name: cemi-cli
+Version: 0.1.1
+Summary: CEMI: An Edge AI MLOps Tool for Model Compression and Embedded Deployment
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: click>=8.0
+Requires-Dist: msal>=1.0
+Requires-Dist: requests>=2.31
+Requires-Dist: rich>=13.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+
+# CEMI CLI
+
+## Install
+
+### From PyPI
+
+The recommended public distribution is the published package from PyPI:
+
+```bash
+pip install cemi-cli
+```
+
+This is the preferred install path because it:
+
+- avoids a source checkout
+- avoids requiring Node/Vite on tester machines
+- installs the exact published CLI package
+
+### From source (developer install only)
+
+Install locally (from repo root):
+
+```bash
+pip install -e ./cli
+```
+
+## Local-only quick start (canonical flow)
+
+1. **Install**: install from PyPI with `pip install cemi-cli`, or use `pip install -e ./cli` for development.
+2. **In your script**: `create_writer(project="...", log_dir="...")` — runs go to `log_dir/runs/`, artifacts to `log_dir/artifacts/`. Default `log_dir` is `.cemi`.
+3. **Start the gateway** (same path as `log_dir`): `cemi gateway` or `cemi gateway --save-dir /path/to/log_dir`.
+4. **Open the workspace**: `cemi view` or open `http://127.0.0.1:3141/workspace`. No login; runs and metrics appear in the UI.
+
+**Contract**: Writer and gateway must use the same directory: Writer writes `log_dir/runs/<run_id>.jsonl` and `log_dir/artifacts/<run_id>/...`; gateway reads that directory via `--save-dir` (default `.cemi`).
+
+**Defaults (local):** Default directory is `.cemi` (current working directory). Default gateway port is `3141`. Default project is `default`. To use another directory, pass the same path to `create_writer(log_dir=...)` and `cemi gateway --save-dir <path>`.
+
+**Custom port:** If you run `cemi gateway -p 3142`, set `CEMI_LOCAL_SERVER_URL=http://127.0.0.1:3142` so the Writer's artifact URLs (from `add_local_file_artifact`) point to the same server. Otherwise the UI may 404 when loading artifacts.
+
+**Verify install:** From repo root: `pip install -e ./cli && pip install -e './cli[dev]' && pytest cli/tests/ -q` to run tests.
+
+**Troubleshooting:** For common issues (no runs, artifact 404, custom port), see the root README **Troubleshooting** section.
+
+---
+
+## Monitoring another repo (e.g. compression-engine)
+
+If you have a separate training or compression repo (e.g. **compression-engine** with MNIST benchmarks, PTQ, QAT), you can monitor its experiments in CEMI by adding only **Writer calls** to your existing code—no new training recipes.
+
+**Compression-engine (actual repo):** Entry point is `python -m engine.main --config <config.yaml>` with optional `--log-dir .cemi`. The YAML config sets `benchmark.models` (e.g. resnet18, mobilenetv2, vit-tiny) and `benchmark.compression.method` (ptq or qat). One run creates one CEMI run per model. Example configs: `benchmark_config_cemi_test.yaml`, `benchmark_config_PTQ.yaml`, `benchmark_config_QAT.yaml`.
+
+### What to add in the training repo (only Writer calls)
+
+1. **Create a writer** — Either:
+   - `create_writer(project="...", log_dir=...)` with `log_dir` from your script’s CLI arg (e.g. `--log-dir`, default `.cemi`), or
+   - When run under `cemi start`, use `create_writer_from_env()` so the script uses `CEMI_SAVE_DIR` and `CEMI_RUN_ID` set by the CLI.
+2. **Per run** — In your existing training/inference loop:
+   - `start_run(name=..., tags=...)` (e.g. tags: `method=ptq`, `method=qat`, `model=resnet18`).
+   - `log_parameter` and `log_metric` in the same places you already compute them.
+   - `add_local_file_artifact(path=..., kind="model")` for ONNX or checkpoints so the UI can serve them.
+   - `log_summary_metrics(...)`, `end_run(status="succeeded"|"failed")`, `emit_run_record()`.
+
+Do not wire a new training recipe; only instrument existing loops.
+
+### How to run (pip user, minimal env)
+
+Run from your **compression-engine repo** directory (where your training/benchmark script lives). No CEMI repo script to run—your code does the training; CEMI only records it.
+
+- **Option A — Gateway and view first, then run your script:**  
+  Terminal 1: `cemi gateway --save-dir .cemi` (or `cemi view --save-dir .cemi` to start gateway and open UI).  
+  Terminal 2: `python -m engine.main --config benchmark_config_cemi_test.yaml --log-dir .cemi`
+
+- **Option B — One command (gateway + UI + your script):**  
+  `cemi start --save-dir .cemi --project compression-engine -- python -m engine.main --config benchmark_config_cemi_test.yaml`  
+  Your script uses `create_writer_from_env()` so it picks up `CEMI_SAVE_DIR` and `CEMI_RUN_ID`; no need to pass `--log-dir` when run via `cemi start`.
+
+### Success criteria
+
+Run **your** training/benchmark script from the compression-engine repo (the code that runs is in that repo, not in CEMI). Three runs (e.g. 3 models on MNIST, one PTQ, one QAT) should appear in the CEMI workspace with metrics and artifacts. Use the same `log_dir` / `--save-dir` for the Writer and the gateway.
+
+**Where to run:** From the **compression-engine repo** directory: `cemi start --save-dir .cemi --project compression-engine -- python -m engine.main --config benchmark_config_cemi_test.yaml`. Config sets models and compression method (ptq/qat); one CEMI run per model. CEMI does not provide a separate “demo run” script—you run your existing code with writer calls added.
+
+---
+
+## Package behavior
+
+This package is intentionally local-only by default.
+
+| Command | Auth required? | Data destination |
+|---------|----------------|------------------|
+| `cemi start -- python train.py` | No | Local files under `save_dir` plus the local gateway |
+| `cemi view` | No | Opens the local workspace UI pointing at the local gateway |
+| `cemi gateway` | No | Serves the local workspace and reads local run/artifact files |
+| `cemi stop` | No | Stops local background services started by the CLI |
+
+---
+
+## Local-first flow (no account)
+
+1. Install the CLI and add the Writer to your training script (see **Writer usage** below).
+2. Start a run with your script:
+   - `cemi start -- python train.py` — Ensures a local gateway is available, sets a local run id, wires env vars, and opens the workspace UI.
+
+In local mode, the workspace UI is served directly by the gateway at:
+
+- `http://127.0.0.1:3141/workspace`
+
+When the workspace UI calls `GET /api/health` on the configured `VITE_API_BASE_URL`, the local gateway responds with:
+
+```json
+{ "status": "ok", "mode": "local" }
+```
+
+This tells the UI to **skip any login flow** and go straight to the `/workspace` area to visualize your local runs.
+
+No login required; everything stays on your machine.
+
+---
+
+## Retroactive local viewing (job already running)
+
+If your training/validation job is **already running** and writing to a directory, use the same path for the gateway:
+
+- Your code uses `create_writer(project="...", log_dir="/path/to/dir")` so events are in `log_dir/runs/<run_id>.jsonl` and `log_dir/artifacts/<run_id>/...`.
+- Later (in another terminal, or a machine that can read that directory), start the gateway with the same path: `cemi gateway --save-dir /path/to/dir`, then `cemi view` or open `http://127.0.0.1:3141/workspace`.
+
+Example:
+
+```python
+from cemi.writer import create_writer
+
+writer = create_writer(project="demo", log_dir="/tmp/cemi-demo")
+writer.start_run(name="retroactive-demo")
+writer.log_metric(name="loss", value=0.5, step=1)
+writer.emit_run_record()
+```
+
+Then later:
+
+```bash
+cemi gateway --save-dir /tmp/cemi-demo
+cemi view
+```
+
+---
+
+## Environment variables
+
+**For local-only use:** none required. Prefer passing `log_dir` to `create_writer(project, log_dir)` so your script and the gateway share the same directory. Optional env vars (used by `create_writer_from_env()` and `cemi start`):
+
+- **CEMI_SAVE_DIR** — Base directory for runs and artifacts; same as `log_dir` in `create_writer`. Gateway `--save-dir` should match.
+- **CEMI_LOCAL_DIR** — Override for run JSONL directory (default: `<log_dir>/runs`).
+- **CEMI_ARTIFACTS_DIR** — Override for artifacts (default: `<log_dir>/artifacts`).
+- **CEMI_LOCAL_SERVER_URL** — URL of the local gateway if you want the Writer to stream live to `cemi gateway`.
+- **CEMI_SINK** — Local sink selection such as `local` or `local+local_server`.
+
+---
+
+## Usage
+
+```bash
+cemi                    # welcome banner + usage
+cemi help               # help and usage
+cemi config             # show local config
+cemi start -- python train.py
+                        # local run: ensure gateway, open workspace UI, run your command
+cemi view               # open local workspace UI (no run created)
+cemi gateway            # start local gateway server (read .cemi/runs, accept POST /api/events)
+cemi gateway --save-dir /path/to/save_dir
+                        # retroactive local: read /path/to/save_dir/runs and serve /path/to/save_dir/artifacts
+cemi view --save-dir /path/to/save_dir
+                        # open workspace UI for that directory (gateway must use same path)
+cemi view --dev-ui       # use Vite dev server for workspace (only when running from repo; after pip install use embedded workspace)
+cemi --help              # full CLI help
+```
+
+---
+
+## Writer usage
+
+**Local-only (recommended):** use `create_writer(project, log_dir)` so runs and artifacts go to `log_dir/runs/` and `log_dir/artifacts/`. Use the same path with `cemi gateway --save-dir <log_dir>` so the workspace can read them.
+
+```python
+from cemi.writer import create_writer
+
+writer = create_writer(project="my-project", log_dir=".cemi")  # default log_dir is .cemi
+writer.start_run(name="My run")
+writer.log_parameter(key="learning_rate", value=0.001)
+writer.log_metric(name="loss", value=0.5, step=1)
+writer.log_summary_metrics({"final_accuracy": 0.95})
+writer.emit_run_record()
+writer.end_run(status="succeeded")
+writer.emit_run_record()
+```
+
+**With CLI:** when you run `cemi start -- python train.py`, the CLI sets `CEMI_RUN_ID`, `CEMI_PROJECT_ID`, `CEMI_SAVE_DIR`, and `CEMI_LOCAL_SERVER_URL`. Use `create_writer_from_env()` so the Writer uses those env vars and the same local directory as the gateway.
+
+---
+
+## Local data and operations
+
+### What gets written to disk
+
+- Run snapshots are appended as JSONL under `save_dir/runs/<run_id>.jsonl`.
+- Copied artifacts are stored under `save_dir/artifacts/<run_id>/`.
+- Local CLI config is stored under `~/.cemi/config.json`.
+- PID files used by `cemi stop` are stored under `~/.cemi/pids/`.
+
+### Where files live by default
+
+- If you do not pass `log_dir` or `--save-dir`, the default save directory is `.cemi` in the current working directory.
+- The default layout is:
+
+```text
+.cemi/
+  runs/
+  artifacts/
+```
+
+- Per-user local state lives under `~/.cemi/`.
+
+### Artifact sensitivity
+
+Artifacts may include model binaries, checkpoints, reports, or copied local files.
+Treat `add_local_file_artifact()` as publishing that file to anyone who can access
+your local gateway on your machine. Do not attach secrets, credentials, private
+datasets, or files you would not want copied into the artifact store.
+
+### Gateway bind address
+
+The gateway should stay bound to `127.0.0.1` only. It is
+intended for local use on the same machine and should not be exposed on a LAN
+or public interface.
+
+### Browser behavior
+
+- `cemi view` opens the browser automatically after the local gateway starts.
+- `cemi start` opens the browser automatically before it launches your command.
+- `cemi gateway` does not open the browser by itself.
+
+### Stop background processes
+
+Use:
+
+```bash
+cemi stop
+```
+
+This stops background gateway/frontend processes started by `cemi start --dev-ui`
+or `cemi view --dev-ui`.
+
+### Clear old state
+
+To clear old local run data for the current project directory:
+
+```bash
+rm -rf .cemi
+```
+
+To clear per-user CLI state:
+
+```bash
+rm -rf ~/.cemi
+```
+
+Only do this if you are sure you no longer need the saved runs, artifacts, config,
+or PID files.
+
+### Reset a broken local setup
+
+1. Run `cemi stop`.
+2. Remove stale project state with `rm -rf .cemi` if needed.
+3. Remove per-user state with `rm -rf ~/.cemi` if config or PID files are stale.
+4. Reinstall the published package from PyPI.
+
+### Uninstall
+
+```bash
+pip uninstall cemi-cli
+```
+
+If you also want to remove local state, delete `.cemi` in your project directory
+and `~/.cemi` in your home directory.
+
+---
+
+## Model PTQ (ONNXRuntime) quickstart (local)
+
+This is a quick local path for **baseline vs INT8 PTQ** monitoring.
+
+### 0) Install the CLI (one-time)
+
+From repo root:
+
+```bash
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -e ./cli
+```
+
+### 1) Start the local gateway
+
+Terminal A:
+
+```bash
+export CEMI_LOCAL_DIR=".cemi/runs"
+export CEMI_ARTIFACTS_DIR=".cemi/artifacts"
+cemi gateway
+```
+
+### 2) Open the embedded workspace UI
+
+In a browser, open:
+
+- `http://127.0.0.1:3141/workspace`
+
+Or run:
+
+```bash
+cemi view
+```
+
+### 3) Log two runs (baseline + int8 PTQ) with ONNX artifacts
+
+Terminal C (repo root, venv active):
+
+```bash
+# Baseline
+python3 scripts/model_ptq_onnxrt_demo.py \
+  --variant baseline \
+  --model-onnx /path/to/baseline.onnx \
+  --summary-metric accuracy=0.765 \
+  --summary-metric loss=0.42 \
+  --summary-metric energy_j=12.5
+
+# INT8 PTQ
+python3 scripts/model_ptq_onnxrt_demo.py \
+  --variant int8_ptq \
+  --model-onnx /path/to/int8_ptq.onnx \
+  --summary-metric accuracy=0.753 \
+  --summary-metric loss=0.45 \
+  --summary-metric energy_j=10.9
+```
+
+What you get:
+- Runs appear in the workspace Runs table immediately (local-only).
+- You can open a run and the **Graph** tab will load the `.onnx` artifact in the Netron viewer (fetched from the local gateway).
+- Latency samples / throughput will be logged automatically if `onnxruntime` + `numpy` are installed; otherwise you can log your own metrics from your training loop using the Writer.
+