artemisllmbench 0.1.0__tar.gz

artemisllmbench-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 TurinTech AI
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

artemisllmbench-0.1.0/PKG-INFO

@@ -0,0 +1,145 @@
+Metadata-Version: 2.4
+Name: artemisllmbench
+Version: 0.1.0
+Summary: LLM benchmark that proves optimization gains — correctness validation + performance measurement for any OpenAI-compatible endpoint
+Author-email: TurinTech AI <neda@turintech.ai>
+License-Expression: MIT
+Project-URL: Homepage, https://pypi.org/project/artemisllmbench
+Project-URL: Documentation, https://pypi.org/project/artemisllmbench
+Project-URL: Bug Tracker, https://turintech.ai
+Keywords: llm,benchmark,vllm,inference,performance,validation
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: System :: Benchmark
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: httpx>=0.28
+Requires-Dist: click>=8.0
+Requires-Dist: rich>=13.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: numpy>=1.24
+Provides-Extra: dashboard
+Requires-Dist: streamlit>=1.30; extra == "dashboard"
+Requires-Dist: plotly>=5.0; extra == "dashboard"
+Provides-Extra: full
+Requires-Dist: artemisllmbench[dashboard]; extra == "full"
+Requires-Dist: sentence-transformers>=2.2; extra == "full"
+Dynamic: license-file
+
+# Artemis LLM Benchmark
+
+A Python CLI for **correctness validation and performance benchmarking** of LLM serving endpoints. Works with any OpenAI-compatible server — vLLM, Ollama, llama.cpp, and more.
+
+---
+
+## Install
+
+```bash
+pip install artemisllmbench                    # core
+pip install "artemisllmbench[dashboard]"       # + Streamlit dashboard
+pip install "artemisllmbench[full]"            # + dashboard + semantic similarity
+```
+
+> `[full]` adds `sentence-transformers` for semantic similarity checks (Layer 3 validity).
+
+---
+
+## What It Does
+
+Artemis answers two questions after you optimize an LLM endpoint:
+- **Did the optimization preserve correctness?** — multi-layer validity checks on every response
+- **What is the publishable performance number?** — reproducible latency, throughput, and goodput metrics
+
+---
+
+## Quick Start
+
+**Validate a single endpoint** — correctness + performance in one command:
+
+```bash
+artemisllmbench validate \
+  --endpoint http://localhost:9000 \
+  --model Qwen/Qwen2.5-7B-Instruct \
+  --hardware a100
+```
+
+**Compare stock vs optimized** — sequential benchmarking with full GPU resources for each:
+
+```bash
+artemisllmbench compare \
+  --endpoint-a http://localhost:9000 \
+  --endpoint-b http://localhost:9001 \
+  --model Qwen/Qwen2.5-7B-Instruct \
+  --hardware a100
+```
+
+**Split-session compare** — when the stock endpoint is already torn down:
+
+```bash
+# Session 1: save the stock baseline
+artemisllmbench baseline --endpoint http://localhost:9000 --model <model> --hardware a100
+
+# Session 2: run the optimized candidate
+artemisllmbench candidate --endpoint http://localhost:9000 --model <model> --hardware a100
+```
+
+The dashboard launches automatically after each run. Open it at `http://<your-ip>:8501`.
+
+---
+
+## Key Features
+
+- **Multi-layer validity** — sanity, structural, semantic (embedding similarity ≥ 0.92), and exact-match checks catch regressions that latency numbers alone miss
+- **Reproducible metrics** — TTFT, P95/P99 latency, ITL (inter-token latency), throughput, CV, drift, and spike detection
+- **SLO / goodput tracking** — set `--slo-ttft` and `--slo-latency` thresholds; get the % of requests that met them
+- **Streamlit dashboard** — live progress, side-by-side results, analytics charts, and a live response comparison panel
+- **Fast mode** — `--fast` cuts runtime by ~75% for quick iteration checks
+- **Cross-machine support** — endpoints can be on different hosts or different hardware
+
+---
+
+## Common Flags
+
+| Flag | Description |
+|------|-------------|
+| `--fast` | Reduced runs (~75% faster). For quick checks only. |
+| `--production` | Full 50-run sequential + concurrent load (default). |
+| `--slo-ttft <ms>` | TTFT SLO threshold — enables goodput reporting. |
+| `--slo-latency <ms>` | End-to-end latency SLO threshold. |
+| `--plots` | ASCII charts inline in terminal output. |
+| `--live` | Rich terminal live view during concurrent phases. |
+| `--no-dashboard` | Skip auto-launching Streamlit. |
+| `--port N` | Streamlit port (default: 8501). |
+
+---
+
+## Validity Layers
+
+| Layer | Check | On failure |
+|-------|-------|------------|
+| 1 Sanity | Non-empty, complete sentence, token bounds | Hard fail |
+| 2 Structural | JSON/Python syntax where required | Hard fail |
+| 3 Semantic | Cosine similarity ≥ 0.92 vs. reference | Hard fail / warning |
+| 4 Exact match | String equality (`control_prompt_v1` only) | Warning |
+
+---
+
+## Pre-flight Conformance Check
+
+```bash
+artemisllmbench check-conformance --endpoint http://localhost:9000
+```
+
+Verifies your endpoint speaks the required OpenAI-compatible SSE format before a full benchmark run.
+
+---
+
+**Full documentation and source:** `artemisllmbench --help` or `artemisllmbench <command> --help`

artemisllmbench-0.1.0/PYPI_README.md

@@ -0,0 +1,109 @@
+# Artemis LLM Benchmark
+
+A Python CLI for **correctness validation and performance benchmarking** of LLM serving endpoints. Works with any OpenAI-compatible server — vLLM, Ollama, llama.cpp, and more.
+
+---
+
+## Install
+
+```bash
+pip install artemisllmbench                    # core
+pip install "artemisllmbench[dashboard]"       # + Streamlit dashboard
+pip install "artemisllmbench[full]"            # + dashboard + semantic similarity
+```
+
+> `[full]` adds `sentence-transformers` for semantic similarity checks (Layer 3 validity).
+
+---
+
+## What It Does
+
+Artemis answers two questions after you optimize an LLM endpoint:
+- **Did the optimization preserve correctness?** — multi-layer validity checks on every response
+- **What is the publishable performance number?** — reproducible latency, throughput, and goodput metrics
+
+---
+
+## Quick Start
+
+**Validate a single endpoint** — correctness + performance in one command:
+
+```bash
+artemisllmbench validate \
+  --endpoint http://localhost:9000 \
+  --model Qwen/Qwen2.5-7B-Instruct \
+  --hardware a100
+```
+
+**Compare stock vs optimized** — sequential benchmarking with full GPU resources for each:
+
+```bash
+artemisllmbench compare \
+  --endpoint-a http://localhost:9000 \
+  --endpoint-b http://localhost:9001 \
+  --model Qwen/Qwen2.5-7B-Instruct \
+  --hardware a100
+```
+
+**Split-session compare** — when the stock endpoint is already torn down:
+
+```bash
+# Session 1: save the stock baseline
+artemisllmbench baseline --endpoint http://localhost:9000 --model <model> --hardware a100
+
+# Session 2: run the optimized candidate
+artemisllmbench candidate --endpoint http://localhost:9000 --model <model> --hardware a100
+```
+
+The dashboard launches automatically after each run. Open it at `http://<your-ip>:8501`.
+
+---
+
+## Key Features
+
+- **Multi-layer validity** — sanity, structural, semantic (embedding similarity ≥ 0.92), and exact-match checks catch regressions that latency numbers alone miss
+- **Reproducible metrics** — TTFT, P95/P99 latency, ITL (inter-token latency), throughput, CV, drift, and spike detection
+- **SLO / goodput tracking** — set `--slo-ttft` and `--slo-latency` thresholds; get the % of requests that met them
+- **Streamlit dashboard** — live progress, side-by-side results, analytics charts, and a live response comparison panel
+- **Fast mode** — `--fast` cuts runtime by ~75% for quick iteration checks
+- **Cross-machine support** — endpoints can be on different hosts or different hardware
+
+---
+
+## Common Flags
+
+| Flag | Description |
+|------|-------------|
+| `--fast` | Reduced runs (~75% faster). For quick checks only. |
+| `--production` | Full 50-run sequential + concurrent load (default). |
+| `--slo-ttft <ms>` | TTFT SLO threshold — enables goodput reporting. |
+| `--slo-latency <ms>` | End-to-end latency SLO threshold. |
+| `--plots` | ASCII charts inline in terminal output. |
+| `--live` | Rich terminal live view during concurrent phases. |
+| `--no-dashboard` | Skip auto-launching Streamlit. |
+| `--port N` | Streamlit port (default: 8501). |
+
+---
+
+## Validity Layers
+
+| Layer | Check | On failure |
+|-------|-------|------------|
+| 1 Sanity | Non-empty, complete sentence, token bounds | Hard fail |
+| 2 Structural | JSON/Python syntax where required | Hard fail |
+| 3 Semantic | Cosine similarity ≥ 0.92 vs. reference | Hard fail / warning |
+| 4 Exact match | String equality (`control_prompt_v1` only) | Warning |
+
+---
+
+## Pre-flight Conformance Check
+
+```bash
+artemisllmbench check-conformance --endpoint http://localhost:9000
+```
+
+Verifies your endpoint speaks the required OpenAI-compatible SSE format before a full benchmark run.
+
+---
+
+**Full documentation and source:** `artemisllmbench --help` or `artemisllmbench <command> --help`

artemisllmbench-0.1.0/README.md

@@ -0,0 +1,353 @@
+# Artemis LLM Benchmark
+
+[![PyPI](https://img.shields.io/pypi/v/artemisllmbench)](https://pypi.org/project/artemisllmbench/)
+
+A standalone Python CLI for service-level correctness validation and performance benchmarking of LLM serving endpoints. Works with any OpenAI-compatible server (vLLM, Ollama, llama.cpp, etc.).
+
+---
+
+## What This Benchmark Does
+
+Artemis is the **final validation gate** — it answers two questions: did the optimization preserve correctness, and what is the publishable performance number?
+
+**The correct workflow:**
+
+```
+Project benchmark (llama-bench / vLLM scripts / benchmark_app)
+  → Guides optimization opportunities
+
+[Apply optimizations]
+
+Project benchmark again → Confirms internal improvement
+
+Artemis LLM Benchmark  ← this tool
+  → Validates improvement holds at service level
+  → Confirms correctness is preserved
+  → Produces the publishable, comparable number
+```
+
+**Use cases:**
+
+| Use case | Command |
+|----------|---------|
+| Validate a single optimized endpoint (correctness + performance) | `validate` |
+| Compare Endpoint A vs Endpoint B side-by-side in one session | `compare` |
+| Compare across sessions or machines (stock already torn down) | `baseline` + `candidate` |
+| Export existing results to the dashboard without re-running | `export-dashboard` |
+
+---
+
+## Install
+
+```bash
+pip install artemisllmbench                    # core
+pip install "artemisllmbench[dashboard]"       # + Streamlit dashboard
+pip install "artemisllmbench[full]"            # + dashboard + semantic similarity
+```
+
+> `[full]` includes `sentence-transformers` for semantic similarity checks (Layer 3 validity). Requires **sentence-transformers ≥ 3.0** and a compatible `huggingface_hub`. If you see `cannot import name 'cached_download'`, upgrade: `pip install --upgrade sentence-transformers`.
+
+**Development (from source):**
+
+```bash
+python -m venv .venv
+.venv/bin/pip install -e ".[full]"
+```
+
+`artemisllmbench` is available after install. In development you can also invoke directly with `python benchmark.py`.
+
+---
+
+## Commands
+
+### `validate` — Single endpoint
+
+Validate correctness and measure performance for one endpoint.
+
+```bash
+artemisllmbench validate \
+  --endpoint http://localhost:9000 \
+  --model Qwen/Qwen2.5-7B-Instruct \
+  --hardware a100
+```
+
+The dashboard launches automatically. Open it at `http://<your-ip>:8501`.
+
+---
+
+### `compare` — Stock vs optimized
+
+Benchmark both endpoints sequentially — each with full dedicated resources — then produce a side-by-side report.
+
+```bash
+artemisllmbench compare \
+  --endpoint-a http://localhost:9000 \
+  --endpoint-b http://localhost:9001 \
+  --model Qwen/Qwen2.5-7B-Instruct \
+  --hardware a100
+```
+
+**Why sequential, not simultaneous?** LLM inference is memory-bandwidth bound. Running two models simultaneously splits GPU VRAM and bandwidth, producing numbers that represent neither endpoint accurately. Sequential measurement with full resources for each gives reproducible, representative results.
+
+**Flow:**
+
+1. Full benchmark on Endpoint A (sequential validation → concurrent load)
+2. CLI pauses — prepare Endpoint B (optionally tear down Endpoint A to free memory)
+3. Full benchmark on Endpoint B
+4. Side-by-side comparison report printed; dashboard updated automatically
+
+**Cross-machine compare** — endpoints do not need to be on the same machine:
+
+```bash
+artemisllmbench compare \
+  --endpoint-a http://192.168.1.10:9000 \
+  --endpoint-b http://192.168.1.20:9000 \
+  --model Qwen/Qwen2.5-7B-Instruct \
+  --hardware a100 \
+  --hardware-b h100
+```
+
+`--hardware-b` labels the optimized side separately in the dashboard (e.g., `NVIDIA A100 80GB → NVIDIA H100 80GB`). If omitted, both sides use the same hardware label.
+
+**Verifying endpoint reachability** before running:
+
+```bash
+curl -s --max-time 5 http://<ip>:<port>/v1/models
+```
+
+Expected: JSON with the model name. If it hangs → port is blocked (check firewall / `ufw allow <port>`). If "connection refused" → vLLM not running or bound to `127.0.0.1` instead of `0.0.0.0`.
+
+---
+
+### `baseline` + `candidate` — Split sessions
+
+Use when Endpoint A is already torn down, or when the two endpoints run on different days or machines.
+
+```bash
+# Step 1: benchmark Endpoint A, save result
+artemisllmbench baseline \
+  --endpoint http://localhost:9000 \
+  --model Qwen/Qwen2.5-7B-Instruct \
+  --hardware a100
+
+# Step 2: benchmark Endpoint B — auto-loads saved baseline for comparison
+artemisllmbench candidate \
+  --endpoint http://localhost:9000 \
+  --model Qwen/Qwen2.5-7B-Instruct \
+  --hardware a100
+```
+
+After `candidate` completes, the dashboard shows the same compare-mode layout as `compare` — side-by-side charts, delta table, and the live comparison panel.
+
+For most cases, prefer `compare` — it runs both in one session and produces cleaner results.
+
+---
+
+### Common flags
+
+| Flag | Description |
+|------|-------------|
+| `--fast` | Fast mode: 5 warmup + reduced runs. Cuts runtime by ~75%. Good for quick checks — not for production comparisons. |
+| `--production` | Production mode: full warmup + 50 sequential runs. Default — explicit flag is optional. |
+| `--slo-ttft <ms>` | SLO threshold for TTFT. Reports **goodput** — % of requests that satisfy all configured SLOs. |
+| `--slo-latency <ms>` | SLO threshold for end-to-end latency. Used together with `--slo-ttft` or alone. |
+| `--hardware-b <hw>` | Label Endpoint B hardware separately (`compare` only) |
+| `--live` | Rich terminal dashboard during concurrent phases |
+| `--plots` | ASCII time-series and histogram charts in terminal report |
+| `--rps N` | Override target requests-per-second |
+| `--requests N` | Override total concurrent request count |
+| `--no-dashboard` | Skip auto-launching Streamlit |
+| `--port N` | Streamlit port (default: 8501) |
+
+**Goodput example** — answers "did the optimization improve SLO compliance?":
+
+```bash
+artemisllmbench compare \
+  --endpoint-a http://localhost:9000 \
+  --endpoint-b http://localhost:9001 \
+  --model Qwen/Qwen2.5-7B-Instruct \
+  --hardware a100 \
+  --slo-ttft 200 \
+  --slo-latency 500
+```
+
+Terminal output per scenario:
+```
+PASS: P95=142ms  Goodput: 98.7%
+```
+
+Dashboard Results tab shows a goodput compliance card with Endpoint A vs Endpoint B percentages and the pp delta.
+
+Before a run starts, Artemis sends one timing probe and prints an estimate:
+
+```
+Mode: PRODUCTION
+Estimated runtime:
+  Sequential   7 scenarios     ~95 min
+  Concurrent   6 scenarios     ~18 min
+  Total                        ~113 min
+
+Tip: add --fast to cut this to ~25 min
+```
+
+---
+
+## Dashboard
+
+Launches automatically with every command. Open at `http://<your-ip>:8501`.
+
+### Tabs
+
+**Live** — real-time progress during a run:
+- Single-run mode: one progress panel showing phase, scenario, metrics as they arrive
+- Compare mode: frozen Endpoint A panel (top) + live Endpoint B panel (bottom), both visible simultaneously. During the CLI pause between phases, a waiting placeholder appears automatically.
+
+**Results** — after the run completes:
+- `validate`: validity gate → output correctness → performance metrics table (TTFT, P95/P99 latency, CV, ITL mean/P95) → goodput card (if `--slo-ttft`/`--slo-latency` were set)
+- `compare` / `candidate`: optimization impact banner → output correctness → side-by-side metrics table with deltas (includes ITL and goodput) → goodput compliance card → live side-by-side comparison panel
+
+**Analytics** — detailed charts:
+- TTFT P50/P95/P99 grouped bar charts (Endpoint A vs Endpoint B in compare mode)
+- Queue pressure: sequential → concurrent P95 delta
+- Latency stability (CV%) with 5% / 20% thresholds
+- Isolation probes: prefill throughput, decode throughput, prefix cache detection
+- RPS saturation curve
+
+The Results and Analytics tabs are cleared during active runs and populate automatically once the benchmark and export complete — no manual refresh needed.
+
+### Live side-by-side comparison panel
+
+Available in `compare` and `candidate` modes under the Results tab. The panel only appears when there is something to show:
+
+- **Both endpoints live** — responses stream directly from both models in real time
+- **Recorded responses available** — replays responses captured during the benchmark run at actual benchmark resources
+- **Neither** — the panel is hidden entirely (this is the case in `validate` mode and when a `baseline`/`candidate` result is viewed without a live endpoint)
+
+Select a preset prompt or type your own, then click **Run comparison** to see responses stream side by side with TTFT and tok/s metrics.
+
+### Manual launch
+
+```bash
+artemisllmbench dashboard
+```
+
+### Export existing results
+
+```bash
+artemisllmbench export-dashboard \
+  --baseline baselines/<baseline-file>.json \
+  --candidate baselines/<candidate-file>.json
+```
+
+---
+
+## How It Works
+
+### Scenarios
+
+All scenarios are frozen, versioned YAML files. A version bump is required to change a scenario.
+
+| Scenario | Input | Output | Measures |
+|----------|-------|--------|----------|
+| `small_prompt_v1` | "What is the capital of Japan?" | ~20 tokens | TTFT, scheduling latency, serving overhead |
+| `large_prompt_v1` | ~1500-token technical content | ~120 tokens | PagedAttention, KV-cache efficiency, prefill throughput |
+| `long_context_v1` | ~6500-token context + factual question | ~60 tokens | Memory pressure, attention scaling |
+| `control_prompt_v1` | "What is 144 divided by 12?" | "12" (exact) | Determinism, precision drift (fp32 → bf16) |
+| `mixed_realistic_v1` | 26-prompt pool: code, reasoning, explanation, Q&A | 25–400 tokens | Realistic workload distribution, variable-length ITL |
+
+**Isolation probes** (run but not shown as primary metrics):
+
+| Probe | Measures |
+|-------|---------|
+| `prefill_probe_v1` | Cold vs. warm prefill throughput (tokens/s) — detects hugepage activation |
+| `decode_probe_v1` | Token generation throughput (tokens/s) and scheduling overhead |
+| `prefix_cache_probe_v1` | KV-cache prefix reuse — speedup ratio cold → warm |
+
+### Execution order
+
+For each scenario:
+
+1. **Warmup** (discarded) — GPU cache init, CUDA graph compilation, JIT
+2. **Sequential ×50** → validity gate + per-request latency distribution
+3. **Concurrent at pinned RPS ×500** → throughput metrics (only if sequential passed)
+4. **RPS sweep** (optional `--rps-sweep`) → saturation curve
+
+### Validity layers
+
+| Layer | Check | Failure mode |
+|-------|-------|--------------|
+| 1 Sanity | Non-empty, complete sentence, token bounds | Hard fail — stops benchmark |
+| 2 Structural | JSON/Python syntax validity where required | Hard fail |
+| 3 Semantic | Embedding cosine similarity ≥ 0.92 vs. reference outputs | Hard fail (<0.85), warning (0.85–0.92) |
+| 4 Exact match | String equality after strip (`control_prompt_v1` only) | Warning only |
+
+Layer 3 uses `sentence-transformers` and handles precision changes (fp32 → bf16) that produce semantically equivalent but not byte-identical outputs.
+
+### Observability signals
+
+| Signal | What it reveals |
+|--------|----------------|
+| **Drift** | Mean latency rising over sequential runs — memory pressure, thermal throttling, KV-cache eviction |
+| **Spikes** | Isolated requests >2.5× median — GC pauses, OS scheduling jitter |
+| **Fat tails** | P99 ≫ P95 — degraded long requests, attention scaling problems |
+| **Bimodal** | Two latency clusters — cache-hit vs. cache-miss path divergence |
+| **ITL (Inter-Token Latency)** | Mean / P95 / P99 time between consecutive tokens — exposes decode-stage jitter from speculative decoding, dynamic batching, or scheduler interference |
+
+Use `--plots` for ASCII time-series and histograms inline in the terminal report.
+Use `--live` for a Rich terminal dashboard during concurrent phases.
+
+---
+
+## Backend Conformance
+
+The benchmark measures a deployed service. It requires an OpenAI-compatible `/v1/chat/completions` endpoint with SSE streaming.
+
+**Pre-flight check:**
+
+```bash
+artemisllmbench check-conformance --endpoint http://localhost:9000
+```
+
+**Adapters** handle SSE format variation (one YAML per backend):
+
+- vLLM: `choices[0].delta.content`
+- llama.cpp: `choices[0].delta.content`
+- `_default.yaml`: fallback for conformant endpoints
+
+Run `artemisllmbench check-conformance --help` for all options.
+
+---
+
+## Development
+
+**Run tests:**
+
+```bash
+pytest tests/
+```
+
+**Project structure:**
+
+```
+benchmark.py                 # CLI entry point (installed as `artemisllmbench`)
+turing_bench/
+├── scenarios/               # Frozen, versioned scenario YAMLs
+├── adapters/                # SSE format configs per backend
+├── runner/                  # Sequential and concurrent execution engines
+├── validity/                # Multi-layer correctness checks
+├── stats/                   # Percentiles, CV, drift, spikes, distribution
+│   ├── live_dashboard.py    #   Rich terminal live view
+│   └── progress_writer.py   #   Writes _progress.json for Streamlit polling
+├── export/                  # Converts result JSON → dashboard config JSON
+├── dashboard/               # Streamlit dashboard (bundled in the package)
+│   └── app.py
+└── report/                  # Baseline manager, report formatter
+dashboard/
+└── data/                    # Dashboard config JSONs (auto-written after runs)
+tests/
+baselines/                   # Saved baseline JSONs (created on first run)
+```
+
+---
+
+**Questions?** Run `artemisllmbench` for the quick-start guide, or `artemisllmbench <command> --help` for full options.