observability-toolkit 2.0.0 → 2.1.0

package/README.md

@@ -1,478 +1,243 @@
 # observability-toolkit
 
-MCP server for observability tooling - query traces, metrics, and logs from local JSONL files for Claude Code sessions. Optionally integrates with SigNoz Cloud for enhanced observability.
+MCP server for observability tooling — query traces, metrics, logs, and LLM events from agentic coding tools. Works with any agent emitting OTel GenAI semantic conventions. Ingest via OTLP to Cloudflare R2 or read from local JSONL.
+
+**Version**: v2.1.0 | **License**: MIT
+
+---
 
 ## Installation
 
 ```bash
+# Claude Code
 claude mcp add observability-toolkit -- npx -y observability-toolkit
-```
 
-Or for local development:
+# Cursor / Windsurf / Continue.dev / Cline — add to MCP config:
+{ "mcpServers": { "observability-toolkit": { "command": "npx", "args": ["-y", "observability-toolkit"] } } }
 
-```bash
-claude mcp add observability-toolkit -- node ~/.claude/mcp-servers/observability-toolkit/dist/server.js
+# Local dev
+node ~/.claude/mcp-servers/observability-toolkit/dist/server.js
 ```
 
+---
+
 ## Tools
 
 | Tool | Description |
 |------|-------------|
-| `obs_query_traces` | Query traces with filtering, regex, numeric operators |
-| `obs_query_metrics` | Query metrics with aggregations (sum, avg, p50, p95, p99, rate) |
-| `obs_query_logs` | Query logs with boolean search, field extraction |
-| `obs_query_llm_events` | Query LLM events with token usage and duration metrics |
+| `obs_query_traces` | Query spans with filtering, regex, numeric operators, agent/tool attributes |
+| `obs_query_metrics` | Query metrics with aggregations (sum, avg, p50, p95, p99, rate), time buckets |
+| `obs_query_logs` | Query logs with boolean search, field extraction, negation |
+| `obs_query_llm_events` | Query LLM events with token usage, duration, provider/model filters |
 | `obs_query_evaluations` | Query evaluation events with aggregations and groupBy |
 | `obs_query_verifications` | Query human verification events for EU AI Act compliance |
-| `obs_health_check` | Check telemetry system health with cache statistics |
-| `obs_context_stats` | Get context window utilization stats |
-| `obs_get_trace_url` | Get SigNoz trace viewer URL (requires SigNoz) |
-| `obs_setup_claudeignore` | Add entries to .claudeignore |
+| `obs_query_regressions` | Detect quality metric regressions via EWMA drift and consecutive breach tracking |
+| `obs_query_metric_histograms` | Query OTLP histogram bucket distributions by metric name |
+| `obs_health_check` | Telemetry system health with cache statistics |
+| `obs_context_stats` | Context window utilization stats |
+| `obs_token_budget` | Context utilization, cache hit rate, headroom per model/session with alert levels |
+| `obs_hallucination_detection` | Hallucination risk from evaluation telemetry — rates, scores, model/method breakdowns |
+| `obs_multi_agent_coordination` | Delegation depth, fan-out ratio, handoff latency, agent token usage |
+| `obs_routing_telemetry` | Model distribution, cost savings, fallback rate, routing latency |
+| `obs_estimate_cost` | Token cost estimation across models |
+| `obs_audit_trail` | Query audit trail events (SHA-256 hash chain) |
+| `obs_manage_datasets` | Create, list, get, delete evaluation datasets (trace promotion) |
+| `obs_inject_evaluations` | Inject evaluation events into local telemetry |
+| `obs_ingest_spans` | Ingest spans to cloud backend via OTLP protobuf |
+| `obs_ingest_traces` | Push complete OTel traces (resourceSpans) with service metadata |
 | `obs_export_langfuse` | Export evaluations to Langfuse via OTLP HTTP |
+| `obs_export_phoenix` | Export evaluations to Arize Phoenix via OTLP HTTP |
+| `obs_export_datadog` | Export evaluations to Datadog LLM Observability |
+| `obs_export_confident` | Export evaluations to Confident AI |
+| `obs_get_trace_url` | Get trace viewer URL |
+| `obs_setup_claudeignore` | Add entries to .claudeignore |
+
+---
 
 ## Configuration
 
-| Variable | Description | Default |
-|----------|-------------|---------|
-| `TELEMETRY_DIR` | Local telemetry directory | `~/.claude/telemetry` |
-| `SIGNOZ_URL` | SigNoz instance URL | - |
-| `SIGNOZ_API_KEY` | SigNoz API key | - |
-| `CACHE_TTL_MS` | Query cache TTL in milliseconds | `60000` |
-| `RETENTION_DAYS` | Days to retain telemetry files | `7` |
-| `LANGFUSE_ENDPOINT` | Langfuse OTLP endpoint URL | - |
-| `LANGFUSE_PUBLIC_KEY` | Langfuse public key | - |
-| `LANGFUSE_SECRET_KEY` | Langfuse secret key | - |
-
-## Usage Examples
-
-### Query Traces
-
-```javascript
-// Basic query
-obs_query_traces({ limit: 10 })
-
-// Filter by trace ID
-obs_query_traces({ traceId: "abc123..." })
-
-// Filter by service and duration
-obs_query_traces({ serviceName: "claude-code", minDurationMs: 100 })
-
-// Regex pattern matching
-obs_query_traces({ spanNameRegex: "^http\\..*" })
-
-// Numeric attribute filtering
-obs_query_traces({
-  numericFilter: [
-    { attribute: "http.status_code", operator: "gte", value: 400 }
-  ]
-})
-
-// Existence checks
-obs_query_traces({
-  attributeExists: ["error.message"],
-  attributeNotExists: ["http.response.body"]
-})
-
-// OTel GenAI agent/tool filters
-obs_query_traces({ agentName: "Explore", toolName: "Read" })
-obs_query_traces({ operationName: "execute_tool", toolCallId: "toolu_123" })
+```bash
+export OBTOOL_API_KEY="obtk_YOUR_KEY_HERE"
+export OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest.integritystudio.ai"
+# Or: doppler run --project integrity-studio --config dev -- npm start
 ```
 
-### Query Logs
-
-```javascript
-// Basic severity filter
-obs_query_logs({ severity: "ERROR", limit: 20 })
-
-// Boolean search (AND)
-obs_query_logs({
-  searchTerms: ["timeout", "connection"],
-  searchOperator: "AND"
-})
-
-// Boolean search (OR)
-obs_query_logs({
-  searchTerms: ["error", "warning", "critical"],
-  searchOperator: "OR"
-})
-
-// Field extraction from JSON logs
-obs_query_logs({
-  extractFields: ["user.id", "request.method", "response.status"]
-})
-
-// Exclude patterns
-obs_query_logs({
-  search: "error",
-  excludeSearch: "health-check"
-})
-```
+| Variable | Default | Notes |
+|----------|---------|-------|
+| `OBTOOL_API_KEY` | — | Required for cloud backend tools |
+| `OTEL_EXPORTER_OTLP_ENDPOINT` | — | Required for cloud telemetry export |
+| `BACKEND_TYPE` | `cloud` | Query/ingest backend: `local` or `cloud` |
+| `TELEMETRY_DIR` | `./.otel` | Local telemetry directory (cwd-relative) |
+| `CACHE_TTL_MS` | `60000` | Query cache TTL |
+| `RETENTION_DAYS` | `7` | Telemetry file retention |
+| `OBTOOL_API_URL` | — | Cloud backend query URL |
+| `OBTOOL_INGEST_URL` | — | Cloud ingest URL |
 
-### Query Metrics
+See [docs/ENVIRONMENT_SETUP.md](docs/ENVIRONMENT_SETUP.md) for full setup.
 
-```javascript
-// Basic query
-obs_query_metrics({ metricName: "session.context.size" })
+---
 
-// Aggregations
-obs_query_metrics({ metricName: "http.duration", aggregation: "avg" })
-obs_query_metrics({ metricName: "http.duration", aggregation: "p95" })
-obs_query_metrics({ metricName: "requests.count", aggregation: "rate" })
+## Data Sources
 
-// Time bucket grouping
-obs_query_metrics({
-  metricName: "token.usage",
-  aggregation: "sum",
-  timeBucket: "1h",
-  groupBy: ["model"]
-})
+### Local JSONL (default)
 
-// Percentiles
-obs_query_metrics({ metricName: "latency", aggregation: "p99" })
-```
+Scans `./.otel/` (cwd-relative; override with `TELEMETRY_DIR`) plus project-local `.claude/telemetry/`, `telemetry/`, `.telemetry/`. All directories are only included if they exist on disk at query time. Supports gzip. Compatible with Claude Code natively and any agent using the OTel file exporter.
 
-### Query LLM Events
+File patterns: `{traces,logs,metrics,llm-events,evaluations,verifications}-YYYY-MM-DD.jsonl[.gz]`
 
-```javascript
-// Basic query
-obs_query_llm_events({ limit: 20 })
+### Cloud Backend (optional)
 
-// Filter by model and provider
-obs_query_llm_events({ model: "claude-3-opus", provider: "anthropic" })
+All query tools accept `backend: 'local' | 'cloud'` (default: `cloud`). Cloud queries `obtool-api` (D1/R2) via `OBTOOL_API_URL` + `OBTOOL_API_KEY`. Circuit breaker protects against cascading failures.
 
-// OTel GenAI operation types
-obs_query_llm_events({ operationName: "chat" })
-obs_query_llm_events({ operationName: "invoke_agent" })
+### Data Pipelines
 
-// Filter by conversation
-obs_query_llm_events({ conversationId: "conv-abc123" })
+**Local backend** (`backend: 'local'`):
 
-// Combine filters
-obs_query_llm_events({
-  operationName: "chat",
-  provider: "anthropic",
-  conversationId: "conv-abc123"
-})
+```
+Claude Code hooks / OTel SDK
+         │ write
+         ▼
+  ./.otel/<path>.jsonl       ← FileSpanExporter (SDK self-telemetry; path is caller-configured)
+  ./.otel/*.jsonl            ← hook-written JSONL (TELEMETRY_DIR primary)
+  .claude/telemetry/         ← legacy project-local hook spans (if present)
+  telemetry/ .telemetry/     ← other project-local dirs (if present)
+         │ read (getTelemetryDirectories union)
+         ▼
+  obs_query_* tools
+         │ enrich
+         ▼
+  derive → judge → sync-to-kv.ts → Cloudflare KV
 ```
 
-### Multi-Provider LLM Support
+**Cloud backend** (`backend: 'cloud'`):
 
-Query events from any LLM provider using OTel GenAI standard identifiers:
+```
+hooks → OTLP HTTP → ingest.integritystudio.ai → R2 → D1 → api.integritystudio.ai
+```
 
-```javascript
-// Anthropic Claude
-obs_query_llm_events({ provider: "anthropic", model: "claude-3-opus" })
+---
 
-// OpenAI
-obs_query_llm_events({ provider: "openai", model: "gpt-4o" })
+## Services
 
-// Google Gemini
-obs_query_llm_events({ provider: "gcp.gemini", model: "gemini-1.5-pro" })
+### Ingest Worker (`obtool-ingest`)
 
-// Mistral AI
-obs_query_llm_events({ provider: "mistral_ai", model: "mistral-large" })
+Cloudflare Worker (Hono v4) receiving OTLP protobuf telemetry. Deployed at `ingest.integritystudio.ai`.
 
-// Cohere
-obs_query_llm_events({ provider: "cohere", model: "command-r-plus" })
+- `POST /v1/{traces,metrics,logs}` — OTLP protobuf ingest (gzip supported)
+- R2 storage: `telemetry/{signal}/{YYYY-MM-DD}/{HH}/batch-{ts}-{uuid8}.jsonl`
+- SHA-256 bearer token auth, KV idempotency (5min TTL)
 
-// AWS Bedrock (multi-model)
-obs_query_llm_events({ provider: "aws.bedrock" })
+```bash
+cd services/obtool-ingest && npm run dev | npm test | npm run deploy
+```
 
-// Azure OpenAI
-obs_query_llm_events({ provider: "azure.ai.openai" })
+### API Worker (`obtool-api`)
 
-// Local models (Ollama)
-obs_query_llm_events({ provider: "ollama", model: "llama3:8b" })
+Cloudflare Worker (Hono v4) querying D1/R2. Deployed at `api.integritystudio.ai`.
 
-// Groq
-obs_query_llm_events({ provider: "groq", model: "llama-3.3-70b" })
-```
+Routes: `/v1/{traces,metrics,logs,sessions,cost,datasets}` + histogram and raw span drill-down. Cursor-based pagination, per-key rate limiting.
 
-**Provider Fallback Chain**: The toolkit uses OTel GenAI v1.39 compliant attribute lookup:
-1. `gen_ai.provider.name` (primary)
-2. `gen_ai.system` (legacy OTel)
-3. `provider` (custom/fallback)
-
-### Health Check with Cache Stats
-
-```javascript
-obs_health_check({ verbose: true })
-
-// Returns:
-{
-  "status": "ok",
-  "backends": { ... },
-  "cache": {
-    "traces": { "hits": 10, "misses": 5, "hitRate": 0.67, "size": 15, "evictions": 0 },
-    "logs": { "hits": 8, "misses": 12, "hitRate": 0.4, "size": 20, "evictions": 2 },
-    "metrics": { "hits": 0, "misses": 0, "hitRate": 0, "size": 0, "evictions": 0 },
-    "llmEvents": { "hits": 0, "misses": 0, "hitRate": 0, "size": 0, "evictions": 0 }
-  }
-}
+```bash
+cd services/obtool-api && npm run dev | npm test | npm run deploy
 ```
 
-## Features
-
-### Query Capabilities
-
-| Feature | Description |
-|---------|-------------|
-| Percentile Aggregations | p50, p95, p99 for metrics |
-| Time Bucket Grouping | 1m, 5m, 1h, 1d buckets for trend analysis |
-| Rate Calculations | Per-second rate of change |
-| Numeric Operators | gt, gte, lt, lte, eq for attribute filtering |
-| Regex Patterns | Advanced span name filtering |
-| Boolean Search | AND/OR operators for log queries |
-| Field Extraction | Extract JSON paths from structured logs |
-| Negation Filters | Exclude matching spans/logs |
-| Existence Checks | Filter by attribute presence |
-
-### OTel Compliance
-
-| Feature | Description |
-|---------|-------------|
-| severityNumber | Standard OTel severity levels |
-| statusCode | UNSET, OK, ERROR for spans |
-| Histogram Buckets | Full histogram distribution support |
-| InstrumentationScope | Library/module metadata |
-| Span Links | Cross-trace relationships |
-| Exemplars | Metric-to-trace correlation |
-| Aggregation Temporality | DELTA, CUMULATIVE support |
-
-### OTel GenAI Semantic Conventions (10/10 compliance)
-
-| Feature | Description |
-|---------|-------------|
-| `gen_ai.operation.name` | Filter by chat, embeddings, invoke_agent, execute_tool |
-| `gen_ai.provider.name` | Provider fallback: gen_ai.provider.name → gen_ai.system → provider |
-| `gen_ai.conversation.id` | Filter LLM events by conversation ID |
-| `gen_ai.agent.id/name` | Filter traces by agent attributes |
-| `gen_ai.tool.name/call.id` | Filter traces by tool attributes |
-| `gen_ai.response.model` | Actual model that responded |
-| `gen_ai.response.finish_reasons` | Why generation stopped |
-| `gen_ai.request.temperature` | Sampling temperature |
-| `gen_ai.request.max_tokens` | Maximum output tokens |
-| Percentiles | p50, p95, p99, rate aggregations |
-
-### Supported LLM Providers
-
-| Provider ID | Description | Example Models |
-|-------------|-------------|----------------|
-| `anthropic` | Anthropic Claude | claude-3-opus, claude-3-sonnet, claude-3-haiku |
-| `openai` | OpenAI GPT | gpt-4o, gpt-4-turbo, gpt-3.5-turbo, o1, o1-mini |
-| `gcp.gemini` | Google AI Studio | gemini-1.5-pro, gemini-1.5-flash, gemini-2.0-flash |
-| `gcp.vertex_ai` | Google Vertex AI | gemini-pro, claude-3-opus (via Vertex) |
-| `aws.bedrock` | AWS Bedrock | claude-3-sonnet, titan-text, llama-3 |
-| `azure.ai.openai` | Azure OpenAI | gpt-4-deployment, gpt-35-turbo |
-| `mistral_ai` | Mistral AI | mistral-large, mistral-small, codestral |
-| `cohere` | Cohere | command-r-plus, command-r, embed-english |
-| `groq` | Groq | llama-3.3-70b, mixtral-8x7b |
-| `ollama` | Ollama (local) | llama3, mistral, codellama |
-| `together_ai` | Together AI | llama-3-70b, mixtral-8x7b |
-| `fireworks_ai` | Fireworks AI | llama-v3-70b, mixtral-8x7b |
-| `huggingface` | HuggingFace | Various open models |
-| `replicate` | Replicate | Various hosted models |
-| `perplexity` | Perplexity | sonar-pro, sonar |
-
-**Note**: Custom provider identifiers are also supported for internal or unlisted LLM services.
-
-### Performance
-
-| Feature | Description |
-|---------|-------------|
-| Query Caching | LRU cache with configurable TTL |
-| File Indexing | .idx sidecars for fast lookups |
-| Gzip Support | Transparent decompression of .jsonl.gz files |
-| BatchWriter | Buffered writes to reduce I/O |
-| Streaming | Early termination for large files |
-| Parallel Queries | Concurrent multi-directory queries |
-| Cursor Pagination | Efficient large result set handling |
-
-### Internal Observability
-
-| Feature | Description |
-|---------|-------------|
-| Cache Metrics | Hit/miss/eviction tracking |
-| Query Timing | Slow query warnings (>500ms) |
-| Circuit Breaker Logging | State transition visibility |
-| Health Check Stats | Cache statistics in health output |
-
-### Security
-
-| Feature | Description |
-|---------|-------------|
-| Query Escaping | ClickHouse-specific escaping, 22-pattern blocklist |
-| Memory Limits | MAX_RESULTS_IN_MEMORY=10000, streaming aggregation |
-| Input Validation | limit≤1000, date range≤365 days, regex limits |
-| Type Safety | NaN/Infinity rejection, explicit type assertions |
-
-See [docs/security.md](docs/security.md) for details.
+### API Provisioning
 
-## Data Sources
+Two-worker system for Flutter client API key lifecycle.
 
-### Local JSONL (Default)
+- **Sender Worker** (public): signup / signin / provision; signs payloads with HMAC-SHA256 before forwarding to receiver
+- **Receiver Worker** (`services/api-provisioning-receiver/`): verifies HMAC signature + replay window; validates email format, registrable domain, and MX records via Cloudflare DoH; authenticates JWT via Auth0 `/userinfo`; upserts team organization (`type='team'`, `current_plan=tier`); adds user to org membership; calls `api-keys-create` Supabase Edge Function
+- **`api-keys-create`** (Supabase/Deno): generates `obtk_` token, inserts `api_keys` with `tier` from request body, syncs to Cloudflare KV
 
-Scans multiple telemetry directories:
-- **Global**: `~/.claude/telemetry/` (always checked)
-- **Project-local**: `.claude/telemetry/`, `telemetry/`, `.telemetry/`
+**Tier values** (`api_key_tier` DB enum): `starter | growth | enterprise` — defaults to `starter`.
 
-File patterns (supports gzip compression):
-- `traces-YYYY-MM-DD.jsonl` / `.jsonl.gz`
-- `logs-YYYY-MM-DD.jsonl` / `.jsonl.gz`
-- `metrics-YYYY-MM-DD.jsonl` / `.jsonl.gz`
-- `llm-events-YYYY-MM-DD.jsonl` / `.jsonl.gz`
+**Receiver env bindings**: `AUTH0_DOMAIN`, `SUPABASE_URL`, `SUPABASE_ANON_KEY`, `SUPABASE_SERVICE_ROLE_KEY`, `SHARED_SECRET`.
 
-### SigNoz Cloud (Optional)
+See [docs/api-provisioning-flutter-contract.md](docs/api-provisioning-flutter-contract.md) | [docs/auth/api-key-provisioning.md](docs/auth/api-key-provisioning.md) | [docs/api-provisioning-security.md](docs/api-provisioning-security.md).
 
-When configured, queries SigNoz Cloud API with:
-- Circuit breaker protection
-- Cursor-based pagination
-- Response time tracking
+```bash
+cd services/e2e && npm test   # 7 files, 30 E2E scenarios (api-key-auth, dashboard-auth, receiver-security, sender-receiver, …)
+```
 
-## OTLP Export
+---
 
-Export data in OpenTelemetry format:
+## Evaluation Libraries
 
-```javascript
-// Export traces
-const otlpTraces = await backend.exportTracesOTLP({ startDate: "2026-01-28" });
+### LLM-as-Judge
 
-// Export logs
-const otlpLogs = await backend.exportLogsOTLP({ severity: "ERROR" });
+G-Eval (chain-of-thought + logprob normalization), QAG faithfulness, position bias mitigation (`mitigatedPairwiseEval`), panel evaluation, circuit breaker + retry. Zod schemas co-located as single source of truth.
 
-// Export metrics
-const otlpMetrics = await backend.exportMetricsOTLP({ metricName: "http.duration" });
-```
+See [docs/quality/llm-as-judge.md](docs/quality/llm-as-judge.md).
 
-### Langfuse Export (v1.8.6+)
-
-Export evaluations to Langfuse for unified tracing and evaluation analysis:
-
-```javascript
-// Export all evaluations from last 7 days
-obs_export_langfuse({})
-
-// Export with filters
-obs_export_langfuse({
-  evaluationName: "quality",
-  scoreMin: 0.8,
-  limit: 500,
-  batchSize: 100
-})
-
-// Dry run to preview export
-obs_export_langfuse({
-  startDate: "2026-01-28",
-  dryRun: true
-})
-
-// Override credentials (for testing)
-obs_export_langfuse({
-  endpoint: "https://cloud.langfuse.com",
-  publicKey: "pk-lf-...",
-  secretKey: "sk-lf-..."
-})
-```
+### Agent-as-Judge
 
-Features:
-- Batched OTLP HTTP export with retry logic
-- Memory protection (400MB warn, 600MB abort)
-- Progress logging for large exports
-- Credential sanitization in error messages
-- DNS rebinding protection
+ProceduralJudge (fixed pipeline, early termination), ReactiveJudge (adaptive routing, LRU state), tool verification (selection 40% / args 30% / result 30%), trajectory efficiency analysis, multi-agent handoff scoring.
 
-## Evaluation Libraries
+See [docs/quality/agent-as-judge.md](docs/quality/agent-as-judge.md).
 
-### LLM-as-Judge (`src/lib/llm-as-judge.ts`)
+### Quality Pipeline (Hooks)
 
-Single-pass LLM evaluation for output quality:
+- **T1 rule-based**: `tool_correctness`, `evaluation_latency`, `task_completion` — every invocation, zero cost
+- **T2 LLM judge**: `relevance`, `coherence`, `faithfulness`, `hallucination` — sampled, budget-controlled
+- **Divergence detection**: entropy-based bimodal alerts for `relevance`, `coherence`, `task_completion`
+- **Regression detection**: post-T2 inline EWMA drift check, emits `quality.degradation_confirmed` OTel event
+- **Meta-evaluation**: explanation quality scoring via `evaluateExplanationQuality()` (R6.2)
 
-```typescript
-import { gEval, qagEvaluate, JudgeCircuitBreaker } from './lib/llm-as-judge.js';
+---
 
-// G-Eval pattern with chain-of-thought
-const result = await gEval(testCase, criteria, llmFn);
+## Dashboard
 
-// QAG faithfulness evaluation
-const faithfulness = await qagEvaluate(testCase, llmFn);
+React 19 + Vite 8 in `dashboard/` (git submodule). Hono API on `:3001`, Auth0 Universal Login, role-based access via Supabase.
 
-// Production circuit breaker
-const breaker = new JudgeCircuitBreaker(5, 60000);
-const result = await breaker.evaluate(() => gEval(...));
-```
+Routes: `/` (overview), `/metrics/:name`, `/role/:roleName`, `/correlations`, `/coverage`, `/traces/:traceId`, `/sessions/:sessionId`, `/agents/:sessionId`, `/compliance`
 
-### Agent-as-Judge (`src/lib/agent-as-judge.ts`)
-
-Multi-step agent evaluation with trajectory analysis:
-
-```typescript
-import {
-  verifyToolCalls,
-  aggregateStepScores,
-  analyzeTrajectory,
-  collectiveConsensus,
-  ProceduralJudge,
-  ReactiveJudge,
-} from './lib/agent-as-judge.js';
-
-// Verify tool call correctness
-const verifications = verifyToolCalls(actions, expectedTools);
-
-// Analyze agent trajectory efficiency
-const metrics = analyzeTrajectory({ actions, expectedSteps: 5 });
-
-// Multi-agent consensus evaluation
-const consensus = await collectiveConsensus(judges, { id: 'eval-1' }, {
-  rounds: 3,
-  convergenceThreshold: 0.05,
-});
-
-// Procedural multi-stage evaluation
-const proceduralJudge = new ProceduralJudge([
-  { name: 'syntax', evaluate: syntaxChecker },
-  { name: 'semantic', evaluate: semanticAnalyzer },
-]);
-const result = await proceduralJudge.evaluate(evaluand);
-
-// Reactive specialist-based evaluation
-const reactiveJudge = new ReactiveJudge(router, specialists, deepDiveSpecialists);
-const result = await reactiveJudge.evaluate(evaluand);
-```
+Deployed as Cloudflare Pages (frontend) + Worker (`quality-metrics-api`). Data synced from local pipeline via `sync-to-kv.ts`.
 
-### Query Agent Evaluations
-
-```javascript
-// Filter by agent ID/name
-obs_query_evaluations({
-  agentId: 'agent-123',
-  agentName: 'TaskRunner',
-  evaluationName: 'tool_correctness',
-})
-
-// Response includes agent-specific fields
-{
-  stepScores: [{ step: 0, score: 0.9, explanation: '...' }],
-  toolVerifications: [{ toolName: 'search', toolCorrect: true, score: 1.0 }],
-  trajectoryLength: 5,
-}
+```bash
+cd dashboard && npm run dev          # :5173 + API :3001
+cd dashboard && npm run populate -- --seed
 ```
 
+---
+
+## Integrations
+
+| Platform | Method | Status |
+|----------|--------|--------|
+| Claude Code | Native MCP | Full |
+| Cursor / Windsurf / Continue.dev / Cline | MCP config | Full |
+| Any OTel agent | OTLP → local JSONL | Full |
+| obtool-ingest | OTLP → Cloudflare R2 | Full |
+| Langfuse / Phoenix / Datadog / Confident AI | OTLP / HTTP export | Export only |
+
+OTel GenAI semconv v1.40.0 compliance. 15 LLM providers supported (`anthropic`, `openai`, `gcp.gemini`, `gcp.vertex_ai`, `aws.bedrock`, `azure.ai.openai`, `mistral_ai`, `cohere`, `groq`, `ollama`, `together_ai`, `fireworks_ai`, `huggingface`, `replicate`, `perplexity`).
+
+---
+
 ## Development
 
 ```bash
-cd ~/.claude/mcp-servers/observability-toolkit
-npm install
-npm run build
-npm test     # 3254 tests
-npm run start
+npm install && npm run build && npm test
 ```
 
+See [docs/repomix/token-tree.txt](docs/repomix/token-tree.txt) for the full file tree with token counts.
+
+---
+
 ## Documentation
 
-- [docs/changelog/](docs/changelog/) - Version history and changelogs
-- [docs/reliability/security.md](docs/reliability/security.md) - Security controls and hardening
-- [docs/quality/llm-as-judge.md](docs/quality/llm-as-judge.md) - LLM-as-Judge architecture
-- [docs/quality/agent-as-judge.md](docs/quality/agent-as-judge.md) - Agent-as-Judge architecture
-- [docs/backlog/](docs/backlog/) - Feature backlog and roadmap
-- [docs/changelog/SESSION_HISTORY.md](docs/changelog/SESSION_HISTORY.md) - Development session logs
-- [docs/Summary.md](docs/Summary.md) - Full documentation index
+| Doc | Description |
+|-----|-------------|
+| [docs/CHANGELOG.md](docs/CHANGELOG.md) | Version index v2.0.1 → v3.0.14 |
+| [docs/roadmap/README.md](docs/roadmap/README.md) | Roadmap, research directions, architecture docs index |
+| [docs/otel-v2/otel-genai-attribute-reference.md](docs/otel-v2/otel-genai-attribute-reference.md) | OTel GenAI attribute reference (v1.40.0) |
+| [docs/otel-v2/agent-span-hierarchies.md](docs/otel-v2/agent-span-hierarchies.md) | Agent span model and hierarchy patterns |
+| [docs/otel-v2/schema-migration.md](docs/otel-v2/schema-migration.md) | JSONL → OTLP migration |
+| [docs/otel-v3/llm-evaluation-frameworks.md](docs/otel-v3/llm-evaluation-frameworks.md) | Langfuse, Phoenix, DeepEval, Datadog comparison |
+| [docs/quality/llm-as-judge.md](docs/quality/llm-as-judge.md) | LLM-as-Judge architecture |
+| [docs/quality/agent-as-judge.md](docs/quality/agent-as-judge.md) | Agent-as-Judge architecture |
+| [docs/api-provisioning-flutter-contract.md](docs/api-provisioning-flutter-contract.md) | Flutter client integration |
+| [docs/auth/api-key-provisioning.md](docs/auth/api-key-provisioning.md) | End-to-end provisioning flow diagram |
+| [docs/api-provisioning-security.md](docs/api-provisioning-security.md) | Security properties and threat model |
+| [docs/reliability/security.md](docs/reliability/security.md) | Security controls |
+| [docs/hooks-integration.md](docs/hooks-integration.md) | Hooks system integration — producer-consumer architecture, type chain, path coupling |

package/dist/__tests__/find-constant-dedup.test.d.ts

@@ -0,0 +1,11 @@
+/**
+ * Tests for constant deduplication CI check
+ *
+ * Validates that check-constant-dedup.sh correctly:
+ * 1. Exits with code 0 when no duplicates found
+ * 2. Exits with code 1 when duplicates exist
+ * 3. Exits with code 42 on script errors
+ * 4. Provides helpful error messages
+ */
+export {};
+//# sourceMappingURL=find-constant-dedup.test.d.ts.map

package/dist/__tests__/find-constant-dedup.test.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"find-constant-dedup.test.d.ts","sourceRoot":"","sources":["../../src/__tests__/find-constant-dedup.test.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG"}