smartcontext-proxy 0.1.0

package/SPEC.md

@@ -0,0 +1,915 @@
+# SmartContext Proxy — Technical Specification v2.0
+
+## Goal
+
+Self-configuring, provider-agnostic transparent proxy between LLM clients and providers. Operates like a network firewall — intercepts traffic, applies context optimization logic, forwards transparently. Zero-config `npx` install, works out of the box.
+
+## Core Principle: Transparent Firewall
+
+```
+Client App  ──►  SmartContext Proxy  ──►  LLM Provider
+  (unchanged)     (intercept+optimize)     (any provider)
+```
+
+The client doesn't know SmartContext exists. The provider doesn't know SmartContext exists. SmartContext sits in the middle, reads the conversation, replaces bloated history with optimized context, and forwards. Like a firewall — but for tokens.
+
+## 1. Zero-Config Bootstrap
+
+### Install & Run
+```bash
+npx smartcontext-proxy
+```
+
+That's it. On first run:
+
+1. **Auto-detect providers**: Scan env vars (`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `GOOGLE_API_KEY`, `OPENROUTER_API_KEY`, `OLLAMA_HOST`). Each detected key = one supported provider.
+2. **Auto-select embedding**: Check for local Ollama (`localhost:11434`) → use `nomic-embed-text`. No Ollama → use built-in ONNX runtime (`@xenova/transformers` with `nomic-embed-text-v1.5`). Zero external dependencies either way.
+3. **Auto-select storage**: Embedded LanceDB (zero config, writes to `~/.smartcontext/data/`). No server, no setup.
+4. **Start proxy**: Listen on `localhost:4800`. Print one line: `SmartContext listening on http://localhost:4800 — providers: anthropic, openai, ollama`
+5. **Generate config**: Write `~/.smartcontext/config.json` with detected settings. User can edit later if needed.
+
+### Client Integration
+Change one env var:
+```bash
+# Before
+ANTHROPIC_API_URL=https://api.anthropic.com
+
+# After
+ANTHROPIC_API_URL=http://localhost:4800/v1/anthropic
+```
+
+Or for OpenAI-compatible clients:
+```bash
+OPENAI_BASE_URL=http://localhost:4800/v1/openai
+```
+
+### Self-Configuration via LLM
+If config is ambiguous (multiple providers, unclear defaults), SmartContext can use a cheap local model (Ollama) or the cheapest available cloud model to:
+- Analyze the user's typical usage pattern (from first few intercepted requests)
+- Suggest optimal tier thresholds
+- Auto-tune chunk sizes based on actual conversation structure
+
+## 2. Provider-Agnostic Architecture
+
+### Request Flow (Firewall Model)
+
+```
+Inbound Request
+     │
+     ▼
+┌─────────────────┐
+│  Format Detect   │  ← Auto-detect: Anthropic Messages / OpenAI Chat / Google GenerateContent
+│  (by URL path    │     /v1/anthropic/* → Anthropic format
+│   or headers)    │     /v1/openai/*   → OpenAI format
+└────────┬────────┘     /v1/google/*   → Google format
+         │              /v1/ollama/*   → Ollama format
+         ▼
+┌─────────────────┐
+│  Parse to        │  ← Normalize all formats to internal CanonicalMessage[]
+│  Canonical       │     { role, content, metadata, timestamp }
+│  Format          │
+└────────┬────────┘
+         │
+         ▼
+┌─────────────────┐
+│  Context         │  ← The core logic:
+│  Optimizer       │     1. Extract system prompt (keep stable for KV-cache)
+│                  │     2. Keep Tier 1 (last N exchanges) verbatim
+│                  │     3. Embed user query → retrieve Tier 2 from vector store
+│                  │     4. Pack into token budget
+│                  │     5. Append Tier 3 summaries if space remains
+└────────┬────────┘
+         │
+         ▼
+┌─────────────────┐
+│  Serialize to    │  ← Convert back to original provider format
+│  Provider Format │     (same format as inbound — transparent proxy)
+└────────┬────────┘
+         │
+         ▼
+┌─────────────────┐
+│  Forward to      │  ← SSE stream-through for streaming requests
+│  Provider        │     Async post-index after response complete
+└────────┬────────┘
+         │
+         ▼
+┌─────────────────┐
+│  Index Exchange  │  ← Embed + store the full exchange (async, non-blocking)
+│  (async)         │     Write raw log to disk
+└─────────────────┘
+```
+
+### Provider Modules
+
+Each provider is a module implementing `ProviderAdapter`:
+
+```typescript
+interface ProviderAdapter {
+  name: string;
+  detect(req: IncomingMessage): boolean;           // Can this adapter handle this request?
+  parseRequest(body: any): CanonicalRequest;        // Provider format → canonical
+  serializeRequest(canonical: CanonicalRequest): any; // Canonical → provider format
+  forwardUrl(originalPath: string): string;         // Where to forward
+  parseStreamChunk(chunk: Buffer): StreamEvent;     // Parse SSE chunks
+  serializeStreamChunk(event: StreamEvent): Buffer; // Re-serialize SSE
+}
+```
+
+Built-in adapters: `anthropic`, `openai`, `google`, `ollama`.
+Custom adapters: drop a `.js` file into `~/.smartcontext/adapters/`.
+
+### Canonical Message Format
+
+```typescript
+interface CanonicalMessage {
+  role: 'system' | 'user' | 'assistant' | 'tool';
+  content: string | ContentBlock[];  // text, images, tool_use, tool_result
+  timestamp?: number;
+  metadata?: {
+    provider?: string;
+    model?: string;
+    tokens?: number;
+    files?: string[];      // file paths mentioned
+    tools?: string[];      // tools used
+    sessionId?: string;
+  };
+}
+
+interface CanonicalRequest {
+  messages: CanonicalMessage[];
+  systemPrompt?: string;
+  model: string;
+  stream: boolean;
+  maxTokens?: number;
+  temperature?: number;
+  tools?: any[];
+  rawHeaders: Record<string, string>;  // preserved for forwarding
+  providerAuth: string;                // API key for forwarding
+}
+```
+
+## 3. Context Optimization Engine
+
+### Tiered Strategy
+
+| Tier | What | Token Budget | Source |
+|------|------|-------------|--------|
+| **T0** | System prompt | Unlimited (stable prefix) | From request, never modified |
+| **T1** | Hot context | Last 3 exchanges verbatim | From request |
+| **T2** | Warm context | Top-K retrieved chunks | Vector store (semantic search) |
+| **T3** | Cold context | Session/project summaries | Pre-computed summaries |
+
+### Token Budget Algorithm
+
+```
+available = model_context_limit - system_prompt_tokens - response_reserve
+t1_tokens = sum(last_3_exchanges)
+t2_budget = available - t1_tokens - t3_reserve
+t3_reserve = min(500, available * 0.05)
+
+// Fill T2 greedily by relevance score until budget exhausted
+for chunk in retrieved_chunks_sorted_by_score:
+    if t2_used + chunk.tokens <= t2_budget:
+        include(chunk)
+        t2_used += chunk.tokens
+
+// Fill T3 with remaining space
+if remaining > 100:
+    include(session_summaries, limit=remaining)
+```
+
+### Retrieval Pipeline
+
+1. **Embed query**: User's last message → embedding vector
+2. **Candidate retrieval**: Top-20 from vector store (cosine similarity)
+3. **File-path boost**: If query mentions a file path, chunks containing that path get +0.2 boost. File-path inertia: if recent exchanges focused on a file, keep boosting it.
+4. **Recency boost**: Current session chunks get +0.15, last-hour chunks get +0.05
+5. **Dedup**: Chunks with similarity >0.92 → keep most recent
+6. **Confidence gate**: If best chunk score < 0.55, skip retrieval entirely → pass original context through (graceful degradation)
+7. **Min chunks**: Always include at least 3 chunks if they pass threshold 0.55
+
+### Chunking
+
+- **Unit**: One user-assistant exchange = one chunk
+- **Long responses**: Split at paragraph boundaries if >2000 tokens. Keep code blocks atomic.
+- **Metadata per chunk**: `{ sessionId, timestamp, files[], tools[], summary(first 100 chars) }`
+- **Overlap**: Last sentence of prev chunk prepended to next chunk
+
+## 4. Streaming Architecture
+
+Non-negotiable: zero perceived latency overhead.
+
+```
+Client ←──SSE──── SmartContext ←──SSE──── Provider
+         (pass-through)         (pass-through)
+
+Timeline:
+0ms    Client sends request
+5ms    SmartContext intercepts, optimizes context
+15ms   Forward to provider (optimized request, fewer tokens)
+20ms   Provider starts streaming response
+20ms   SmartContext passes first SSE chunk to client
+...    Stream continues transparently
+done   SmartContext asynchronously indexes the exchange
+```
+
+The optimization happens BEFORE the provider call (5-15ms for embed + retrieve). The streaming response is passed through byte-for-byte with zero buffering.
+
+## 5. Storage Architecture
+
+### Plugin System
+
+```typescript
+interface StorageAdapter {
+  name: string;
+
+  // Vector operations
+  upsertChunks(chunks: Chunk[]): Promise<void>;
+  search(embedding: number[], options: SearchOptions): Promise<ScoredChunk[]>;
+
+  // Raw log operations
+  appendLog(sessionId: string, exchange: Exchange): Promise<void>;
+  getSessionLog(sessionId: string): Promise<Exchange[]>;
+
+  // Summary operations
+  upsertSummary(sessionId: string, summary: string): Promise<void>;
+  getSummaries(sessionIds: string[]): Promise<Summary[]>;
+
+  // Lifecycle
+  initialize(config: any): Promise<void>;
+  close(): Promise<void>;
+}
+```
+
+### Built-in Adapters
+
+| Adapter | Config | Use Case |
+|---------|--------|----------|
+| `lancedb` (default) | Zero-config, `~/.smartcontext/data/` | npx users, single machine |
+| `opensearch` | `{ url: "http://..." }` | Teams, existing ES/OS infra |
+| `qdrant` | `{ url: "http://..." }` | ML teams with Qdrant |
+| `filesystem` | `{ path: "..." }` | Minimal, logs only, no vector search |
+
+### Embedding Plugin
+
+```typescript
+interface EmbeddingAdapter {
+  name: string;
+  dimensions: number;
+  embed(texts: string[]): Promise<number[][]>;
+  initialize(config: any): Promise<void>;
+}
+```
+
+| Adapter | Config | Use Case |
+|---------|--------|----------|
+| `onnx` (default) | Zero-config, downloads model on first run | npx users, no GPU |
+| `ollama` | `{ url: "http://localhost:11434", model: "nomic-embed-text" }` | Local Ollama users |
+| `remote-ollama` | `{ url: "http://beast:11434", model: "nomic-embed-text" }` | Our setup (Beast PC) |
+
+## 6. Configuration
+
+### Auto-Generated Config (`~/.smartcontext/config.json`)
+
+```jsonc
+{
+  // Auto-detected on first run, editable
+  "proxy": {
+    "port": 4800,
+    "host": "127.0.0.1"
+  },
+  "providers": {
+    // Auto-discovered from env vars
+    "anthropic": { "apiKey": "env:ANTHROPIC_API_KEY" },
+    "openai": { "apiKey": "env:OPENAI_API_KEY" }
+  },
+  "embedding": {
+    // Auto-selected: ollama if available, else onnx
+    "adapter": "ollama",
+    "config": { "url": "http://localhost:11434", "model": "nomic-embed-text" }
+  },
+  "storage": {
+    // Default: zero-config lancedb
+    "adapter": "lancedb",
+    "config": { "path": "~/.smartcontext/data" }
+  },
+  "context": {
+    "tier1_exchanges": 3,           // Hot: last N exchanges kept verbatim
+    "tier2_max_chunks": 10,         // Warm: max retrieved chunks
+    "tier2_min_score": 0.55,        // Minimum similarity for retrieval
+    "tier3_token_reserve": 500,     // Cold: tokens reserved for summaries
+    "recency_boost": 0.15,          // Boost for current session chunks
+    "filepath_boost": 0.20,         // Boost for file-path matches
+    "dedup_threshold": 0.92,        // Near-duplicate merge threshold
+    "confidence_gate": 0.55,        // Below this: skip retrieval, pass-through
+    "response_reserve_tokens": 8192 // Reserve for model response
+  },
+  "logging": {
+    "level": "info",
+    "raw_logs": true,               // Store full conversation logs
+    "metrics": true,                 // Token savings, latency stats
+    "debug_headers": false           // X-SmartContext-* headers in responses
+  }
+}
+```
+
+### Process Management
+
+**Foreground (default)** — like any dev server:
+```bash
+npx smartcontext-proxy          # Starts in foreground, Ctrl+C stops
+```
+
+**Daemon mode** — runs in background:
+```bash
+npx smartcontext-proxy start    # Start as background daemon
+npx smartcontext-proxy stop     # Stop daemon (sends SIGTERM)
+npx smartcontext-proxy restart  # Restart daemon
+npx smartcontext-proxy status   # Show: running/stopped, PID, uptime, stats
+```
+
+Daemon mechanics:
+- PID file: `~/.smartcontext/smartcontext.pid`
+- Stdout/stderr: `~/.smartcontext/logs/proxy.log`
+- `start` detaches process, writes PID, exits immediately
+- `stop` reads PID file, sends SIGTERM, waits for graceful shutdown (flush metrics, close storage)
+- `status` checks PID alive + shows stats from metrics endpoint
+- Graceful shutdown on SIGTERM/SIGINT: finish in-flight requests (5s timeout), flush index queue, close storage, remove PID file
+
+**System service (optional)** — for always-on:
+```bash
+npx smartcontext-proxy install-service   # Generate systemd/launchd service file
+npx smartcontext-proxy uninstall-service # Remove service
+```
+- macOS: generates LaunchAgent plist in `~/Library/LaunchAgents/`
+- Linux: generates systemd user service in `~/.config/systemd/user/`
+- Auto-start on boot, auto-restart on crash
+
+### CLI
+
+```bash
+npx smartcontext-proxy                    # Start foreground (Ctrl+C to stop)
+npx smartcontext-proxy start              # Start daemon
+npx smartcontext-proxy stop               # Stop daemon
+npx smartcontext-proxy restart            # Restart daemon
+npx smartcontext-proxy status             # Running? PID, uptime, savings stats
+npx smartcontext-proxy install-service    # Install system service (auto-start)
+npx smartcontext-proxy uninstall-service  # Remove system service
+npx smartcontext-proxy --port 8080        # Custom port (foreground)
+npx smartcontext-proxy --config ./my.json # Custom config
+npx smartcontext-proxy index <file>       # Index existing session logs
+npx smartcontext-proxy providers          # List detected providers
+npx smartcontext-proxy benchmark          # Run retrieval quality benchmark
+```
+
+## 7. Adapter System (Plugin & Play)
+
+### How Adapters Work
+
+Adapters are npm packages following naming convention `smartcontext-adapter-*`:
+
+```bash
+# Install OpenSearch adapter
+npm install -g smartcontext-adapter-opensearch
+
+# Install Qdrant adapter
+npm install -g smartcontext-adapter-qdrant
+
+# SmartContext auto-discovers installed adapters
+npx smartcontext-proxy
+# Output: "Discovered adapters: opensearch, qdrant"
+```
+
+### Our OC Adapter
+
+For our system, we build `smartcontext-adapter-openclaw`:
+
+```bash
+npm install -g smartcontext-adapter-openclaw
+```
+
+This adapter:
+- **Storage**: Uses OpenSearch on Castle (auto-discovers from `ES_URL` env var or OC config)
+- **Embedding**: Uses Beast Ollama (auto-discovers from OC agent config)
+- **Sessions**: Reads OC gateway session logs for initial indexing
+- **Dashboard**: Exposes metrics to dashboard-ts via existing OpenSearch indices
+- **Auth**: Reads OC auth-profiles for provider API keys
+
+Config for our setup becomes just:
+```jsonc
+{
+  "adapter": "openclaw",
+  "config": { "ocHome": "~/.openclaw" }  // Everything else auto-discovered
+}
+```
+
+## 8. Control Panel & Observability
+
+SmartContext ships with a built-in web dashboard. The user sees real value from minute one.
+
+### 8.1 Web Dashboard (built-in)
+
+Accessible at `http://localhost:4800` (same port as proxy, root path serves UI).
+Single-page app, embedded in the binary — no extra dependencies, no build step.
+Built with vanilla HTML/CSS/JS (no React/Vue) — inlined into the server, <50KB total.
+
+#### Dashboard Screens
+
+**Home / Status**
+```
+┌─────────────────────────────────────────────────────┐
+│  SmartContext Proxy                    ● Running     │
+│                                    [Pause] [Stop]   │
+├─────────────────────────────────────────────────────┤
+│                                                     │
+│  💰 Total Saved          ⚡ Requests Today          │
+│  $63.00                  142                        │
+│  4.2M tokens             avg 68% savings            │
+│                                                     │
+│  📊 Savings Over Time (7-day chart)                 │
+│  ▁▃▅▆▇█▇▆▇██▇                                     │
+│                                                     │
+│  🔌 Providers            💾 Storage                 │
+│  anthropic: ● active     chunks: 8,943              │
+│  openai: ● active        sessions: 142              │
+│  ollama: ● active        disk: 234 MB               │
+│                                                     │
+│  ⏱ Performance                                      │
+│  Latency overhead: 12ms p50 / 18ms p95              │
+│  Embedding: ollama (nomic-embed-text)               │
+│  Cache hit rate: 73%                                │
+└─────────────────────────────────────────────────────┘
+```
+
+**Live Feed** — real-time request stream:
+```
+┌─────────────────────────────────────────────────────┐
+│  Live Feed                          [Auto-scroll ✓] │
+├─────────────────────────────────────────────────────┤
+│  14:23:05  anthropic/opus    45.2K → 12.1K  -73%   │
+│            Retrieved: 7 chunks (top: 0.89)    12ms  │
+│  14:22:58  openai/gpt-4o    28.1K → 9.8K   -65%   │
+│            Retrieved: 5 chunks (top: 0.82)     8ms  │
+│  14:22:41  anthropic/sonnet  8.2K → 8.2K   pass    │
+│            ⚠ Below threshold, pass-through     2ms  │
+│  14:22:30  ollama/qwen3     12.0K → 4.1K   -66%   │
+│            Retrieved: 4 chunks (top: 0.77)    15ms  │
+└─────────────────────────────────────────────────────┘
+```
+
+Clicking a row expands to show: original messages, what was retrieved, what was cut, final assembled context. Full transparency.
+
+**Sessions** — per-session breakdown:
+- Session list with timestamps, request count, total savings
+- Click session → see all exchanges, retrieval decisions, chunk scores
+- Export session as JSON
+
+**Savings Report** — the money page:
+```
+┌─────────────────────────────────────────────────────┐
+│  Savings Report                     [Export CSV]     │
+├─────────────────────────────────────────────────────┤
+│                                                     │
+│  This Month                                         │
+│  ┌──────────────────────────────┐                   │
+│  │ Without SmartContext: $412   │                    │
+│  │ With SmartContext:    $127   │                    │
+│  │ ─────────────────────────── │                    │
+│  │ You saved:           $285   │  ← big, green     │
+│  │ Savings rate:         69%   │                    │
+│  └──────────────────────────────┘                   │
+│                                                     │
+│  By Provider                                        │
+│  anthropic   $198 saved  (72% reduction)            │
+│  openai       $67 saved  (61% reduction)            │
+│  ollama        $0 saved  (local, free)              │
+│                                                     │
+│  By Model                                           │
+│  claude-opus-4-6    $142 saved  (most expensive)    │
+│  claude-sonnet-4-6   $56 saved                      │
+│  gpt-4o              $67 saved                      │
+│                                                     │
+│  Projection (if current usage continues)            │
+│  Next month: ~$290 saved                            │
+│  Next year:  ~$3,480 saved                          │
+│                                                     │
+└─────────────────────────────────────────────────────┘
+```
+
+**Settings** — editable from UI:
+- Context tuning (tier sizes, thresholds, boosts)
+- Provider management (add/remove API keys)
+- Storage config
+- Logging level
+- Pause/resume individual providers
+- Changes write to `~/.smartcontext/config.json`
+
+#### Dashboard Tech Stack
+- Vanilla HTML + CSS + minimal JS (no framework)
+- Served by the proxy server itself (same port, root path)
+- All HTML/CSS/JS inlined into a single TypeScript file (`src/ui/dashboard.ts`)
+- Data via WebSocket from proxy (real-time updates) + REST API for history
+- Charts: lightweight `<canvas>` drawing, no chart library dependency
+- Works offline, no CDN, no external resources
+
+### 8.2 System Tray (optional, separate package)
+
+For users who want a tray icon:
+```bash
+npm install -g smartcontext-tray
+```
+
+Tray icon shows:
+- Green dot = running, yellow = paused, red = stopped
+- Click → opens web dashboard in default browser
+- Right-click menu: Pause / Resume / Stop / Open Dashboard / Quit
+- Tooltip: "SmartContext: 142 requests, $63 saved today"
+
+Built with `trayhost` (lightweight, no Electron). Separate package because tray requires native deps — core proxy stays zero-native-deps.
+
+### 8.3 API Endpoints (programmatic access)
+
+All dashboard data available via REST:
+
+```
+GET /_sc/status          → { state: "running"|"paused", uptime, pid }
+GET /_sc/stats           → { requests, savings, latency, storage }
+GET /_sc/stats/daily     → [ { date, requests, tokens_saved, cost_saved } ... ]
+GET /_sc/stats/providers → { anthropic: {...}, openai: {...} }
+GET /_sc/stats/models    → { "claude-opus-4-6": {...}, "gpt-4o": {...} }
+GET /_sc/feed            → WebSocket: real-time request stream
+GET /_sc/sessions        → [ { id, started, requests, savings } ... ]
+GET /_sc/sessions/:id    → { exchanges: [...], chunks_retrieved: [...] }
+GET /_sc/config          → current config (keys redacted)
+PUT /_sc/config          → update config (partial merge)
+POST /_sc/pause          → pause proxy (pass-through all requests)
+POST /_sc/resume         → resume optimization
+POST /_sc/stop           → graceful shutdown
+```
+
+### 8.4 Pause Mode
+
+When paused:
+- Proxy still runs and forwards all requests
+- Context optimization disabled — requests pass through unmodified
+- Indexing continues (still learning from conversations)
+- Dashboard shows "PAUSED" badge
+- Useful for: debugging, comparing with/without, temporary disable
+
+### 8.5 Response Headers (opt-in)
+
+When `logging.debug_headers: true`:
+```
+X-SmartContext-Savings: 73%
+X-SmartContext-Original-Tokens: 45200
+X-SmartContext-Optimized-Tokens: 12100
+X-SmartContext-Retrieved-Chunks: 7
+X-SmartContext-Top-Score: 0.89
+X-SmartContext-Cache-Hit: prefix
+X-SmartContext-Latency-Ms: 12
+X-SmartContext-Mode: optimized|pass-through|paused
+```
+
+### 8.6 Dashboard Integration (OC Adapter)
+For our setup, the OC adapter additionally:
+- Writes metrics to `smartcontext-metrics` OpenSearch index
+- Dashboard-ts gets a SmartContext tab reading from this index
+- Same data, native dashboard look & feel
+
+## 9. Test Mode & LLM-Assisted Diagnostics
+
+### 9.1 A/B Test Mode
+
+```bash
+npx smartcontext-proxy --test-mode
+# or from dashboard: Settings → Enable Test Mode
+```
+
+In test mode, every request is sent **twice**:
+
+```
+Client Request
+     │
+     ├──► Path A: SmartContext optimized → Provider → Response A (returned to client)
+     │
+     └──► Path B: Original unmodified   → Provider → Response B (stored, not returned)
+```
+
+The client always gets Response A (optimized). Response B is stored for comparison.
+
+**What gets compared:**
+- **Semantic similarity**: embed both responses, compute cosine similarity. Score >0.95 = equivalent quality. Score <0.85 = potential retrieval miss.
+- **Token delta**: how many tokens saved (A vs B input)
+- **Latency delta**: overhead of optimization path
+- **Content diff**: structured diff of responses (key facts present/missing)
+- **Tool use match**: if A and B call the same tools with same arguments
+
+**Dashboard in Test Mode:**
+
+```
+┌─────────────────────────────────────────────────────┐
+│  A/B Test Results                    [Export JSON]   │
+├─────────────────────────────────────────────────────┤
+│                                                     │
+│  Total comparisons: 87                              │
+│  Quality match (>0.95): 79 (91%)     ← green       │
+│  Minor diff (0.85-0.95): 6 (7%)     ← yellow      │
+│  Significant diff (<0.85): 2 (2%)   ← red          │
+│                                                     │
+│  Avg token savings: 64%                             │
+│  Avg latency overhead: 14ms                         │
+│                                                     │
+│  ⚠ Significant diffs (click to inspect):            │
+│  #43  anthropic/opus  0.78  "missed DB schema..."   │
+│  #71  openai/gpt-4o   0.82  "lost function sig..."  │
+│                                                     │
+└─────────────────────────────────────────────────────┘
+```
+
+Clicking a diff row shows side-by-side: optimized context vs full context, what chunks were retrieved, what was missing, both responses.
+
+**Cost note:** Test mode doubles API costs. Dashboard shows estimated extra cost. User can limit: `--test-mode --test-sample 20%` (randomly sample 20% of requests for A/B).
+
+### 9.2 Verbose Logging
+
+```bash
+npx smartcontext-proxy --verbose
+# or config: logging.level = "debug"
+# or dashboard: Settings → Log Level → Debug
+```
+
+Debug logs per request:
+
+```
+[14:23:05.001] REQ #142 anthropic/claude-opus-4-6
+[14:23:05.002]   Original: 45,200 tokens (23 messages)
+[14:23:05.003]   System prompt: 2,100 tokens (stable, cache-eligible)
+[14:23:05.004]   Tier 1: kept last 3 exchanges (4,800 tokens)
+[14:23:05.008]   Embedding query: "fix the auth middleware bug" → 768-dim vector (6ms)
+[14:23:05.012]   Retrieval: 20 candidates, 7 above threshold
+[14:23:05.012]     #1 score=0.89 session=abc123 "auth middleware refactor from yesterday"
+[14:23:05.012]     #2 score=0.84 session=abc123 "JWT validation edge case discussion"
+[14:23:05.012]     #3 score=0.81 session=def456 "middleware stack architecture overview"
+[14:23:05.012]     #4 score=0.77 session=abc123 "auth test failures and fixes"
+[14:23:05.012]     #5 score=0.72 session=ghi789 "similar bug in payment middleware"
+[14:23:05.012]     #6 score=0.68 filepath-boost=+0.20 → 0.88 "src/middleware/auth.ts changes"
+[14:23:05.012]     #7 score=0.61 session=abc123 "general project setup"
+[14:23:05.013]   Dedup: merged #1 and #4 (similarity 0.93)
+[14:23:05.013]   Budget: 38,300 available → packed 6 chunks (5,200 tokens)
+[14:23:05.013]   Tier 3: 1 session summary (320 tokens)
+[14:23:05.014]   Final context: 12,420 tokens (savings: 72.5%)
+[14:23:05.014]   Forwarding to api.anthropic.com
+[14:23:07.891]   Response: 1,842 tokens, streaming completed (2,877ms)
+[14:23:07.892]   Indexing exchange async...
+[14:23:07.910]   Indexed: 1 chunk (18ms)
+```
+
+Logs written to:
+- `~/.smartcontext/logs/proxy.log` (standard)
+- `~/.smartcontext/logs/debug.log` (verbose, only when enabled)
+- `~/.smartcontext/logs/requests/` (per-request JSON dumps when `logging.request_dumps: true`)
+
+Per-request JSON dump (for forensic analysis):
+
+```jsonc
+{
+  "id": 142,
+  "timestamp": "2026-03-29T14:23:05.001Z",
+  "provider": "anthropic",
+  "model": "claude-opus-4-6",
+  "original": {
+    "messages": 23,
+    "tokens": 45200,
+    "system_prompt_tokens": 2100
+  },
+  "optimized": {
+    "messages": 12,
+    "tokens": 12420,
+    "tier1_tokens": 4800,
+    "tier2_tokens": 5200,
+    "tier3_tokens": 320,
+    "system_prompt_tokens": 2100
+  },
+  "retrieval": {
+    "query_embedding_ms": 6,
+    "search_ms": 4,
+    "candidates": 20,
+    "above_threshold": 7,
+    "after_dedup": 6,
+    "after_budget": 6,
+    "top_score": 0.89,
+    "chunks": [
+      { "id": "chunk_abc_17", "score": 0.89, "tokens": 850, "session": "abc123", "preview": "auth middleware refactor..." },
+      // ...
+    ]
+  },
+  "savings_pct": 72.5,
+  "latency_overhead_ms": 14,
+  "response_tokens": 1842,
+  // In test mode, also includes:
+  "ab_test": {
+    "response_b_tokens": 1910,
+    "semantic_similarity": 0.97,
+    "quality_match": true
+  }
+}
+```
+
+### 9.3 LLM-Assisted Diagnostics
+
+SmartContext can use a cheap LLM to analyze its own behavior and help debug issues.
+
+#### Auto-Diagnosis (on significant quality diff)
+
+When A/B test detects similarity <0.85, or when user flags a bad response:
+
+1. SmartContext collects the full context: original messages, what was retrieved, what was cut, both responses
+2. Sends to a diagnostic LLM (cheapest available: local Ollama, or Haiku-tier cloud model)
+3. LLM analyzes and produces a diagnostic report
+
+```
+┌─────────────────────────────────────────────────────┐
+│  🔍 Diagnostic Report — Request #43                 │
+├─────────────────────────────────────────────────────┤
+│                                                     │
+│  Problem: Response quality degraded (similarity     │
+│  0.78). Model missed database schema context.       │
+│                                                     │
+│  Root cause: The DB schema was discussed in          │
+│  exchange #7 (45 min ago) but the embedding          │
+│  similarity to current query was only 0.52 —         │
+│  below the 0.55 threshold. The schema discussion     │
+│  used technical column names while the current       │
+│  query uses business-level terminology.              │
+│                                                     │
+│  Recommended fixes:                                  │
+│  1. Lower tier2_min_score to 0.50 for this          │
+│     session type (schema discussions)                │
+│  2. Add keyword fallback: if query contains          │
+│     table/column names, grep raw logs                │
+│  3. Consider hybrid retrieval: semantic + keyword    │
+│                                                     │
+│  ⚡ Auto-fix available:                              │
+│  [Apply fix #1] [Apply fix #2] [Ignore]             │
+│                                                     │
+└─────────────────────────────────────────────────────┘
+```
+
+#### Diagnostic Commands
+
+```bash
+npx smartcontext-proxy diagnose              # Analyze last 100 requests, find issues
+npx smartcontext-proxy diagnose --request 43 # Diagnose specific request
+npx smartcontext-proxy diagnose --tune       # Suggest config tuning based on usage patterns
+```
+
+From dashboard: any request in Live Feed or A/B results has a "🔍 Diagnose" button.
+
+#### What the Diagnostic LLM Analyzes
+
+| Trigger | What LLM Receives | What LLM Returns |
+|---------|-------------------|-------------------|
+| Quality diff <0.85 | Both contexts, both responses, retrieval scores | Root cause, fix suggestions |
+| User clicks "Diagnose" | Full request dump (JSON) | Plain-language analysis |
+| `diagnose --tune` | Aggregate stats, score distributions, miss patterns | Config recommendations with reasoning |
+| First 50 requests (onboarding) | Request patterns, conversation structure | Auto-tune suggestions for chunk size, thresholds |
+
+#### Auto-Tuning
+
+After accumulating 50+ requests, SmartContext can auto-tune itself:
+
+```bash
+npx smartcontext-proxy auto-tune
+```
+
+The diagnostic LLM analyzes:
+- Score distribution (are thresholds too high/low?)
+- Miss patterns (what type of context gets lost?)
+- Chunk size effectiveness (too big = wasteful, too small = lost context)
+- Provider-specific patterns (some models need more context than others)
+
+Produces a tuning report with specific config changes. User approves or rejects each.
+
+Dashboard: Settings → "🧪 Auto-Tune" button. Shows proposed changes with before/after predictions.
+
+#### LLM Selection for Diagnostics
+
+Priority (cheapest first):
+1. Local Ollama (qwen3-coder-next, kimi-k2.5) — free, fast
+2. Ollama Cloud — cheap, fast
+3. Cheapest detected cloud provider (Haiku, GPT-4o-mini)
+4. Skip diagnostics if no cheap LLM available
+
+Diagnostic calls are **never** sent through SmartContext itself (avoid recursion). Direct API calls to the diagnostic LLM.
+
+## 10. Graceful Degradation
+
+SmartContext must NEVER make things worse:
+
+| Failure | Behavior |
+|---------|----------|
+| Vector store down | Pass-through original request unmodified |
+| Embedding fails | Pass-through original request unmodified |
+| No chunks above threshold | Pass-through original request unmodified |
+| Provider unreachable | Return error (same as without proxy) |
+| Config missing | Auto-generate defaults and start |
+| First run, empty index | Pass-through until enough data indexed |
+
+The proxy is **additive only**. If anything in the optimization pipeline fails, the original request goes through untouched. The user never sees degraded quality from SmartContext — worst case they get exactly what they'd get without it.
+
+## 10. Security
+
+- API keys never stored in plaintext — reference env vars (`env:ANTHROPIC_API_KEY`)
+- Proxy listens on localhost by default (no network exposure)
+- Raw logs encrypted at rest (AES-256, key derived from machine ID)
+- No telemetry, no phone-home, no analytics
+- All data stays local unless user explicitly configures remote storage
+
+## 11. Project Structure
+
+```
+smartcontext-proxy/
+├── src/
+│   ├── index.ts              # Entry point, CLI, daemon management
+│   ├── proxy/
+│   │   ├── server.ts         # HTTP/SSE proxy server + UI serving
+│   │   ├── router.ts         # Route to correct provider adapter
+│   │   ├── stream.ts         # SSE pass-through logic
+│   │   └── pause.ts          # Pause/resume state management
+│   ├── providers/
+│   │   ├── types.ts          # ProviderAdapter interface
+│   │   ├── anthropic.ts      # Anthropic Messages API
+│   │   ├── openai.ts         # OpenAI Chat Completions
+│   │   ├── google.ts         # Google GenerateContent
+│   │   └── ollama.ts         # Ollama native API
+│   ├── context/
+│   │   ├── optimizer.ts      # Core context optimization logic
+│   │   ├── canonical.ts      # Canonical message format
+│   │   ├── chunker.ts        # Message chunking
+│   │   ├── retriever.ts      # Vector search + scoring
+│   │   └── budget.ts         # Token budget allocation
+│   ├── storage/
+│   │   ├── types.ts          # StorageAdapter interface
+│   │   ├── lancedb.ts        # Default embedded storage
+│   │   └── filesystem.ts     # Fallback: raw logs only
+│   ├── embedding/
+│   │   ├── types.ts          # EmbeddingAdapter interface
+│   │   ├── onnx.ts           # Built-in ONNX embedding
+│   │   └── ollama.ts         # Ollama embedding
+│   ├── config/
+│   │   ├── auto-detect.ts    # Provider/embedding/storage discovery
+│   │   ├── schema.ts         # Config validation
+│   │   └── defaults.ts       # Default values
+│   ├── metrics/
+│   │   ├── collector.ts      # Request/response metrics
+│   │   ├── endpoint.ts       # /_sc/* REST API
+│   │   └── history.ts        # Persistent metrics (daily/monthly)
+│   ├── ui/
+│   │   ├── dashboard.ts      # Generates HTML/CSS/JS (inlined, no deps)
+│   │   ├── ws-feed.ts        # WebSocket live feed server
+│   │   └── api.ts            # /_sc/* route handlers (pause/stop/config)
+│   ├── daemon/
+│   │   ├── process.ts        # Fork/detach, PID file, signal handling
+│   │   └── service.ts        # Generate systemd/launchd service files
+│   └── adapters/
+│       └── loader.ts         # Discover & load external adapters
+├── adapters/
+│   └── openclaw/             # Our adapter (separate npm package)
+│       ├── index.ts
+│       ├── storage.ts        # OpenSearch storage
+│       ├── embedding.ts      # Beast Ollama embedding
+│       └── session-importer.ts # Import OC session logs
+├── package.json
+├── tsconfig.json
+└── README.md
+```
+
+## 12. Synergy with Our Stack
+
+| Our Component | SmartContext Integration |
+|--------------|------------------------|
+| **OC Gateway** | SmartContext sits between OC and Anthropic/Gemini APIs. OC config points `baseUrl` to SmartContext. |
+| **Beast PC** | Remote Ollama embedding via `remote-ollama` adapter. Faster than ONNX, zero cost. |
+| **OpenSearch (Castle)** | `openclaw` adapter stores chunks + metrics in OS. Dashboard reads them. |
+| **Session logs** | `session-importer.ts` indexes historical OC sessions on first setup. |
+| **Dashboard** | New SmartContext tab: savings graph, retrieval quality, per-cron breakdown. |
+| **Cron jobs** | Each cron call goes through SmartContext → cross-session context for recurring tasks. |
+| **A2A Bridge** | Agent-to-agent messages indexed → agents share context automatically. |
+
+## 13. Benchmark Plan
+
+Before public release, benchmark on 10 real CC sessions (2 per type):
+
+| Session Type | What to Measure |
+|-------------|----------------|
+| Bug fix (short) | Retrieval precision, latency overhead |
+| Feature build (long) | Token savings %, quality retention |
+| Cron/monitoring | Cross-session context value |
+| Multi-file refactor | File-path boost effectiveness |
+| Learning/research | Summary tier value |
+
+Metrics per session:
+- **Semantic similarity**: SmartContext response vs full-context response (cosine sim of embeddings)
+- **Token ratio**: optimized / original
+- **Latency**: p50, p95 overhead
+- **Retrieval precision**: manually scored relevance of retrieved chunks (1-5 scale)