kodelyth-ecc 1.5.9 → 1.7.0

package/rules/common/cost-aware-model-routing.md

@@ -0,0 +1,152 @@
+# Cost-Aware Model Routing — Auto-Pick the Right Model Tier per Task
+
+This rule teaches the AI to **classify the current task and recommend the right model tier** instead of defaulting to one expensive model for everything. Pairs with the `agent-intent-routing` rule (agent selection) and the `token-budget` safety hook (spend control) to keep ECC sessions both effective AND affordable.
+
+> **The 80/20 of cost in agent workflows:** 80% of turns are trivial (rename, format, doc, summary, list-files) and burn frontier-model tokens for no benefit. Routing them to a fast/cheap tier saves 60–80% of monthly spend with no measurable quality loss.
+
+---
+
+## The Rule
+
+Before composing your response, classify the task by **scope, ambiguity, and risk**, then either:
+
+1. **Route silently** — the user already picked a model that matches the tier (do nothing).
+2. **Suggest a tier change** — emit a one-line recommendation when the active model is wildly mismatched (frontier model on a doc-typo fix, or a tiny model on a security audit).
+
+Always be transparent. Never silently switch models. Always teach the user how to make the decision next time.
+
+---
+
+## Tier definitions
+
+| Tier | Examples | Default models (provider-agnostic) |
+|---|---|---|
+| **trivial** | Single-line rename, format-only, fix typo, add JSDoc, generate plain summary, list files, regex tweak, version bump, capitalisation. Output usually < 50 lines. | Claude Haiku · Gemini Flash · GPT-4.1-mini · Qwen 2.5 Coder 7B (local) · Ollama (Llama 3.1 8B). |
+| **standard** | Code review of a single PR, write a small unit test, refactor one file, fix a non-critical bug, write a focused doc section, design a small data model. | Claude Sonnet · GPT-4.1 · Gemini Pro · Qwen 2.5 Coder 32B. |
+| **hard** | Architecture decision, debug-blitz of intermittent prod bug, security audit, multi-file refactor (3+ files), incident postmortem, design review, devil-mode adversarial sweep, agent harness changes, anything `code-architect`/`incident-commander`/`debug-detective` would own. | Claude Opus · GPT-5 · Gemini Ultra · best-available frontier model. |
+
+**The names are reference points** — pick the equivalent tier in whatever provider stack the user has configured. The classification is what matters, not the brand.
+
+---
+
+## Classification heuristics (signals to weigh)
+
+Use these signals together — single signals are noisy, three or four together are reliable:
+
+### Trivial signals
+- One file, < 50 lines of context
+- Pure formatting / renaming / documentation work
+- No tests need updating
+- The user's question fits in a single sentence
+- No ambiguity ("rename `getUserName` to `getUserDisplayName`")
+- Verbs: format, rename, capitalize, fix typo, add comment, list, count, summarize
+
+### Standard signals
+- 1–3 files in scope
+- Behavior change + tests required
+- One subsystem
+- Verbs: refactor, review, fix bug, write test, optimize, document, migrate (small)
+- The user pasted a stack trace from one place
+
+### Hard signals
+- Cross-cutting concern (auth, payments, telemetry, retries, concurrency)
+- Multiple subsystems / 4+ files
+- Security implications (PII, secrets, authz, supply chain)
+- Production / incident framing ("urgent", "down", "leaking", "live")
+- Architectural framing ("decide", "design", "should we", "trade-off")
+- The user invoked `code-architect`, `incident-commander`, `debug-detective`, `security-reviewer`, `chaos-engineer`, `backdoor-hunter`, `prompt-injection-hunter`
+- Devil-mode parallel commands
+- The session has already burned > 30k tokens (compounding context)
+- Agent handoff chains involving 3+ specialists
+
+If signals are **mixed or contradictory**, default to **standard** and surface the ambiguity to the user.
+
+---
+
+## Output format when suggesting a tier change
+
+Emit exactly one block, then proceed normally:
+
+```
+[model-router] task=<trivial|standard|hard> · suggested=<tier> · current=<active model or "unknown">
+  why: <one short sentence on the dominant signals>
+  next: <how to switch — provider-specific, e.g. "use --model claude-haiku" or "switch to /model haiku">
+```
+
+**Only emit this block when the active model is mismatched.** If the user is already on the right tier, route silently.
+
+---
+
+## Per-team configurability
+
+Teams override the defaults via env vars or `.kodelyth/router.json`:
+
+### Env vars
+
+| Variable | Effect |
+|---|---|
+| `KODELYTH_ROUTER=off` | Disable the rule entirely. The AI never suggests model changes. |
+| `KODELYTH_ROUTER_TRIVIAL=<model-id>` | Override trivial tier. |
+| `KODELYTH_ROUTER_STANDARD=<model-id>` | Override standard tier. |
+| `KODELYTH_ROUTER_HARD=<model-id>` | Override hard tier. |
+| `KODELYTH_ROUTER_DEFAULT=<trivial\|standard\|hard>` | Default tier for ambiguous tasks (default `standard`). |
+
+### Project file: `.kodelyth/router.json`
+
+```json
+{
+  "trivial":  "claude-haiku-4-5-20251001",
+  "standard": "claude-sonnet-4-6",
+  "hard":     "claude-opus-4-1",
+  "default":  "standard",
+  "notes":    "Set by infra team. Bump trivial→sonnet for any task that touches /payments."
+}
+```
+
+A `notes` field is purely human — the AI surfaces it when emitting a recommendation so the team's reasoning rides along.
+
+---
+
+## Hard rules (never break these)
+
+1. **Security audits, incident triage, and devil-mode never downgrade.** If the active task involves `security-reviewer`, `incident-commander`, `debug-detective`, `prompt-injection-hunter`, `supply-chain-auditor`, `secret-hunter`, `backdoor-hunter`, `chaos-engineer`, `jailbreak-tester`, `code-stealer-detector`, or `license-violation-finder` — recommend **hard** tier even if other signals look trivial.
+2. **Production framing wins.** "Down", "leaking", "broken in prod", "incident", "outage" → hard.
+3. **No silent demotion of multi-file refactors** — if 3+ files are in scope, never recommend trivial.
+4. **Respect explicit user choice.** If the user said "use haiku" or "stay on sonnet", do not override. Surface only if a new turn introduces signals that conflict.
+5. **Budget-aware:** if the `token-budget` safety hook reports usage > 70% of budget, bias one tier downward when ambiguous to extend the session.
+
+---
+
+## Examples
+
+```
+User: "rename getUserName to getUserDisplayName everywhere"
+[model-router] task=trivial · suggested=trivial · current=opus
+  why: single rename, mechanical, no behavior change.
+  next: switch to /model claude-haiku to save ~80% on this turn.
+```
+
+```
+User: "production is down and the auth service is throwing 502s, help"
+[model-router] task=hard · suggested=hard · current=haiku
+  why: production incident with security-relevant subsystem.
+  next: switch to /model claude-opus before continuing — this is incident-commander territory.
+```
+
+```
+User: "review this 40-line PR"
+(active model: sonnet — silent route, no block emitted)
+```
+
+---
+
+## Why this rule exists
+
+LLM cost is the #1 reason teams stop using agent toolkits. Two complementary primitives:
+
+- The **token-budget safety hook** prevents catastrophic blow-ups (hard cap per session).
+- This **routing rule** prevents the slow leak (frontier-model overuse on trivial work).
+
+Together they typically cut LLM bills by 50–70% with no measurable quality drop on standard work.
+
+For implementation patterns inside user code (your own pipelines that call LLMs), see the [`cost-aware-llm-pipeline`](../../skills/cost-aware-llm-pipeline/SKILL.md) skill.

package/scripts/dashboard/data.js

@@ -0,0 +1,353 @@
+// scripts/dashboard/data.js
+//
+// Local observability dashboard: pure aggregators (v1.7.0).
+//
+// All data the dashboard renders comes from:
+//   - The MCP catalog (filesystem reads of agents/, skills/, commands/, rules/, bundles/)
+//   - The BM25 memory store (~/.kodelyth/memory/)
+//   - The evolve signal streams (~/.kodelyth/evolve/)
+//   - The token-budget hook state (${KODELYTH_TOKEN_BUDGET_DIR})
+//   - The orchestration session dirs (.orchestration/<session>/)
+//
+// Every aggregator is pure: takes its inputs explicitly OR uses well-defined
+// env-overridable defaults. No global mutable state. Errors are swallowed
+// and replaced with sane empty defaults — the dashboard must never crash
+// because one data source is unavailable.
+'use strict';
+
+const fs   = require('fs');
+const os   = require('os');
+const path = require('path');
+
+const ROOT = path.resolve(__dirname, '..', '..');
+
+// Lazy + defensive requires — the dashboard renders even if one module is broken.
+function safeRequire(rel) {
+  try { return require(path.join(ROOT, rel)); } catch { return null; }
+}
+
+const memoryStore = safeRequire('scripts/memory/store.js');
+const catalog     = safeRequire('scripts/mcp/catalog.js');
+const evolveStats = safeRequire('scripts/evolve/stats.js');
+const evolveProposals = safeRequire('scripts/evolve/proposals.js');
+
+// ── helpers ──────────────────────────────────────────────────────────────────
+
+function defaultMemoryDir() {
+  return process.env.KODELYTH_MEMORY_DIR || path.join(os.homedir(), '.kodelyth', 'memory');
+}
+function defaultEvolveDir() {
+  return process.env.KODELYTH_EVOLVE_DIR || path.join(os.homedir(), '.kodelyth', 'evolve');
+}
+function defaultBudgetDir() {
+  return process.env.KODELYTH_TOKEN_BUDGET_DIR || path.join(os.homedir(), '.kodelyth', 'token-budget');
+}
+function defaultCoordRoot() {
+  return process.env.KODELYTH_COORDINATION_ROOT || path.join(process.cwd(), '.orchestration');
+}
+
+function safeReadDir(p) {
+  try { return fs.readdirSync(p, { withFileTypes: true }); } catch { return []; }
+}
+function safeStat(p) {
+  try { return fs.statSync(p); } catch { return null; }
+}
+function safeReadJson(p, fallback) {
+  try {
+    return JSON.parse(fs.readFileSync(p, 'utf8'));
+  } catch { return fallback; }
+}
+
+// ── overview ─────────────────────────────────────────────────────────────────
+
+function overview({
+  repoRoot   = ROOT,
+  memoryDir  = defaultMemoryDir(),
+  evolveDir  = defaultEvolveDir(),
+  coordRoot  = defaultCoordRoot(),
+} = {}) {
+  let catalogStats = { agents: 0, skills: 0, commands: 0, rules: 0, bundles: 0 };
+  if (catalog?.stats) {
+    try { catalogStats = catalog.stats(); } catch { /* swallow */ }
+  }
+
+  let memoryTotal = 0;
+  let memoryProjects = 0;
+  if (memoryStore?.stats) {
+    try {
+      const ms = memoryStore.stats();
+      memoryTotal = ms.total || 0;
+      memoryProjects = ms.projects || 0;
+    } catch { /* swallow */ }
+  }
+
+  let proposalsCounts = { pending: 0, accepted: 0, rejected: 0, applied: 0, total: 0 };
+  if (evolveProposals?.readLatest) {
+    try {
+      const all = evolveProposals.readLatest(evolveDir);
+      proposalsCounts.total = all.length;
+      for (const p of all) {
+        if (p.status && proposalsCounts[p.status] !== undefined) {
+          proposalsCounts[p.status] += 1;
+        }
+      }
+    } catch { /* swallow */ }
+  }
+
+  let routingMisses = 0;
+  if (evolveStats?.getRoutingMissStats) {
+    try { routingMisses = evolveStats.getRoutingMissStats(evolveDir).total_misses || 0; } catch { /* */ }
+  }
+
+  let surfaceTotal = 0;
+  if (evolveStats?.getReuseStats) {
+    try { surfaceTotal = evolveStats.getReuseStats(evolveDir).total_surfaces || 0; } catch { /* */ }
+  }
+
+  let sessionsCount = 0;
+  try {
+    if (fs.existsSync(coordRoot)) {
+      sessionsCount = safeReadDir(coordRoot).filter(e => e.isDirectory()).length;
+    }
+  } catch { /* */ }
+
+  return {
+    package_root: repoRoot,
+    storage: {
+      memory_dir:  memoryDir,
+      evolve_dir:  evolveDir,
+      coord_root:  coordRoot,
+      home:        os.homedir(),
+    },
+    catalog: catalogStats,
+    memory:  { total: memoryTotal, projects: memoryProjects },
+    evolve:  {
+      proposals:       proposalsCounts,
+      routing_misses:  routingMisses,
+      surfaces_total:  surfaceTotal,
+    },
+    swarm: {
+      sessions: sessionsCount,
+    },
+    generated_at: new Date().toISOString(),
+  };
+}
+
+// ── memory ───────────────────────────────────────────────────────────────────
+
+// NOTE: the memory store reads `KODELYTH_MEMORY_DIR` once at require time
+// and freezes its path. The dashboard therefore reads from whatever store
+// path was configured at boot — there's no way to redirect mid-process.
+// `memoryDir` here is reported back via /api/overview but cannot be
+// overridden per-call.
+function memoryStats() {
+  if (!memoryStore?.stats) return { total: 0, projects: 0, byLanguage: {}, topTags: [], storageDir: defaultMemoryDir() };
+  try { return memoryStore.stats(); }
+  catch { return { total: 0, projects: 0, byLanguage: {}, topTags: [], storageDir: defaultMemoryDir() }; }
+}
+
+function recentMemories({ limit = 20 } = {}) {
+  if (!memoryStore?.listAll) return [];
+  try {
+    const all = memoryStore.listAll();
+    all.sort((a, b) => String(b.captured_at || '').localeCompare(String(a.captured_at || '')));
+    return all.slice(0, Math.max(1, Math.min(200, limit))).map(m => ({
+      id:           m.id,
+      captured_at:  m.captured_at,
+      problem:      m.problem,
+      approach:     (m.approach || '').slice(0, 240),
+      tags:         m.tags || [],
+      language:     m.language || null,
+      project_path: m.project_path || null,
+      source:       m.source || null,
+    }));
+  } catch { return []; }
+}
+
+function memorySearch({ query, limit = 10 } = {}) {
+  if (!memoryStore?.recall || !query) return [];
+  try {
+    const results = memoryStore.recall(query, { limit: Math.max(1, Math.min(50, limit)) });
+    return results.map(r => ({
+      id:       r.id,
+      score:    Number((r.score || 0).toFixed(3)),
+      problem:  r.problem,
+      approach: (r.approach || '').slice(0, 240),
+      tags:     r.tags || [],
+      language: r.language || null,
+    }));
+  } catch { return []; }
+}
+
+// ── evolve ───────────────────────────────────────────────────────────────────
+
+function evolveSnapshot({ evolveDir = defaultEvolveDir(), proposalLimit = 50 } = {}) {
+  proposalLimit = Math.max(1, Math.min(200, proposalLimit));
+  let reuse = { total_memories_tracked: 0, total_surfaces: 0, last_updated: null, entries: [] };
+  let miss  = { total_misses: 0, unique_prompts: 0, entries: [] };
+  if (evolveStats?.getReuseStats) {
+    try { reuse = evolveStats.getReuseStats(evolveDir); } catch { /* */ }
+  }
+  if (evolveStats?.getRoutingMissStats) {
+    try { miss = evolveStats.getRoutingMissStats(evolveDir); } catch { /* */ }
+  }
+
+  let proposals = [];
+  if (evolveProposals?.readLatest) {
+    try {
+      proposals = evolveProposals.readLatest(evolveDir).slice(0, proposalLimit).map(p => ({
+        id:           p.id,
+        type:         p.type,
+        status:       p.status,
+        created_at:   p.created_at,
+        last_updated: p.last_updated || p.timestamp,
+        target_path:  p.proposal?.target_path || null,
+        rationale:    p.proposal?.rationale || null,
+        applied_path: p.applied_path || null,
+        note:         p.note || null,
+      }));
+    } catch { /* */ }
+  }
+
+  return { reuse, miss, proposals };
+}
+
+// ── catalog ──────────────────────────────────────────────────────────────────
+
+function catalogList({ kind, query = '', limit = 50 } = {}) {
+  if (!catalog) return [];
+  const fnByKind = {
+    agents:   catalog.loadAgents,
+    skills:   catalog.loadSkills,
+    commands: catalog.loadCommands,
+    rules:    catalog.loadRules,
+    bundles:  catalog.loadBundles,
+  };
+  const fn = fnByKind[kind];
+  if (!fn) return [];
+
+  let items;
+  try { items = fn() || []; } catch { return []; }
+
+  const q = String(query || '').toLowerCase().trim();
+  if (q) {
+    items = items.filter(it => {
+      const hay = `${it.name || ''} ${it.description || ''} ${(it.tags || []).join(' ')}`.toLowerCase();
+      return hay.includes(q);
+    });
+  }
+  return items.slice(0, Math.max(1, Math.min(500, limit))).map(it => ({
+    name:        it.name,
+    description: it.description || '',
+    tags:        it.tags || [],
+    path:        it.path || null,
+    origin:      it.origin || null,
+  }));
+}
+
+function catalogCounts() {
+  if (!catalog?.stats) return { agents: 0, skills: 0, commands: 0, rules: 0, bundles: 0 };
+  try { return catalog.stats(); } catch { return { agents: 0, skills: 0, commands: 0, rules: 0, bundles: 0 }; }
+}
+
+// ── sessions (orchestration / swarm) ─────────────────────────────────────────
+
+function sessionsList({ coordRoot = defaultCoordRoot(), limit = 30 } = {}) {
+  if (!fs.existsSync(coordRoot)) return [];
+  const dirs = safeReadDir(coordRoot).filter(e => e.isDirectory());
+  const out = [];
+  for (const d of dirs) {
+    const abs = path.join(coordRoot, d.name);
+    const st = safeStat(abs);
+    if (!st) continue;
+    const workers = safeReadDir(abs).filter(e => e.isDirectory()).map(e => e.name);
+    out.push({
+      session: d.name,
+      path:    abs,
+      modified: st.mtime.toISOString(),
+      workers,
+      worker_count: workers.length,
+    });
+  }
+  out.sort((a, b) => b.modified.localeCompare(a.modified));
+  return out.slice(0, Math.max(1, Math.min(200, limit)));
+}
+
+function sessionDetail({ session, coordRoot = defaultCoordRoot() } = {}) {
+  if (!session || session === '..' || session === '.') return null;
+  const dir = path.join(coordRoot, session);
+  // Containment check — ensure the resolved path stays within coordRoot.
+  if (!dir.startsWith(coordRoot + path.sep) && dir !== coordRoot) return null;
+  if (!fs.existsSync(dir)) return null;
+  const workers = safeReadDir(dir).filter(e => e.isDirectory());
+  return {
+    session,
+    path: dir,
+    workers: workers.map(w => {
+      const wdir = path.join(dir, w.name);
+      const taskPath    = path.join(wdir, 'task.md');
+      const handoffPath = path.join(wdir, 'handoff.md');
+      const statusPath  = path.join(wdir, 'status.md');
+      return {
+        slug:        w.name,
+        has_task:    fs.existsSync(taskPath),
+        has_handoff: fs.existsSync(handoffPath),
+        has_status:  fs.existsSync(statusPath),
+        task_excerpt:    safeReadExcerpt(taskPath, 800),
+        handoff_excerpt: safeReadExcerpt(handoffPath, 800),
+        status_excerpt:  safeReadExcerpt(statusPath, 800),
+      };
+    }),
+  };
+}
+
+function safeReadExcerpt(p, max = 800) {
+  try {
+    const content = fs.readFileSync(p, 'utf8');
+    return content.length > max ? content.slice(0, max) + '\n…(truncated)' : content;
+  } catch { return null; }
+}
+
+// ── token budget snapshot ────────────────────────────────────────────────────
+
+function tokenBudgetSnapshot({ budgetDir = defaultBudgetDir() } = {}) {
+  if (!fs.existsSync(budgetDir)) return { sessions: [], total_tokens: 0 };
+  const files = safeReadDir(budgetDir).filter(e => e.isFile() && e.name.startsWith('budget-') && e.name.endsWith('.json'));
+  const sessions = [];
+  let total = 0;
+  for (const f of files) {
+    const data = safeReadJson(path.join(budgetDir, f.name), null);
+    if (!data) continue;
+    sessions.push({
+      session_id:  data.session_id || f.name.replace(/^budget-|\.json$/g, ''),
+      tokens:      Number(data.tokens) || 0,
+      budget:      data.budget || null,
+      mode:        data.mode || null,
+      last_event:  data.last_event || data.updated_at || null,
+    });
+    total += Number(data.tokens) || 0;
+  }
+  sessions.sort((a, b) => (b.last_event || '').localeCompare(a.last_event || ''));
+  return { sessions: sessions.slice(0, 50), total_tokens: total };
+}
+
+module.exports = {
+  // overview
+  overview,
+  // memory
+  memoryStats,
+  recentMemories,
+  memorySearch,
+  // evolve
+  evolveSnapshot,
+  // catalog
+  catalogList,
+  catalogCounts,
+  // sessions
+  sessionsList,
+  sessionDetail,
+  // token budget
+  tokenBudgetSnapshot,
+  // exposed for tests
+  _internals: { defaultMemoryDir, defaultEvolveDir, defaultBudgetDir, defaultCoordRoot, safeReadDir, safeReadJson, safeReadExcerpt },
+};