mcp-efficiency-engine 0.1.0

package/observability/evals/iteration-value-report.md

@@ -0,0 +1,39 @@
+# Iteration Value Report
+
+- timestamp: 2026-07-05T09:54:54.568327+00:00
+- events: 305
+- events_with_event_id: 247
+- metrics_covered: 7
+- feedback_confirmed: 101
+- confirmed_success: 101
+
+## KPI
+
+- metrics_coverage_rate: 2.83%
+- confirmed_feedback_rate: 40.89%
+- confirmed_success_rate: 100.0%
+- total_tokens: 6474
+- total_cost_usd: 0.0542
+- tokens_per_confirmed_success: 64.1
+- cost_per_confirmed_success_usd: 0.000537
+
+## Breakdown
+
+### Tool mode
+- hybrid: count=2, tokens=2400, cost_usd=0.02
+- local-only: count=4, tokens=4044, cost_usd=0.0342
+- model-only: count=1, tokens=30, cost_usd=0.0
+
+### Model
+- GPT-5.3-Codex: count=7, tokens=6474, cost_usd=0.0542
+
+### Engine
+- Azure RAG Builder: count=1, tokens=1200, cost_usd=0.01
+- CodeGraph: count=4, tokens=2874, cost_usd=0.0242
+- GitNexus: count=1, tokens=1200, cost_usd=0.01
+- Graphify: count=1, tokens=1200, cost_usd=0.01
+
+## Assessment
+
+- has_value_signal: True
+- notes: Value signal available. Track trend, not one-point absolute values.

package/observability/evals/learning-loop-report.json

@@ -0,0 +1,268 @@
+{
+  "timestamp": "2026-07-05T09:54:51.179894+00:00",
+  "source": "C:\\repo\\mcp-efficiency-engine\\observability\\logs\\routing-decisions.jsonl",
+  "feedback_source": "C:\\repo\\mcp-efficiency-engine\\observability\\logs\\learning-feedback.jsonl",
+  "total_events": 305,
+  "pending_events": 197,
+  "confirmed_events": 108,
+  "kpis": {
+    "fallback_rate": 0.1475,
+    "grounded_rate": 0.8492,
+    "success_rate": 1.0,
+    "confidence_avg": 0.945
+  },
+  "thresholds": {
+    "fallback_rate_max": 0.2,
+    "grounded_rate_min": 0.8,
+    "confidence_avg_min": 0.8
+  },
+  "health": {
+    "fallback_ok": true,
+    "grounded_ok": true,
+    "confidence_ok": true
+  },
+  "patterns": [
+    {
+      "pattern": "azure-rag+info",
+      "total": 4,
+      "fallback_rate": 0.0,
+      "grounded_rate": 1.0,
+      "pending_count": 4,
+      "confirmed_count": 0,
+      "success_rate": 0.0,
+      "confidence_avg": 0.0,
+      "engines": {
+        "Azure RAG Builder": 4
+      },
+      "agents": {
+        "rag-azure": 4
+      }
+    },
+    {
+      "pattern": "azure-rag+knowledge-grounded",
+      "total": 52,
+      "fallback_rate": 0.0,
+      "grounded_rate": 1.0,
+      "pending_count": 38,
+      "confirmed_count": 14,
+      "success_rate": 1.0,
+      "confidence_avg": 0.946,
+      "engines": {
+        "Azure RAG Builder": 52
+      },
+      "agents": {
+        "rag-azure": 52
+      }
+    },
+    {
+      "pattern": "backend+bug-fix",
+      "total": 12,
+      "fallback_rate": 0.0,
+      "grounded_rate": 1.0,
+      "pending_count": 0,
+      "confirmed_count": 12,
+      "success_rate": 1.0,
+      "confidence_avg": 0.95,
+      "engines": {
+        "CodeGraph": 12
+      },
+      "agents": {
+        "backend": 12
+      }
+    },
+    {
+      "pattern": "dba+analysis",
+      "total": 51,
+      "fallback_rate": 0.0,
+      "grounded_rate": 1.0,
+      "pending_count": 37,
+      "confirmed_count": 14,
+      "success_rate": 1.0,
+      "confidence_avg": 0.946,
+      "engines": {
+        "Graphify": 51
+      },
+      "agents": {
+        "dba": 51
+      }
+    },
+    {
+      "pattern": "dba+sql",
+      "total": 18,
+      "fallback_rate": 0.0,
+      "grounded_rate": 1.0,
+      "pending_count": 11,
+      "confirmed_count": 7,
+      "success_rate": 1.0,
+      "confidence_avg": 0.95,
+      "engines": {
+        "Graphify": 18
+      },
+      "agents": {
+        "dba": 18
+      }
+    },
+    {
+      "pattern": "dev+bug-fix",
+      "total": 50,
+      "fallback_rate": 0.06,
+      "grounded_rate": 1.0,
+      "pending_count": 47,
+      "confirmed_count": 3,
+      "success_rate": 1.0,
+      "confidence_avg": 0.917,
+      "engines": {
+        "CodeGraph": 50
+      },
+      "agents": {
+        "backend": 50
+      }
+    },
+    {
+      "pattern": "dev+info",
+      "total": 2,
+      "fallback_rate": 0.0,
+      "grounded_rate": 0.5,
+      "pending_count": 2,
+      "confirmed_count": 0,
+      "success_rate": 0.0,
+      "confidence_avg": 0.0,
+      "engines": {
+        "CodeGraph": 2
+      },
+      "agents": {
+        "backend": 2
+      }
+    },
+    {
+      "pattern": "frontend+bug-fix",
+      "total": 1,
+      "fallback_rate": 1.0,
+      "grounded_rate": 0.0,
+      "pending_count": 1,
+      "confirmed_count": 0,
+      "success_rate": 0.0,
+      "confidence_avg": 0.0,
+      "engines": {
+        "CodeGraph": 1
+      },
+      "agents": {
+        "frontend-agent": 1
+      }
+    },
+    {
+      "pattern": "frontend+feature",
+      "total": 14,
+      "fallback_rate": 0.0,
+      "grounded_rate": 1.0,
+      "pending_count": 2,
+      "confirmed_count": 12,
+      "success_rate": 1.0,
+      "confidence_avg": 0.95,
+      "engines": {
+        "CodeGraph": 14
+      },
+      "agents": {
+        "frontend-agent": 14
+      }
+    },
+    {
+      "pattern": "incremental-json-projection",
+      "total": 7,
+      "fallback_rate": 0.0,
+      "grounded_rate": 1.0,
+      "pending_count": 0,
+      "confirmed_count": 7,
+      "success_rate": 1.0,
+      "confidence_avg": 0.91,
+      "engines": {
+        "CodeGraph": 7
+      },
+      "agents": {
+        "autodocs-agent": 7
+      }
+    },
+    {
+      "pattern": "iot+architecture",
+      "total": 1,
+      "fallback_rate": 0.0,
+      "grounded_rate": 1.0,
+      "pending_count": 1,
+      "confirmed_count": 0,
+      "success_rate": 0.0,
+      "confidence_avg": 0.0,
+      "engines": {
+        "GitNexus/CodeGraph + Graphify": 1
+      },
+      "agents": {
+        "iot": 1
+      }
+    },
+    {
+      "pattern": "legacy+migration",
+      "total": 53,
+      "fallback_rate": 0.7358,
+      "grounded_rate": 0.3208,
+      "pending_count": 39,
+      "confirmed_count": 14,
+      "success_rate": 1.0,
+      "confidence_avg": 0.946,
+      "engines": {
+        "GitNexus": 53
+      },
+      "agents": {
+        "legacy": 53
+      }
+    },
+    {
+      "pattern": "rag+info",
+      "total": 21,
+      "fallback_rate": 0.0952,
+      "grounded_rate": 0.9048,
+      "pending_count": 8,
+      "confirmed_count": 13,
+      "success_rate": 1.0,
+      "confidence_avg": 0.95,
+      "engines": {
+        "CodeGraph": 1,
+        "Graphify": 20
+      },
+      "agents": {
+        "backend": 1,
+        "rag-local": 20
+      }
+    },
+    {
+      "pattern": "unknown-domain+info",
+      "total": 6,
+      "fallback_rate": 0.0,
+      "grounded_rate": 0.0,
+      "pending_count": 6,
+      "confirmed_count": 0,
+      "success_rate": 0.0,
+      "confidence_avg": 0.0,
+      "engines": {
+        "CodeGraph": 6
+      },
+      "agents": {
+        "backend": 6
+      }
+    },
+    {
+      "pattern": "ux-ui+review",
+      "total": 13,
+      "fallback_rate": 0.0,
+      "grounded_rate": 1.0,
+      "pending_count": 1,
+      "confirmed_count": 12,
+      "success_rate": 1.0,
+      "confidence_avg": 0.95,
+      "engines": {
+        "Graphify": 13
+      },
+      "agents": {
+        "ux-ui": 13
+      }
+    }
+  ]
+}

package/observability/evals/learning-loop-report.md

@@ -0,0 +1,37 @@
+# Learning Loop Report
+
+- timestamp: 2026-07-05T09:54:51.179894+00:00
+- total_events: 305
+- pending_events: 197
+- confirmed_events: 108
+
+## Global KPIs
+
+- fallback_rate: 14.75%
+- grounded_rate: 84.92%
+- success_rate: 100.0%
+- confidence_avg: 0.945
+
+## Health
+
+- fallback_ok: True
+- grounded_ok: True
+- confidence_ok: True
+
+## Patterns
+
+- azure-rag+info: total=4, pending=4, confirmed=0, fallback=0.0%, grounded=100.0%, confidence=0.0
+- azure-rag+knowledge-grounded: total=52, pending=38, confirmed=14, fallback=0.0%, grounded=100.0%, confidence=0.946
+- backend+bug-fix: total=12, pending=0, confirmed=12, fallback=0.0%, grounded=100.0%, confidence=0.95
+- dba+analysis: total=51, pending=37, confirmed=14, fallback=0.0%, grounded=100.0%, confidence=0.946
+- dba+sql: total=18, pending=11, confirmed=7, fallback=0.0%, grounded=100.0%, confidence=0.95
+- dev+bug-fix: total=50, pending=47, confirmed=3, fallback=6.0%, grounded=100.0%, confidence=0.917
+- dev+info: total=2, pending=2, confirmed=0, fallback=0.0%, grounded=50.0%, confidence=0.0
+- frontend+bug-fix: total=1, pending=1, confirmed=0, fallback=100.0%, grounded=0.0%, confidence=0.0
+- frontend+feature: total=14, pending=2, confirmed=12, fallback=0.0%, grounded=100.0%, confidence=0.95
+- incremental-json-projection: total=7, pending=0, confirmed=7, fallback=0.0%, grounded=100.0%, confidence=0.91
+- iot+architecture: total=1, pending=1, confirmed=0, fallback=0.0%, grounded=100.0%, confidence=0.0
+- legacy+migration: total=53, pending=39, confirmed=14, fallback=73.58%, grounded=32.08%, confidence=0.946
+- rag+info: total=21, pending=8, confirmed=13, fallback=9.52%, grounded=90.48%, confidence=0.95
+- unknown-domain+info: total=6, pending=6, confirmed=0, fallback=0.0%, grounded=0.0%, confidence=0.0
+- ux-ui+review: total=13, pending=1, confirmed=12, fallback=0.0%, grounded=100.0%, confidence=0.95

package/observability/evals/repo-intake-eval-cases.md

@@ -0,0 +1,23 @@
+# Repo Intake Eval Cases
+
+Casos minimos para validar que el intake registra capacidades y dominios esperados.
+
+## Matriz base
+
+1. `boostDBA` -> `domain=dba`, motor esperado `Graphify`.
+1. `boost_azure_rag` -> `domain=azure-rag`, motor esperado `Azure RAG Builder`.
+1. `boost_iot` -> `domain=iot`, agente esperado `iot`.
+
+## Criterios de aprobacion
+
+1. Repo aparece en `repo-registry/repos.yml` con `approval.status=approved`.
+1. Se genera capability en `repo-intake/generated/v2/<slug>/<version>/capabilities/capability.json`.
+1. No hay errores en `validate-repo-registry.ps1 -Strict`.
+
+## Comandos
+
+```powershell
+pwsh -NoProfile -ExecutionPolicy Bypass -File .\scripts\intake\validate-repo-registry.ps1 -Strict
+.\scripts\intake\run-repo-intake.ps1
+```
+

package/observability/evals/routing-eval-cases.json

@@ -0,0 +1,68 @@
+{
+  "cases": [
+    {
+      "id": "case-login-bug",
+      "input": "Arregla bug de login",
+      "intent": "bug-fix",
+      "domain": "backend",
+      "source_type": "code",
+      "capability": "backend-coding"
+    },
+    {
+      "id": "case-sla-grounding",
+      "input": "Necesito fuentes SLA reales",
+      "intent": "knowledge-grounded",
+      "domain": "azure-rag",
+      "source_type": "corporate-docs",
+      "capability": "azure-rag-enterprise"
+    },
+    {
+      "id": "case-dba-analysis",
+      "input": "Analiza schema SQL",
+      "intent": "analysis",
+      "domain": "dba",
+      "source_type": "technical-docs",
+      "capability": "database-analysis"
+    },
+    {
+      "id": "case-dba-sql-intent",
+      "input": "Analiza esquema SQL de la tabla de pedidos",
+      "intent": "sql",
+      "domain": "dba",
+      "source_type": "technical-docs",
+      "capability": "database-analysis"
+    },
+    {
+      "id": "case-legacy-migration",
+      "input": "Plan de migracion legacy",
+      "intent": "migration",
+      "domain": "legacy",
+      "source_type": "code",
+      "capability": "legacy-migration"
+    },
+    {
+      "id": "case-rag-tech-docs-local",
+      "input": "explica guia tecnica local",
+      "intent": "info",
+      "domain": "rag",
+      "source_type": "technical-docs",
+      "capability": "rag-knowledge"
+    },
+    {
+      "id": "case-frontend-feature",
+      "input": "Implementar nuevo componente de checkout",
+      "intent": "feature",
+      "domain": "frontend",
+      "source_type": "code",
+      "capability": "frontend-coding"
+    },
+    {
+      "id": "case-ux-ui-review",
+      "input": "Revisar consistencia y accesibilidad del design system",
+      "intent": "review",
+      "domain": "ux-ui",
+      "source_type": "technical-docs",
+      "capability": "ux-ui-governance"
+    }
+  ]
+}

package/observability/evals/routing-eval-cases.md

@@ -0,0 +1,25 @@
+# Routing Eval Cases
+
+## Casos base
+
+1. `bug login` -> `backend` + `CodeGraph` + Caveman `full`.
+1. `SLA` -> `rag-azure` + `Azure RAG Builder` + `sources` no vacio.
+1. `arquitectura` -> `rag-local`/`dba` segun dominio con `Graphify`.
+1. `legacy migration` -> `legacy` + `GitNexus` + `hitl.required=true` cuando haya alto impacto/fallback.
+
+## Suite ejecutable JSON-first
+
+- `observability/evals/routing-eval-cases.json`
+
+## Runner
+
+```powershell
+py -3 .\scripts\intake\run-routing-evals.py
+```
+
+## Criterios de pase
+
+1. `cases_failed = 0`.
+1. Coincidencia exacta de `agent` y `engine` por caso.
+1. `prompt.exists = true` en todos los casos.
+

package/observability/evals/routing-eval-report.json

@@ -0,0 +1,72 @@
+{
+  "timestamp": "2026-07-05T09:53:37.247596+00:00",
+  "cases_total": 8,
+  "cases_passed": 8,
+  "cases_failed": 0,
+  "results": [
+    {
+      "id": "case-login-bug",
+      "ok": true,
+      "event_id": "2ffca0b3-6e27-4044-af22-a18213974750",
+      "feedback_recorded": true,
+      "feedback_output": "Feedback recorded for event_id: 2ffca0b3-6e27-4044-af22-a18213974750\nOutput: C:\\repo\\mcp-efficiency-engine\\observability\\logs\\learning-feedback.jsonl\n",
+      "output": "{\n  \"event_id\": \"2ffca0b3-6e27-4044-af22-a18213974750\",\n  \"timestamp\": \"2026-07-05T09:53:28.805628+00:00\",\n  \"input\": \"Arregla bug de login\",\n  \"intent\": \"bug-fix\",\n  \"domain\": \"backend\",\n  \"source_type\": \"code\",\n  \"agent\": \"backend\",\n  \"engine\": \"CodeGraph\",\n  \"optimization_profile\": \"strict+full\",\n  \"fallback\": false,\n  \"grounded\": true,\n  \"sources\": [\n    \"repo:boost_backend\"\n  ],\n  \"notes\": \"selected_by=capability\",\n  \"prompt\": {\n    \"selected\": \".github/prompts/backend.fix-bug.prompt.md\",\n    \"exists\": true,\n    \"selection_mode\": \"auto\"\n  },\n  \"skill\": {\n    \"selected\": \".github/skills/backend-coding/SKILL.md\",\n    \"exists\": true,\n    \"selection_mode\": \"auto\"\n  },\n  \"optimization\": {\n    \"token_saver\": \"always_on\",\n    \"token_saver_profile\": \"strict\",\n    \"caveman\": \"always_on\",\n    \"caveman_profile\": \"full\",\n    \"sources_preserved\": true,\n    \"context_reduction_strategy\": \"symbol\"\n  },\n  \"memory\": {\n    \"selected\": [\n      \"repo-memory\",\n      \"routing-memory\",\n      \"code-memory\"\n    ],\n    \"reason\": \"code route uses structural retrieval\"\n  },\n  \"requirements\": [\n    \"token-saver-mcp\",\n    \"codebase-memory-mcp\",\n    \"codegraph-command\",\n    \"codegraph-index\"\n  ],\n  \"learning\": {\n    \"used_pattern\": \"backend+bug-fix\",\n    \"success\": null,\n    \"outcome_status\": \"pending\",\n    \"fallback\": false,\n    \"confidence\": 0.9\n  },\n  \"hitl\": {\n    \"mode\": \"always_on_auto\",\n    \"required\": false,\n    \"action\": \"none\",\n    \"reason\": \"low_risk_route\"\n  },\n  \"telemetry\": {\n    \"iteration_metrics_recorded\": false,\n    \"metrics_output\": \"\"\n  }\n}\nEvent appended to: C:\\repo\\mcp-efficiency-engine\\observability\\logs\\routing-decisions.jsonl\n"
+    },
+    {
+      "id": "case-sla-grounding",
+      "ok": true,
+      "event_id": "eff45ff1-30e0-4f86-acc1-8c2898e52527",
+      "feedback_recorded": true,
+      "feedback_output": "Feedback recorded for event_id: eff45ff1-30e0-4f86-acc1-8c2898e52527\nOutput: C:\\repo\\mcp-efficiency-engine\\observability\\logs\\learning-feedback.jsonl\n",
+      "output": "{\n  \"event_id\": \"eff45ff1-30e0-4f86-acc1-8c2898e52527\",\n  \"timestamp\": \"2026-07-05T09:53:29.907978+00:00\",\n  \"input\": \"Necesito fuentes SLA reales\",\n  \"intent\": \"knowledge-grounded\",\n  \"domain\": \"azure-rag\",\n  \"source_type\": \"corporate-docs\",\n  \"agent\": \"rag-azure\",\n  \"engine\": \"Azure RAG Builder\",\n  \"optimization_profile\": \"evidence-first+evidence-first\",\n  \"fallback\": false,\n  \"grounded\": true,\n  \"sources\": [\n    \"repo:boost_azure_rag\"\n  ],\n  \"notes\": \"selected_by=capability\",\n  \"prompt\": {\n    \"selected\": \".github/prompts/azure-rag.query.prompt.md\",\n    \"exists\": true,\n    \"selection_mode\": \"auto\"\n  },\n  \"skill\": {\n    \"selected\": \".github/skills/azure-rag-enterprise/SKILL.md\",\n    \"exists\": true,\n    \"selection_mode\": \"auto\"\n  },\n  \"optimization\": {\n    \"token_saver\": \"always_on\",\n    \"token_saver_profile\": \"evidence-first\",\n    \"caveman\": \"always_on\",\n    \"caveman_profile\": \"evidence-first\",\n    \"sources_preserved\": true,\n    \"context_reduction_strategy\": \"chunk\"\n  },\n  \"memory\": {\n    \"selected\": [\n      \"repo-memory\",\n      \"routing-memory\",\n      \"enterprise-memory\"\n    ],\n    \"reason\": \"corporate evidence route\"\n  },\n  \"requirements\": [\n    \"token-saver-mcp\",\n    \"codebase-memory-mcp\",\n    \"azure-rag-connector\"\n  ],\n  \"learning\": {\n    \"used_pattern\": \"azure-rag+knowledge-grounded\",\n    \"success\": null,\n    \"outcome_status\": \"pending\",\n    \"fallback\": false,\n    \"confidence\": 0.9\n  },\n  \"hitl\": {\n    \"mode\": \"always_on_auto\",\n    \"required\": false,\n    \"action\": \"none\",\n    \"reason\": \"low_risk_route\"\n  },\n  \"telemetry\": {\n    \"iteration_metrics_recorded\": false,\n    \"metrics_output\": \"\"\n  }\n}\nEvent appended to: C:\\repo\\mcp-efficiency-engine\\observability\\logs\\routing-decisions.jsonl\n"
+    },
+    {
+      "id": "case-dba-analysis",
+      "ok": true,
+      "event_id": "c2173480-9c3f-4e5e-a40b-17bc3c8effaf",
+      "feedback_recorded": true,
+      "feedback_output": "Feedback recorded for event_id: c2173480-9c3f-4e5e-a40b-17bc3c8effaf\nOutput: C:\\repo\\mcp-efficiency-engine\\observability\\logs\\learning-feedback.jsonl\n",
+      "output": "{\n  \"event_id\": \"c2173480-9c3f-4e5e-a40b-17bc3c8effaf\",\n  \"timestamp\": \"2026-07-05T09:53:30.851086+00:00\",\n  \"input\": \"Analiza schema SQL\",\n  \"intent\": \"analysis\",\n  \"domain\": \"dba\",\n  \"source_type\": \"technical-docs\",\n  \"agent\": \"dba\",\n  \"engine\": \"Graphify\",\n  \"optimization_profile\": \"balanced+lite\",\n  \"fallback\": false,\n  \"grounded\": true,\n  \"sources\": [\n    \"repo:boost_dba\",\n    \"graphify:index\"\n  ],\n  \"notes\": \"selected_by=capability\",\n  \"prompt\": {\n    \"selected\": \".github/prompts/dba.query-review.prompt.md\",\n    \"exists\": true,\n    \"selection_mode\": \"auto\"\n  },\n  \"skill\": {\n    \"selected\": \".github/skills/database-analysis/SKILL.md\",\n    \"exists\": true,\n    \"selection_mode\": \"auto\"\n  },\n  \"optimization\": {\n    \"token_saver\": \"always_on\",\n    \"token_saver_profile\": \"balanced\",\n    \"caveman\": \"always_on\",\n    \"caveman_profile\": \"lite\",\n    \"sources_preserved\": true,\n    \"context_reduction_strategy\": \"manifest\"\n  },\n  \"memory\": {\n    \"selected\": [\n      \"repo-memory\",\n      \"routing-memory\",\n      \"knowledge-memory\"\n    ],\n    \"reason\": \"technical docs route\"\n  },\n  \"requirements\": [\n    \"token-saver-mcp\",\n    \"codebase-memory-mcp\",\n    \"graphify-index\"\n  ],\n  \"learning\": {\n    \"used_pattern\": \"dba+analysis\",\n    \"success\": null,\n    \"outcome_status\": \"pending\",\n    \"fallback\": false,\n    \"confidence\": 0.9\n  },\n  \"hitl\": {\n    \"mode\": \"always_on_auto\",\n    \"required\": false,\n    \"action\": \"none\",\n    \"reason\": \"low_risk_route\"\n  },\n  \"telemetry\": {\n    \"iteration_metrics_recorded\": false,\n    \"metrics_output\": \"\"\n  }\n}\nEvent appended to: C:\\repo\\mcp-efficiency-engine\\observability\\logs\\routing-decisions.jsonl\n"
+    },
+    {
+      "id": "case-dba-sql-intent",
+      "ok": true,
+      "event_id": "7a5907f7-edf3-477f-82a4-4f354648a1ae",
+      "feedback_recorded": true,
+      "feedback_output": "Feedback recorded for event_id: 7a5907f7-edf3-477f-82a4-4f354648a1ae\nOutput: C:\\repo\\mcp-efficiency-engine\\observability\\logs\\learning-feedback.jsonl\n",
+      "output": "{\n  \"event_id\": \"7a5907f7-edf3-477f-82a4-4f354648a1ae\",\n  \"timestamp\": \"2026-07-05T09:53:31.748935+00:00\",\n  \"input\": \"Analiza esquema SQL de la tabla de pedidos\",\n  \"intent\": \"sql\",\n  \"domain\": \"dba\",\n  \"source_type\": \"technical-docs\",\n  \"agent\": \"dba\",\n  \"engine\": \"Graphify\",\n  \"optimization_profile\": \"balanced+lite\",\n  \"fallback\": false,\n  \"grounded\": true,\n  \"sources\": [\n    \"repo:boost_dba\",\n    \"graphify:index\"\n  ],\n  \"notes\": \"selected_by=capability\",\n  \"prompt\": {\n    \"selected\": \".github/prompts/dba.query-review.prompt.md\",\n    \"exists\": true,\n    \"selection_mode\": \"auto\"\n  },\n  \"skill\": {\n    \"selected\": \".github/skills/database-analysis/SKILL.md\",\n    \"exists\": true,\n    \"selection_mode\": \"auto\"\n  },\n  \"optimization\": {\n    \"token_saver\": \"always_on\",\n    \"token_saver_profile\": \"balanced\",\n    \"caveman\": \"always_on\",\n    \"caveman_profile\": \"lite\",\n    \"sources_preserved\": true,\n    \"context_reduction_strategy\": \"manifest\"\n  },\n  \"memory\": {\n    \"selected\": [\n      \"repo-memory\",\n      \"routing-memory\",\n      \"knowledge-memory\"\n    ],\n    \"reason\": \"technical docs route\"\n  },\n  \"requirements\": [\n    \"token-saver-mcp\",\n    \"codebase-memory-mcp\",\n    \"graphify-index\"\n  ],\n  \"learning\": {\n    \"used_pattern\": \"dba+sql\",\n    \"success\": null,\n    \"outcome_status\": \"pending\",\n    \"fallback\": false,\n    \"confidence\": 0.9\n  },\n  \"hitl\": {\n    \"mode\": \"always_on_auto\",\n    \"required\": false,\n    \"action\": \"none\",\n    \"reason\": \"low_risk_route\"\n  },\n  \"telemetry\": {\n    \"iteration_metrics_recorded\": false,\n    \"metrics_output\": \"\"\n  }\n}\nEvent appended to: C:\\repo\\mcp-efficiency-engine\\observability\\logs\\routing-decisions.jsonl\n"
+    },
+    {
+      "id": "case-legacy-migration",
+      "ok": true,
+      "event_id": "d8577530-5805-413f-b4c3-2c244800f5e2",
+      "feedback_recorded": true,
+      "feedback_output": "Feedback recorded for event_id: d8577530-5805-413f-b4c3-2c244800f5e2\nOutput: C:\\repo\\mcp-efficiency-engine\\observability\\logs\\learning-feedback.jsonl\n",
+      "output": "{\n  \"event_id\": \"d8577530-5805-413f-b4c3-2c244800f5e2\",\n  \"timestamp\": \"2026-07-05T09:53:32.678151+00:00\",\n  \"input\": \"Plan de migracion legacy\",\n  \"intent\": \"migration\",\n  \"domain\": \"legacy\",\n  \"source_type\": \"code\",\n  \"agent\": \"legacy\",\n  \"engine\": \"GitNexus\",\n  \"optimization_profile\": \"strict+full\",\n  \"fallback\": false,\n  \"grounded\": true,\n  \"sources\": [\n    \"repo:boost_architect_forensics_expert\",\n    \"gitnexus:index\"\n  ],\n  \"notes\": \"selected_by=capability\",\n  \"prompt\": {\n    \"selected\": \".github/prompts/legacy.impact-analysis.prompt.md\",\n    \"exists\": true,\n    \"selection_mode\": \"auto\"\n  },\n  \"skill\": {\n    \"selected\": \".github/skills/legacy-migration/SKILL.md\",\n    \"exists\": true,\n    \"selection_mode\": \"auto\"\n  },\n  \"optimization\": {\n    \"token_saver\": \"always_on\",\n    \"token_saver_profile\": \"strict\",\n    \"caveman\": \"always_on\",\n    \"caveman_profile\": \"full\",\n    \"sources_preserved\": true,\n    \"context_reduction_strategy\": \"symbol\"\n  },\n  \"memory\": {\n    \"selected\": [\n      \"repo-memory\",\n      \"routing-memory\",\n      \"code-memory\"\n    ],\n    \"reason\": \"code route uses structural retrieval\"\n  },\n  \"requirements\": [\n    \"token-saver-mcp\",\n    \"codebase-memory-mcp\",\n    \"gitnexus-command\"\n  ],\n  \"learning\": {\n    \"used_pattern\": \"legacy+migration\",\n    \"success\": null,\n    \"outcome_status\": \"pending\",\n    \"fallback\": false,\n    \"confidence\": 0.9\n  },\n  \"hitl\": {\n    \"mode\": \"always_on_auto\",\n    \"required\": true,\n    \"action\": \"request_human_confirmation\",\n    \"reason\": \"high_impact:migration\"\n  },\n  \"telemetry\": {\n    \"iteration_metrics_recorded\": false,\n    \"metrics_output\": \"\"\n  }\n}\nEvent appended to: C:\\repo\\mcp-efficiency-engine\\observability\\logs\\routing-decisions.jsonl\n"
+    },
+    {
+      "id": "case-rag-tech-docs-local",
+      "ok": true,
+      "event_id": "3ed21e66-70e2-430d-9b21-2835424db5f2",
+      "feedback_recorded": true,
+      "feedback_output": "Feedback recorded for event_id: 3ed21e66-70e2-430d-9b21-2835424db5f2\nOutput: C:\\repo\\mcp-efficiency-engine\\observability\\logs\\learning-feedback.jsonl\n",
+      "output": "{\n  \"event_id\": \"3ed21e66-70e2-430d-9b21-2835424db5f2\",\n  \"timestamp\": \"2026-07-05T09:53:33.597089+00:00\",\n  \"input\": \"explica guia tecnica local\",\n  \"intent\": \"info\",\n  \"domain\": \"rag\",\n  \"source_type\": \"technical-docs\",\n  \"agent\": \"rag-local\",\n  \"engine\": \"Graphify\",\n  \"optimization_profile\": \"balanced+lite\",\n  \"fallback\": false,\n  \"grounded\": true,\n  \"sources\": [\n    \"graphify:index\"\n  ],\n  \"notes\": \"selected_by=domain_default_expected\",\n  \"prompt\": {\n    \"selected\": \".github/prompts/rag.knowledge-answer.prompt.md\",\n    \"exists\": true,\n    \"selection_mode\": \"auto\"\n  },\n  \"skill\": {\n    \"selected\": \".github/skills/rag-knowledge/SKILL.md\",\n    \"exists\": true,\n    \"selection_mode\": \"auto\"\n  },\n  \"optimization\": {\n    \"token_saver\": \"always_on\",\n    \"token_saver_profile\": \"balanced\",\n    \"caveman\": \"always_on\",\n    \"caveman_profile\": \"lite\",\n    \"sources_preserved\": true,\n    \"context_reduction_strategy\": \"manifest\"\n  },\n  \"memory\": {\n    \"selected\": [\n      \"repo-memory\",\n      \"routing-memory\",\n      \"knowledge-memory\"\n    ],\n    \"reason\": \"technical docs route\"\n  },\n  \"requirements\": [\n    \"token-saver-mcp\",\n    \"codebase-memory-mcp\",\n    \"graphify-index\"\n  ],\n  \"learning\": {\n    \"used_pattern\": \"rag+info\",\n    \"success\": null,\n    \"outcome_status\": \"pending\",\n    \"fallback\": false,\n    \"confidence\": 0.9\n  },\n  \"hitl\": {\n    \"mode\": \"always_on_auto\",\n    \"required\": false,\n    \"action\": \"none\",\n    \"reason\": \"low_risk_route\"\n  },\n  \"telemetry\": {\n    \"iteration_metrics_recorded\": false,\n    \"metrics_output\": \"\"\n  }\n}\nEvent appended to: C:\\repo\\mcp-efficiency-engine\\observability\\logs\\routing-decisions.jsonl\n"
+    },
+    {
+      "id": "case-frontend-feature",
+      "ok": true,
+      "event_id": "57258e0f-0d70-4b10-b5d3-6dfc7354c8f1",
+      "feedback_recorded": true,
+      "feedback_output": "Feedback recorded for event_id: 57258e0f-0d70-4b10-b5d3-6dfc7354c8f1\nOutput: C:\\repo\\mcp-efficiency-engine\\observability\\logs\\learning-feedback.jsonl\n",
+      "output": "{\n  \"event_id\": \"57258e0f-0d70-4b10-b5d3-6dfc7354c8f1\",\n  \"timestamp\": \"2026-07-05T09:53:34.727844+00:00\",\n  \"input\": \"Implementar nuevo componente de checkout\",\n  \"intent\": \"feature\",\n  \"domain\": \"frontend\",\n  \"source_type\": \"code\",\n  \"agent\": \"frontend-agent\",\n  \"engine\": \"CodeGraph\",\n  \"optimization_profile\": \"strict+full\",\n  \"fallback\": false,\n  \"grounded\": true,\n  \"sources\": [\n    \"repo:boost_frontend\"\n  ],\n  \"notes\": \"selected_by=capability\",\n  \"prompt\": {\n    \"selected\": \".github/prompts/frontend.code.prompt.md\",\n    \"exists\": true,\n    \"selection_mode\": \"auto\"\n  },\n  \"skill\": {\n    \"selected\": \".github/skills/frontend-coding/SKILL.md\",\n    \"exists\": true,\n    \"selection_mode\": \"auto\"\n  },\n  \"optimization\": {\n    \"token_saver\": \"always_on\",\n    \"token_saver_profile\": \"strict\",\n    \"caveman\": \"always_on\",\n    \"caveman_profile\": \"full\",\n    \"sources_preserved\": true,\n    \"context_reduction_strategy\": \"symbol\"\n  },\n  \"memory\": {\n    \"selected\": [\n      \"repo-memory\",\n      \"routing-memory\",\n      \"code-memory\"\n    ],\n    \"reason\": \"code route uses structural retrieval\"\n  },\n  \"requirements\": [\n    \"token-saver-mcp\",\n    \"codebase-memory-mcp\",\n    \"codegraph-command\",\n    \"codegraph-index\"\n  ],\n  \"learning\": {\n    \"used_pattern\": \"frontend+feature\",\n    \"success\": null,\n    \"outcome_status\": \"pending\",\n    \"fallback\": false,\n    \"confidence\": 0.9\n  },\n  \"hitl\": {\n    \"mode\": \"always_on_auto\",\n    \"required\": false,\n    \"action\": \"none\",\n    \"reason\": \"low_risk_route\"\n  },\n  \"telemetry\": {\n    \"iteration_metrics_recorded\": false,\n    \"metrics_output\": \"\"\n  }\n}\nEvent appended to: C:\\repo\\mcp-efficiency-engine\\observability\\logs\\routing-decisions.jsonl\n"
+    },
+    {
+      "id": "case-ux-ui-review",
+      "ok": true,
+      "event_id": "a7c73bfe-7d7c-4634-ac0e-c19026f90180",
+      "feedback_recorded": true,
+      "feedback_output": "Feedback recorded for event_id: a7c73bfe-7d7c-4634-ac0e-c19026f90180\nOutput: C:\\repo\\mcp-efficiency-engine\\observability\\logs\\learning-feedback.jsonl\n",
+      "output": "{\n  \"event_id\": \"a7c73bfe-7d7c-4634-ac0e-c19026f90180\",\n  \"timestamp\": \"2026-07-05T09:53:36.434832+00:00\",\n  \"input\": \"Revisar consistencia y accesibilidad del design system\",\n  \"intent\": \"review\",\n  \"domain\": \"ux-ui\",\n  \"source_type\": \"technical-docs\",\n  \"agent\": \"ux-ui\",\n  \"engine\": \"Graphify\",\n  \"optimization_profile\": \"balanced+lite\",\n  \"fallback\": false,\n  \"grounded\": true,\n  \"sources\": [\n    \"repo:boost_design_ux_with_intent\",\n    \"graphify:index\"\n  ],\n  \"notes\": \"selected_by=capability\",\n  \"prompt\": {\n    \"selected\": \".github/prompts/ux-ui.review.prompt.md\",\n    \"exists\": true,\n    \"selection_mode\": \"auto\"\n  },\n  \"skill\": {\n    \"selected\": \".github/skills/ux-ui-governance/SKILL.md\",\n    \"exists\": true,\n    \"selection_mode\": \"auto\"\n  },\n  \"optimization\": {\n    \"token_saver\": \"always_on\",\n    \"token_saver_profile\": \"balanced\",\n    \"caveman\": \"always_on\",\n    \"caveman_profile\": \"lite\",\n    \"sources_preserved\": true,\n    \"context_reduction_strategy\": \"manifest\"\n  },\n  \"memory\": {\n    \"selected\": [\n      \"repo-memory\",\n      \"routing-memory\",\n      \"knowledge-memory\"\n    ],\n    \"reason\": \"technical docs route\"\n  },\n  \"requirements\": [\n    \"token-saver-mcp\",\n    \"codebase-memory-mcp\",\n    \"graphify-index\"\n  ],\n  \"learning\": {\n    \"used_pattern\": \"ux-ui+review\",\n    \"success\": null,\n    \"outcome_status\": \"pending\",\n    \"fallback\": false,\n    \"confidence\": 0.9\n  },\n  \"hitl\": {\n    \"mode\": \"always_on_auto\",\n    \"required\": false,\n    \"action\": \"none\",\n    \"reason\": \"low_risk_route\"\n  },\n  \"telemetry\": {\n    \"iteration_metrics_recorded\": false,\n    \"metrics_output\": \"\"\n  }\n}\nEvent appended to: C:\\repo\\mcp-efficiency-engine\\observability\\logs\\routing-decisions.jsonl\n"
+    }
+  ]
+}

package/observability/evaluation.md

@@ -0,0 +1,34 @@
+# Evaluation
+
+Evalua inputs esperados vs agente/motor real y perfil de optimizacion aplicado.
+
+## Objetivo
+
+- Validar que routing, grounding y optimizacion cumplen contrato.
+- Detectar regresiones antes de impactar operacion.
+
+## Fuente de verdad
+
+- Casos: `observability/evals/routing-eval-cases.json`
+- Runner: `scripts/intake/run-routing-evals.py`
+- Reporte: `observability/evals/routing-eval-report.json`
+- Eventos: `observability/logs/routing-decisions.jsonl`
+
+## Criterios de aprobacion
+
+1. `cases_failed = 0`.
+1. `prompt.exists = true` en todos los casos.
+1. `agent` y `engine` esperados por caso.
+1. `hitl.required = true` en rutas de alto impacto.
+
+## Semaforo
+
+- `OK`: `cases_failed = 0`.
+- `WARN`: 1 caso fallado o desvio leve de perfil.
+- `FAIL`: 2+ casos fallados o HITL incorrecto en alto riesgo.
+
+## Ejecucion
+
+```powershell
+py -3 .\scripts\intake\run-routing-evals.py
+```

package/observability/logs/copilot-usage-ingest-report.json

@@ -0,0 +1,13 @@
+{
+  "timestamp": "2026-07-05T09:54:53.104912+00:00",
+  "session_log": "C:\\Users\\sergio.hierro.cinos\\AppData\\Roaming\\Code\\User\\workspaceStorage\\52c8c5ec726bf4047d40cb00b7487a0b\\GitHub.copilot-chat\\debug-logs\\013fa51d-1313-48db-b022-d8df5b6f81cd\\main.jsonl",
+  "chat_session_log": "C:\\Users\\sergio.hierro.cinos\\AppData\\Roaming\\Code\\User\\workspaceStorage\\52c8c5ec726bf4047d40cb00b7487a0b\\chatSessions\\013fa51d-1313-48db-b022-d8df5b6f81cd.jsonl",
+  "metrics_log": "observability/logs/iteration-metrics.jsonl",
+  "scanned_events": 9,
+  "usage_events_detected": 8,
+  "inserted": 8,
+  "updated_existing": 0,
+  "skipped_duplicates": 0,
+  "status": "ok",
+  "note": "If status=no-usage-fields-found, this VS Code debug log currently does not expose per-turn token usage in a parseable shape."
+}