mcp-efficiency-engine 0.1.1 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,20 @@
1
+ #!/usr/bin/env sh
2
+ set -eu
3
+
4
+ REPO_ROOT=$(git rev-parse --show-toplevel 2>/dev/null || true)
5
+ if [ -z "$REPO_ROOT" ]; then
6
+ exit 0
7
+ fi
8
+
9
+ if command -v pwsh >/dev/null 2>&1; then
10
+ pwsh -NoProfile -ExecutionPolicy Bypass -File "$REPO_ROOT/scripts/ops/post-commit-refresh.ps1"
11
+ exit 0
12
+ fi
13
+
14
+ if command -v powershell >/dev/null 2>&1; then
15
+ powershell -NoProfile -ExecutionPolicy Bypass -File "$REPO_ROOT/scripts/ops/post-commit-refresh.ps1"
16
+ exit 0
17
+ fi
18
+
19
+ # No PowerShell available; skip silently to avoid blocking commits.
20
+ exit 0
@@ -136,6 +136,28 @@ Archivo `telemetry/config.json`:
136
136
 
137
137
  Si falta configuración, se omite automáticamente.
138
138
 
139
+ ## Flujo post-commit para proyectos consumidores
140
+
141
+ El install host configura hooks git locales (`core.hooksPath=.githooks`).
142
+
143
+ Hook incluido:
144
+
145
+ - `post-commit` -> `scripts/ops/post-commit-refresh.ps1`
146
+
147
+ Regla:
148
+
149
+ - Solo actua cuando el commit toca rutas bajo `projects/`.
150
+
151
+ Pipeline ejecutado en ese caso:
152
+
153
+ 1. AutoDocs incremental (`scripts/wiki/compiler_main.py`)
154
+ 2. Refresh de reportes de learning/value (`scripts/learning/*.py`)
155
+ 3. Publicacion de snapshots KPI a LangSmith (`scripts/ops/publish-langsmith-kpis.py`)
156
+
157
+ Log de ejecucion:
158
+
159
+ - `observability/logs/session/post-commit-refresh-*.json`
160
+
139
161
  ### Cómo deshabilitar exporters
140
162
 
141
163
  - Desactivar todo: `TELEMETRY_ENABLED=false`
package/README.md CHANGED
@@ -192,6 +192,16 @@ Comportamiento esperado:
192
192
  - si no existe `repo-registry/repos.yml`, pregunta por owner/prefix y si quieres registrar un repo inicial para intake
193
193
  - si no añades repos en ese momento, deja el registry plantilla listo para añadirlos despues y rerun de intake
194
194
 
195
+ Nota npm (entornos con politicas de scripts):
196
+
197
+ - si `npm` bloquea `install`/`postinstall` (por ejemplo con `allow-scripts`), el scaffold/bootstrapping puede no ejecutarse automaticamente
198
+ - en ese caso, aprueba scripts (`npm approve-scripts`) o ejecuta manualmente:
199
+
200
+ ```powershell
201
+ npx mcp-efficiency-engine install
202
+ npx mcp-efficiency-engine validate -PortableMode
203
+ ```
204
+
195
205
  Tambien puedes relanzar la instalacion manualmente sobre el proyecto actual:
196
206
 
197
207
  ```powershell
@@ -229,6 +239,32 @@ py -3 .\scripts\intake\agent-pipeline-preflight.py
229
239
  py -3 .\scripts\intake\validate-repo-registry.py --strict
230
240
  ```
231
241
 
242
+ ### Flujo automatico al hacer commit en projects/
243
+
244
+ Cuando instalas el engine en un proyecto host (`mcpee install`), se configura `core.hooksPath=.githooks` con un `post-commit` que ejecuta `scripts/ops/post-commit-refresh.ps1`.
245
+
246
+ Comportamiento del hook:
247
+
248
+ - si el ultimo commit no toca `projects/`, no hace nada
249
+ - si detecta cambios en `projects/`, ejecuta:
250
+ - `scripts/wiki/compiler_main.py` (AutoDocs incremental)
251
+ - `scripts/learning/learning-loop-report.py`
252
+ - `scripts/learning/iteration-value-report.py`
253
+ - `scripts/ops/publish-langsmith-kpis.py` (best effort)
254
+
255
+ Artefactos/resultados:
256
+
257
+ - AutoDocs actualizado en `autodocs/generated` y `autodocs/site`
258
+ - reportes de observabilidad actualizados en `observability/evals`
259
+ - snapshots KPI publicados a LangSmith para dashboards
260
+ - resumen local en `observability/logs/session/post-commit-refresh-*.json`
261
+
262
+ Instalacion manual de hooks (si necesitas reprovisionar):
263
+
264
+ ```powershell
265
+ .\scripts\setup\install-project-hooks.ps1
266
+ ```
267
+
232
268
  ## Flujo De Intake
233
269
 
234
270
  `repo-intake` soporta dos modos:
@@ -344,6 +380,68 @@ $env:LANGSMITH_ENABLED='false'
344
380
  $env:TELEMETRY_EXPORTERS='console,json'
345
381
  ```
346
382
 
383
+ Troubleshooting rapido: no aparecen dashboards/runs en LangSmith
384
+
385
+ Importante: en LangSmith, los runs se validan primero en `Tracing` (y opcionalmente `Monitoring`). La seccion `Custom Dashboards` no se autogenera por defecto; puede aparecer vacia aunque la telemetria este funcionando correctamente.
386
+
387
+ Checklist minimo (debe cumplirse todo):
388
+
389
+ - `LANGSMITH_ENABLED=true`
390
+ - `LANGSMITH_API_KEY` definido
391
+ - `LANGSMITH_PROJECT` definido
392
+ - `TELEMETRY_EXPORTERS=console,json,langsmith`
393
+ - el flujo que ejecutas realmente emite telemetria (por ejemplo `hi.ps1`, intake o routing-evals)
394
+
395
+ Comprobacion en PowerShell:
396
+
397
+ ```powershell
398
+ Write-Host "LANGSMITH_ENABLED=$env:LANGSMITH_ENABLED"
399
+ Write-Host "LANGSMITH_PROJECT=$env:LANGSMITH_PROJECT"
400
+ Write-Host "TELEMETRY_EXPORTERS=$env:TELEMETRY_EXPORTERS"
401
+ ```
402
+
403
+ Si `LANGSMITH_WORKSPACE_ID` no coincide con tu workspace real, los runs pueden quedar en otro workspace y "no verse" en la UI esperada.
404
+
405
+ Verificacion local (aunque LangSmith falle):
406
+
407
+ - revisa que se siguen generando logs en `observability/logs/` (la app no debe romperse por un fallo del exporter)
408
+ - si hay trazas locales pero no runs remotos, el problema es de configuracion/conectividad de LangSmith y no del flujo principal
409
+
410
+ ### Alinear KPIs locales con LangSmith
411
+
412
+ Para enviar a LangSmith los KPIs que ya calcula el engine en local (`learning-loop-report.json` e `iteration-value-report.json`) y poder construir dashboards con esa señal:
413
+
414
+ ```powershell
415
+ npm run langsmith:kpis
416
+ ```
417
+
418
+ Este comando publica runs de resumen con tags `mcpee`, `kpi`, `dashboard` y nombres:
419
+
420
+ - `KPI::LearningLoop`
421
+ - `KPI::IterationValue`
422
+ - `KPI::AlignmentSnapshot`
423
+
424
+ Con eso puedes filtrar en `Tracing` por `tag:kpi` y montar dashboards manuales en `Custom Dashboards` sobre esos runs.
425
+
426
+ Separacion plataforma vs proyecto consumidor:
427
+
428
+ - los runs KPI incluyen metadata y tags de scope automaticamente
429
+ - metadata: `host_project`, `host_project_slug`, `telemetry_scope` (`platform` o `consumer`)
430
+ - tags: `scope:<valor>` y `host:<slug>`
431
+
432
+ Ejemplos de filtro para un dashboard de proyecto consumidor:
433
+
434
+ - `Tag contains kpi`
435
+ - `Tag contains scope:consumer`
436
+ - `Tag contains host:<slug-del-proyecto>`
437
+
438
+ Si quieres forzar nombre de proyecto host (por ejemplo en CI):
439
+
440
+ ```powershell
441
+ $env:MCPEE_HOST_PROJECT='mi-proyecto-app'
442
+ npm run langsmith:kpis
443
+ ```
444
+
347
445
  Variables de entorno soportadas:
348
446
 
349
447
  - `TELEMETRY_ENABLED`
@@ -6,6 +6,7 @@ const { spawnSync } = require("node:child_process");
6
6
 
7
7
  const packageRoot = path.resolve(__dirname, "..");
8
8
  const scaffoldEntries = [
9
+ ".githooks",
9
10
  ".github",
10
11
  ".vscode",
11
12
  "autodocs/README.md",
@@ -260,6 +261,15 @@ function runBootstrap(targetRoot) {
260
261
  return runPowerShell(bootstrapScript, [], targetRoot);
261
262
  }
262
263
 
264
+ function installProjectHooks(targetRoot) {
265
+ const hookScript = path.join(targetRoot, "scripts", "setup", "install-project-hooks.ps1");
266
+ if (!fs.existsSync(hookScript)) {
267
+ return 0;
268
+ }
269
+
270
+ return runPowerShell(hookScript, [], targetRoot);
271
+ }
272
+
263
273
  function runHostInstall(rawOptions) {
264
274
  const options = rawOptions;
265
275
  const targetRoot = normalizePath(options.targetDir);
@@ -284,6 +294,11 @@ function runHostInstall(rawOptions) {
284
294
  return initStatus;
285
295
  }
286
296
 
297
+ const hookStatus = installProjectHooks(targetRoot);
298
+ if (hookStatus !== 0) {
299
+ return hookStatus;
300
+ }
301
+
287
302
  process.stdout.write("[mcpee] Bootstrap omitido. Puedes ejecutar .\\scripts\\bootstrap-portable.cmd mas tarde.\n");
288
303
  return 0;
289
304
  }
@@ -294,11 +309,21 @@ function runHostInstall(rawOptions) {
294
309
  return initStatus;
295
310
  }
296
311
 
312
+ const hookStatus = installProjectHooks(targetRoot);
313
+ if (hookStatus !== 0) {
314
+ return hookStatus;
315
+ }
316
+
297
317
  process.stdout.write("[mcpee] Modo no interactivo detectado. Se inicializo el registry plantilla y se omitio bootstrap interactivo.\n");
298
318
  return 0;
299
319
  }
300
320
 
301
- return runBootstrap(targetRoot);
321
+ const bootstrapStatus = runBootstrap(targetRoot);
322
+ if (bootstrapStatus !== 0) {
323
+ return bootstrapStatus;
324
+ }
325
+
326
+ return installProjectHooks(targetRoot);
302
327
  }
303
328
 
304
329
  function runHostInstallFromCli(argv = process.argv.slice(2)) {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "mcp-efficiency-engine",
3
- "version": "0.1.1",
3
+ "version": "0.1.2",
4
4
  "description": "Motor de orquestacion para agentes MCP con routing por dominio y bootstrap portable.",
5
5
  "license": "MIT",
6
6
  "type": "commonjs",
@@ -9,6 +9,7 @@
9
9
  "mcp-efficiency-engine": "./bin/mcpee.js"
10
10
  },
11
11
  "files": [
12
+ ".githooks/",
12
13
  "bin/",
13
14
  "telemetry/",
14
15
  "autodocs/README.md",
@@ -50,6 +51,7 @@
50
51
  "hi": "node ./bin/mcpee.js hi",
51
52
  "bye": "node ./bin/mcpee.js bye",
52
53
  "intake": "node ./bin/mcpee.js intake",
54
+ "langsmith:kpis": "py -3 ./scripts/ops/publish-langsmith-kpis.py",
53
55
  "pack:check": "npm pack --dry-run"
54
56
  },
55
57
  "engines": {
package/requirements.txt CHANGED
@@ -1,2 +1,3 @@
1
1
  graphifyy[mcp]
2
2
  langsmith
3
+ mcp
@@ -73,6 +73,26 @@ function Test-CommandAvailable {
73
73
  return [bool](Get-Command $Name -ErrorAction SilentlyContinue)
74
74
  }
75
75
 
76
+ function Ensure-SetupPrerequisites {
77
+ if ($script:SetupAttempted) {
78
+ if (-not $script:SetupSucceeded) {
79
+ throw 'setup-prerequisites was already attempted and failed earlier in this run'
80
+ }
81
+ return $true
82
+ }
83
+
84
+ $script:SetupAttempted = $true
85
+ Write-Host '[info] Missing prerequisite detected. Running setup-prerequisites automatically...' -ForegroundColor DarkYellow
86
+ & pwsh -NoProfile -ExecutionPolicy Bypass -File .\scripts\setup\setup-prerequisites.ps1 -PortableMode
87
+ if ($LASTEXITCODE -ne 0) {
88
+ $script:SetupSucceeded = $false
89
+ throw "setup-prerequisites failed with exit code $LASTEXITCODE"
90
+ }
91
+
92
+ $script:SetupSucceeded = $true
93
+ return $true
94
+ }
95
+
76
96
  function Invoke-ObservabilityRetentionCleanup {
77
97
  param(
78
98
  [string]$RepoRoot,
@@ -320,6 +340,8 @@ Set-Location $repoRoot
320
340
 
321
341
  $script:Steps = @()
322
342
  $script:HasFailures = $false
343
+ $script:SetupAttempted = $false
344
+ $script:SetupSucceeded = $false
323
345
  $script:StepLogs = [ordered]@{}
324
346
  $script:RetentionCleanupReport = [ordered]@{
325
347
  applied = $false
@@ -367,6 +389,10 @@ else {
367
389
 
368
390
  if (-not $SkipGraphRefresh) {
369
391
  Invoke-Step -Name 'Refresh codegraph index sync' -Action {
392
+ if (-not (Get-Command codegraph -ErrorAction SilentlyContinue)) {
393
+ [void](Ensure-SetupPrerequisites)
394
+ }
395
+
370
396
  if (-not (Get-Command codegraph -ErrorAction SilentlyContinue)) {
371
397
  throw 'codegraph command not found'
372
398
  }
@@ -548,6 +574,10 @@ else {
548
574
 
549
575
  if (-not $SkipCodegraphStatus) {
550
576
  Invoke-Step -Name 'Check codegraph status' -Action {
577
+ if (-not (Get-Command codegraph -ErrorAction SilentlyContinue)) {
578
+ [void](Ensure-SetupPrerequisites)
579
+ }
580
+
551
581
  if (-not (Get-Command codegraph -ErrorAction SilentlyContinue)) {
552
582
  throw 'codegraph command not found'
553
583
  }
@@ -835,6 +835,19 @@ else {
835
835
 
836
836
  if (-not $SkipMcpStartupChecks) {
837
837
  Invoke-Step -Name 'Validate MCP servers start/listen capability' -Action {
838
+ $requiredCommands = @(
839
+ 'token-saver-mcp',
840
+ 'codebase-memory-mcp',
841
+ 'codegraph',
842
+ 'npx'
843
+ )
844
+
845
+ $missingCommands = @($requiredCommands | Where-Object { -not (Test-CommandAvailable -Name $_) })
846
+ if ($missingCommands.Count -gt 0) {
847
+ Write-Host ("[info] Missing MCP prerequisites detected ({0}). Running setup-prerequisites automatically..." -f ($missingCommands -join ', ')) -ForegroundColor DarkYellow
848
+ [void](Ensure-SetupPrerequisites)
849
+ }
850
+
838
851
  $py = Get-PythonCommand
839
852
  $pyCmd = $py[0]
840
853
  $pyParts = @()
@@ -0,0 +1,110 @@
1
+ param(
2
+ [switch]$Force,
3
+ [switch]$SkipLangSmith,
4
+ [switch]$SkipLearningReports,
5
+ [switch]$SkipAutoDocs
6
+ )
7
+
8
+ Set-StrictMode -Version Latest
9
+ $ErrorActionPreference = 'Stop'
10
+
11
+ function Get-PythonCommand {
12
+ if (Get-Command py -ErrorAction SilentlyContinue) {
13
+ return @('py', '-3')
14
+ }
15
+ if (Get-Command python -ErrorAction SilentlyContinue) {
16
+ return @('python')
17
+ }
18
+ throw 'Python is required (py or python not found in PATH).'
19
+ }
20
+
21
+ function Invoke-PythonScript {
22
+ param(
23
+ [string]$ScriptPath,
24
+ [string[]]$Args = @(),
25
+ [switch]$IgnoreErrors
26
+ )
27
+
28
+ $python = Get-PythonCommand
29
+ & $python[0] $python[1] $ScriptPath @Args
30
+ $exitCode = $LASTEXITCODE
31
+ if ($exitCode -ne 0 -and -not $IgnoreErrors) {
32
+ throw "Command failed ($exitCode): $ScriptPath"
33
+ }
34
+ return $exitCode
35
+ }
36
+
37
+ $repoRoot = (& git rev-parse --show-toplevel 2>$null)
38
+ if (-not $repoRoot) {
39
+ Write-Host '[mcpee] post-commit-refresh skipped: git repo not detected.'
40
+ exit 0
41
+ }
42
+
43
+ Set-Location $repoRoot
44
+
45
+ $changedFiles = (& git diff-tree --no-commit-id --name-only -r HEAD 2>$null)
46
+ if ($LASTEXITCODE -ne 0) {
47
+ Write-Host '[mcpee] post-commit-refresh skipped: unable to inspect last commit.'
48
+ exit 0
49
+ }
50
+
51
+ $projectChanges = @($changedFiles | Where-Object { $_ -match '^projects/' })
52
+ if (-not $Force -and $projectChanges.Count -eq 0) {
53
+ Write-Host '[mcpee] post-commit-refresh skipped: no changes under projects/ in last commit.'
54
+ exit 0
55
+ }
56
+
57
+ $started = Get-Date
58
+ $status = 'ok'
59
+ $errors = @()
60
+ $steps = @()
61
+
62
+ try {
63
+ if (-not $SkipAutoDocs) {
64
+ Write-Host '[mcpee] AutoDocs: compiling incremental projection...'
65
+ Invoke-PythonScript -ScriptPath '.\scripts\wiki\compiler_main.py'
66
+ $steps += 'autodocs'
67
+ }
68
+
69
+ if (-not $SkipLearningReports) {
70
+ Write-Host '[mcpee] Observability: refreshing learning reports...'
71
+ Invoke-PythonScript -ScriptPath '.\scripts\learning\learning-loop-report.py'
72
+ Invoke-PythonScript -ScriptPath '.\scripts\learning\iteration-value-report.py'
73
+ $steps += 'learning-reports'
74
+ }
75
+
76
+ if (-not $SkipLangSmith) {
77
+ Write-Host '[mcpee] LangSmith: publishing KPI snapshots...'
78
+ Invoke-PythonScript -ScriptPath '.\scripts\ops\publish-langsmith-kpis.py' -IgnoreErrors
79
+ $steps += 'langsmith-kpis'
80
+ }
81
+ }
82
+ catch {
83
+ $status = 'failed'
84
+ $errors += $_.Exception.Message
85
+ Write-Warning ("[mcpee] post-commit-refresh failed: {0}" -f $_.Exception.Message)
86
+ }
87
+
88
+ $finished = Get-Date
89
+ $duration = [math]::Round((($finished - $started).TotalSeconds), 2)
90
+
91
+ $summary = [ordered]@{
92
+ timestamp = $finished.ToString('o')
93
+ operation = 'post-commit-refresh'
94
+ status = $status
95
+ duration_sec = $duration
96
+ changed_projects_files = $projectChanges.Count
97
+ steps = $steps
98
+ errors = $errors
99
+ }
100
+
101
+ $sessionDir = Join-Path $repoRoot 'observability/logs/session'
102
+ New-Item -ItemType Directory -Path $sessionDir -Force | Out-Null
103
+ $stamp = Get-Date -Format 'yyyyMMdd-HHmmss'
104
+ $reportPath = Join-Path $sessionDir ("post-commit-refresh-$stamp.json")
105
+ $summary | ConvertTo-Json -Depth 8 | Set-Content -Path $reportPath -Encoding UTF8
106
+
107
+ Write-Host ("[mcpee] post-commit-refresh {0}. report: {1}" -f $status, $reportPath)
108
+
109
+ # Never block commit history after it was created.
110
+ exit 0
@@ -0,0 +1,302 @@
1
+ from __future__ import annotations
2
+
3
+ import json
4
+ import os
5
+ import sys
6
+ import uuid
7
+ from datetime import datetime, timezone
8
+ from pathlib import Path
9
+ from typing import Any
10
+
11
+ REPO_ROOT = Path(__file__).resolve().parents[2]
12
+ if str(REPO_ROOT) not in sys.path:
13
+ sys.path.insert(0, str(REPO_ROOT))
14
+
15
+ from telemetry.config import load_config
16
+
17
+
18
+ def _read_json(path: Path) -> dict[str, Any]:
19
+ if not path.exists():
20
+ return {}
21
+ try:
22
+ payload = json.loads(path.read_text(encoding="utf-8"))
23
+ except Exception:
24
+ return {}
25
+ return payload if isinstance(payload, dict) else {}
26
+
27
+
28
+ def _iso_now() -> datetime:
29
+ return datetime.now(timezone.utc)
30
+
31
+
32
+ def _slug(value: str) -> str:
33
+ return "".join(ch.lower() if ch.isalnum() else "-" for ch in value).strip("-")
34
+
35
+
36
+ def _resolve_host_scope(repo_root: Path) -> tuple[str, str]:
37
+ host_project = (os.getenv("MCPEE_HOST_PROJECT") or repo_root.name or "unknown-project").strip()
38
+ host_slug = _slug(host_project) or "unknown-project"
39
+ platform_slugs = {"mcp-efficiency-engine", "efficiency", "mcpee"}
40
+ scope = "platform" if host_slug in platform_slugs else "consumer"
41
+ return host_project, scope
42
+
43
+
44
+ def _flatten_kpis(payload: dict[str, Any]) -> dict[str, Any]:
45
+ flat: dict[str, Any] = {}
46
+ for key, value in payload.items():
47
+ flat[f"kpi_{key}"] = value
48
+ return flat
49
+
50
+
51
+ def _publish_run(
52
+ client: Any,
53
+ *,
54
+ project: str,
55
+ name: str,
56
+ inputs: dict[str, Any],
57
+ outputs: dict[str, Any],
58
+ tags: list[str],
59
+ metadata: dict[str, Any] | None = None,
60
+ total_cost: float | None = None,
61
+ total_tokens: int | None = None,
62
+ run_type: str = "chain",
63
+ ) -> str:
64
+ now = _iso_now()
65
+ run_id = str(uuid.uuid4())
66
+ payload: dict[str, Any] = {
67
+ "id": run_id,
68
+ "project_name": project,
69
+ "name": name,
70
+ "run_type": run_type,
71
+ "inputs": inputs,
72
+ "outputs": outputs,
73
+ "start_time": now,
74
+ "end_time": now,
75
+ "tags": tags,
76
+ "extra": {"metadata": metadata or {}},
77
+ }
78
+
79
+ if total_cost is not None:
80
+ payload["total_cost"] = float(total_cost)
81
+ payload["prompt_cost"] = 0.0
82
+ payload["completion_cost"] = float(total_cost)
83
+ if total_tokens is not None:
84
+ payload["total_tokens"] = int(total_tokens)
85
+ # For LLM-compatible dashboard metrics, expose token fields explicitly.
86
+ payload["prompt_tokens"] = 0
87
+ payload["completion_tokens"] = int(total_tokens)
88
+
89
+ usage_metadata: dict[str, Any] = {
90
+ "input_tokens": int(payload.get("prompt_tokens", 0) or 0),
91
+ "output_tokens": int(payload.get("completion_tokens", 0) or 0),
92
+ "total_tokens": int(payload.get("total_tokens", 0) or 0),
93
+ "total_cost": float(payload.get("total_cost", 0.0) or 0.0),
94
+ }
95
+
96
+ outputs_kpi = payload.setdefault("outputs", {})
97
+ outputs_kpi["usage_metadata"] = usage_metadata
98
+
99
+ meta = payload.setdefault("extra", {}).setdefault("metadata", {})
100
+ meta["usage_metadata"] = usage_metadata
101
+
102
+ client.create_run(**payload)
103
+ return run_id
104
+
105
+
106
+ def _publish_feedback_metrics(client: Any, *, run_id: str, metrics: dict[str, Any], source: str) -> None:
107
+ for key, raw_value in metrics.items():
108
+ try:
109
+ value = float(raw_value)
110
+ except Exception:
111
+ continue
112
+
113
+ try:
114
+ client.create_feedback(
115
+ run_id=run_id,
116
+ key=str(key),
117
+ score=value,
118
+ comment=f"mcpee kpi:{source}",
119
+ )
120
+ except Exception:
121
+ # Feedback is optional. Keep publishing resilient.
122
+ continue
123
+
124
+
125
+ def main() -> int:
126
+ repo_root = REPO_ROOT
127
+ cfg = load_config(repo_root)
128
+ host_project, telemetry_scope = _resolve_host_scope(repo_root)
129
+ host_slug = _slug(host_project)
130
+
131
+ if not cfg.langsmith.enabled:
132
+ print("LangSmith disabled in effective config. Nothing to publish.")
133
+ return 0
134
+
135
+ if not cfg.langsmith.api_key.strip() or not cfg.langsmith.project.strip():
136
+ print("LangSmith config missing api_key or project. Nothing to publish.")
137
+ return 0
138
+
139
+ try:
140
+ from langsmith import Client # type: ignore
141
+ except Exception as exc:
142
+ print(f"langsmith SDK is not available: {exc}")
143
+ return 1
144
+
145
+ kwargs: dict[str, Any] = {"api_key": cfg.langsmith.api_key}
146
+ if cfg.langsmith.endpoint:
147
+ kwargs["api_url"] = cfg.langsmith.endpoint
148
+ if cfg.langsmith.workspace_id:
149
+ kwargs["workspace_id"] = cfg.langsmith.workspace_id
150
+
151
+ client = Client(**kwargs)
152
+
153
+ learning_path = repo_root / "observability" / "evals" / "learning-loop-report.json"
154
+ value_path = repo_root / "observability" / "evals" / "iteration-value-report.json"
155
+
156
+ learning = _read_json(learning_path)
157
+ value = _read_json(value_path)
158
+
159
+ published = 0
160
+
161
+ if learning:
162
+ learning_kpis = learning.get("kpis", {}) if isinstance(learning.get("kpis", {}), dict) else {}
163
+ learning_flat_kpis = _flatten_kpis(learning_kpis)
164
+ run_id = _publish_run(
165
+ client,
166
+ project=cfg.langsmith.project,
167
+ name="KPI::LearningLoop",
168
+ inputs={
169
+ "source_file": str(learning_path),
170
+ "timestamp": learning.get("timestamp"),
171
+ },
172
+ outputs={
173
+ "kpis": learning_kpis,
174
+ **learning_flat_kpis,
175
+ "health": learning.get("health", {}),
176
+ "totals": {
177
+ "total_events": learning.get("total_events"),
178
+ "confirmed_events": learning.get("confirmed_events"),
179
+ "pending_events": learning.get("pending_events"),
180
+ },
181
+ },
182
+ tags=[
183
+ "mcpee",
184
+ "kpi",
185
+ "learning-loop",
186
+ "dashboard",
187
+ f"scope:{telemetry_scope}",
188
+ f"host:{host_slug}",
189
+ ],
190
+ metadata={
191
+ "kpi_source": "learning-loop",
192
+ "kpi_kind": "snapshot",
193
+ "host_project": host_project,
194
+ "host_project_slug": host_slug,
195
+ "telemetry_scope": telemetry_scope,
196
+ **learning_flat_kpis,
197
+ },
198
+ )
199
+ _publish_feedback_metrics(client, run_id=run_id, metrics=learning_flat_kpis, source="learning-loop")
200
+ published += 1
201
+
202
+ if value:
203
+ value_kpis = value.get("kpis", {}) if isinstance(value.get("kpis", {}), dict) else {}
204
+ value_flat_kpis = _flatten_kpis(value_kpis)
205
+ run_id = _publish_run(
206
+ client,
207
+ project=cfg.langsmith.project,
208
+ name="KPI::IterationValue",
209
+ inputs={
210
+ "source_file": str(value_path),
211
+ "timestamp": value.get("timestamp"),
212
+ },
213
+ outputs={
214
+ "kpis": value_kpis,
215
+ **value_flat_kpis,
216
+ "totals": value.get("totals", {}),
217
+ "assessment": value.get("assessment", {}),
218
+ },
219
+ tags=[
220
+ "mcpee",
221
+ "kpi",
222
+ "iteration-value",
223
+ "dashboard",
224
+ f"scope:{telemetry_scope}",
225
+ f"host:{host_slug}",
226
+ ],
227
+ metadata={
228
+ "kpi_source": "iteration-value",
229
+ "kpi_kind": "snapshot",
230
+ "host_project": host_project,
231
+ "host_project_slug": host_slug,
232
+ "telemetry_scope": telemetry_scope,
233
+ **value_flat_kpis,
234
+ },
235
+ total_cost=float(value_kpis.get("total_cost_usd", 0.0) or 0.0),
236
+ total_tokens=int(value_kpis.get("total_tokens", 0) or 0),
237
+ run_type="llm",
238
+ )
239
+ _publish_feedback_metrics(client, run_id=run_id, metrics=value_flat_kpis, source="iteration-value")
240
+ published += 1
241
+
242
+ if learning or value:
243
+ learning_kpis = learning.get("kpis", {}) if isinstance(learning.get("kpis", {}), dict) else {}
244
+ value_kpis = value.get("kpis", {}) if isinstance(value.get("kpis", {}), dict) else {}
245
+ learning_flat_kpis = _flatten_kpis(learning_kpis)
246
+ value_flat_kpis = _flatten_kpis(value_kpis)
247
+ run_id = _publish_run(
248
+ client,
249
+ project=cfg.langsmith.project,
250
+ name="KPI::AlignmentSnapshot",
251
+ inputs={
252
+ "source": "local-observability-evals",
253
+ "learning_present": bool(learning),
254
+ "value_present": bool(value),
255
+ },
256
+ outputs={
257
+ "learning_kpis": learning_kpis,
258
+ "value_kpis": value_kpis,
259
+ **{f"learning_{k}": v for k, v in learning_flat_kpis.items()},
260
+ **{f"value_{k}": v for k, v in value_flat_kpis.items()},
261
+ "published_at": _iso_now().isoformat(),
262
+ },
263
+ tags=[
264
+ "mcpee",
265
+ "kpi",
266
+ "alignment",
267
+ "dashboard",
268
+ f"scope:{telemetry_scope}",
269
+ f"host:{host_slug}",
270
+ ],
271
+ metadata={
272
+ "kpi_source": "alignment",
273
+ "kpi_kind": "snapshot",
274
+ "host_project": host_project,
275
+ "host_project_slug": host_slug,
276
+ "telemetry_scope": telemetry_scope,
277
+ **{f"learning_{k}": v for k, v in learning_flat_kpis.items()},
278
+ **{f"value_{k}": v for k, v in value_flat_kpis.items()},
279
+ },
280
+ total_cost=float(value_kpis.get("total_cost_usd", 0.0) or 0.0),
281
+ total_tokens=int(value_kpis.get("total_tokens", 0) or 0),
282
+ run_type="llm",
283
+ )
284
+ alignment_feedback = {
285
+ **{f"learning_{k}": v for k, v in learning_flat_kpis.items()},
286
+ **{f"value_{k}": v for k, v in value_flat_kpis.items()},
287
+ }
288
+ _publish_feedback_metrics(client, run_id=run_id, metrics=alignment_feedback, source="alignment")
289
+ published += 1
290
+
291
+ print(
292
+ f"Published KPI runs: {published} to project '{cfg.langsmith.project}' "
293
+ f"(host_project='{host_project}', scope='{telemetry_scope}')."
294
+ )
295
+ if published == 0:
296
+ print("No local KPI report JSON files were found in observability/evals.")
297
+
298
+ return 0
299
+
300
+
301
+ if __name__ == "__main__":
302
+ raise SystemExit(main())
@@ -0,0 +1,33 @@
1
+ param(
2
+ [switch]$Force
3
+ )
4
+
5
+ Set-StrictMode -Version Latest
6
+ $ErrorActionPreference = 'Stop'
7
+
8
+ $repoRoot = (& git rev-parse --show-toplevel 2>$null)
9
+ if (-not $repoRoot) {
10
+ throw 'Not inside a git repository.'
11
+ }
12
+
13
+ Set-Location $repoRoot
14
+ $hooksDir = Join-Path $repoRoot '.githooks'
15
+ $postCommitHook = Join-Path $hooksDir 'post-commit'
16
+
17
+ if (-not (Test-Path $postCommitHook)) {
18
+ throw "Missing hook template: $postCommitHook"
19
+ }
20
+
21
+ $currentHooksPath = (& git config --local --get core.hooksPath 2>$null)
22
+ if (-not $Force -and $currentHooksPath -and $currentHooksPath -ne '.githooks') {
23
+ Write-Warning "core.hooksPath is already set to '$currentHooksPath'. Skipping update. Use -Force to override."
24
+ exit 0
25
+ }
26
+
27
+ & git config --local core.hooksPath .githooks
28
+ if ($LASTEXITCODE -ne 0) {
29
+ throw 'Failed to configure core.hooksPath.'
30
+ }
31
+
32
+ Write-Host '[mcpee] Git hooks configured: core.hooksPath=.githooks'
33
+ Write-Host '[mcpee] Hook active: post-commit -> scripts/ops/post-commit-refresh.ps1'
@@ -111,12 +111,36 @@ function Install-PythonRequirements {
111
111
 
112
112
  function Resolve-PythonCommand {
113
113
  if (Test-Command 'py') {
114
- return @('py', '-3.14')
114
+ return @('py', '-3')
115
115
  }
116
116
  if (Test-Command 'python') {
117
117
  return @('python')
118
118
  }
119
- throw 'Python 3.14 is required for graphify MCP. Install Python first: https://www.python.org/downloads/'
119
+ throw 'Python 3 is required for MCP scripts. Install Python first: https://www.python.org/downloads/'
120
+ }
121
+
122
+ function Ensure-PythonModuleInstalled {
123
+ param(
124
+ [Parameter(Mandatory = $true)][string]$PythonCommand,
125
+ [string[]]$PythonArgs = @(),
126
+ [Parameter(Mandatory = $true)][string]$Module,
127
+ [Parameter(Mandatory = $true)][string]$Package
128
+ )
129
+
130
+ if (Test-PythonImport -PythonCommand $PythonCommand -PythonArgs $PythonArgs -Module $Module) {
131
+ Write-Host "[ok] Python module '$Module' already available"
132
+ return
133
+ }
134
+
135
+ Write-Host "[install] $PythonCommand $($PythonArgs -join ' ') -m pip install $Package"
136
+ & $PythonCommand @PythonArgs -m pip install $Package
137
+ if ($LASTEXITCODE -ne 0) {
138
+ throw "Failed to install Python package '$Package'"
139
+ }
140
+
141
+ if (-not (Test-PythonImport -PythonCommand $PythonCommand -PythonArgs $PythonArgs -Module $Module)) {
142
+ throw "Python module '$Module' is still unavailable after installing '$Package'"
143
+ }
120
144
  }
121
145
 
122
146
  Write-Host '== MCP platform prerequisites setup =='
@@ -159,21 +183,24 @@ if (-not $SkipRepomix) {
159
183
  Install-ToolFromManifest -Tool $toolByCommand['repomix']
160
184
  }
161
185
 
162
- if (-not $SkipGraphify) {
163
- $py = Resolve-PythonCommand
164
- $pyCmd = $py[0]
165
- $pyArgs = @()
166
- if ($py.Length -gt 1) {
167
- $pyArgs = $py[1..($py.Length - 1)]
168
- }
186
+ $py = Resolve-PythonCommand
187
+ $pyCmd = $py[0]
188
+ $pyArgs = @()
189
+ if ($py.Length -gt 1) {
190
+ $pyArgs = $py[1..($py.Length - 1)]
191
+ }
169
192
 
170
- if ($py.Length -gt 1) {
171
- Write-Host "[setup] Using Python launcher: $($py -join ' ')"
172
- }
173
- else {
174
- Write-Host "[setup] Using Python launcher: $pyCmd"
175
- }
193
+ if ($py.Length -gt 1) {
194
+ Write-Host "[setup] Using Python launcher: $($py -join ' ')"
195
+ }
196
+ else {
197
+ Write-Host "[setup] Using Python launcher: $pyCmd"
198
+ }
199
+
200
+ # Core observability dependency used by KPI publishing and telemetry export.
201
+ Ensure-PythonModuleInstalled -PythonCommand $pyCmd -PythonArgs $pyArgs -Module 'langsmith' -Package 'langsmith'
176
202
 
203
+ if (-not $SkipGraphify) {
177
204
  Install-PythonRequirements -PythonCommand $pyCmd -PythonArgs $pyArgs -RequirementsPath 'requirements.txt'
178
205
 
179
206
  if (-not (Test-PythonImport -PythonCommand $pyCmd -PythonArgs $pyArgs -Module 'graphify.serve')) {