savant-code 0.0.19 → 0.0.20

package/dev/fids/archive/FID-2026-0803-006-code-map-audit-hygiene.md

@@ -1,243 +0,0 @@
-# FID: Code-Map Package Audit — Guard Fix, Loader Resilience, Hygiene (CM-1…CM-9)
-
-**Filename:** `FID-2026-0803-006-code-map-audit-hygiene.md`
-**ID:** FID-2026-0803-006
-**Severity:** medium
-**Status:** verified
-**Created:** 2026-08-03
-**Author:** Savant
-
-**Summary:**
-Audit of `packages/code-map` (tree-sitter indexing + language detection; ~900 lines read 0-EOF). Two
-MEDIUM findings — (CM-1) a dead `call in {}` prototype-key guard in `buildTokenCallers` that lets a
-real-world `toString`/`valueOf`/`hasOwnProperty` token collision crash the whole code map (SDK
-degrades to empty scores with only a warning), and (CM-2) the module-level `UnifiedLanguageLoader`
-caches a `Parser.init` rejection forever, silently disabling tree-sitter for the process lifetime with
-no surfaced diagnostic — plus seven LOW hygiene/robustness findings (CM-3 dead wasm-path fallback loop,
-CM-4 pointless rethrow, CM-5 lazy-init race, CM-6 language-table gaps, CM-7 TOCTOU + misclassified read
-errors, CM-8 unnecessary cast, CM-9 eval-style `new Function` shim). Baseline: 50/50 tests, typecheck
-clean, reachable via `sdk/src/run-state.ts` (Law 4 verified).
-
----
-
-## Environment (RED — findings with evidence)
-
-All in `packages/code-map/src/`; baseline verified: `bun test` 50 pass / 0 fail, `bun run typecheck`
-exit 0, reachability `getFileTokenScores` → `sdk/src/run-state.ts:167` (dynamic import, wrapped in
-try/catch at `:171-174`) + SDK index re-export + `sdk/smoke-test-dist.ts:126`.
-
-### CM-1 — MEDIUM — dead `call in {}` guard can crash the code map
-
-`parse.ts:311` — `if (!definingFile || callingFile === definingFile || call in {}) continue`. The guard
-was meant to skip Object.prototype-colliding tokens, but `'toString' in {}` is **false** (only
-inherited `__proto__` is caught). Trigger path: a project where `toString`/`valueOf`/`hasOwnProperty`
-appears as both a defined identifier (a class overriding `toString` — common) and a call
-(`obj.toString()`). Then `callersByToken[call]` returns the *truthy* `Object.prototype.toString`
-function → `callerFiles.length (1) < 25` passes → `callerFiles.includes(...)` → **TypeError:
-callerFiles.includes is not a function** → `getFileTokenScores` rejects → SDK catches
-(`run-state.ts:171-174`) → entire code map silently empty with a warning. `constructor` is only
-shielded incidentally because `IGNORE_TOKENS` excludes it from scoring.
-
-### CM-2 — MEDIUM — loader caches a rejection; tree-sitter silently disabled forever
-
-`languages.ts:209-216,294` — `unifiedLoader.parserReady` is created eagerly at module scope
-(`this.parserReady = initTreeSitterForNode()`). If `Parser.init` rejects (wasm missing/unreadable —
-including the self-heal download failing once), every subsequent `getLanguageConfig` rejects forever,
-even if the wasm becomes available, and `getLanguageConfig`'s catch (`:301-308`) returns `undefined`
-logging only under `DEBUG_PARSING`. Net: symbol scoring silently disabled for the process lifetime with
-no surfaced cause; plus a possible unhandled-rejection race at module scope.
-
-### CM-3 — LOW — dead fallback loop in `resolveWasmPath`
-
-`languages.ts:174-181` — `for (const wasmPath of possiblePaths) { try { return wasmPath } catch {
-continue } }`: `return` can never throw, so the loop always returns `possiblePaths[0]` and the
-"try each path" intent never runs (the `catch` is dead scaffolding).
-
-### CM-4 — LOW — pointless rethrow
-
-`languages.ts:227` — `catch (err) { throw err }` around the entire `createLanguageConfig` init body is
-a no-op; remove the wrapper.
-
-### CM-5 — LOW — lazy-init race in `createLanguageConfig`
-
-`languages.ts:243-258` — concurrent calls both observe `!cfg.parser` and double-run
-`Language.load` + `new Parser()` + `new Query()`. Currently serial inside `getFileTokenScores`'s loop,
-but `getLanguageConfig` is public API (hosts could `Promise.all` files). Dedupe with a per-config
-in-flight promise.
-
-### CM-6 — LOW — language-table gaps
-
-`languages.ts` `languageTable` — `.mjs`/`.cjs` (ESM/CJS modules — ubiquitous in modern TS/JS projects)
-and `.mts`/`.cts` are unmapped, so those files are silently unindexed. `.c`/`.cc`/`.cxx`/`.h`, Kotlin,
-Swift, PHP, Scala, CSS etc. are absent — but `@vscode/tree-sitter-wasm/wasm/` (verified) ships only
-cpp/c-sharp/css/javascript/typescript, and the repo has no C/CSS queries, so those are correctly
-documented as out of scope rather than half-added.
-
-### CM-7 — LOW — TOCTOU + misclassified read errors
-
-`parse.ts:255-262` — `fs.statSync(filePath).size` then `fs.readFileSync(filePath, 'utf8')`: the file can
-change or vanish between the two calls (uncaught ENOENT lands in `parseTokensWithLimits`'s catch →
-`emptyParsedTokens(false)` at `:243`). The readFile-provider rejection path returns
-`emptyParsedTokens(false)` at `:183` too — a failed read is classified as *parsed*, consuming a
-`MAX_PARSE_FILES` slot with zero tokens. Should be `skipped: true`.
-
-### CM-8 — LOW — unnecessary cast
-
-`parse.ts:387` — `(tree as { delete?: () => void }).delete?.()`. `Tree.delete(): void` exists in
-web-tree-sitter 0.25.10 types (verified `node_modules/web-tree-sitter/tree-sitter.d.ts:338`). Use
-`tree.delete()` directly; keep the `finally` (memory hygiene is correct).
-
-### CM-9 — LOW — eval-style `__dirname` shim
-
-`utils.ts:2` — `new Function('try { return __dirname; } ...')`. Works in the private runtime but is
-CSP/sandbox-hostile and unwrapped at the call site (`resolveWasmPath` calls it outside any try).
-Add an outer try/catch and keep the documented eslint-disable.
-
----
-
-## Root Cause
-
-Two classes: (1) a broken guard expression (`call in {}` — always-false except `__proto__`) that was
-never exercised by the test corpus (no test uses Object.prototype-colliding tokens), and (2) a
-module-level singleton that treats an init failure as terminal state without surfacing it. The rest are
-accumulated small dead-code/robustness warts in a package that predates the current quality program.
-
-## Proposed Solution (after approval — audit-only now)
-
-1. **CM-1** — `parse.ts:311`: `call in {}` → `call in Object.prototype` (one token; covers
-   `constructor`/`toString`/`valueOf`/`hasOwnProperty`/`isPrototypeOf`/`propertyIsEnumerable`/
-   `toLocaleString`/`__proto__`).
-2. **CM-2** — `languages.ts`: rework `UnifiedLanguageLoader` — lazy `initParser` that stores the
-   in-flight promise (concurrent callers share it) but clears it on rejection so a later call retries;
-   no eager module-scope promise (removes the unhandled-rejection risk). Add a one-time `console.warn`
-   in `getLanguageConfig`'s catch (with eslint-disable) so a silently-degrading subsystem surfaces
-   its cause once per process.
-3. **CM-3** — `languages.ts:174-181`: collapse the dead loop to `return possiblePaths[0]` with a
-   comment (behavior-identical — the loop always returned the first path).
-4. **CM-4** — `languages.ts`: remove the `try { … } catch (err) { throw err }` wrapper in
-   `createLanguageConfig`.
-5. **CM-5** — `languages.ts`: add optional `initPromise?: Promise<void>` to `LanguageConfig`; in
-   `createLanguageConfig`, `cfg.initPromise ??= (async () => { …init… })()` then `await` it, clearing
-   it in a catch so a failed init can retry.
-6. **CM-6** — `languages.ts` `languageTable`: add `.mjs`/`.cjs` → javascript wasm/query and
-   `.mts`/`.cts` → typescript wasm/query. Document the out-of-scope boundary (C-family, Kotlin, Swift,
-   PHP, CSS) in a table comment.
-7. **CM-7** — `parse.ts`: readFile-provider rejection → `emptyParsedTokens(true)`; wrap the stat-path
-   `readFileSync` in try/catch → `null` (skipped) on failure, closing the TOCTOU window.
-8. **CM-8** — `parse.ts:387`: `tree.delete()` directly.
-9. **CM-9** — `utils.ts`: wrap the `new Function` invocation in try/catch returning `undefined` on
-   failure (callers already fall back to `process.cwd()`).
-
-**Non-goals:** adding C/Kotlin/Swift/PHP/CSS grammars (wasm/query authoring is a separate feature
-track); worker-thread parallel parsing (design note, larger change); changing the SDK's degraded-mode
-handling of empty scores.
-
-## Files To Be Changed (implementation stage)
-
-- `packages/code-map/src/parse.ts` (CM-1, CM-7, CM-8)
-- `packages/code-map/src/languages.ts` (CM-2, CM-3, CM-4, CM-5, CM-6)
-- `packages/code-map/src/utils.ts` (CM-9)
-- Tests: `packages/code-map/__tests__/parse.test.ts` or `integration.test.ts` (CM-1 regression —
-  `toString`-collision corpus must not throw), `__tests__/languages.test.ts` (CM-6 — new extensions
-  resolve), optional concurrent-init identity test (CM-5)
-
-## Verification
-
-- [x] Recon: 5 source files read 0-EOF; tests 50/50; typecheck clean; reachability verified
-      (`run-state.ts:167`, SDK index export, smoke-test asserts `getFileTokenScores`)
-- [x] `call in {}` semantics confirmed (`'toString' in {}` === false); `Tree.delete(): void` confirmed
-      in web-tree-sitter types (d.ts:338); wasm manifest confirmed (`@vscode/tree-sitter-wasm/wasm/`)
-- [x] No implementation files modified during this audit (audit-only)
-- [x] Implementation: code-map 51 pass / 0 fail (incl. 1 new CM-1 regression), `bun run typecheck` clean,
-      zero-warning ESLint (full-repo `--max-warnings 0` exit 0), Prettier clean on changed files,
-      `bun run lint:md` exit 0
-- [x] Implementation: SDK smoke caveat noted (smoke-test-dist requires a built dist absent from the
-      tree; CJS tree-sitter harness failure pre-existing/environmental — `getFileTokenScores` export
-      verified at source level via typecheck); independent AUDIT via code-reviewer (clean); CHANGELOG
-      entry added; FID archived
-
-## Perfection Loop
-
-### Loop 1
-
-- **RED:** Completed 2026-08-03 — findings CM-1…CM-9 catalogued with file:line evidence and runtime
-  verification (50/50 tests, clean typecheck, `Tree.delete` exists, wasm manifest inspected, SDK
-  caller + degradation path confirmed at `run-state.ts:171-174`).
-- **GREEN:** Sequential thinking over each finding (see Summary). Missed-question check ("What
-  questions should I have asked when this FID was created, but failed to?"):
-  1. *Is the `call in {}` crash actually reachable with real tokens?* — Yes: `toString`/`valueOf`/
-     `hasOwnProperty` are both ordinary identifiers and Object.prototype keys; `constructor` is only
-     shielded by `IGNORE_TOKENS` scoring exclusion. The `callersByToken[call]` truthy-function path is
-     confirmed (`length 1 < 25` then missing `.includes` → TypeError).
-  2. *Does the crash surface loudly?* — No: the SDK catches it and continues with empty scores
-     (warning only). So CM-1 is feature-loss, not a hard crash — severity MEDIUM, not CRITICAL.
-  3. *Is the CM-2 retry safe against double-init?* — Yes: concurrent callers share one in-flight
-     promise (stored before await); only a rejection clears it for retry.
-  4. *Does collapsing `resolveWasmPath` change behavior?* — No: the loop always returned
-     `possiblePaths[0]` (its try/catch could never throw); collapse is behavior-identical.
-- **AUDIT:** Document-audit before implementation: CM-1 fix is a one-token expression change
-  verified against `Object.prototype` member semantics; CM-2/CM-5 fixes follow the loader's existing
-  promise-dedup pattern (`parserReady`) and are race-checked (single in-flight promise, clear-on-
-  rejection); CM-6 additions reuse existing wasm/query pairs (no new grammars); CM-7's `skipped: true`
-  classification is consistent with `getFileTokenScores`'s `if (parsed.skipped) continue` guard;
-  CM-8 verified against the shipped type declarations. Call-graph evidence in Verification. Zero new
-  public exports beyond the optional `LanguageConfig.initPromise` field (backward-compatible).
-- **SELF-CORRECT:** Initial scan flagged CM-6 as "language coverage gaps" broadly; corrected during
-  GREEN: the wasm manifest inspection bounded the fix to the four free JS/TS-family extensions and
-  moved C-family/Kotlin/Swift/etc. to documented non-goals (writing new grammars is a feature track,
-  not hygiene). CM-1 severity held at MEDIUM after confirming the SDK's catch-and-degrade path.
-- **SELF-CORRECT (implementation):** CM-8 deviated from the FID's literal "use `tree.delete()`
-  directly" — the optional call was retained (`tree.delete?.()`): web-tree-sitter's Tree declares
-  `delete()`, but structurally-compatible test mocks omit it (8 mock tests failed on the direct call),
-  and the cast (the actual Law-6 violation) is gone. An editing slip during CM-8 briefly dropped the
-  call entirely (leaving only a comment — a production memory-leak regression); caught and restored in
-  the immediate follow-up edit. CM-7 was completed in response to the independent AUDIT: the TOCTOU
-  try/catch now wraps `statSync` as well as `readFileSync`, so a file that vanishes before the stat
-  also classifies as skipped rather than parsed. SDK `smoke-test-dist` was found to require a built
-  dist absent from the working tree — its CJS tree-sitter failure (`import_code_map is not defined`)
-  is pre-existing/environmental, not caused by this FID.
-
-## Lessons Learned
-
-1. A guard expression that is always-false (`call in {}`) is indistinguishable from no guard at all —
-   prototype-key collisions (`toString`, `valueOf`) are a real, common-input crash class. When
-   writing "skip Object.prototype keys" logic, verify with `Object.prototype` explicitly.
-2. Module-level singletons that cache a rejected init promise turn a transient failure into permanent
-   silent disablement; clear-on-rejection + a one-time warn turns it into a retryable, diagnosable
-   condition.
-
-## Resolution
-
-- **Fixed By:** Savant (operator-approved implementation — FID presented for approval, approved, then
-  implemented)
-- **Fixed Date:** 2026-08-03
-- **Fix Description:** CM-1 — `parse.ts` `buildTokenCallers` guard `call in {}` → `call in
-  Object.prototype` (covers constructor/toString/valueOf/hasOwnProperty/etc.; the old guard only
-  caught `__proto__` and let a real `toString` collision crash the code map). CM-2 —
-  `UnifiedLanguageLoader` rewritten: lazy `initParser` sharing one in-flight promise across concurrent
-  callers, clearing it on rejection so a later call retries (no more permanent silent disablement),
-  plus a one-time `console.warn` in `getLanguageConfig` surfacing the cause; no eager module-scope
-  promise (removes the unhandled-rejection risk). CM-3 — dead wasm-path fallback loop collapsed to
-  `return possiblePaths[0]` (behavior-identical). CM-4 — no-op `catch (err) { throw err }` removed.
-  CM-5 — `LanguageConfig.initPromise` dedupes concurrent `createLanguageConfig` inits, cleared on
-  failure for retry. CM-6 — `.mjs`/`.cjs` → javascript and `.mts`/`.cts` → typescript rows added to
-  `languageTable` (same grammars; C-family/Kotlin/Swift/CSS documented out-of-scope). CM-7 — read
-  failures (provider-reject and fs paths) classify as `skipped: true` instead of consuming a parse
-  slot; the fs path now does stat+read in one try/catch window (TOCTOU closed, including a vanished-
-  before-stat file). CM-8 — the `as { delete?: … }` cast removed; `tree.delete?.()` retained (the
-  declared type has `delete()`, the optional call guards structurally-compatible mocks). CM-9 —
-  `getDirnameDynamically` wrapped in an outer try/catch (CSP/sandbox-safe, callers already fall back
-  to `process.cwd()`).
-- **Tests Added:** `integration.test.ts` +1 — CM-1 regression: a `toString`-defined + `toString`-
-  called corpus must resolve without throwing and produce no `toString` caller entry (fails on the old
-  code). `languages.test.ts` — `languageTable.length` 10→12 and the `expectedLanguages` list extended
-  for `.mjs`/`.cjs`/`.mts`/`.cts`. Existing 8 mock parse tests exercised the CM-8 guard (they fail on
-  a direct `tree.delete()`), confirming the retained `?.`.
-- **Verified By:** Savant — independent AUDIT via code-reviewer (clean — no CRITICAL/HIGH/MEDIUM;
-  CM-7 statSync window completed in response; CM-8 `?.` deviation documented). Gate suite: code-map
-  51 pass / 0 fail, code-map typecheck clean, full-repo `bun x eslint . --max-warnings 0` exit 0,
-  `bun run lint:md` exit 0, Prettier clean. SDK smoke caveat: `smoke-test-dist` requires a built dist
-  (absent in the tree); the CJS tree-sitter failure is pre-existing/environmental; the
-  `getFileTokenScores` export assertion holds at source level (typecheck).
-- **Commit/PR:** None (working tree)
-- **Archived:** Yes

package/dev/fids/archive/FID-2026-0803-007-evals-benchmark-audit-hygiene.md

@@ -1,296 +0,0 @@
-# FID: Evals Benchmark Runner Audit (EV-1…EV-11)
-
-**Filename:** `FID-2026-0803-007-evals-benchmark-audit-hygiene.md`
-**ID:** FID-2026-0803-007
-**Severity:** high
-**Status:** verified
-**Created:** 2026-08-03
-**Author:** Savant
-
-**Summary:**
-Audit of the evals benchmark harness (`evals/benchmark/*` + `evals/logger.ts`) for
-correctness, error handling, and type safety. Headline: **the evals package fails
-typecheck** (`tsc --noEmit` exits 2) with two errors that are invisible to the
-CI hard gate (AGENTS.md gates only sdk/common/agent-runtime/cli). Both broken
-lines are botched "fixes" from a prior audit that were never verified to compile.
-
----
-
-## Findings
-
-### EV-1 (HIGH) — evals package typecheck is broken, invisible to CI
-
-`bun run typecheck` exits **2** with two errors. Neither surfaced because evals is
-not in the AGENTS.md hard gate, and both were introduced by the
-FID-2026-0802-006 "fix" pass without a compile check.
-
-### EV-1a (HIGH) — lying cast in trace-analyzer that fails tsc
-
-`evals/benchmark/trace-analyzer.ts:222`:
-
-```ts
-return output.value as unknown as {
-  overallAnalysis: string
-  agentFeedback: unknown[]
-}
-```
-
-The cast target `agentFeedback: unknown[]` **does not satisfy the declared return
-type** (`Array<{ agentId; strengths; weaknesses; recommendations }>`) → TS2322.
-The FID-0802-006 DEBT1 comment claims this was the "concrete cast instead of
-`any`" fix, but it both weakens the type and does not compile. Downstream,
-`format-output.ts:159-193` (`formatTraceAnalysis`) reads `feedback.agentId` and
-`feedback.strengths.length` directly — malformed output would TypeError.
-
-**Fix:** define `TraceAnalyzerResultSchema` with zod (mirroring `JudgingResultSchema`
-in judge.ts) and `safeParse` the value; on failure fall through to the existing
-error branch (which already returns the correct fallback shape).
-
-### EV-1b (HIGH) — pino "fix" that never compiled
-
-`evals/logger.ts:21`:
-
-```ts
-}) => ReturnType<(typeof pino)['destination']>
-```
-
-`destination` does not exist on `typeof pino` in the installed pino 9.6 types →
-TS2339:21:34. pino 9 removed static `destination`/`transport` from its type
-declarations (they exist at runtime). The `PinoWithStaticDestination` intersection
-references the missing member in its own return type.
-
-**Fix:** import `DestinationStream` (exported — verified `pino.d.ts:282`) and type
-the return as `DestinationStream`:
-
-```ts
-type PinoWithStaticDestination = typeof pino & {
-  destination: (opts: {
-    dest: string
-    mkdir: boolean
-    sync: boolean
-  }) => DestinationStream
-}
-```
-
-### EV-2 (MEDIUM) — JudgingResultSchema defined but never used
-
-`evals/benchmark/judge.ts:12` defines and exports `JudgingResultSchema`; the zod
-schema is **dead code** — `runSingleJudge` at `:203` does
-`judgeResult.output.value as JudgingResult` with zero runtime validation.
-Malformed model output (missing `analysis`, string `overallScore`) flows straight
-into score averaging → NaN aggregates, or a TypeError in
-`format-output.ts:183-187`. The schema was clearly built for this purpose.
-
-**Fix:** `const parsed = JudgingResultSchema.safeParse(judgeResult.output.value)`;
-on failure log + `return null` (falls into the existing all-failed branch).
-
-### EV-3 (MEDIUM) — timeouts don't abort the underlying work
-
-`withTimeout` (`common/src/util/promise.ts:68`) races and rejects on timeout, but
-the inner `client.run` promise **keeps executing** — a timed-out 20-min judge
-keeps burning API dollars and keeps the event loop alive (process won't exit).
-Call sites: `judge.ts:160` (20 min), `trace-analyzer.ts:187` (20 min),
-`meta-analyzer.ts:198` (30 min), `agent-runner.ts:64` (60 min, wraps repo setup +
-runner).
-
-The SDK already supports abort: `sdk/src/run.ts:181` (`signal?: AbortSignal`),
-`:294` and `:546` check `signal?.aborted`.
-
-**Fix:**
-
-- `judge.ts`, `trace-analyzer.ts`, `meta-analyzer.ts`: drop the `withTimeout`
-  wrapper; pass `signal: AbortSignal.timeout(<ms>)` in the `client.run` options.
-  On abort, `run()` rejects with an AbortError → existing catch branches handle it.
-- `agent-runner.ts`: keep the outer `withTimeout` as the overall budget, and
-  additionally thread an `AbortSignal` into `SavantCodeRunner` (new optional
-  constructor opt) → its `client.run` call, so the inner LLM loop actually stops.
-- **Out of scope (documented):** external CLI runners (claude/codex/opencode) —
-  those spawn child processes; kill semantics are a separate mechanism.
-
-### EV-4 (MEDIUM) — dead, mislabeled judge agent
-
-`judge.ts:134-136`: `judgeAgents['judge-sonnet']` has `id: 'judge-claude'`
-(key/id mismatch), and only gpt+gemini are ever invoked (`:211-215`). The
-"2 judges in parallel" comment is accurate — the sonnet entry is unreachable dead
-code with a confusing name. `Object.values(judgeAgents)` also passes it as an
-available definition on every run.
-
-**Fix:** delete the `'judge-sonnet'` entry.
-
-### EV-5 (MEDIUM) — median picker always favors the higher scorer
-
-`judge.ts:276-277`: with 2 valid results sorted ascending,
-`medianIndex = floor(2/2) = 1` → the analysis text always comes from the
-*higher*-scoring judge, never the lower — a systematic bias when judges disagree.
-
-**Fix:** `const medianIndex = Math.floor((sortedResults.length - 1) / 2)` — true
-lower median (identical for odd n, conservative for even n; text cannot be
-averaged).
-
-### EV-6 (LOW) — `catch (error: any)` on exec failure
-
-`agent-runner.ts:185`: the caught error from `execAsync` has
-`code/stdout/stderr`. `any` bypasses the error contract.
-
-**Fix:** `catch (error)` + narrow:
-`const execError = error as { code?: number; stdout?: string; stderr?: string }`.
-
-### EV-7 (LOW) — 9× `any[]` where `AgentDefinition[]` fits
-
-`run-benchmark.ts:55,59`, `trace-analyzer.ts:118`, `agent-runner.ts:40`,
-`lessons-extractor.ts:21`, `meta-analyzer.ts:151`, `eval-task-generator.ts:107`,
-`runners/savant.ts:15,25` — all just pass agent definitions into `client.run`,
-which already types them as `AgentDefinition[]` (SDK export, already imported in
-every one of these files). Also `setup-test-repo.ts:44` (`options: any`) and
-`gen-repo-eval.ts:30` (`(c: any)`).
-
-**Fix:** replace with `AgentDefinition[]` / `{ sha: string }` / typed options.
-Compiler-arbitrated (typecheck fails if any site needs more than the import).
-
-### EV-8 (LOW) — untyped map callbacks in truncateTrace
-
-`trace-utils.ts:14,18,38`: `(item: any)` / `(file: any)` in `truncateTrace` map
-callbacks. The tool-result output union exists in the codebase.
-
-**Fix:** type as the tool-result output item shape (narrowed local type) with the
-`?.` access already present.
-
-### EV-9 (LOW) — meta-analyzer cast feeds manual validation (downgraded)
-
-`meta-analyzer.ts:278`: `output.value as unknown as Record<string, unknown>`.
-0-EOF read shows the downstream code **already defensively validates** (typeof /
-Array.isArray guards, string filters) — so this is NOT a crash path like EV-2.
-
-**Fix:** for consistency with EV-1a/EV-2, define `MetaAnalyzerResultSchema` (zod)
-and parse; or document the manual validation as intentional. Implement the zod
-variant only if it lands cleanly with the EV-1a schema work; otherwise leave with
-a confirming comment.
-
-### EV-10 (LOW) — bare `catch {` swallows diagnostics
-
-`analyze-task-scores.ts:98,165`, `filter-supplemental-files.ts:23,70`:
-`catch {` with no error binding.
-
-**Fix:** `catch (error)` + `console.warn`/`getErrorObject` (pattern already used
-elsewhere in these files).
-
-### EV-11 (LOW) — partial agent trace lost on timeout/error
-
-`agent-runner.ts:141-149`: on abort/exception, `trace` stays empty and `cost`
-stays 0 — up to 60 min of work discarded with no record of what the agent did
-before dying. `SavantCodeRunner` collects `steps` locally and only returns them
-on success.
-
-**Fix:** add optional `traceSink?: AgentStep[]` constructor opt to
-`SavantCodeRunner`; push each event to both local `steps` and the sink.
-`agent-runner` passes its `trace` array → partial trace survives an abort, and
-the existing error string is recorded.
-
----
-
-## Perfection Loop trace (Loop 1)
-
-- **RED** — Four-pass scan. All findings verified against live code: `tsc` run
-  twice (exit 2, both errors reproduced); `JudgingResultSchema` usage grep (only
-  definition + type-infer); `AbortSignal` support verified in `sdk/src/run.ts`
-  (181/294/546); pino `DestinationStream` export verified in `pino.d.ts:282`;
-  `Runner` interface + `SavantCodeRunner.run()` read 0-EOF; `localAgentDefinitions`
-  dataflow traced main.ts → run-benchmark → agent-runner → savant.ts (all `any[]`,
-  all feeding `client.run`'s typed param); `format-output.ts:159-193` consumer
-  confirmed. Zero CRITICAL; nothing data-destroying.
-- **GREEN** — Missed-questions answered: *is the typecheck failure real or a
-  harness artifact?* (real — plain `tsc --noEmit -p .` exits 2); *would EV-2
-  crash or just produce NaN?* (both possible — NaN averages or a TypeError in
-  format-output); *is EV-3 worth SDK surgery?* (no — SDK already supports
-  `signal`); *does removing judge-sonnet change the 2-judge ensemble?* (no —
-  only gpt+gemini are ever invoked); *is EV-5's lower-median unbiased?* (no
-  selection scheme is unbiased for n=2; lower-median is deterministic and
-  conservative, and identical to current behavior for odd n).
-- **AUDIT** — EV-1a/1b are compiler-arbitrated (typecheck fails if the fix is
-  wrong); EV-2 mirrors the already-present fallback branch (all-failed → zeroed
-  result); EV-3 abort path lands in existing catch branches (no new control
-  flow); EV-11's `traceSink` is a backward-compatible optional constructor opt
-  (existing callers unaffected); zod schemas follow the judge.ts pattern already
-  in the file. Zero new public SDK/config surface.
-- **SELF-CORRECT** — EV-9 downgraded from "lying cast" to "defensive-validated,
-  optional schema" after the 0-EOF read of meta-analyzer.ts; EV-3 scoped to the
-  three direct `client.run` sites + SavantCodeRunner (external CLI runners
-  documented out of scope); EV-5 fix chosen as lower-median index rather than
-  "average the two analyses" (text cannot be averaged); EV-11 fix shaped as a
-  trace sink rather than restructuring `RunnerResult`.
-
-**Files touched on approval:** 10 files — `judge.ts`, `trace-analyzer.ts`,
-`meta-analyzer.ts`, `agent-runner.ts`, `trace-utils.ts`, `logger.ts`,
-`run-benchmark.ts`, `eval-task-generator.ts`, `lessons-extractor.ts`,
-`runners/savant.ts` (+ `setup-test-repo.ts`/`gen-repo-eval.ts` if EV-7's one-off
-anys are in scope).
-
-**Verification on approval:** evals typecheck (must be exit 0 — the headline),
-evals tests (67 v2 tests), sdk/common typechecks (touched files import from
-them), full-repo ESLint `--max-warnings 0`, Prettier, markdownlint, independent
-code-reviewer pass, CHANGELOG entry + LEARNINGS + session summary, archive.
-
----
-
-## Resolution
-
-**Implemented:** 2026-08-03, all EV-1…EV-11 as specified.
-
-- **EV-1a** — `trace-analyzer.ts`: added `TraceAnalyzerResultSchema` (zod,
-  mirroring `JudgingResultSchema`) and `safeParse`; malformed output falls
-  through to the existing error branch. Replaces the `unknown[]` cast that
-  failed TS2322.
-- **EV-1b** — `logger.ts`: `PinoWithStaticDestination.destination` now returns
-the exported `DestinationStream` type (verified `pino.d.ts:861`; `pino()
-accepts it at :828`). Fixes TS2339:21:34.
-- **EV-2** — `judge.ts`: `JudgingResultSchema.safeParse` in `runSingleJudge`;
-  failure → `return null` → existing all-failed branch.
-- **EV-3** — `judge.ts`/`trace-analyzer.ts`/`meta-analyzer.ts`/`lessons-extractor.ts`:
-  `withTimeout(client.run(...))` replaced with `signal: AbortSignal.timeout(...)`
-  (SDK already supports abort — `sdk/src/run.ts:181/294/546`); `agent-runner.ts`
-  keeps the outer `withTimeout` and additionally threads a 60-min
-  `AbortSignal.timeout` + `traceSink` into `SavantCodeRunner` → its `client.run`.
-  **Recorded extension:** `lessons-extractor.ts:197` had the identical
-  `withTimeout` pattern (missed in the scan) and was fixed for consistency.
-  **Recorded nuance:** the abort error message replaces the descriptive
-  "timed out after N minutes" text in logs — acceptable.
-- **EV-4** — `judge.ts`: `'judge-sonnet'` entry deleted (`Object.values` now
-  yields the 2 used judges).
-- **EV-5** — `judge.ts`: `Math.floor((n-1)/2)` (true lower median).
-  **Recorded behavior change:** for n=2 the analysis text now comes from the
-  lower scorer — deliberate per the approved spec.
-- **EV-6** — `agent-runner.ts`: `catch (error: any)` → narrowed
-  `{ code?; stdout?; stderr?; message? }`.
-- **EV-7** — `any[]` → `AgentDefinition[]` at 9 sites + `options: any` →
-  `ExecFileSyncOptions` + `(c: any)` → `{ sha: string }`. Boundary verified:
-  `LoadedAgentDefinition extends AgentDefinition`.
-- **EV-8** — `trace-utils.ts`: typed with `ToolResultOutput` +
-  `isJsonObject` guard; truncation semantics preserved.
-- **EV-9** — `meta-analyzer.ts`: confirming comment only (manual defensive
-  validation is intentional — the FID's documented fallback branch).
-- **EV-10** — `analyze-task-scores.ts:98,165`: `catch (error)` +
-  `console.warn`. **Recorded deviation:** the two `filter-supplemental-files.ts`
-  sites are expected-failure probes (file-not-at-commit / commit-may-already-
-exist) — a warn per missing file would spam, so bare `catch {` retained with
-  intent comments instead of the FID's blanket `catch (error) + warn`.
-- **EV-11** — `SavantCodeRunner` gains optional `traceSink?: AgentStep[]`;
-  `agent-runner.ts` passes its `trace` so partial steps survive an abort, and
-  skips the success-path push for savant (no double-push — keyed on
-  `externalAgentType`).
-
-**Verification (all green):**
-
-- **evals typecheck: exit 0** (was 2 — the headline fix)
-- evals tests: 67 pass / 0 fail · sdk + common typechecks: 0 errors
-- full-repo ESLint `--max-warnings 0`: exit 0 · Prettier: clean · markdownlint: exit 0
-- Independent code review (code-reviewer-deepseek-flash): clean — no
-  CRITICAL/HIGH/MEDIUM; three recorded notes above all captured in this Resolution.
-
-**SELF-CORRECT during implementation:**
-- EV-10 `filter-supplemental-files.ts` narrowed from "add warn everywhere" to
-  "diagnostics for real failures, intent comments for expected-failure probes".
-- EV-3 scope extended to `lessons-extractor.ts` (identical pattern discovered).
-- EV-11 shaped as a `traceSink` (backward-compatible optional ctor opt) with a
-  double-push guard instead of restructuring `RunnerResult`.
-
-**Fixed Date:** 2026-08-03 · **Author:** Savant

package/dev/fids/archive/FID-2026-0803-009-echo-enforcement-doc-drift.md

@@ -1,112 +0,0 @@
-# FID: ECHO Enforcement Layer Documentation Drift (EC-1..EC-4)
-
-**Filename:** `FID-2026-0803-009-echo-enforcement-doc-drift.md`
-**ID:** FID-2026-0803-009
-**Severity:** low
-**Status:** analyzed
-**Created:** 2026-08-03
-**Author:** Savant
-
-**Summary:**
-Audit of the ECHO Protocol enforcement layer (`agents/savant/savant.ts`, `protocol.config.yaml`,
-`ARCHITECTURE.md`, `ECHO.md` v0.2.0) found the *runtime* enforcement fully ECHO-compliant (FSM gating in
-`tool-executor.ts:386-467`, SoD holds — Forge has no bash, Detective has no write tools, Verifier is zero-tool) with
-**4 LOW documentation-drift items** in the bootstrap doc `ECHO.md` and one config comment. No CRITICAL/HIGH/MEDIUM.
-
----
-
-## RED — Findings
-
-### EC-1 (LOW) — ECHO.md Researcher row is stale
-
-`ECHO.md:64` lists the Researcher tools as `web_search, read_url` only. There are **two** researcher agents:
-`researcher-web` (`toolNames: ['web_search', 'read_url']`, researcher-web.ts:21) and `researcher-docs`
-(`toolNames: ['read_docs']`, researcher-docs.ts:22). Since ECHO.md is the bootstrap doc the Orchestrator reads, it
-never learns `read_docs` exists. `ARCHITECTURE.md:33` already documents this correctly
-(`web_search, read_url (web); read_docs (docs)`).
-
-### EC-2 (LOW) — ECHO.md "9 specialized agents" omits the 4 infra spawnables
-
-`ECHO.md:49-51` states the harness "enforces the Perfection Loop through 9 specialized agents", but `savant.ts:123-137`
-`spawnableAgents` actually lists **13**: the 9 canonical + `basher`, `tmux-cli`, `browser-use`, `context-pruner`
-(infrastructure helpers). `ARCHITECTURE.md:212-238` covers this ("Helper Tool Libraries (Filesystem-Only)") and
-explicitly warns "Future checklists/audits should not confuse them." ECHO.md is silent on the distinction.
-
-### EC-3 (LOW) — ECHO.md Forge "Restricted Tools" cell lists bash it never has
-
-`ECHO.md:59` Forge restricted cell reads `spawn_agents, bash (destructive), ask_user`. `forge.ts:33` is
-`toolNames: ['write_file', 'str_replace', 'set_output']` with **zero** `bash` references in the file — bash is never
-granted, so listing it as "restricted" is misleading for a bootstrap doc. The SoD table (ECHO.md:72) correctly says
-"No bash (test) access".
-
-### EC-4 (LOW) — protocol.config.yaml `commands.build` is a partial build
-
-`protocol.config.yaml:28` `build: 'bun run ci'` compiles only SDK + Savant-Free (`package.json:29`:
-`ci: "bun run build:sdk && bun run build:savant-free"`) — the CLI, agent-runtime, code-map, database, and
-llm-providers are not compiled. The inline comment acknowledges this ("use type_check + lint_md for quality gates"),
-but ECHO Law 1 defines `commands.build` as "your compile/build command". Verified: **no code reads `commands.build`**
-— it is a pure documentation surface for agents, so a comment fix is safe and sufficient.
-
----
-
-## GREEN — Proposed Fixes (minimal, docs/config only)
-
-| # | File | Change |
-|---|---|---|
-| **EC-1** | `ECHO.md:64` | Researcher tools cell: `web_search, read_url` → `web_search, read_url (web); read_docs (docs)` (match ARCHITECTURE.md:33) |
-| **EC-2** | `ECHO.md:49-51` | Add footnote after roster intro: 4 infra spawnables (`basher`, `tmux-cli`, `browser-use`, `context-pruner`) are helper/infra agents not in the 9-agent roster — see ARCHITECTURE.md → Helper Tool Libraries |
-| **EC-3** | `ECHO.md:59` | Forge restricted cell: `spawn_agents, bash (destructive), ask_user` → `spawn_agents, ask_user` (bash never granted) |
-| **EC-4** | `protocol.config.yaml:28` | Rewrite comment: build = release-artifact build (SDK + Savant-Free); full 9-workspace compile gate is `type_check` |
-
-**Rejected alternatives (recorded):**
-
-- Changing `commands.build` value to the full `type_check` chain — rejected: `bun run ci` is the release-pipeline
-  entrypoint used by CI; the semantic split (build artifacts vs compile gate) is the correct model, it only needed
-  documentation. Comment-only fix, zero runtime risk.
-- Deleting/rewriting the ECHO.md roster table wholesale — rejected: 7/9 rows are accurate; only Researcher (EC-1) and
-  Forge restricted cell (EC-3) need edits. Minimal-change principle (Law: MINIMAL changes).
-
----
-
-## AUDIT — Verification Plan
-
-1. **markdownlint** on `ECHO.md` (`bun run lint:md`) — table edits must not break MD tables (MD055/MD056).
-2. **YAML parse** of `protocol.config.yaml` (`bun -e` with `yaml` pkg) — comment rewrite must not break the file.
-3. **Table alignment** — re-grep the roster table: Researcher row contains `read_docs`, Forge cell no longer lists
-   `bash`, footnote present after intro.
-4. **No stale refs** — confirm no other tracked `.md` repeats the stale `web_search, read_url` Researcher cell
-   (known: only ECHO.md:64; ARCHITECTURE.md:33 is already correct).
-5. **No code change** — this FID touches docs/config only; typecheck not required (comment-only + markdown edits).
-
-**Double-audit requirement:** static (grep verification) + runtime (markdownlint + YAML parse) — two independent
-methods per AUDIT phase.
-
----
-
-## Resolution
-
-**Status:** verified — **Fixed Date:** 2026-08-03 · **Verified By:** Savant
-(double audit: static grep + markdownlint/YAML parse)
-
-**Changes applied (all 4, per approved GREEN section):**
-
-- **EC-1** — `ECHO.md:69` Researcher row tools cell → `web_search, read_url (web); read_docs (docs)`
-  (matches ARCHITECTURE.md:33 and the two real researcher agents).
-- **EC-2** — `ECHO.md:55-58` added roster-intro footnote: 4 infra spawnables (`basher`, `tmux-cli`,
-  `browser-use`, `context-pruner`) are NOT independent ECHO conversation roles → ARCHITECTURE.md →
-  "Helper Tool Libraries".
-- **EC-3** — `ECHO.md:64` Forge restricted cell → `spawn_agents, ask_user` (bash never granted;
-  forge.ts:33 has zero bash). SoD table row (ECHO.md:77 "No bash (test) access") now consistent with the roster.
-- **EC-4** — `protocol.config.yaml:28` build comment rewritten: release-artifact build (SDK + Savant-Free)
-  vs. full 9-workspace compile gate (`type_check`).
-
-**Verification (double audit — two independent methods):**
-
-1. **Static grep** — each edit verified in place: Researcher row carries `read_docs (docs)`, Forge cell no
-   longer lists `bash`, footnote present after roster intro, `commands.build` comment rewritten. No other
-   tracked `.md` carries the stale Researcher cell (only ARCHITECTURE.md:33, already correct).
-2. **Runtime** — `bun run lint:md` exits 0 after FID spacing fixes; `protocol.config.yaml` parses via `yaml`
-   pkg (`YAML_OK commands.build = bun run ci`).
-
-**Scope guard:** doc/config only — no TypeScript touched, so typecheck not required (per AUDIT plan item 5).
-No code changed; nothing to call-graph verify (Law 4 N/A).