auditor-lambda 0.6.12 → 0.8.0

package/docs/product.md

@@ -1,5 +1,9 @@
 # Product
 
+> Normative definition: [`spec/audit-goals.md`](../spec/audit-goals.md) — product
+> identity, invariants, deterministic/LLM boundaries, and completion. This page is
+> the product overview.
+
 ## Canonical surface
 
 The primary product is `/audit-code` in conversation.
@@ -12,8 +16,8 @@ Normal product usage should:
 - keep semantic review with the active conversation agent by default
 - advance the audit automatically until it completes or no further automatic progress is possible
 
-The CLI is backend infrastructure, a local development harness, and a
-repo-local fallback. It is not the preferred end-user mental model.
+The CLI is backend infrastructure, a local development harness, and a repo-local
+fallback. It is not the preferred end-user mental model.
 
 ## Supported surfaces
 
@@ -48,17 +52,14 @@ provider adapters such as `claude-code`, `opencode`, `subprocess-template`, and
 
 ## Language strategy
 
-Packet quality should not depend on one language ecosystem. JavaScript,
-TypeScript, and Python can receive the richest early support because they are
-common in current usage, but every language analyzer must write into the same
-language-neutral graph and artifact contracts.
+Packet quality should not depend on one language ecosystem. Every language
+analyzer must write into the same language-neutral graph and artifact contracts;
+JS/TS and Python get the richest early support only because they are common.
 
 Do not keep expanding support by adding one bespoke parser per ecosystem unless
-there is concrete repository demand or a high-value deterministic signal. The
-current breadth of package and workspace manifest hints is enough to validate
-the packetization approach. The next product goal is to make graph planning
-observable, maintainable, and extensible through generic ownership hints rather
-than through an open-ended list of file-format handlers.
+there is concrete repository demand or a high-value deterministic signal. Prefer
+making graph planning observable and extensible through generic ownership hints
+over an open-ended list of file-format handlers.
 
 The shared graph should model:
 
@@ -71,111 +72,43 @@ The shared graph should model:
   subprocesses
 - edge confidence, direction, and reason
 
-Graph evidence should be treated in tiers:
+Graph evidence is tiered, strongest first:
 
-- deterministic directed edges, such as imports, entrypoints, route handlers,
-  test/source links, and resolved analyzer references
-- deterministic ownership edges, such as package, module, project, or subsystem
-  roots
+- deterministic directed edges (imports, entrypoints, route handlers,
+  test/source links, resolved analyzer references)
+- deterministic ownership edges (package, module, project, or subsystem roots)
 - analyzer-supplied ownership roots, normalized into graph reference edges
-- language-agnostic semantic affinity, such as shared unusual domain terms,
-  nearby paths, identifier overlap, or embeddings
-
-Semantic affinity can help rank `boundary_files`, explain possible context, and
-highlight missing deterministic extraction. It should not merge packets on
-frequency alone because common tokens like `user`, `request`, `client`,
-`config`, and `error` often connect unrelated code.
-
-Language-specific adapters should enrich the graph without changing packet or
-result contracts:
+- language-agnostic semantic affinity (shared unusual domain terms, nearby
+  paths, identifier overlap, embeddings)
 
-- JS/TS: TypeScript compiler API, package manifests, import/export edges, route
-  conventions, test adjacency
-- Python: local import statement parsing, package/module resolution,
-  pytest/unittest adjacency, and future framework route conventions
-- Other ecosystems: prefer analyzer-supplied ownership roots, ctags/tree-sitter,
-  LSP output, or existing external analyzer data before adding new bespoke
-  manifest parsers
+Semantic affinity can rank `boundary_files`, explain possible context, and
+highlight missing extraction — but it must not merge packets on frequency alone,
+because common tokens (`user`, `request`, `client`, `config`, `error`) connect
+unrelated code.
 
-The fallback should remain useful even when a language has no deep analyzer:
-manifest files, path structure, tests, config, and external analyzer output can
-still seed a graph with lower-confidence edges.
-
-Deterministic tool runners should be project-config aware. For example, ESLint
-syntax-resolution should run only when the repository has repo-local ESLint
-configuration, not merely because an ESLint binary is installed.
+The fallback must stay useful even when a language has no deep analyzer:
+manifests, path structure, tests, config, and external analyzer output can seed a
+graph with lower-confidence edges. Deterministic tool runners should be
+project-config aware — e.g. ESLint syntax-resolution runs only when the repo has
+local ESLint configuration, not merely because the binary is installed.
 
 ## Packet planning
 
-`AuditTask` remains the deterministic coverage identity. `ReviewPacket` is the
-worker-facing unit of understanding.
-
-The next packetization phase should:
-
-- use planner observability to tune which edge kinds change grouping, which
-  files stay boundary-only, and which extractor gaps leave weakly explained
-  packets
-- extend and exercise the generic ownership-root input so external analyzers
-  can say "these files belong to module root X" without a new parser for every
-  ecosystem
-- keep graph and manifest parser code modular before broadening it further
-- exercise deterministic Python import, package, and test/source graph support
-  on fixture and real repositories to find the next highest-value gaps
-- use language-agnostic semantic affinity only as low-authority context unless
-  corroborated by deterministic graph evidence
+`AuditTask` is the deterministic coverage identity; `ReviewPacket` is the
+worker-facing unit of understanding. Packetization aims for packets that read as
+coherent code-ownership or execution-flow units, not merely budget-sized bundles:
+
 - build packets around coherent subsystems and execution flows
-- keep shared fan-in files visible as context instead of letting them merge too
-  much of the repository into one packet
-- distinguish strong edges from weak or heuristic edges
-- group tests with the code they verify when that helps review quality
-- include packet rationale, key edges, entrypoints, and boundary files
-- track packet-quality metrics such as cohesion, fan-in/fan-out, boundary
-  crossings, orphan tasks, weak-packet gap and extension counts, risk
-  concentration, and largest unexplained packet
-
-The practical success bar is that packets feel like reviewable code ownership
-or execution-flow units, not merely budget-sized bundles.
-
-## Production readiness
-
-The package publication path is operational. The release gate, packaged install
-smoke tests, and GitHub Actions Trusted Publishing path are routine
-maintenance. The remaining production work is product confidence rather than a
-new contract shape.
-
-Readiness should be judged through three checks:
-
-- field-trial quality: run real repositories through planning, validate
-  artifacts, and use `audit_plan_metrics.json` to track packet count, weak
-  packet count, average cohesion, merge edge kinds, and weak-packet samples
-- full-loop behavior: prove `next-step` capability routing, packet dispatch,
-  worker review, `submit-packet`, `merge-and-ingest`, selective deepening,
-  runtime validation, and final `audit-report.md` promotion in at least one
-  real host flow
-- release hygiene: keep `npm run verify:release`, linked smoke, packaged
-  smoke, tarball preview, and Trusted Publishing green from a clean checkout
-
-Extractor work should follow field-trial evidence. Fix deterministic graph gaps
-when metrics show them, prefer analyzer-supplied ownership roots before new
-manifest parsers, and keep semantic affinity as context unless deterministic
-evidence corroborates it.
-
-The current production-readiness focus is:
-
-- use the remediator packet-dispatch loop and Polar runtime-confirmed loop as
-  regression evidence for Windows runtime execution, runtime follow-up, final
-  synthesis, and report-promotion behavior
-- use the remediator contract-link field trial as regression evidence that
-  small schema, workflow, package script, and type contract suites can become
-  graph evidence without broad directory merges
-- rerun `remediator-lambda` after its Windows `EBUSY` test cleanup issue is
-  fixed
-- keep exercising analyzer ownership roots on real repositories before adding
-  ecosystem-specific manifest parsers
-- keep host setup claims aligned with verified Codex, Claude Desktop, OpenCode,
-  VS Code, and Antigravity behavior
-- split high-concentration implementation files only after the packetization
-  and schema contracts stay easy to review
+- keep shared fan-in files visible as context rather than merging large parts of
+  the repo into one packet
+- distinguish strong (deterministic) edges from weak or heuristic ones
+- group tests with the code they verify when it aids review
+- carry packet rationale, key edges, entrypoints, and boundary files
+- prefer the generic ownership-root contract (analyzers naming module roots) over
+  a new parser per ecosystem, and keep graph/manifest parsing modular
+
+Planner observability (`audit_plan_metrics.json`: cohesion, fan-in/out, boundary
+crossings, weak-packet gaps) is how extraction gaps are found and prioritized.
 
 ## Non-goals
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "auditor-lambda",
-  "version": "0.6.12",
+  "version": "0.8.0",
   "private": false,
   "description": "Portable hybrid code-auditing framework for arbitrary repositories.",
   "type": "module",

package/schemas/audit_findings.schema.json

@@ -45,7 +45,8 @@
           "confidence",
           "lens",
           "summary",
-          "affected_files"
+          "affected_files",
+          "evidence"
         ],
         "properties": {
           "id": { "type": "string" },
@@ -76,7 +77,7 @@
           },
           "impact": { "type": "string" },
           "likelihood": { "type": "string" },
-          "evidence": { "type": "array", "items": { "type": "string" } },
+          "evidence": { "type": "array", "minItems": 1, "items": { "type": "string" } },
           "reproduction": { "type": "array", "items": { "type": "string" } },
           "systemic": { "type": "boolean" },
           "related_findings": { "type": "array", "items": { "type": "string" } },

package/schemas/dispatch_quota.schema.json

@@ -101,6 +101,7 @@
     "quota_source_snapshot": {
       "type": ["object", "null"],
       "description": "Real-time usage snapshot from a QuotaSource, if available.",
+      "additionalProperties": false,
       "properties": {
         "remaining_pct": { "type": ["number", "null"] },
         "reset_at": { "type": ["string", "null"], "format": "date-time" },
@@ -113,6 +114,7 @@
     "backoff_state": {
       "type": ["object", "null"],
       "description": "Exponential backoff state for repeated rate-limit errors.",
+      "additionalProperties": false,
       "properties": {
         "consecutive_429_count": { "type": "integer", "minimum": 0 },
         "current_cooldown_ms": { "type": "integer", "minimum": 0 },

package/schemas/external_analyzer_results.schema.json

@@ -58,8 +58,8 @@
             "enum": ["critical", "high", "medium", "low", "info"]
           },
           "path": { "type": "string" },
-          "line_start": { "type": "integer" },
-          "line_end": { "type": "integer" },
+          "line_start": { "type": "integer", "minimum": 1 },
+          "line_end": { "type": "integer", "minimum": 1 },
           "summary": { "type": "string" },
           "rule": { "type": "string" },
           "raw": {}

package/schemas/repo_manifest.schema.json

@@ -15,7 +15,7 @@
       },
       "additionalProperties": false
     },
-    "generated_at": { "type": "string" },
+    "generated_at": { "type": "string", "format": "date-time" },
     "files": {
       "type": "array",
       "items": {

package/docs/handoff.md

@@ -1,204 +0,0 @@
-# Handoff
-
-Current pickup note for the next implementation agent. Keep durable product
-direction in `docs/product.md`, engineering workflow in `docs/development.md`,
-contracts in `docs/contracts.md`, and operator steps in
-`docs/operator-guide.md`.
-
-## Current State
-
-The docs refresh remains consolidated under `docs/`; do not restore old
-phase-specific docs unless asked. Checked-in `dist/` is expected to be rebuilt
-after TypeScript changes.
-
-Graph-informed packetization is in place and observable through
-`review_packets.json` and `audit_plan_metrics.json`: packet entrypoints,
-key edges, boundary files, quality, merge/boundary edge kinds, weak packet
-counts, gap counts, extension counts, and bounded samples are all emitted.
-
-Latest completed slice:
-
-- completed the remediator-lambda audit end-to-end after refreshing stale
-  artifacts:
-  - `run-to-completion` refreshed file disposition, auto-fix, structure,
-    planning, runtime validation, and synthesis
-  - resolved all additional runtime/selective-deepening handoffs; final
-    `audit_tasks.json` had 81 tasks, all complete, 0 pending
-  - fixed target-side Windows/runtime validation noise in remediator-lambda:
-    `src/phases/plan.ts` no longer invokes `npx vitest`/`npx jest` in temp
-    roots without `package.json`; `tests/phase-plan.test.ts` has a longer
-    cleanup retry/hook timeout; `vitest.config.ts` excludes generated
-    audit/provider directories from test discovery
-  - `npm test` in `C:\Code\remediator-lambda` now passes: 5 test files,
-    51 tests
-  - final synthesis promoted `C:\Code\remediator-lambda\audit-report.md`
-    (47 findings, 16 work blocks), and `.audit-artifacts` was cleaned by
-    completion
-  - `node C:\Code\auditor-lambda\dist\index.js validate --root
-    C:\Code\remediator-lambda --artifacts-dir
-    C:\Code\remediator-lambda\.audit-artifacts` reports `issue_count: 0`
-
-Prior completed remediator slice:
-
-- completed the remediator-lambda final selective-deepening round:
-  - created dispatch run `20260509T180000000Z_audit_tasks_completed_002`
-    for the two remaining pending tasks
-  - submitted packet
-    `lens-steward-security:security-reliability:packet-1-cfa943527d`;
-    accepted 2 result entries, `finding_count: 0`
-  - `merge-and-ingest` accepted 2 result entries, rejected 0,
-    `spurious_file_count: 0`, `finding_count: 0`
-  - `audit_tasks.json` now has 73 tasks, all `complete`, 0 pending
-  - `audit-code validate` on the remediator artifact bundle reports
-    `issue_count: 0`
-
-Prior completed implementation slice:
-
-- fixed `merge-and-ingest` to treat unexpected files in `task-results/` as
-  warnings rather than hard failures; subagents sometimes write a spurious
-  packet-level result file alongside per-task `submit-packet` submissions —
-  the unexpected file check now emits a stderr warning and increments
-  `spurious_file_count` in the output JSON, but does not block ingestion when
-  all backend-assigned result files are present and valid
-- added regression test: `merge-and-ingest proceeds despite unexpected files
-  in task-results/`; test count: 199 passing
-- fixed Windows `EBUSY` test cleanup in `remediator-lambda/tests/phase-plan.test.ts`:
-  `enumerateTestFiles` calls `spawnSync("npx vitest ...")` in the temp dir;
-  on Windows the child process handle lingers briefly after return, causing
-  `rm()` in `afterEach` to EBUSY; added `rmWithRetry` (5 attempts, 100ms×n
-  backoff) used in both `beforeEach` and `afterEach`; remediator-lambda now
-  passes all 153 tests cleanly
-
-Prior completed slice:
-
-- added `python-test-util-suite-link` edges: `.py` files co-located in a
-  `utils/`, `helpers/`, or `support/` subdirectory within an `isTestPath`
-  directory are chained as a suite (same bounded-suite pattern as existing
-  TypeScript type / JSON schema / package-script suites); `conftest.py` is
-  excluded from the predicate
-- confidence: `0.72`; direction: `undirected`
-- added 3 focused unit tests; rebuilt checked-in `dist/`
-
-Field evidence (Polar-CV-KAN):
-
-- canonical run: `.audit-artifacts/polar-python-util-suite-20260509`
-  (7 packets, 1.000 cohesion, 2 weak packets)
-- `python-test-util-suite-link` produces 2 intra-unit edges within the
-  `tests-utils` packet (`assertions.py → mocks.py`, `mocks.py → test_data.py`)
-- `tests-utils` packet: `internal_edge_count` 0 → 2; `cohesion_score` 0 → 1;
-  `unexplained_file_count` 3 → 0; no longer a weak packet
-- Polar metrics: 7 packets, **1.000 cohesion** (up from 0.857), **2 weak
-  packets** (down from 3)
-- 2 remaining weak packets are `unexplained_files` type; genuinely isolated
-  files (`.auditorignore`, `experiments/domains/__init__.py`,
-  `experiments/summarize_results.py`) cannot be linked without false positives
-
-Field evidence (remediator-lambda):
-
-- baseline: `.audit-artifacts/remediator-yaml-refs-20260508`
-- remediator metrics stable: 62 tasks, 3 packets, 1.000 cohesion, 0 weak
-  packets; `python-test-util-suite-link` adds 0 edges (TypeScript repo, no
-  `.py` files)
-- remediator full audit loop completed: `.audit-artifacts/` (in-progress run
-  `20260509T153435008Z_audit_tasks_completed_006` + deepening run
-  `20260509T155225210Z_audit_tasks_completed_001`); first round produced 42
-  findings across 65 tasks; deepening round added 4 findings across 6 tasks
-- remediator deepening `merge-and-ingest` retry succeeded after the spurious
-  file fix:
-  - command used:
-    `node C:\Code\auditor-lambda\dist\index.js merge-and-ingest --run-id 20260509T155225210Z_audit_tasks_completed_001 --root C:\Code\remediator-lambda --artifacts-dir C:\Code\remediator-lambda\.audit-artifacts`
-  - accepted 6 result entries, rejected 0, `spurious_file_count: 1`,
-    `finding_count: 4`
-  - result ingestion progressed and added 2 selective deepening tasks
-- current remediator artifact state after retry: `audit_results_ingested`
-  present, `audit_tasks_completed` satisfied, `requeue_tasks.json` empty,
-  `audit_tasks.json` has 73 tasks with 0 pending after final selective
-  deepening run `20260509T180000000Z_audit_tasks_completed_002`
-- final selective deepening verified the existing `src/types/workerSession.ts`
-  security findings and upheld the reliability no-finding result for
-  `src/types/sessionConfig.ts`, `src/types/workerResult.ts`, and
-  `src/types/workerSession.ts`; it added 0 findings
-- current remediator packet metrics: 73 tasks, 3 packets, 1.000 cohesion,
-  1 weak packet with 1 unexplained file
-- final refreshed remediator audit completed with 81 tasks, all complete, and
-  final `audit-report.md` at repo root. Runtime validation was confirmed after
-  excluding generated audit/provider directories from Vitest discovery; the
-  earlier `EBUSY` output was environmental noise from generated worktrees.
-
-## Verification
-
-Completed:
-
-```bash
-npm run build
-npm test   # 199 passing
-node C:\Code\auditor-lambda\dist\index.js merge-and-ingest --run-id 20260509T180000000Z_audit_tasks_completed_002 --root C:\Code\remediator-lambda --artifacts-dir C:\Code\remediator-lambda\.audit-artifacts
-node C:\Code\auditor-lambda\dist\index.js validate --root C:\Code\remediator-lambda --artifacts-dir C:\Code\remediator-lambda\.audit-artifacts
-npm test   # in C:\Code\remediator-lambda, 51 passing
-node C:\Code\auditor-lambda\dist\index.js run-to-completion --root C:\Code\remediator-lambda --artifacts-dir C:\Code\remediator-lambda\.audit-artifacts --max-runs 10
-node C:\Code\auditor-lambda\dist\index.js validate --root C:\Code\remediator-lambda --artifacts-dir C:\Code\remediator-lambda\.audit-artifacts
-```
-
-## Files Touched Recently
-
-- `src/cli.ts` — `cmdMergeAndIngest`: unexpected files → warning, not failure
-- `tests/audit-code-wrapper.test.mjs` — new regression test
-- `dist/` — rebuilt
-- `C:\Code\remediator-lambda\src\phases\plan.ts` — avoid test-runner
-  enumeration in roots without `package.json`
-- `C:\Code\remediator-lambda\tests\phase-plan.test.ts` — sturdier
-  `rmWithRetry` helper and longer hook timeout
-- `C:\Code\remediator-lambda\vitest.config.ts` — exclude generated
-  audit/provider directories from test discovery
-- `C:\Code\remediator-lambda\audit-report.md` — final promoted report
-- `docs/handoff.md`
-
-## Next Steps
-
-1. The 2 remaining weak packets in Polar (`experiments-domains` with 5
-   unexplained files, `tests-tiny-files` with 3 unexplained files) share the
-   same genuinely isolated files (`.auditorignore`,
-   `experiments/domains/__init__.py`, `experiments/summarize_results.py`).
-   No extractor can address these without false positives; treat as floor.
-   Only revisit if a future field trial on a different repo surfaces the same
-   pattern in fixable form.
-2. Remediator-lambda field trial is closed. Review the final
-   `C:\Code\remediator-lambda\audit-report.md` only if you need product
-   remediation planning; no audit-code backend work remains for that run.
-3. Run the release/publish flow only when intentionally cutting a version.
-
-## Cautions
-
-- `AuditTask` remains the deterministic coverage identity; `ReviewPacket`
-  should not replace result ingestion contracts.
-- Weak graph edges, semantic affinity, and shared token frequency should remain
-  context unless deterministic graph evidence corroborates them.
-- Boundary files are evidence hints. Worker prompts should continue to
-  discourage broad reads outside the packet.
-- Keep suite links bounded and evidence-led; do not turn same-directory
-  proximity into a broad packet merge rule.
-- `conftest-link` fires only when conftest.py is inside a `isTestPath`
-  directory; root-level conftest.py is deliberately excluded to avoid O(n)
-  fan-out to all Python files.
-- `yaml-path-reference-link` only matches string values ending in config
-  extensions (`.yaml`, `.yml`, `.json`, `.toml`) that resolve to an existing
-  file in the repo; absolute URLs and values without `/` are excluded.
-- `python-test-util-suite-link` predicate requires all four conditions: `.py`
-  extension, NOT a conftest, parent dir name in `{utils, helpers, support}`,
-  and the parent dir's normalized path passes `isTestPath`. Do not broaden the
-  dir-name set without field evidence from a real repository.
-- `python-test-util-suite-link` edges appear as intra-unit edges (not counted
-  in `merge_edge_kind_counts`) when all suite files belong to the same unit.
-  This is correct — the edges still increment `internal_edge_count` and clear
-  the weak-packet flag. Absence from merge counts does not mean the edges are
-  inactive.
-- `merge-and-ingest` unexpected files now warn to stderr and increment
-  `spurious_file_count` in the output JSON. They do not cause ingestion to
-  fail. The check is still present to make spurious writes visible.
-- In this sandbox, running the wrapper from `C:\Code\auditor-lambda` with an
-  absolute remediator root hit an `EPERM` while overwriting the existing
-  remediator run `audit-results.json`; invoking the built CLI directly from
-  `C:\Code\remediator-lambda` succeeded. Treat this as an execution-environment
-  wrinkle unless it reproduces outside the sandbox.
-- Final remediator completion cleaned `.audit-artifacts`; use the promoted
-  repo-root `audit-report.md` and `validate` output as the source of truth.