@mutagent/helix 0.1.0-alpha.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/skills/mutagent-helix/SKILL.md +113 -0
- package/.claude/skills/mutagent-helix/scripts/cli/codex-transcode.ts +143 -0
- package/.claude/skills/mutagent-helix/scripts/cli/doctor.ts +111 -0
- package/.claude/skills/mutagent-helix/scripts/cli/init.ts +382 -0
- package/.claude/skills/mutagent-helix/scripts/cli/run.sh +61 -0
- package/CLAUDE.md +193 -0
- package/LICENSE +201 -0
- package/NOTICE +22 -0
- package/README.md +209 -0
- package/assets/adl-loop.svg +162 -0
- package/assets/system-status.svg +422 -0
- package/bin/mutagent-cli.mjs +9950 -0
- package/bin/mutagent-helix.mjs +78 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/CLAUDE.md +28 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/SKILL.md +138 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/assets/templates/agentspec.yaml.tpl +247 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/references/frameworks/doc-pins.md +53 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/references/principles.md +91 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/references/requirements.yaml +295 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/references/workflows/orchestrator-protocol.md +302 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/cli/doctor.ts +116 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/cli/init.ts +230 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/cli/run.sh +60 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/config/resolve-spec-dir.ts +199 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/contract/agentspec.schema.ts +455 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/setup/detect.ts +67 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/validate/validate-spec.ts +93 -0
- package/mutagent-agentspec/LICENSE +201 -0
- package/mutagent-agentspec/NOTICE +19 -0
- package/mutagent-agentspec/README.md +44 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/SKILL.md +100 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/assets/agents/ai-architect.md +312 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/assets/agents/ai-engineer.md +357 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/assets/templates/build-report.md.tpl +39 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/references/principles.md +31 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/references/workflows/build-protocol.md +61 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/references/workflows/improve-handoff.md +23 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/scripts/cli/run.sh +60 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/scripts/handoff/validate-build-input.ts +72 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/scripts/sync-spec/check-sync-spec.ts +499 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/scripts/verify/spec-impl-coverage.ts +182 -0
- package/mutagent-builder/CLAUDE.md +25 -0
- package/mutagent-builder/README.md +26 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/README.md +192 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/SKILL.md +352 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/agents/ai-architect.md +256 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/agents/ai-engineer.md +344 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/agents/diagnostics-analyzer.md +576 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/agents/discovery.md +425 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/audit.json.tpl +24 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/audit.md.tpl +49 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/config.yaml.tpl +100 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/pr-body.md.tpl +56 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/report.html.tpl +327 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/README.md +27 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/iter-N-handover.md.tpl +125 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/self-diagnosis-contract.v0.1.0.yaml.tpl +234 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/spec.yaml.tpl +152 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/team.yaml.tpl +127 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/wave-N-dashboard.html.tpl +1712 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.1-nl-query.md +59 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.2-tier0-progress.md +58 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.3-analysis-progress.md +44 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.4-html-report.md +61 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.5-rca-detail.md +69 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.6-stale-warning.md +53 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.1-welcome.md +60 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.2-source-platform-picker.md +41 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.3-source-auth.md +75 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.4-target-picker.md +48 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.5-ask-tool.md +43 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.6-config-review.md +57 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.7-filter-defaults.md +52 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.8-first-run.md +59 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/examples/sample-findings.json +92 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/examples/sample-meta.json +47 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/adapter-strategy.md +75 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/config-migration.md +84 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/config.md +159 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/filter-search-matrix.md +66 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/harness-knowledge.md +22 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/memory-format.md +78 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/onboarding-decisions.yaml +187 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/operation-inventory.md +59 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/overview.md +167 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/reference.md +119 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/script-index.md +101 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-adapter-contract.md +93 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/claude-code-transcripts.md +66 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/codex-transcripts.md +78 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/codex.md +145 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/install-paths.md +57 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/langfuse.md +178 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/local-jsonl.md +55 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/openobserve.md +102 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/otel.md +63 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/cloud-rest.md +84 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-claude.md +65 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-cloud-agent-sdk.md +92 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-codex.md +72 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-cursor.md +108 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-mastra.md +77 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-opencode.md +70 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/apply-pr-comment-format.md +82 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/diagnostics.md +35 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/handover-contract.md +247 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/onboarding.md +300 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/orchestrator-protocol.md +1861 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/rca.md +331 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/rendering-anatomy.md +314 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/schedule-prep.md +100 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/scope-model.md +128 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/verification-methodology.md +174 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/aggregate/sort-findings.ts +0 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/awareness/blind-spots.ts +167 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/awareness/llm-sample.ts +681 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/doctor.ts +215 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/init.ts +1013 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/install-agents.ts +350 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/run.sh +61 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/shared-templates.ts +79 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/config/load.ts +392 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/config/resolve-credential.ts +120 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/config/schema.ts +641 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/config/validate.ts +134 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/context/build-diagnosis-context.ts +417 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/context/load-context.ts +105 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/contract/types.ts +166 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/enrich/build-render-input.ts +1964 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/enrich/corpus-severity.ts +172 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/enrich/dismissal-match.ts +270 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/enrich/rank-remedies.ts +75 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/hooks/install-hooks.sh +46 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/hooks/pre-push.sh +78 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/invocation/parse-brief.ts +188 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/library/match.ts +114 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/library/paths.ts +95 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/library/store.ts +535 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/library/types.ts +181 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/lint/template-inline-js.ts +267 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/memory/append.ts +188 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/memory/read.ts +218 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/platforms/entity-context.ts +686 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/read-unitf.ts +267 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/trace.ts +1078 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/unitf-adapter.ts +0 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/unitf-types.ts +213 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/report/persist-selections.ts +174 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/report/render.ts +3254 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/run/diagnose.ts +64 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/run/session.ts +167 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/sample/caps.ts +345 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/sample/deep-read-gate.ts +424 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/sample/representative.ts +584 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/scan/objection.ts +224 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/scan/trajectory.ts +489 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/self-diagnostics/dispatch.ts +135 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/self-diagnostics/probe.ts +170 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/setup/detect.ts +307 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/setup/ensure-cli.ts +315 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/setup/reconfigure.ts +77 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/setup/verify-agents.ts +207 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/slicer.ts +166 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/claude-code.ts +17 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/codex.ts +11 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/langfuse.ts +17 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/local-jsonl.ts +12 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/otel.ts +12 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/index.ts +54 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/registry.ts +45 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/types.ts +86 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/stale-detector.ts +74 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/test/render-playwright.cli.ts +477 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/tier0/claude-code.ts +133 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/tier0/langfuse.ts +143 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/tier0-scan.ts +423 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/completeness-check.ts +436 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/doc-refs.ts +190 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/finalize-gate.ts +417 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/findings-contract.ts +628 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/render-js-syntax.ts +188 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/report-checklist.yaml +217 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/trace-shape.ts +262 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/wave6-checklist.ts +326 -0
- package/mutagent-diagnostics/CLAUDE.md +41 -0
- package/mutagent-diagnostics/LICENSE +201 -0
- package/mutagent-diagnostics/NOTICE +18 -0
- package/mutagent-diagnostics/README.md +343 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/SKILL.md +548 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/agents/audit-executor.md +169 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/agents/dataset-builder.md +160 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/agents/discovery.md +425 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/agents/evaluator.md +1108 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/brand/theme.css +213 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/brand/wordmark.html +9 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/context-flow-lens.md +85 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/data-lens.md +47 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/decision-lens.md +48 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/methodology-critic-lens.md +53 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/trajectory-lens.md +44 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/build-review-interface.md +83 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/edd-loop.md +134 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/error-analysis.md +113 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/eval-audit.md +154 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/eval-stage.md +168 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/generate-synthetic-data.md +81 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/grounded-adjudication.md +221 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/memory-format.md +65 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/methodology.md +201 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/operation-inventory.md +197 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/validate-evaluator.md +125 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/workflows/orchestrator-protocol.md +300 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/write-judge-prompt.md +123 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/behavior-tree.schema.yaml +73 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/dataset.schema.yaml +66 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/edd-change-request.schema.yaml +114 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/eval-matrix.schema.yaml +74 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/flow-graph.schema.yaml +69 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/flow-profile.schema.yaml +49 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/methodology-review.schema.yaml +40 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/scorecard.schema.yaml +85 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/agent-dispatch.ts +0 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/aggregate-discover.ts +543 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/artifact-paths.ts +115 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/assemble-scorecard.ts +172 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/build-dataset.ts +186 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/build-evals.ts +93 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/build-review-ui.ts +393 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/check-method-router.ts +170 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/aggregate.ts +112 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/audit-run.ts +182 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/doctor.ts +211 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/dogfood.ts +133 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/init.ts +601 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/methodology-review.ts +122 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/prep.ts +295 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/profile-subject.ts +165 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/run.sh +56 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/variance-check.ts +105 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/code-eval.ts +248 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/code-quality-verdict.ts +285 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/codegen-evals.ts +173 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cold-start-project.ts +151 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cold-start-sampler.ts +267 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/config/load.ts +343 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/config/schema.ts +380 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/agentspec-evals.ts +85 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/dataset.ts +149 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/eval-engine.ts +118 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/eval-matrix.ts +577 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/eval-types.ts +1074 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/flow-graph.ts +194 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/types.ts +303 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/validation.ts +193 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/derive-dataset.ts +152 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/determine-outcome.ts +219 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/diff-discriminate.ts +160 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/discover-criteria.ts +348 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/edd/change-request.ts +232 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/edd/edd-types.ts +210 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/edd/variance-gate.ts +186 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/emit-completeness.ts +111 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/eval-engine.ts +160 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/evaluate.ts +333 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/flow-graph.ts +230 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/judge-prompt-template.ts +253 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/judge-provider.ts +135 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/lint-grounding.ts +229 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/lint-uniformity.ts +168 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/living-suite.ts +109 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/load-bundle.ts +203 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/load-profile-vocab.ts +64 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/load-profile.ts +106 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/mask.ts +138 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/materialize-dataset.ts +113 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/matrix-judge.ts +750 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/memory/append.ts +215 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/memory/read.ts +168 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/persist-eval-criteria.ts +232 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/prep-tasks.ts +125 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/profile-subject.ts +310 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/publish-report.ts +131 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/read-unitf-traces.ts +81 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/render-build-cards.ts +195 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/render-discover-report.ts +1640 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/render-eval-report.ts +3838 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/render-report.ts +212 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/resolve-credential.ts +110 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/resolve-ref.ts +98 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/result-verify.ts +129 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/route-failures.ts +330 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/run-deterministic.ts +367 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/run-evaluate.ts +719 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/run-judge.ts +155 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/run-pipeline.ts +175 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/sample-traces.ts +210 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/self-audit.ts +387 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/source-map.ts +106 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/subject-profile.ts +134 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/substrate.ts +162 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/sync-eval-criteria.ts +244 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/ui-slots.ts +119 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/unitf-to-evaltrace.ts +284 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/validate-judge.ts +358 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/variance-compare.ts +177 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/subjects/mutagent-diagnostics/behavior-tree.yaml +140 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/subjects/mutagent-diagnostics/eval-matrix.yaml +1270 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/subjects/mutagent-diagnostics/methodology-review.yaml +105 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/workflows/audit.workflow.js +82 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/workflows/data-leak.workflow.js +236 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/workflows/variance.workflow.js +163 -0
- package/mutagent-evaluator/CLAUDE.md +45 -0
- package/mutagent-evaluator/LICENSE +201 -0
- package/mutagent-evaluator/NOTICE +23 -0
- package/mutagent-evaluator/README.md +88 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/SKILL.md +287 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/scripts/contracts/amend-request.ts +503 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/scripts/improve-loop-run.ts +304 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/scripts/loop-state-cli.ts +300 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/scripts/loop-state.ts +440 -0
- package/mutagent-optimize/CLAUDE.md +49 -0
- package/mutagent-optimize/README.md +26 -0
- package/mutagent-orchestrator/CLAUDE.md +102 -0
- package/mutagent-orchestrator/README.md +20 -0
- package/mutagent-orchestrator/assets/agents/discovery.md +425 -0
- package/mutagent-orchestrator/assets/agents/dogfood-monitor.md +266 -0
- package/mutagent-orchestrator/assets/agents/monitor.md +258 -0
- package/mutagent-orchestrator/orchestrator.md +782 -0
- package/mutagent-orchestrator/references/MONITORING.md +209 -0
- package/mutagent-orchestrator/references/config-fields.md +132 -0
- package/mutagent-orchestrator/references/config-migration.md +132 -0
- package/mutagent-orchestrator/references/index-registry.md +92 -0
- package/mutagent-orchestrator/routing.yaml +391 -0
- package/mutagent-orchestrator/scripts/config-schema.ts +1010 -0
- package/mutagent-orchestrator/scripts/dispatch.ts +439 -0
- package/mutagent-orchestrator/scripts/dogfood/extract-feedback.ts +203 -0
- package/mutagent-orchestrator/scripts/dogfood/reconstruct-trajectory.ts +327 -0
- package/mutagent-orchestrator/scripts/dogfood/render-dogfood-report.ts +347 -0
- package/mutagent-orchestrator/scripts/dogfood/resolve-subagents.ts +160 -0
- package/mutagent-orchestrator/scripts/dogfood/run-report.ts +264 -0
- package/mutagent-orchestrator/scripts/dogfood/slack-thread.ts +106 -0
- package/mutagent-orchestrator/scripts/dogfood/types.ts +257 -0
- package/mutagent-orchestrator/scripts/eval-routing.ts +753 -0
- package/mutagent-orchestrator/scripts/gate.ts +219 -0
- package/mutagent-orchestrator/scripts/handover-contract.ts +362 -0
- package/mutagent-orchestrator/scripts/index/build-index.ts +351 -0
- package/mutagent-orchestrator/scripts/index/render-index.ts +110 -0
- package/mutagent-orchestrator/scripts/monitor/slack-notify.ts +111 -0
- package/mutagent-orchestrator/scripts/monitor/triggers.ts +162 -0
- package/mutagent-orchestrator/scripts/onboarding-check.ts +369 -0
- package/mutagent-orchestrator/scripts/render-roster.ts +308 -0
- package/mutagent-orchestrator/scripts/resolve-credential.ts +185 -0
- package/mutagent-orchestrator/scripts/resolve-paths.ts +114 -0
- package/mutagent-orchestrator/scripts/slack/format.ts +63 -0
- package/mutagent-orchestrator/scripts/slack/post.ts +115 -0
- package/mutagent-orchestrator/scripts/sync-index.ts +311 -0
- package/package.json +64 -0
|
@@ -0,0 +1,1861 @@
|
|
|
1
|
+
# Orchestrator Protocol — Inline Procedure for the Parent Session
|
|
2
|
+
|
|
3
|
+
> **Authority**: SKILL.md §0 (Setup Detection), PR-022 (Self-Diagnostics [INTERNAL])
|
|
4
|
+
>
|
|
5
|
+
> **Why this file exists**: Claude Code sub-agents cannot dispatch other sub-agents
|
|
6
|
+
> (Agent tool) or invoke AskUserQuestion. Coordinator-class orchestration therefore
|
|
7
|
+
> MUST run in the parent session — the one that invoked the skill. This protocol
|
|
8
|
+
> is loaded inline by the parent session after §0 detects a complete config.
|
|
9
|
+
>
|
|
10
|
+
> **Operator voice-stamp T1**: "The Diagnostic skill must perform the Orchestrator
|
|
11
|
+
> protocols. Sub-Agents probably cannot dispatch Sub-Agents. That's why they also
|
|
12
|
+
> cannot do AskUserQuestion. To keep it on the main session, we probably need to
|
|
13
|
+
> use the skill." — explicit endorsement of this inline approach.
|
|
14
|
+
>
|
|
15
|
+
> Loaded when: `scripts/setup/detect.ts` returns `state: "complete"` AND the caller
|
|
16
|
+
> is NOT in `--reconfigure` mode.
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
> **⚠ Wave-6 methodology is MANDATORY.** All Wave-6 scripts (`scripts/invocation/parse-brief.ts`,
|
|
21
|
+
> `scripts/awareness/*`, `scripts/library/match.ts`, `scripts/sample/caps.ts`,
|
|
22
|
+
> `scripts/sample/deep-read-gate.ts`) MUST be invoked at the steps where they appear below
|
|
23
|
+
> (Steps 3a, 3.5, 4.5, 5.5, and the Step 6 pre-gate) unless a documented exemption applies
|
|
24
|
+
> (see `references/workflows/rca.md` Gate Exemption Taxonomy). Skipping a Wave-6 script with
|
|
25
|
+
> inline prose reasoning is a protocol violation — the deep-read gate in particular HARD-REFUSES.
|
|
26
|
+
|
|
27
|
+
---
|
|
28
|
+
|
|
29
|
+
<!-- TODO: P3 cross-platform capability probe — see iter-10 tab ⑥
|
|
30
|
+
Step 0 below is reserved for the boot probe that verifies Agent + AskUserQuestion
|
|
31
|
+
capabilities are available in the current runtime before proceeding.
|
|
32
|
+
P3 (DEFERRED) will implement cross-platform detection:
|
|
33
|
+
- Claude Code: Agent + AskUserQuestion native
|
|
34
|
+
- Codex: Task + chat-choice fallback
|
|
35
|
+
- Cursor/generic: capability-based probe + cannot-orchestrate sentinel
|
|
36
|
+
Until P3 lands, proceed assuming parent session has Agent + AskUserQuestion available.
|
|
37
|
+
-->
|
|
38
|
+
|
|
39
|
+
## Step 0 — Capability Check [P3 TODO]
|
|
40
|
+
|
|
41
|
+
*(Boot probe DEFERRED — see TODO comment above. For now: assume parent session has
|
|
42
|
+
Agent dispatch + AskUserQuestion available. If Agent dispatch fails at runtime,
|
|
43
|
+
surface the error and halt with guidance to use Claude Code.)*
|
|
44
|
+
|
|
45
|
+
## Step 1 — Config Detection
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
Bash("scripts/cli/run.sh scripts/setup/detect.ts")
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Expect `state: "complete"`. If partial or missing, halt and route to
|
|
52
|
+
`references/workflows/onboarding.md`.
|
|
53
|
+
|
|
54
|
+
Check `InitDescriptor.agentsBoundary`:
|
|
55
|
+
- `"missing"` → display install instruction + halt
|
|
56
|
+
- `"pending-restart"` → display restart instruction + halt
|
|
57
|
+
- `"invalid"` → display re-install instruction + halt
|
|
58
|
+
- `"ready"` → proceed
|
|
59
|
+
|
|
60
|
+
> **W9-10 (lean onboarding):** Phase-5b agent install is non-blocking. `scripts/cli/init.ts`
|
|
61
|
+
> owns agent install (`pnpx … init` installs skill + agents). Onboarding's Phase 5b only
|
|
62
|
+
> checks / offers install if agents are missing — it never mandates it. See
|
|
63
|
+
> `references/workflows/onboarding.md` Phase 5b. Do NOT edit onboarding.md here; B5 owns it.
|
|
64
|
+
|
|
65
|
+
## Step 1.5 — Source/Target Selection (MANDATORY — run at run START)
|
|
66
|
+
|
|
67
|
+
The `global.sources[]` / `global.targets[]` catalog can hold MORE THAN ONE entry.
|
|
68
|
+
The skill binds ONE source (source-consumer role) + ONE target (target-writer role)
|
|
69
|
+
per run. `scripts/setup/detect.ts` (Step 1) already resolved the binding and reports
|
|
70
|
+
its outcome — this step ACTS on it. Selection precedence (matches the orchestrator +
|
|
71
|
+
evaluator exactly — implemented in `scripts/config/load.ts` `selectByRole`):
|
|
72
|
+
|
|
73
|
+
1. **explicit `--source <name>` / `--target <name>`** — if the operator's invocation
|
|
74
|
+
named one, that wins outright. A name that matches no catalog entry is an ERROR
|
|
75
|
+
(`unknown-name`) → detect.ts reports `state: partial` with the valid names; halt +
|
|
76
|
+
surface. **This skips the prompt.**
|
|
77
|
+
2. **exactly one entry** → auto-bound (`status: bound`). No prompt.
|
|
78
|
+
3. **exactly one `default: true`** among many → `resolved-default` (F2 CONFIRM). The
|
|
79
|
+
default is **PRESELECTED** (a usable binding EXISTS — NON-FATAL for the gate), but
|
|
80
|
+
the skill **SHOULD CONFIRM it** ("use default `<X>`?") before proceeding — the
|
|
81
|
+
preselected option is the default; the other catalog names are override choices.
|
|
82
|
+
This is a **confirm** ASK, DISTINCT from the pick-which ASK in case 5.
|
|
83
|
+
4. **>1 `default: true`** → `multiple-defaults` CONFIG ERROR → detect.ts reports
|
|
84
|
+
`state: partial`; halt + tell the operator to keep exactly one default.
|
|
85
|
+
5. **>1 entry, 0 `default: true`, no `--source`/`--target`** → `needs-selection`.
|
|
86
|
+
detect.ts keeps `state: complete` (a source EXISTS — this is NON-FATAL for the
|
|
87
|
+
onboarding gate) and surfaces the candidate names in the DetectResult fields
|
|
88
|
+
**`sourceNeedsSelection[]`** and **`targetNeedsSelection[]`**. When either is
|
|
89
|
+
non-empty, the skill **MUST PROMPT** the operator to **pick which** before proceeding.
|
|
90
|
+
|
|
91
|
+
### The ASK (platform `ask_tool`)
|
|
92
|
+
|
|
93
|
+
There are TWO ASK shapes, keyed off the `selectByRole` status per role (Case 3 vs 5).
|
|
94
|
+
Both use the platform ASK — `AskUserQuestion` on Claude Code, the chat multi-choice
|
|
95
|
+
fallback on other runtimes (per `config.ask_tool`). One question per un-picked role.
|
|
96
|
+
|
|
97
|
+
**(a) Pick-which** — `status: needs-selection` (Case 5; `detect.sourceNeedsSelection`
|
|
98
|
+
and/or `targetNeedsSelection` non-empty). No preselection — the operator picks one:
|
|
99
|
+
|
|
100
|
+
```
|
|
101
|
+
AskUserQuestion({
|
|
102
|
+
questions: [
|
|
103
|
+
{
|
|
104
|
+
question: "Multiple trace sources are configured — which one should this run diagnose?",
|
|
105
|
+
header: "Source",
|
|
106
|
+
multiSelect: false,
|
|
107
|
+
options: detect.sourceNeedsSelection.map((name) => ({
|
|
108
|
+
label: name,
|
|
109
|
+
description: "global.sources[] entry — bind this source for the run",
|
|
110
|
+
preview: `scripts/cli/run.sh scripts/setup/detect.ts --source ${name}`,
|
|
111
|
+
})),
|
|
112
|
+
},
|
|
113
|
+
// …and a second question for the target when detect.targetNeedsSelection is non-empty.
|
|
114
|
+
],
|
|
115
|
+
})
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
**(b) Confirm-the-default** — `status: resolved-default` (Case 3). A default is already
|
|
119
|
+
preselected (`sourceSelection.bound`); the operator confirms it or overrides. The
|
|
120
|
+
preselected default is the FIRST option (the confirm answer); the remaining
|
|
121
|
+
`sourceSelection.candidates` are the override choices:
|
|
122
|
+
|
|
123
|
+
```
|
|
124
|
+
AskUserQuestion({
|
|
125
|
+
questions: [
|
|
126
|
+
{
|
|
127
|
+
question: "Source `<default>` is the configured default — use it for this run?",
|
|
128
|
+
header: "Source",
|
|
129
|
+
multiSelect: false,
|
|
130
|
+
options: [
|
|
131
|
+
{
|
|
132
|
+
label: `<default> (default)`,
|
|
133
|
+
description: "Confirm the configured default:true source",
|
|
134
|
+
preview: `scripts/cli/run.sh scripts/setup/detect.ts --source <default>`,
|
|
135
|
+
},
|
|
136
|
+
// …the other sourceSelection.candidates as override options.
|
|
137
|
+
],
|
|
138
|
+
},
|
|
139
|
+
// …and a confirm question for the target when its status is resolved-default.
|
|
140
|
+
],
|
|
141
|
+
})
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
> Confirm (b) is preselected → default-accept is one keystroke; pick (a) has no
|
|
145
|
+
> preselection. Non-interactive/CI runs (an explicit `--source`/`--target`, or a
|
|
146
|
+
> `--yes`/assume-default escape) skip BOTH asks — the binding is already unambiguous.
|
|
147
|
+
|
|
148
|
+
Re-run detection with the operator's pick threaded as the override so the binding is
|
|
149
|
+
now unambiguous:
|
|
150
|
+
|
|
151
|
+
```bash
|
|
152
|
+
Bash("scripts/cli/run.sh scripts/setup/detect.ts --source <picked> --target <picked>")
|
|
153
|
+
# expect state:complete, sourceNeedsSelection:[] , targetNeedsSelection:[]
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
> `default: true` on exactly one `global.sources[]` (and one `global.targets[]`) entry
|
|
157
|
+
> downgrades the pick-which prompt to a one-keystroke **confirm** (Case 3 · ASK (b)) —
|
|
158
|
+
> it PRESELECTS but still confirms (F2). To skip the prompt ENTIRELY, pass an explicit
|
|
159
|
+
> `--source`/`--target` (or a `--yes`/assume-default escape in CI) — those are the only
|
|
160
|
+
> full prompt-skip paths.
|
|
161
|
+
|
|
162
|
+
The bound source flows into Step 2 (`<source-cli>` = the picked source's platform CLI).
|
|
163
|
+
|
|
164
|
+
## Step 2 — Score-Scale Auto-Discovery (iter-8)
|
|
165
|
+
|
|
166
|
+
Before any score-based filter, probe the platform's scoring scale:
|
|
167
|
+
|
|
168
|
+
```bash
|
|
169
|
+
Bash("<source-cli> scores list --limit 100 --json")
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
Classify scale type:
|
|
173
|
+
- `boolean` — negative = `false`/`0`/`"down"`
|
|
174
|
+
- `discrete-1-5` — negative = ≤ 2 (below midpoint 3)
|
|
175
|
+
- `discrete-1-10` — negative = ≤ 4 (below midpoint 5)
|
|
176
|
+
- `continuous-0-1` — negative = < 0.5
|
|
177
|
+
- `categorical` — ask operator to specify
|
|
178
|
+
|
|
179
|
+
Cache: `.mutagent/diagnostics/cache/scale-{platform}.json`. Re-probe weekly.
|
|
180
|
+
|
|
181
|
+
If ambiguous: AskUserQuestion for clarification.
|
|
182
|
+
|
|
183
|
+
## Step 3 — NL → TraceFilter Translation
|
|
184
|
+
|
|
185
|
+
The operator provides natural language. Translate:
|
|
186
|
+
|
|
187
|
+
```
|
|
188
|
+
Operator: "diagnose all sessions with negative feedback last week for the search-agent"
|
|
189
|
+
|
|
190
|
+
Parent session reasons:
|
|
191
|
+
agentId: "search-agent"
|
|
192
|
+
startTime: 7daysAgo
|
|
193
|
+
hasFeedback: true
|
|
194
|
+
scoreBelow: <from scale probe>
|
|
195
|
+
|
|
196
|
+
Filter → CLI query (executed at the Helix PRE-STAGE, before the skill is dispatched):
|
|
197
|
+
mutagent-cli trace fetch --platform <p> --agent-id search-agent --from 7daysAgo \
|
|
198
|
+
--has-feedback --score-below {threshold} \
|
|
199
|
+
--export .mutagent/traces/<queryId>/traces.jsonl
|
|
200
|
+
→ writes traces.jsonl (UniTF) + manifest.json; both ride into the skill via
|
|
201
|
+
HandoverBundle.inputs[]. The skill READS them at Step 3.7 — it never fetches.
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
The NL→filter translation is what parameterizes the `mutagent-cli trace fetch`
|
|
205
|
+
query at the pre-stage; the skill itself receives the pre-produced JSONL. Run
|
|
206
|
+
`mutagent-cli trace --help` to discover the fetch/count/export surface.
|
|
207
|
+
|
|
208
|
+
Reference: `references/filter-search-matrix.md` for per-platform CLI flags.
|
|
209
|
+
|
|
210
|
+
### I-011: Multi-Q AskUserQuestion batch for ambiguous queries
|
|
211
|
+
|
|
212
|
+
**When the NL query forks into 3+ distinct filter interpretations**, use a SINGLE
|
|
213
|
+
multi-question `AskUserQuestion` call — NOT separate sequential questions. Batch all
|
|
214
|
+
forks in one call per `feedback_ask_user_question_with_previews`:
|
|
215
|
+
|
|
216
|
+
> Rule: every option MUST attach a concrete `preview` block (CLI invocation, filter JSON,
|
|
217
|
+
> or expected trace count estimate). Short labels alone are too vague to pick from.
|
|
218
|
+
|
|
219
|
+
**Example multi-Q AskUserQuestion** for "diagnose the bad sessions from last week":
|
|
220
|
+
|
|
221
|
+
```
|
|
222
|
+
AskUserQuestion({
|
|
223
|
+
questions: [
|
|
224
|
+
{
|
|
225
|
+
question: "Which time window? (last week is ambiguous — platform timezone may differ)",
|
|
226
|
+
header: "Time range",
|
|
227
|
+
multiSelect: false,
|
|
228
|
+
options: [
|
|
229
|
+
{
|
|
230
|
+
label: "Last 7 calendar days",
|
|
231
|
+
description: "From 2026-05-23 00:00 UTC to now",
|
|
232
|
+
preview: '--from 2026-05-23T00:00:00Z --to now'
|
|
233
|
+
},
|
|
234
|
+
{
|
|
235
|
+
label: "Last Mon–Sun week",
|
|
236
|
+
description: "Calendar week 2026-05-19 to 2026-05-25",
|
|
237
|
+
preview: '--from 2026-05-19T00:00:00Z --to 2026-05-26T00:00:00Z'
|
|
238
|
+
},
|
|
239
|
+
{
|
|
240
|
+
label: "Last 48 hours (incident window)",
|
|
241
|
+
description: "Focused on the most recent activity",
|
|
242
|
+
preview: '--from 2026-05-28T00:00:00Z --to now'
|
|
243
|
+
}
|
|
244
|
+
]
|
|
245
|
+
},
|
|
246
|
+
{
|
|
247
|
+
question: "Which signal should be primary? (determines slicing strategy)",
|
|
248
|
+
header: "Signal type",
|
|
249
|
+
multiSelect: false,
|
|
250
|
+
options: [
|
|
251
|
+
{
|
|
252
|
+
label: "Negative feedback only",
|
|
253
|
+
description: "Only traces with explicit user thumbs-down or score < threshold",
|
|
254
|
+
preview: '--has-feedback --score-below 0.4'
|
|
255
|
+
},
|
|
256
|
+
{
|
|
257
|
+
label: "All negative signals",
|
|
258
|
+
description: "Feedback + errors + high latency (broader coverage)",
|
|
259
|
+
preview: '--has-feedback --has-error --latency-above 10000'
|
|
260
|
+
},
|
|
261
|
+
{
|
|
262
|
+
label: "Low-score traces only",
|
|
263
|
+
description: "Score-filtered, no feedback required",
|
|
264
|
+
preview: '--score-below 0.4'
|
|
265
|
+
}
|
|
266
|
+
]
|
|
267
|
+
},
|
|
268
|
+
{
|
|
269
|
+
question: "Scope to a specific agent?",
|
|
270
|
+
header: "Agent scope",
|
|
271
|
+
multiSelect: false,
|
|
272
|
+
options: [
|
|
273
|
+
{
|
|
274
|
+
label: "All agents",
|
|
275
|
+
description: "No agent filter — full session scope",
|
|
276
|
+
preview: '(no --agent-id flag)'
|
|
277
|
+
},
|
|
278
|
+
{
|
|
279
|
+
label: "search-agent only",
|
|
280
|
+
description: "Scope to the search agent (TraceFilter.skillAgentScope)",
|
|
281
|
+
preview: '--agent-id search-agent'
|
|
282
|
+
},
|
|
283
|
+
{
|
|
284
|
+
label: "orchestrator + analyzers",
|
|
285
|
+
description: "The diagnostics skill's own agent set",
|
|
286
|
+
preview: '--agent-id diagnostics-orchestrator,diagnostics-analyzer'
|
|
287
|
+
}
|
|
288
|
+
]
|
|
289
|
+
}
|
|
290
|
+
]
|
|
291
|
+
})
|
|
292
|
+
```
|
|
293
|
+
|
|
294
|
+
> **Design note**: batching all 3 forks into a single AskUserQuestion reduces round-trips.
|
|
295
|
+
> The operator sees all ambiguities at once and resolves them in one interaction instead of
|
|
296
|
+
> 3 sequential questions. This is the multi-Q AskUserQuestion pattern from
|
|
297
|
+
> `feedback_ask_user_question_with_previews`.
|
|
298
|
+
|
|
299
|
+
**Simple query (no fork)**: if the query has exactly ONE clear interpretation, proceed
|
|
300
|
+
directly to CLI invocation without AskUserQuestion. Only batch-ask when genuine ambiguity exists.
|
|
301
|
+
|
|
302
|
+
**Default filter with confirmation**: if the query is partially clear, propose a default
|
|
303
|
+
filter and use a single-question AskUserQuestion with a preview block showing the exact CLI
|
|
304
|
+
invocation. Do NOT use prose questions — always attach the `preview` block.
|
|
305
|
+
|
|
306
|
+
## Step 3a — Parse Brief + Scope Resolution (Wave-6 R2.6, D2, W11-06/07 — MANDATORY)
|
|
307
|
+
|
|
308
|
+
Before reasoning about the filter inline, parse the operator's verbatim brief
|
|
309
|
+
through the published parser — do NOT free-parse:
|
|
310
|
+
|
|
311
|
+
```bash
|
|
312
|
+
Bash("scripts/cli/run.sh scripts/invocation/parse-brief.ts \"<verbatim operator brief>\"")
|
|
313
|
+
# → { agent?, timeWindow?, focus?, residual, scopeType: 'skill'|'agent'|null, entity? }
|
|
314
|
+
```
|
|
315
|
+
|
|
316
|
+
Store the verbatim brief in `runMeta.operatorInvocation` (D2). Use the parsed
|
|
317
|
+
fields (`agent`, `timeWindow`, `focus`) to build the TraceFilter in Step 3. If
|
|
318
|
+
`residual` is non-empty AND materially affects the filter, surface to operator
|
|
319
|
+
via a single AskUserQuestion.
|
|
320
|
+
|
|
321
|
+
### Step 3a.1 — Load AutoMemory + context links (PHASE 3 + 4 — run START)
|
|
322
|
+
|
|
323
|
+
At run START, load two shared run-context surfaces (both COMMITTED, project-level):
|
|
324
|
+
|
|
325
|
+
```bash
|
|
326
|
+
# AutoMemory — operator feedback + tool-lessons, filtered to { diagnose, general }.
|
|
327
|
+
Bash("scripts/cli/run.sh scripts/memory/read.ts \"$PROJECT_ROOT\" diagnose")
|
|
328
|
+
# → MemoryEntry[] (name, description, type, lifecycle, created, body)
|
|
329
|
+
|
|
330
|
+
# Context links — global.context[] + lifecycle.diagnostics.context[], de-duped.
|
|
331
|
+
Bash("scripts/cli/run.sh scripts/context/load-context.ts \"$PROJECT_ROOT\"")
|
|
332
|
+
# → { links[], globalLinks[], stageLinks[], migrationRequired }
|
|
333
|
+
```
|
|
334
|
+
|
|
335
|
+
- **AutoMemory** (`scripts/memory/read.ts`) surfaces prior operator feedback about
|
|
336
|
+
the TOOL (e.g. "stop surfacing low-tagging-rate") + standing preferences. Inject
|
|
337
|
+
the returned entries' bodies into your reasoning so past feedback steers this run.
|
|
338
|
+
Subject = the tool + operator, NOT the diagnosed agent — see
|
|
339
|
+
`references/memory-format.md`.
|
|
340
|
+
- **Context links** (`scripts/context/load-context.ts`) are supplementary docs the
|
|
341
|
+
operator wired (triage runbook, product context). Read the linked `path`s and
|
|
342
|
+
consult them per each link's `when`. If `migrationRequired` is true on either,
|
|
343
|
+
STOP and run `references/config-migration.md` first.
|
|
344
|
+
|
|
345
|
+
### W11-07: Scope picker (skill vs agent)
|
|
346
|
+
|
|
347
|
+
After parse-brief completes, determine the diagnostic scope using this decision tree:
|
|
348
|
+
|
|
349
|
+
| Case | Action |
|
|
350
|
+
|---|---|
|
|
351
|
+
| `scopeType` is non-null ("skill" or "agent") | Use `scopeType` directly — no AskUserQuestion |
|
|
352
|
+
| `scopeType` is null AND operator provides no further context | AskUserQuestion: skill vs agent vs all-traces (see scope-model.md) |
|
|
353
|
+
|
|
354
|
+
**AskUserQuestion scope picker template** (only when scopeType is null):
|
|
355
|
+
|
|
356
|
+
```
|
|
357
|
+
AskUserQuestion({
|
|
358
|
+
questions: [{
|
|
359
|
+
question: "What are you diagnosing?",
|
|
360
|
+
header: "Diagnostic scope",
|
|
361
|
+
multiSelect: false,
|
|
362
|
+
options: [
|
|
363
|
+
{
|
|
364
|
+
label: "A skill (e.g. mutagent-diagnostics)",
|
|
365
|
+
description: "Diagnose a skill's own operational traces",
|
|
366
|
+
preview: "scope: skill — uses skill identity from config.agents[]"
|
|
367
|
+
},
|
|
368
|
+
{
|
|
369
|
+
label: "An agent (e.g. search-agent, orchestrator)",
|
|
370
|
+
description: "Diagnose a named agent's runtime traces",
|
|
371
|
+
preview: "scope: agent — uses agent identity from config.agents[]"
|
|
372
|
+
},
|
|
373
|
+
{
|
|
374
|
+
label: "All traces (no scope filter)",
|
|
375
|
+
description: "Run across all traces without scoping to a named entity",
|
|
376
|
+
preview: "scope: all-traces — escape hatch; no identity resolution"
|
|
377
|
+
}
|
|
378
|
+
]
|
|
379
|
+
}]
|
|
380
|
+
})
|
|
381
|
+
```
|
|
382
|
+
|
|
383
|
+
### W11-07: Agent identity resolution
|
|
384
|
+
|
|
385
|
+
After scope is determined and entity is known (from `parsed.entity` or the scope picker answer):
|
|
386
|
+
|
|
387
|
+
1. Look up `entity` in `config.agents[]` via `resolveEntityIdentity(entity, config.agents)`.
|
|
388
|
+
2. If a match is found, annotate `EntityContext.identity` with the resolved pointers.
|
|
389
|
+
3. The identity pointers drive platform-specific trace fetching (Langfuse `traceName`/`tags`,
|
|
390
|
+
OTel `serviceName`/`resourceAttrs`).
|
|
391
|
+
4. If no match: proceed with name-based trace matching (existing behavior — no change).
|
|
392
|
+
|
|
393
|
+
Full model: `references/workflows/scope-model.md`.
|
|
394
|
+
|
|
395
|
+
## Step 3.5 — Awareness Layer (Wave-6 R2.2, W11-05 — MANDATORY unless priors)
|
|
396
|
+
|
|
397
|
+
BEFORE the Step 4 primary-signal pick, run the awareness mini-sample + blind-spot
|
|
398
|
+
scan so the slicing strategy is grounded in observed evidence, not assumptions:
|
|
399
|
+
|
|
400
|
+
```bash
|
|
401
|
+
# B2-fix (verified against each script's import.meta.main argv parser):
|
|
402
|
+
# • llm-sample.ts reads its trace metadata via the REQUIRED --metadata flag (NOT a
|
|
403
|
+
# positional path) and the sample size via --awareness-sample <n> (alias --size).
|
|
404
|
+
# OMIT it to accept the V3-1 default of 50 — a substantive deep read (see below).
|
|
405
|
+
# • blind-spots.ts reads the awareness-surfaced signals via the REQUIRED --findings
|
|
406
|
+
# flag (a JSON array STRING or a file path); the optional --awareness-fired marks
|
|
407
|
+
# a fresh run vs a prior-based skip.
|
|
408
|
+
Bash("scripts/cli/run.sh scripts/awareness/llm-sample.ts --metadata /tmp/traces-meta.json --output /tmp/awareness-sample.json")
|
|
409
|
+
Bash("scripts/cli/run.sh scripts/awareness/blind-spots.ts --findings '[\"signal-a\",\"signal-b\"]' --awareness-fired true --output /tmp/blind-spots.json")
|
|
410
|
+
```
|
|
411
|
+
|
|
412
|
+
- `llm-sample.ts` — the awareness deep-read LLM mini-sample. `--metadata` is required;
|
|
413
|
+
the sample size comes from `--awareness-sample <n>` (alias `--size`) and **defaults
|
|
414
|
+
to 50** (V3-1, `AWARENESS_SAMPLE_DEFAULT`) when omitted. RATIONALE: Tier-0 already
|
|
415
|
+
catches LOUD faults; the deep read exists to surface the signals Tier-0 CANNOT see so
|
|
416
|
+
the primary signal is picked from Tier-0 ∪ deep-read. N=5 leaves the deep read too thin
|
|
417
|
+
to surface those other signals. Pass a small N (e.g. `--awareness-sample 5`) only for
|
|
418
|
+
loud-only runs. **Skip ONLY on confirmed priors** (a prior library match for this
|
|
419
|
+
entity, see Step 4.5). The Block-B cascade (expanded discovery + ledger-subtracted
|
|
420
|
+
reserve) activates only when `--run-id` is also passed (optionally with `--entity
|
|
421
|
+
<name>` for the ledger-subtraction marginal-reads path — see Step 6.5).
|
|
422
|
+
- **V3-2 — cascade first-pass seed scales with the ORIGINAL run's depth.** On the
|
|
423
|
+
`--run-id` cascade, when no explicit `--awareness-sample`/`--size` is pinned, the
|
|
424
|
+
first-pass N is `min(originalSampleN, 50)` — where `originalSampleN` is the awareness
|
|
425
|
+
depth the ORIGINAL run (the one named by `--run-id`) actually used — instead of the
|
|
426
|
+
old fixed 12. Recovery reads that run's `run-meta.json.awarenessSampleSize`; pass
|
|
427
|
+
`--config-root <host-root>` (where `.mutagent/diagnostics/` lives) so it can find it.
|
|
428
|
+
When the size can't be recovered (legacy run / unset field) it falls back to **50**
|
|
429
|
+
(NOT 12) and logs the reason to stderr.
|
|
430
|
+
- **Recording (makes recovery work for future cascades):** record the effective
|
|
431
|
+
awareness depth into THIS run's run-meta via `finalizeRunSession(session, {
|
|
432
|
+
awarenessSampleSize: <effective N> })` (new optional `RunMeta.awarenessSampleSize`
|
|
433
|
+
field). A later `--run-id` cascade over this run then recovers it. Legacy runs that
|
|
434
|
+
never recorded it simply take the 50 fallback.
|
|
435
|
+
- `blind-spots.ts` — emits the blind-spot table consumed by Methodology Step 1.5
|
|
436
|
+
in the rendered report. `--findings` is the awareness-surfaced signal list (JSON
|
|
437
|
+
array string or path); pass `--awareness-fired false` on a prior-based skip.
|
|
438
|
+
|
|
439
|
+
### F2 (UI-1op) — ASSIGN the producer output into runMeta (MANDATORY, not just a witness)
|
|
440
|
+
|
|
441
|
+
The witness stamp below proves awareness RAN; it does NOT carry the producer output
|
|
442
|
+
into the report. The awareness mini-sample + blind-spot table are LLM/runtime-produced
|
|
443
|
+
(the `findings`/`discoveredSignals` come from the analyzer agent's LLM pass — the
|
|
444
|
+
deterministic enricher CANNOT recompute them). They reach the Methodology tab ONLY
|
|
445
|
+
when the orchestrator assigns them into `findings.runMeta` BEFORE the Step 8.5 enricher
|
|
446
|
+
call. The enricher PRESERVES `runMeta.awarenessSample` / `runMeta.blindSpots` verbatim
|
|
447
|
+
on its passthrough (it never drops them), but it never invents them.
|
|
448
|
+
|
|
449
|
+
After running awareness (and BEFORE building `/tmp/findings.json` for Step 8.5), assign:
|
|
450
|
+
|
|
451
|
+
```ts
|
|
452
|
+
// awarenessSample: package the selected traces + the analyzer-surfaced signals
|
|
453
|
+
// (LLM output) with an injected timestamp via buildAwarenessSample().
|
|
454
|
+
findings.runMeta.awarenessSample = buildAwarenessSample(
|
|
455
|
+
selectedTraces, // from selectAwarenessTraces() (deterministic)
|
|
456
|
+
discoveredSignals, // from the analyzer agent's LLM mini-sample pass
|
|
457
|
+
awarenessFiredAtIso // injected ISO timestamp
|
|
458
|
+
);
|
|
459
|
+
|
|
460
|
+
// blindSpots: the Tier-0-measurable-vs-blind-spot taxonomy table.
|
|
461
|
+
findings.runMeta.blindSpots = buildBlindSpots({
|
|
462
|
+
awarenessFindings: discoveredSignals,
|
|
463
|
+
awarenessFired: true,
|
|
464
|
+
}).rows;
|
|
465
|
+
```
|
|
466
|
+
|
|
467
|
+
On a **prior-based SKIP** (Step 4.5 match), assign the placeholder instead so the
|
|
468
|
+
Methodology renders the explicit "library priors exist; awareness skipped" panel:
|
|
469
|
+
|
|
470
|
+
```ts
|
|
471
|
+
findings.runMeta.blindSpots = buildBlindSpots({ awarenessFindings: [], awarenessFired: false }).rows;
|
|
472
|
+
// (leave awarenessSample undefined — the renderer's SKIP placeholder handles it)
|
|
473
|
+
```
|
|
474
|
+
|
|
475
|
+
> A bare witness stamp with no `runMeta.awarenessSample` / `runMeta.blindSpots`
|
|
476
|
+
> assignment is the F2 regression: the renderer sees `undefined` and emits the SKIP
|
|
477
|
+
> placeholder even though awareness ran. The assignment above is what makes the
|
|
478
|
+
> anti-tunnel-vision proof VISIBLE (Wave-6 awareness methodology, PR-036).
|
|
479
|
+
|
|
480
|
+
### W11-05: Awareness witness stamp (MANDATORY)
|
|
481
|
+
|
|
482
|
+
After running or consciously skipping awareness, write the awareness-witness stamp:
|
|
483
|
+
|
|
484
|
+
```bash
|
|
485
|
+
# Fresh run — awareness ran:
|
|
486
|
+
writeAwarenessWitnessStamp(reportDir, { isFreshRun: true, awarenessRan: true })
|
|
487
|
+
|
|
488
|
+
# Prior-based skip (confirmed library match from Step 4.5):
|
|
489
|
+
writeAwarenessWitnessStamp(reportDir, {
|
|
490
|
+
isFreshRun: false,
|
|
491
|
+
awarenessRan: false,
|
|
492
|
+
exemptionReason: "library prior exists for this entity (Step 4.5 match)"
|
|
493
|
+
})
|
|
494
|
+
```
|
|
495
|
+
|
|
496
|
+
The Step 8.9 checklist (wave6-checklist.ts) reads this stamp and HARD-FAILS
|
|
497
|
+
(regardless of isClientReport) if isFreshRun=true + awarenessRan=false +
|
|
498
|
+
no exemptionReason. This makes "awareness ran" enforceable, not just documented.
|
|
499
|
+
|
|
500
|
+
## Step 3.6 — Trace Acquisition, STANDALONE runs (V3-9 — delegate-or-artifact)
|
|
501
|
+
|
|
502
|
+
When the skill is invoked **standalone** (a bare `*diagnose` — no Helix, no
|
|
503
|
+
`HandoverBundle.inputs[]` handed in), traces are acquired by EXACTLY ONE of the
|
|
504
|
+
**3 legal acquisition paths** (adl-v3 PRD §4, G-DELEG):
|
|
505
|
+
|
|
506
|
+
| # | Path | Who fetches | When |
|
|
507
|
+
|---|---|---|---|
|
|
508
|
+
| a | **Helix pre-fetch** | Helix, BEFORE dispatch — slice rides in via `HandoverBundle.inputs[]` | orchestrated runs (not this step) |
|
|
509
|
+
| b | **`*discover` dispatch** | the `discovery` system agent | orchestrated collection |
|
|
510
|
+
| c | **Standalone: artifact intake OR vendored-Discovery spawn** | a pre-staged artifact, or the vendored `discovery` sub-agent | THIS step |
|
|
511
|
+
|
|
512
|
+
**The standalone flow (path c) — run in this order:**
|
|
513
|
+
|
|
514
|
+
1. **Artifact intake first.** Check the documented intake location
|
|
515
|
+
`.mutagent/traces/<queryId>/` for an existing `traces.jsonl` (+
|
|
516
|
+
`traces.manifest.json`). If present → consume it (proceed to Step 3.7);
|
|
517
|
+
do NOT spawn Discovery, do NOT re-fetch.
|
|
518
|
+
2. **Else DELEGATE to the vendored Discovery agent.** Spawn the sub-agent this
|
|
519
|
+
skill ships for exactly this purpose — `assets/agents/discovery.md`
|
|
520
|
+
(installed at the project's `.claude/agents/discovery.md`) — via the Task
|
|
521
|
+
tool (`subagent_type: "discovery"`), handing it the TraceQuery you resolved
|
|
522
|
+
at Step 3 (platform + filters + window + `--export
|
|
523
|
+
.mutagent/traces/<queryId>/traces.jsonl`). The agent runs
|
|
524
|
+
`mutagent-cli trace {count,fetch --export}` INSIDE its own context and hands
|
|
525
|
+
back the `traces.jsonl` + `traces.manifest.json` pair. Then proceed to
|
|
526
|
+
Step 3.7 on that handover.
|
|
527
|
+
|
|
528
|
+
> **HARD RULE (the ONE illegal path):** the diagnostics stage itself NEVER runs
|
|
529
|
+
> `mutagent-cli trace fetch` / `trace export` (nor any wrapper of them — `node
|
|
530
|
+
> …/mutagent-cli.mjs`, `bunx`, …) inline in its own session. Fetching is
|
|
531
|
+
> Discovery's job; the stage only READS handovers. Sizing reads
|
|
532
|
+
> (`trace count` / `list` / `chains`) for shaping the query are permitted.
|
|
533
|
+
> Enforcement: SimuLatte SL-6 (G-DELEG) transcript-gates this — an inline
|
|
534
|
+
> main-chain fetch/export is an automatic V0 FAIL naming the tool-call line.
|
|
535
|
+
|
|
536
|
+
## Step 3.7 — Read the UniTF handover (MANDATORY precondition — PRD-MP-01)
|
|
537
|
+
|
|
538
|
+
Traces arrive **already fetched + normalized** as a UniTF JSONL (+ optional
|
|
539
|
+
`manifest.json`), handed in via the `HandoverBundle.inputs[]` — or, on a
|
|
540
|
+
standalone run, via the Step 3.6 delegate-or-artifact flow. Helix produces
|
|
541
|
+
them BEFORE dispatch by running `mutagent-cli trace fetch --export …` (see the
|
|
542
|
+
pre-stage flow in `../../../../mutagent-tools/references/MIGRATION-diagnostics-evaluator.md`
|
|
543
|
+
§4.2). **The skill NEVER fetches and NEVER hand-parses a raw platform export**
|
|
544
|
+
(3-legal-paths hard rule, Step 3.6). Inline `jq`/`sed`/`awk` pre-mapping is
|
|
545
|
+
**FORBIDDEN**.
|
|
546
|
+
|
|
547
|
+
The skill reads the handover via `scripts/normalize/read-unitf.ts`, which
|
|
548
|
+
projects each `UnifiedTrace` record — through `unitf-adapter.ts` — into the
|
|
549
|
+
skill's canonical shapes. It produces the SAME downstream artifacts the pipeline
|
|
550
|
+
has always depended on:
|
|
551
|
+
|
|
552
|
+
1. **`traces-metadata.json`** (`TraceMetadata[]`) — consumed by Tier-0 +
|
|
553
|
+
slicer (Step 4 / Step 5).
|
|
554
|
+
2. **`entity-context.json`** (`EntityContext`) — consumed by the enricher at
|
|
555
|
+
Step 8.5a via `--entity-context`. This is the ONLY authorized producer;
|
|
556
|
+
the enricher must NEVER synthesize `diagnosedEntity` from inline heuristics.
|
|
557
|
+
The projection reads it from the UniTF `ext.agent` block.
|
|
558
|
+
|
|
559
|
+
### The handover inputs (from `HandoverBundle.inputs[]`)
|
|
560
|
+
|
|
561
|
+
| `inputs[]` entry | What |
|
|
562
|
+
|---|---|
|
|
563
|
+
| `kind:"trace", id:"trace-list"` | `traces.jsonl` — one `UnifiedTrace` per line (NDJSON) |
|
|
564
|
+
| `kind:"config", id:"trace-manifest"` | `manifest.json` (`format:"unitf@0.1"`, count, truncated) — OPTIONAL |
|
|
565
|
+
|
|
566
|
+
### Read the handover via the INTERNAL CLI transport
|
|
567
|
+
|
|
568
|
+
`read-unitf.ts` exposes an `import.meta.main` transport (INTERNAL — run.sh only,
|
|
569
|
+
NOT the product `mutagent` CLI). One deterministic call produces BOTH downstream
|
|
570
|
+
artifacts — no inline `jq`/`bun -e` hand-wiring. It is platform-agnostic: every
|
|
571
|
+
source platform (langfuse · local-ndjson · claude-code · codex · otel) already
|
|
572
|
+
went through the CLI's per-platform adapter, so there is a SINGLE read path here:
|
|
573
|
+
|
|
574
|
+
```bash
|
|
575
|
+
Bash("scripts/cli/run.sh scripts/normalize/read-unitf.ts \
|
|
576
|
+
--in <inputs[trace-list].path> \
|
|
577
|
+
--manifest <inputs[trace-manifest].path> \
|
|
578
|
+
--out-metadata /tmp/traces-metadata.json \
|
|
579
|
+
--out-entity /tmp/entity-context.json")
|
|
580
|
+
```
|
|
581
|
+
|
|
582
|
+
- `--in` — the handed-over UniTF `.jsonl` (one `UnifiedTrace` per line). Blank
|
|
583
|
+
lines skipped; non-UniTF / unparseable lines are tolerated-but-visible
|
|
584
|
+
(dropped-count + samples to stderr), mirroring the old local-jsonl tolerance.
|
|
585
|
+
- `--manifest` — OPTIONAL. Format / count / truncation mismatches surface as
|
|
586
|
+
WARNINGS to stderr but NEVER abort a partial export (partial-load behavior).
|
|
587
|
+
- `--out-metadata` — `TraceMetadata[]` (Tier-0 + slicer input).
|
|
588
|
+
- `--out-entity` — the `EntityContext` (consumed verbatim at Step 8.5a).
|
|
589
|
+
- ≥1 `--out-*` is required. Deterministic — no clock/random/network/LLM.
|
|
590
|
+
|
|
591
|
+
This is the AUTHORIZED producer of `entity-context.json`. `diagnosedEntity` is
|
|
592
|
+
never hand-wired and never silently dropped (the OP-7 guarantee is preserved
|
|
593
|
+
across the UniTF flip).
|
|
594
|
+
|
|
595
|
+
### Fetching moved UP to the CLI — discover the surface
|
|
596
|
+
|
|
597
|
+
The skill no longer owns any per-platform fetch/normalize code (those adapters
|
|
598
|
+
now live + are tested in `@mutagent/tools`). Agents/operators who need to see how
|
|
599
|
+
the JSONL was produced — or produce one manually for local dev — run:
|
|
600
|
+
|
|
601
|
+
```bash
|
|
602
|
+
mutagent-cli trace --help # discover the fetch/count/export surface
|
|
603
|
+
mutagent-cli trace count --platform <p> --agent <name> --since <window>
|
|
604
|
+
mutagent-cli trace fetch --platform <p> --agent <name> --since <window> \
|
|
605
|
+
--export .mutagent/traces/<queryId>/traces.jsonl # writes traces.jsonl + manifest.json
|
|
606
|
+
```
|
|
607
|
+
|
|
608
|
+
> ⚠️ This cheat-sheet is for HUMANS and the Discovery agent — it is **NOT a
|
|
609
|
+
> license for the running diagnostics stage to fetch inline**. During a run,
|
|
610
|
+
> acquisition goes through Step 3.6 (artifact intake or Discovery dispatch) —
|
|
611
|
+
> the stage running `trace fetch`/`trace export` itself is the ONE illegal
|
|
612
|
+
> path (V3-9 / G-DELEG).
|
|
613
|
+
|
|
614
|
+
Cross-reference: Wave-5 R1.7 / APPENDIX-A §A.2 — "diagnosedEntity must come
|
|
615
|
+
from the read-unitf EntityContext projection". Step 8.5a (below) is the
|
|
616
|
+
downstream consumer. The self-diag skill-typed entity (Step 12.2) is built by
|
|
617
|
+
`buildSkillSelfEntityContext` in `scripts/normalize/platforms/entity-context.ts`
|
|
618
|
+
(kept — it reads the skill's own SKILL.md + session, not a platform export).
|
|
619
|
+
|
|
620
|
+
## Step 4 — Tier 0 Static Scan (PR-001 — MANDATORY before LLM)
|
|
621
|
+
|
|
622
|
+
```bash
|
|
623
|
+
Bash("scripts/cli/run.sh scripts/tier0-scan.ts /tmp/mutagent-traces.json > /tmp/tier0-out.json")
|
|
624
|
+
```
|
|
625
|
+
|
|
626
|
+
**Do NOT call analyzers until Tier 0 completes.**
|
|
627
|
+
|
|
628
|
+
Tier 0 output: `hasErrorSpike`, `hasLatencySpike`, `hasFeedbackCluster`,
|
|
629
|
+
`estimatedSlots`, `slicingStrategy`.
|
|
630
|
+
|
|
631
|
+
## Step 4.5 — Library-First Match (Wave-6 R2.3 — Best-Effort Prior Consult)
|
|
632
|
+
|
|
633
|
+
> **W9-02**: Header changed MANDATORY → "Best-Effort Prior Consult". An empty library
|
|
634
|
+
> is valid — proceed fresh + log `runMeta.decisions`. The deep-read gate (Step 6 pre-gate)
|
|
635
|
+
> is UNTOUCHED: empty library → no `priorSignalsRef` → HARD-REQUIRED deep-read (PR-035
|
|
636
|
+
> cross-ref). De-mandate does NOT relax the deep-read floor.
|
|
637
|
+
>
|
|
638
|
+
> Pre-flight caps (`scripts/sample/caps.ts`) run at Step 5.5 BEFORE any Step-5b
|
|
639
|
+
> override; Step 4.5 cannot bypass them.
|
|
640
|
+
|
|
641
|
+
Before slicing, consult the diagnostics library for confirmed priors on this
|
|
642
|
+
entity and merge them into the Tier-0 report (library priors weighted 3×):
|
|
643
|
+
|
|
644
|
+
```bash
|
|
645
|
+
Bash("scripts/cli/run.sh scripts/library/match.ts /tmp/tier0-out.json --entity <name>")
|
|
646
|
+
# → merges library priors (3× weight) into the Tier-0 report's libraryMatches[]
|
|
647
|
+
```
|
|
648
|
+
|
|
649
|
+
A confirmed library prior here is what authorizes skipping the Step 3.5
|
|
650
|
+
`llm-sample.ts` mini-sample.
|
|
651
|
+
|
|
652
|
+
**GUARD (W9-02):** Mandate library-match BEFORE dispatch. If both `priors==0`
|
|
653
|
+
AND `llmReads==0`, FAIL-LOUD — this is the double-zero anti-laziness gate.
|
|
654
|
+
Empty library alone is fine (proceed fresh); double-zero is not.
|
|
655
|
+
|
|
656
|
+
```yaml
|
|
657
|
+
# Step Spec (W9-04)
|
|
658
|
+
stage:
|
|
659
|
+
id: step-4.5-library-match
|
|
660
|
+
gate: { mandatory: false } # W9-02 de-mandate — empty library is valid
|
|
661
|
+
inputs: { tier0_report: /tmp/tier0-out.json, entity_name: "{name}" }
|
|
662
|
+
run: "scripts/cli/run.sh scripts/library/match.ts /tmp/tier0-out.json --entity {name}"
|
|
663
|
+
outputs: { library_matches: "libraryMatches[] # 3x weighted; may be empty" }
|
|
664
|
+
on_result:
|
|
665
|
+
matches_present: { action: merge-priors-3x, side_effect: "authorizes reducing (NOT skipping) the deep-read floor" }
|
|
666
|
+
matches_empty: { action: proceed-fresh, route: step-5,
|
|
667
|
+
# W13-C (D-5): canonical decision shape is {step, choice, rationale, timestamp}
|
|
668
|
+
# (scripts/normalize/trace.ts RunMeta.decisions) — NOT {stepId, decision, reason}.
|
|
669
|
+
log: { target: runMeta.decisions, record: { step: "4.5-library-match", choice: "proceed-fresh (0 priors)", rationale: "library returned no matches for this entity" } } }
|
|
670
|
+
on_fail: { action: skip-and-log, route: step-5 }
|
|
671
|
+
# Step 6 deep-read pre-gate is the mandatory:true exemplar:
|
|
672
|
+
# gate: { mandatory: true }; on_fail: { action: HARD-REFUSE, route: "exemption (rca.md) OR halt+ask" }
|
|
673
|
+
# W9-02 GUARD: library de-mandate does NOT touch deep-read-gate.ts. Empty library =>
|
|
674
|
+
# no priorSignalsRef => deep-read HARD-REQUIRED (PR-035 cross-ref).
|
|
675
|
+
```
|
|
676
|
+
|
|
677
|
+
## Step 5 — Dynamic-Cluster Slicing (PR-017)
|
|
678
|
+
|
|
679
|
+
```bash
|
|
680
|
+
Bash("scripts/cli/run.sh scripts/slicer.ts /tmp/tier0-out.json /tmp/mutagent-traces.json --cap 5
|
|
681
|
+
> /tmp/slice-plan.json")
|
|
682
|
+
```
|
|
683
|
+
|
|
684
|
+
Slice plan: array of `{ sliceId, traceIds[], scope }` objects, capped at 5 slices.
|
|
685
|
+
|
|
686
|
+
## Step 5b — Slicer-Override Decision (PRD-MP-02)
|
|
687
|
+
|
|
688
|
+
After the slicer emits its plan, evaluate the override predicate:
|
|
689
|
+
|
|
690
|
+
**Predicate:** `slice-plan.rationale` starts with
|
|
691
|
+
`"Window-based slicing (no a-priori signal)"` **AND**
|
|
692
|
+
`parsedInvocation.focus` (from Step 3a) is non-null.
|
|
693
|
+
|
|
694
|
+
When the predicate fires:
|
|
695
|
+
|
|
696
|
+
1. Switch sampling strategy to `representative.ts` 4-bucket sample
|
|
697
|
+
(worst · median · best · random, 15-floor, worst-weighted) instead
|
|
698
|
+
of the window plan.
|
|
699
|
+
2. Record the override in `runMeta`:
|
|
700
|
+
```json
|
|
701
|
+
{
|
|
702
|
+
"samplingOverride": {
|
|
703
|
+
"from": "window-based",
|
|
704
|
+
"to": "representative-sample",
|
|
705
|
+
"reason": "focus=<parsedInvocation.focus>"
|
|
706
|
+
}
|
|
707
|
+
}
|
|
708
|
+
```
|
|
709
|
+
3. The Methodology tab Decisions row MUST surface this override.
|
|
710
|
+
|
|
711
|
+
**When focus is null**: keep the slicer's window plan unchanged.
|
|
712
|
+
Window-based slicing is the correct fallback for unfocused exploratory runs.
|
|
713
|
+
|
|
714
|
+
> **B4 consumer-boundary shape note (verified against `scripts/sample/representative.ts`).**
|
|
715
|
+
> When the override fires, `representative.ts` returns a `BucketSample` whose
|
|
716
|
+
> `selected` is a `BucketAssignment[]` — each element is the OBJECT
|
|
717
|
+
> `{ traceId, bucket, badness }` (e.g. `{ traceId: "tr-…", bucket: "worst", badness: 0.82 }`),
|
|
718
|
+
> **NOT** a bare trace-id string. Any Step-5b/6 consumer that needs the id must read
|
|
719
|
+
> `assignment.traceId` (e.g. `sample.selected.map(a => a.traceId)`); treating an element
|
|
720
|
+
> as a string yields `undefined` ids downstream. (`bucket` ∈ worst·median·best·random;
|
|
721
|
+
> `badness` is the deterministic per-trace score, surfaced for the determinism proof.)
|
|
722
|
+
|
|
723
|
+
> Earning evidence: F-SD-3 — a prior diagnostic run overrode the
|
|
724
|
+
> 5×390-window plan with hand-rolled `jq` with no protocol home for
|
|
725
|
+
> the decision.
|
|
726
|
+
|
|
727
|
+
## Step 5.5 — Caps (Wave-6 D1 — first-to-trip stop)
|
|
728
|
+
|
|
729
|
+
Evaluate the configured caps before dispatch; the first cap to trip halts further
|
|
730
|
+
sampling/dispatch:
|
|
731
|
+
|
|
732
|
+
```bash
|
|
733
|
+
Bash("scripts/cli/run.sh scripts/sample/caps.ts --config <caps>")
|
|
734
|
+
```
|
|
735
|
+
|
|
736
|
+
Inactive caps (e.g. cost cap, default OFF per D1) are skipped.
|
|
737
|
+
|
|
738
|
+
> **Operator override (PR-048):** the max-trace ceiling defaults to `min(N, 1000)`. To let a
|
|
739
|
+
> run read beyond 1000 traces, pass `--max-trace <N>` (operator-explicit only, never auto) — it
|
|
740
|
+
> raises the active `max_trace_count` ceiling on command and threads into `computeCeiling(N, override)`
|
|
741
|
+
> at the Step 6 escalation loop.
|
|
742
|
+
|
|
743
|
+
## Step 5.7 — Build Diagnosis-Context Lens (W18-context — MANDATORY before analyzer fan-out)
|
|
744
|
+
|
|
745
|
+
> **⚠ Runs BEFORE the Step 6 analyzer fan-out.** Each analyzer MANDATORY-PRE-READs the
|
|
746
|
+
> `diagnosis-context.md` this step produces (see Step 6 dispatch). Build it once per run;
|
|
747
|
+
> every parallel analyzer reads the same lens.
|
|
748
|
+
|
|
749
|
+
**WHY.** An analyzer that starts blind searches the traces with no factual baseline for
|
|
750
|
+
"what IS this thing?" — it can confabulate a failure mode that contradicts the entity's
|
|
751
|
+
own system prompt / tools / source. The diagnosis-context is a GROUNDED LENS (name · scope
|
|
752
|
+
· model · purpose · FULL system prompt · tools · source code when accessible) the analyzer
|
|
753
|
+
reads + understands FIRST, so every finding corroborates against extracted fact, not a
|
|
754
|
+
guess. It is **EXTRACTED FACT ONLY** — no inferred/derived claims, no "prompt is uncached",
|
|
755
|
+
no "latency caused by X". The analyzer corroborates whatever it is handed, so an unverified
|
|
756
|
+
hint becomes a self-fulfilling error (grounding contract — PR-018, now also PR-055
|
|
757
|
+
"Ground cost/cache + intent, never infer" in `.meta/design-principles.md`).
|
|
758
|
+
|
|
759
|
+
Build it with the deterministic (no-LLM) assembler. It consumes the `entity-context.json`
|
|
760
|
+
produced at Step 3.7 + the normalized traces, and writes `diagnosis-context.md`:
|
|
761
|
+
|
|
762
|
+
```bash
|
|
763
|
+
# Verified against scripts/context/build-diagnosis-context.ts import.meta.main:
|
|
764
|
+
# REQUIRED: --entity-context <f> --traces <f> --output <diagnosis-context.md>
|
|
765
|
+
# OPTIONAL: --purpose <text> (operator-stated lens; labeled "operator-stated", never trace-derived)
|
|
766
|
+
# OPTIONAL: --doc <label>:<path> (REPEATABLE — embeds a source file VERBATIM; the codeAccess case)
|
|
767
|
+
Bash("scripts/cli/run.sh scripts/context/build-diagnosis-context.ts \
|
|
768
|
+
--entity-context /tmp/entity-context.json \
|
|
769
|
+
--traces /tmp/normalized-traces.json \
|
|
770
|
+
--output /tmp/diagnosis-context.md \
|
|
771
|
+
[--purpose \"<operator-stated purpose>\"] \
|
|
772
|
+
[--doc SKILL.md:.claude/skills/mutagent-diagnostics/SKILL.md] \
|
|
773
|
+
[--doc references/principles.md:.claude/skills/mutagent-diagnostics/references/principles.md]")
|
|
774
|
+
```
|
|
775
|
+
|
|
776
|
+
- `--entity-context` / `--traces` / `--output` are **REQUIRED** — the script exits non-zero
|
|
777
|
+
(usage message) if any is missing.
|
|
778
|
+
- `--purpose` is OPTIONAL — when supplied it renders under the `## Purpose
|
|
779
|
+
_(provenance: operator-stated)_` section, explicitly labeled stated-intent, NOT a
|
|
780
|
+
trace-verified fact. Omit it and the section records "no operator-stated purpose supplied".
|
|
781
|
+
- `--doc <label>:<path>` is OPTIONAL + **REPEATABLE** (split on the FIRST colon — the label is
|
|
782
|
+
the leading token). Each file is read + PII-sanitized + embedded VERBATIM under `## Source
|
|
783
|
+
Code _(provenance: source-code)_`. Supply the entity's own source (skill → SKILL.md + key
|
|
784
|
+
references; agent → the agent definition) for the codeAccess case; omit entirely for the
|
|
785
|
+
no-codeAccess client case (the FULL extracted system prompt is then the primary ground truth).
|
|
786
|
+
|
|
787
|
+
The script is **Type A — Pure** (no LLM, no clock, no random): same inputs ⇒ byte-identical
|
|
788
|
+
`diagnosis-context.md`. The rendered sections, in fixed order, are: **Identity → Purpose →
|
|
789
|
+
System Prompt (FULL, untruncated) → Tool Inventory → Source Code**, each carrying a provenance
|
|
790
|
+
badge (`trace-extracted` · `source-code` · `operator-stated`) so every fact is traceable to
|
|
791
|
+
its origin. Anything that cannot be grounded is **OMITTED** — there is deliberately no field
|
|
792
|
+
for inferred/derived claims.
|
|
793
|
+
|
|
794
|
+
## Step 6 — Parallel Analyzer Dispatch (PR-005 — cap ≤ 5)
|
|
795
|
+
|
|
796
|
+
### Step 6 pre-gate — Deep-Read Gate (Wave-6 R2.1 — MANDATORY before dispatch)
|
|
797
|
+
|
|
798
|
+
BEFORE dispatching ANY analyzer, run the deep-read gate:
|
|
799
|
+
|
|
800
|
+
```bash
|
|
801
|
+
Bash("scripts/cli/run.sh scripts/sample/deep-read-gate.ts ...")
|
|
802
|
+
```
|
|
803
|
+
|
|
804
|
+
The gate may **HARD-REFUSE** (e.g. zero LLM reads with no prior signals). On
|
|
805
|
+
refusal, either route to a documented exemption (see `references/workflows/rca.md`
|
|
806
|
+
Gate Exemption Taxonomy) or HALT and ASK the operator. **Never** bypass the gate
|
|
807
|
+
with inline prose reasoning.
|
|
808
|
+
|
|
809
|
+
```yaml
|
|
810
|
+
# Step Spec — Step 6 deep-read pre-gate (W9-04)
|
|
811
|
+
stage:
|
|
812
|
+
id: step-6-deep-read-gate
|
|
813
|
+
gate: { mandatory: true }
|
|
814
|
+
inputs: { slice_plan: /tmp/slice-plan.json, caps_config: "<caps>" }
|
|
815
|
+
run: "scripts/cli/run.sh scripts/sample/deep-read-gate.ts ..."
|
|
816
|
+
outputs: { verdict: "DeepReadGateVerdict # {allow, tooThin, coverageWarning}" }
|
|
817
|
+
on_fail: { action: HARD-REFUSE, route: "exemption (rca.md Gate Exemption Taxonomy) OR halt+ask" }
|
|
818
|
+
# mandatory:false exemplar is Step 4.5 above.
|
|
819
|
+
```
|
|
820
|
+
|
|
821
|
+
> **Too-thin guard (W9-07 / PR-048):** when `verdict.tooThin === true` (population ≥ 1000
|
|
822
|
+
> AND `llmReadCount < 100`), surface the `coverageWarning` banner in the rendered report.
|
|
823
|
+
> This is a warning, NOT a refusal — the escalation loop below remediates it.
|
|
824
|
+
|
|
825
|
+
### Step 6 — Trace-Hungry Escalation Loop (W9-09 — PR-048)
|
|
826
|
+
|
|
827
|
+
After the deep-read gate passes, execute the tiered escalation loop using B1 symbols
|
|
828
|
+
(`ESCALATION_RUNGS`, `computeCeiling`, `timeBudgetForTier`, `buildBatchSample`,
|
|
829
|
+
`isSufficient` from `scripts/sample/caps.ts` + `scripts/sample/representative.ts`):
|
|
830
|
+
|
|
831
|
+
```
|
|
832
|
+
# DEFAULT ceiling = min(population, 1000). OPERATOR OVERRIDE (--max-trace <N>): pass it as the
|
|
833
|
+
# 2nd arg to RAISE the ceiling ABOVE 1000 ON COMMAND (operator-explicit only, never auto).
|
|
834
|
+
ceiling = computeCeiling(population, maxTraceOverride?) # min(population, override ?? 1000)
|
|
835
|
+
|
|
836
|
+
# ESCALATION_RUNGS = [50, 100, 250, 500, 1000] — DIP→RAMP. The leading 50 is the cheap DIP
|
|
837
|
+
# first probe: sip 50 traces, then ramp through 100·250·500·1000 only if evidence stays thin.
|
|
838
|
+
for tier in ESCALATION_RUNGS where tier <= ceiling:
|
|
839
|
+
batch = buildBatchSample(population, tier) # worst-weighted; reuses representative.ts buckets
|
|
840
|
+
LLM-read the batch traces
|
|
841
|
+
update findings + append confirmed patterns to library/tier0 detectable-pattern set
|
|
842
|
+
record batches[] entry in runMeta.deepRead
|
|
843
|
+
|
|
844
|
+
if isSufficient(proof, newFailureCategoriesInLastBatch):
|
|
845
|
+
stopReason = "evidence-sufficient"; break
|
|
846
|
+
if elapsed >= timeBudgetForTier(tier): # per-tier {50:300,100:600,250:900,500:1200,1000:1800}
|
|
847
|
+
stopReason = "time-budget"; break
|
|
848
|
+
|
|
849
|
+
else:
|
|
850
|
+
stopReason = "ceiling-reached"
|
|
851
|
+
|
|
852
|
+
# Small stacks: N < 50 (the dip first-rung) => single representative sample at floor 15..N; no forced escalation.
|
|
853
|
+
# Anti-laziness (R-AL-2): population >= 1000 AND llmReadCount < 100 => FLAG / refuse.
|
|
854
|
+
```
|
|
855
|
+
|
|
856
|
+
Record outcome in `runMeta.deepRead`:
|
|
857
|
+
```json
|
|
858
|
+
{
|
|
859
|
+
"deepRead": {
|
|
860
|
+
"population": <N>,
|
|
861
|
+
"tierReached": <tier>,
|
|
862
|
+
"llmReadCount": <count>,
|
|
863
|
+
"coverageConfidence": "high|medium|low",
|
|
864
|
+
"stopReason": "evidence-sufficient|ceiling-reached|time-budget",
|
|
865
|
+
"batches": [{ "tier": 100, "newFailureCategories": 3, "coverageConfidence": "low" }, ...]
|
|
866
|
+
}
|
|
867
|
+
}
|
|
868
|
+
```
|
|
869
|
+
|
|
870
|
+
See `scripts/normalize/trace.ts` (`RunMeta.deepRead`) for the type definition landed by B1.
|
|
871
|
+
|
|
872
|
+
### Step 6 — Analyzer Dispatch via Handover Contract (W9-01)
|
|
873
|
+
|
|
874
|
+
For each slice in `slice-plan.json`, render the `analyzer_dispatch` YAML block from
|
|
875
|
+
`references/workflows/handover-contract.md` (fill all `{templated}` fields) and dispatch:
|
|
876
|
+
|
|
877
|
+
```
|
|
878
|
+
Agent(
|
|
879
|
+
subagent_type: "diagnostics-analyzer",
|
|
880
|
+
run_in_background: true,
|
|
881
|
+
prompt: |
|
|
882
|
+
<rendered analyzer_dispatch YAML from references/workflows/handover-contract.md>
|
|
883
|
+
)
|
|
884
|
+
```
|
|
885
|
+
|
|
886
|
+
The handover contract replaces freeform per-run prompts. Every parallel analyzer
|
|
887
|
+
in a run receives the SAME structured brief — no drift.
|
|
888
|
+
|
|
889
|
+
**W18-context — MANDATORY PRE-READ of `diagnosis-context.md` (every analyzer, before trace analysis).**
|
|
890
|
+
The analyzer brief MUST carry the `diagnosis-context.md` path built at Step 5.7, and the
|
|
891
|
+
analyzer's FIRST step is to **read + understand the diagnosis context to set its failure-mode
|
|
892
|
+
lens BEFORE analyzing traces.** This is non-optional: an analyzer that starts blind can surface
|
|
893
|
+
a failure mode that contradicts the entity's own (extracted) system prompt / tools / source.
|
|
894
|
+
The lens is extracted FACT only — the analyzer corroborates against it; it must NEVER seed or
|
|
895
|
+
corroborate an UNVERIFIED hint from it (there are no hints in it by construction). See
|
|
896
|
+
`references/workflows/handover-contract.md` (`artifacts_in.diagnosis_context`) for the brief
|
|
897
|
+
field and `assets/agents/diagnostics-analyzer.md` (Step 0 — read+understand) for the analyzer's
|
|
898
|
+
pre-read step.
|
|
899
|
+
|
|
900
|
+
**W18-cache — cache-status detection rule (NEVER infer "uncached").** When an analyzer reasons
|
|
901
|
+
about prompt-caching, cost, or token volume, the cache state comes ONLY from the grounded
|
|
902
|
+
cache-token fields on each trace (`cacheStatus` ∈ `hit`|`miss`|`unknown`, derived from
|
|
903
|
+
`cachedInputTokens` / `cacheCreationTokens`; `cacheHitRate` populated only when cache fields
|
|
904
|
+
were present). Rules:
|
|
905
|
+
- `cacheStatus` **absent / `"unknown"`** ⇒ report cache state as **UNKNOWN**, NEVER as
|
|
906
|
+
"uncached". Absence of a cache-token field is NOT evidence of no caching.
|
|
907
|
+
- **NEVER infer** caching (or its absence) from a flat `promptTokens` value or byte sizes.
|
|
908
|
+
The motivating miss: a client agent's caching was active ~89%, but a byte-size inference
|
|
909
|
+
reported "uncached → 408M billed tokens". `miss` is a GROUNDED no-cache-read (cache fields
|
|
910
|
+
present, nothing served) and is distinct from `unknown` — never collapse the two.
|
|
911
|
+
|
|
912
|
+
See `scripts/normalize/trace.ts` (`cacheStatus` / `cachedInputTokens` / `cacheCreationTokens`
|
|
913
|
+
/ `cacheHitRate` + the `CacheStatus` type) for the field definitions.
|
|
914
|
+
|
|
915
|
+
**W13-C (Variance, RC-LLM-PIN) — pin model + temperature on every analyzer dispatch.**
|
|
916
|
+
Unpinned model/temperature was the DOMINANT inter-run variance lever (the same traces
|
|
917
|
+
produced different findings each run). Every analyzer (and the orchestrator's own RCA
|
|
918
|
+
reasoning at Step 8) MUST run under the pinned inference envelope declared in
|
|
919
|
+
`assets/agents/diagnostics-analyzer.md` frontmatter (`inference:`):
|
|
920
|
+
- `temperature: 0` — PINNED unconditionally (deterministic sampling, host-agnostic);
|
|
921
|
+
- `model` — explicit DEFAULT pin (`claude-sonnet-4-6`), configurable per dispatch.
|
|
922
|
+
|
|
923
|
+
Honoring `feedback_model_intent_sacred`: this DECLARES intent — no silent swap, no
|
|
924
|
+
context-optimized routing, no retry-on-failure alternate-model fallback. If the
|
|
925
|
+
orchestrator overrides `model` for a run, the override MUST be explicit and logged to
|
|
926
|
+
`runMeta.decisions` (`step: "6-dispatch"`, the chosen model named in `rationale`).
|
|
927
|
+
Never re-target implicitly; if the pinned model cannot satisfy a constraint, THROW.
|
|
928
|
+
|
|
929
|
+
**Remedy-field contract (W12-08, PR-052 proposed + W13-C D-1 — enforced at Step 7.1).** Every
|
|
930
|
+
remedy the analyzer returns MUST:
|
|
931
|
+
- **link to a target** — `applyTarget` is HARD-required (a code location, or per
|
|
932
|
+
target platform the agent prompt / agent definition) — plus `targetClass`;
|
|
933
|
+
- **focus on an origin** — `rationale`, `whyWorks`, `applyInstructions` (≥1) all
|
|
934
|
+
required, and the finding carries ≥1 structured `assumption`;
|
|
935
|
+
- **declare cost + correctness** — `cost` and `correctness` are REQUIRED
|
|
936
|
+
categoricals (`low|medium|high`). They drive the renderer's header badges AND the
|
|
937
|
+
deterministic `rank` derivation (see Step 8). The analyzer does NOT emit `rank` —
|
|
938
|
+
it is enricher-derived (`scripts/enrich/rank-remedies.ts`); any emitted rank is
|
|
939
|
+
overwritten.
|
|
940
|
+
- **cite the source when findable** — emit a real `diff` (Before/After) when the
|
|
941
|
+
source is accessible; ELSE set `diffStatus ∈ {source-unavailable,
|
|
942
|
+
origin-unknown}`. NEVER fabricate a diff (honors `feedback_model_intent_sacred`).
|
|
943
|
+
|
|
944
|
+
These are the same fields `findings-contract.ts` rejects at Step 7.1; emitting them
|
|
945
|
+
up front avoids a RESEND round-trip.
|
|
946
|
+
|
|
947
|
+
**Never dispatch more than 5 analyzers in parallel.** Log dispatched count.
|
|
948
|
+
|
|
949
|
+
### Step 6 — Single-shot vs fan-out decision rule (PRD-MP-03)
|
|
950
|
+
|
|
951
|
+
Before dispatching, determine analyzer count using these criteria in order:
|
|
952
|
+
|
|
953
|
+
| Criterion | Rule | Preferred count |
|
|
954
|
+
|---|---|---|
|
|
955
|
+
| (a) slicer rationale = `"window-based-no-signal"` AND focus supplied | Step 5b override fired | **1** (single-shot on the representative sample) |
|
|
956
|
+
| (b) All slices share identical scope (no signal variance across slices) | Fan-out adds no coverage | **1** |
|
|
957
|
+
| (c) Sample size ≤ 30 traces total | Single analyzer fits in one context window | **1** |
|
|
958
|
+
| (d) 30 < sample ≤ 50 | Parallel re-read divergence improves coverage | **2** |
|
|
959
|
+
| (e) Default (> 50, diverse slices) | Dispatch up to min(sliceCount, 5) | **min(sliceCount, 5)** |
|
|
960
|
+
|
|
961
|
+
Record the choice in `runMeta`:
|
|
962
|
+
```json
|
|
963
|
+
{
|
|
964
|
+
"dispatch": {
|
|
965
|
+
"analyzerCount": 1,
|
|
966
|
+
"reason": "criterion (b): all slices share identical scope",
|
|
967
|
+
"slicesUsed": ["slice-0", "slice-1"]
|
|
968
|
+
}
|
|
969
|
+
}
|
|
970
|
+
```
|
|
971
|
+
|
|
972
|
+
Single-shot runs MUST cite which criterion fired. The Methodology tab Decisions
|
|
973
|
+
row surfaces `runMeta.dispatch`.
|
|
974
|
+
|
|
975
|
+
**If Agent tool dispatch fails at runtime** (subagent_type not registered):
|
|
976
|
+
fall back to `subagent_type: "general-purpose"` with the content of
|
|
977
|
+
`assets/agents/diagnostics-analyzer.md` inlined as the system prompt body.
|
|
978
|
+
Log `fallback_used: true` in `run-meta.json`.
|
|
979
|
+
|
|
980
|
+
After dispatch, use Monitor tool to await completion:
|
|
981
|
+
```
|
|
982
|
+
Monitor: until test -f /tmp/findings-all.json; do sleep 2; done
|
|
983
|
+
```
|
|
984
|
+
|
|
985
|
+
**Do NOT** use `Bash("sleep N && cat <file>")` — hits harness Blocked:sleep guard.
|
|
986
|
+
|
|
987
|
+
## Step 6.5 — Selection-Hub Context Assembly (Wave-17 — `SignalCensusContext`)
|
|
988
|
+
|
|
989
|
+
After deep-read produces the sampled trace bodies + the analyzer `Finding[]`, the
|
|
990
|
+
orchestrator assembles the **`SignalCensusContext`** that turns deep-read-DISCOVERED
|
|
991
|
+
signals into FIRST-CLASS, evidence-floored census candidates. This is the seam that
|
|
992
|
+
makes the Wave-17 selection flow run end-to-end: the context is built HERE and passed
|
|
993
|
+
to the enricher (Step 8.5) via `--signal-ctx`. Absent → safe-by-default (the enricher
|
|
994
|
+
runs the legacy Tier-0-only census; any discovered signals stay
|
|
995
|
+
*suspected-unconfirmed*, never silently crowned).
|
|
996
|
+
|
|
997
|
+
The context object (consumed by `buildSignalCensus` in
|
|
998
|
+
`scripts/enrich/build-render-input.ts`) has THREE optional fields, each from a distinct
|
|
999
|
+
deterministic source:
|
|
1000
|
+
|
|
1001
|
+
```ts
|
|
1002
|
+
interface SignalCensusContext {
|
|
1003
|
+
corroborations?: TrajectoryCorroboration[]; // R1 evidence floor (mechanical)
|
|
1004
|
+
foldedDigests?: DeepReadLedgerEntry[]; // R2 cross-run ledger (re-floored)
|
|
1005
|
+
sampledCount?: number; // R6 honest prevalence denominator
|
|
1006
|
+
}
|
|
1007
|
+
```
|
|
1008
|
+
|
|
1009
|
+
**(a) `corroborations` — mechanical trajectory scan (R1 evidence floor).**
|
|
1010
|
+
Run the trajectory CLI over the sampled deep-read bodies. It is a pure, deterministic
|
|
1011
|
+
tool-call sequence analyzer (NO LLM, no clock, no random):
|
|
1012
|
+
|
|
1013
|
+
```bash
|
|
1014
|
+
# Verified against scripts/scan/trajectory.ts import.meta.main:
|
|
1015
|
+
# --bodies <traceBodies.json> is REQUIRED (a single TraceBody OR a TraceBody[]).
|
|
1016
|
+
# --out <path> is OPTIONAL — corroborations are written there, else to stdout.
|
|
1017
|
+
Bash("scripts/cli/run.sh scripts/scan/trajectory.ts --bodies /tmp/sampled-bodies.json --out /tmp/corroborations.json")
|
|
1018
|
+
# → TrajectoryCorroboration[] ([{ signal, evidenceRef }, …])
|
|
1019
|
+
```
|
|
1020
|
+
|
|
1021
|
+
Each corroboration pairs a coarse WHY-style `signal` (`loop/latency` · `tool-misuse` ·
|
|
1022
|
+
`handoff-loss` · `prompt-underspec`, per the LOCKED `PATTERN_SIGNAL_MAP`) with an
|
|
1023
|
+
`evidenceRef` pointing at the concrete trace span(s) (`trace:<id>#msg[i,j,…]`). A
|
|
1024
|
+
discovered signal is promoted to **PRIMARY only if** a corroboration maps to its WHAT
|
|
1025
|
+
**and** the `evidenceRef` resolves — LLM assertion alone never reaches PRIMARY (it is
|
|
1026
|
+
capped at SECONDARY). This is the **evidence-cited floor**: mechanical trajectory
|
|
1027
|
+
corroboration, not the analyzer's say-so, is what earns PRIMARY.
|
|
1028
|
+
|
|
1029
|
+
**(b) `foldedDigests` — cross-run ledger fold (R2).**
|
|
1030
|
+
Compute the still-VALID cross-run deep-read digests for the diagnosed entity:
|
|
1031
|
+
|
|
1032
|
+
```ts
|
|
1033
|
+
// scripts/library/store.ts — fold the per-entity deep-read-ledger.json down to the
|
|
1034
|
+
// entries that are still valid (analyzerVersion + entityFingerprint + TTL match).
|
|
1035
|
+
const foldedDigests = foldValidDigests(entityName, {
|
|
1036
|
+
analyzerVersion, // bump invalidates prior digests
|
|
1037
|
+
entityFingerprint, // entity changed under it → invalid
|
|
1038
|
+
nowMs: Date.now(), // injected clock (deterministic in tests)
|
|
1039
|
+
// ttlMs defaults to DEEP_READ_LEDGER_TTL_MS (~30d)
|
|
1040
|
+
});
|
|
1041
|
+
```
|
|
1042
|
+
|
|
1043
|
+
A folded digest is **re-admitted as a discovered candidate but MUST re-pass the SAME
|
|
1044
|
+
evidence floor** (its `evidenceRef` must still resolve + corroborate) — the ledger
|
|
1045
|
+
carries version-stamped digests + validity only; it NEVER decides promotion. This is
|
|
1046
|
+
how a discovered signal seen on a PRIOR run can re-surface without being trusted blindly.
|
|
1047
|
+
|
|
1048
|
+
**(c) `sampledCount` — honest prevalence denominator (R6).**
|
|
1049
|
+
The count of traces actually deep-read this run. A discovered primary's census measure
|
|
1050
|
+
reads *"seen in k/n sampled"* against THIS `n` — never a fabricated corpus rate over the
|
|
1051
|
+
full window. Defaults to the window total when absent (best-effort).
|
|
1052
|
+
|
|
1053
|
+
**Assemble + thread.** The orchestrator writes the assembled object to a temp file and
|
|
1054
|
+
passes it to the enricher at Step 8.5:
|
|
1055
|
+
|
|
1056
|
+
```json
|
|
1057
|
+
// /tmp/signal-ctx.json
|
|
1058
|
+
{
|
|
1059
|
+
"corroborations": [ { "signal": "tool-misuse", "evidenceRef": "trace:tr-7#msg[4,5]" } ],
|
|
1060
|
+
"foldedDigests": [ /* DeepReadLedgerEntry[] from foldValidDigests */ ],
|
|
1061
|
+
"sampledCount": 18
|
|
1062
|
+
}
|
|
1063
|
+
```
|
|
1064
|
+
|
|
1065
|
+
> **Selection-flow summary (Wave-17).** Tier-0 is now a **pre-filter** (cheap candidate
|
|
1066
|
+
> surfacing + the `objection.ts` objection-scan as a sampling-priority hint), NOT the
|
|
1067
|
+
> arbiter of PRIMARY. The hub: **objection-scan + ledger-subtraction sampling** narrow
|
|
1068
|
+
> *where to look*; **mandatory discovery** (deep-read) surfaces WHAT; the **evidence-cited
|
|
1069
|
+
> floor** (mechanical trajectory corroboration) gates promotion; ranking is
|
|
1070
|
+
> **impact-dominant** for discovered signals (impact first, prevalence tiebreak, default
|
|
1071
|
+
> impact ≥ 2 so a fresh discovery never loses to latency-spike on a tie); prevalence is
|
|
1072
|
+
> **honest k/n** over the sampled denominator. When the top discovered signal can't clear
|
|
1073
|
+
> the floor, the enricher emits `suspectedPrimaryUnconfirmed` (R7) so the report shows
|
|
1074
|
+
> *"suspected primary — unconfirmed"* instead of crowning a cheap signal.
|
|
1075
|
+
|
|
1076
|
+
## Step 7 — Aggregate + Deduplicate Findings
|
|
1077
|
+
|
|
1078
|
+
Collect all analyzer output files. Deduplicate by
|
|
1079
|
+
`(traceId, failureOrigin.what + why + where)`. Cluster correlated findings
|
|
1080
|
+
across analyzers. Merge into a single findings array.
|
|
1081
|
+
|
|
1082
|
+
**W13-C (Variance) — deterministic aggregate sort.** Parallel analyzers complete in
|
|
1083
|
+
nondeterministic order, so the merged array's order drifted run-to-run even when the
|
|
1084
|
+
content converged. After dedup, order the array with the deterministic sort so the
|
|
1085
|
+
aggregate (and all downstream order: render, dedup-key emission, byte-identity) is
|
|
1086
|
+
reproducible:
|
|
1087
|
+
|
|
1088
|
+
```bash
|
|
1089
|
+
# Sort the deduped findings into a stable, content-addressed order (Type A pure script).
|
|
1090
|
+
bun scripts/cli/run.sh scripts/aggregate/sort-findings.ts /tmp/findings-all.json /tmp/findings-all.json
|
|
1091
|
+
```
|
|
1092
|
+
|
|
1093
|
+
This is a SORT, not a dedup — R-SI-1 (no new dedup script) is not engaged; dedup stays
|
|
1094
|
+
in-memory in the orchestrator. The sort key is severity → confidence → failureOrigin
|
|
1095
|
+
tuple (what·why·where) → findingId (see `scripts/aggregate/sort-findings.ts`).
|
|
1096
|
+
|
|
1097
|
+
### Step 7.1 — Findings-Contract Gate (W12-07 — post-aggregate, pre-RCA — MANDATORY)
|
|
1098
|
+
|
|
1099
|
+
Before the RCA layer (Step 8), validate the aggregated findings array against the
|
|
1100
|
+
findings contract. This makes the CC-09 force-emit field set (was prose-only in
|
|
1101
|
+
`handover-contract.md` → `required_remedy_fields`) executable, so a field-loss can
|
|
1102
|
+
never reach the renderer.
|
|
1103
|
+
|
|
1104
|
+
```bash
|
|
1105
|
+
# Write the merged findings array to a temp file, then validate.
|
|
1106
|
+
bun scripts/cli/run.sh scripts/validate/findings-contract.ts /tmp/findings-all.json
|
|
1107
|
+
```
|
|
1108
|
+
|
|
1109
|
+
- **Exit 0** → every finding satisfies the contract → proceed to Step 8 (RCA).
|
|
1110
|
+
- **Exit 1** → the validator prints one machine-readable directive per offending
|
|
1111
|
+
finding on stdout, of the form:
|
|
1112
|
+
|
|
1113
|
+
```
|
|
1114
|
+
RESEND <findingId> with <comma-separated missing fields>
|
|
1115
|
+
```
|
|
1116
|
+
|
|
1117
|
+
Remedy-level misses are qualified as `remedy:<remedyId>.<field>` so the analyzer
|
|
1118
|
+
knows where to look. For each directive, **re-dispatch the analyzer for that
|
|
1119
|
+
finding's slice** (handover-contract `on_missing_field`: `max_redispatch: 2`,
|
|
1120
|
+
then `drop with marker INCOMPLETE_FIELDS`). Record each redispatch in
|
|
1121
|
+
`runMeta.redispatches[]` and `runMeta.decisions[]` (`step: "7.1-redispatch"`).
|
|
1122
|
+
|
|
1123
|
+
Required fields enforced (W12-08, PR-052 proposed + W13-C D-1):
|
|
1124
|
+
- **Remedy:** `applyTarget` (HARD-required — every remedy links to a target),
|
|
1125
|
+
`targetClass`, `rationale`, `whyWorks`, `applyInstructions` (≥1), `cost`,
|
|
1126
|
+
`correctness` (both `low|medium|high`), and exactly one of `diff` (when source
|
|
1127
|
+
findable) **or** `diffStatus ∈ {source-unavailable, origin-unknown}` — NEVER a
|
|
1128
|
+
fabricated diff. (`rank` is enricher-derived, NOT gate-checked — see Step 8.)
|
|
1129
|
+
- **Finding:** `findingId`, `actionable`, `failureOrigin`, `whyChain`,
|
|
1130
|
+
`sourceTraceIds`, `referenceIds`, `audience`, and ≥1 structured `assumption`.
|
|
1131
|
+
|
|
1132
|
+
The renderer therefore never has to crash on a malformed finding — the contract
|
|
1133
|
+
catches it upstream here.
|
|
1134
|
+
|
|
1135
|
+
## Step 8 — RCA Layer (PR-002, PR-018, PR-020)
|
|
1136
|
+
|
|
1137
|
+
> **W13-C (Variance, RC-LLM-PIN):** the orchestrator's RCA reasoning here runs under
|
|
1138
|
+
> the SAME pinned inference envelope as the analyzers (`temperature: 0`, explicit
|
|
1139
|
+
> default `model`; see Step 6 dispatch). Clustering (15→N) is unpinned LLM judgment
|
|
1140
|
+
> otherwise — pinning it makes the RCA layer reproducible. Model-intent-sacred:
|
|
1141
|
+
> declare the pin, never silently swap.
|
|
1142
|
+
|
|
1143
|
+
For each finding cluster:
|
|
1144
|
+
|
|
1145
|
+
1. **WHAT** (wrong-output | missing-output | loop | latency-spike | cost-overshoot
|
|
1146
|
+
| format-violation | hallucination | user-complaint | low-score | missing-context)
|
|
1147
|
+
2. **WHY** (prompt-underspec | prompt-overspec | tool-misuse | tool-missing
|
|
1148
|
+
| context-overflow | provider-limit | data-staleness | handoff-loss | dependency-failure)
|
|
1149
|
+
3. **WHERE** (system-prompt | tool-definition | agent-config | routing-config
|
|
1150
|
+
| upstream-data | provider-side | harness-side | user-input)
|
|
1151
|
+
4. **Recursive whys**: keep asking "why" until `isOrigin: true`. No fixed depth.
|
|
1152
|
+
5. **Evidence**: every claim cites a specific trace message index, file:line, or tool call ID.
|
|
1153
|
+
6. **Remedies**: ranked by cost × correctness. W13-C (D-1): this ranking is now
|
|
1154
|
+
implemented in code — the Step 8.5 enricher derives `remedy.rank` deterministically
|
|
1155
|
+
from the analyzer's `cost` + `correctness` categoricals via
|
|
1156
|
+
`scripts/enrich/rank-remedies.ts` (lower rank = higher priority; correctness
|
|
1157
|
+
dominates cost; ties broken by `remedyId`). The analyzer does NOT supply `rank`;
|
|
1158
|
+
deriving it removes an agent-discretion variance source and guarantees the
|
|
1159
|
+
renderer's RANK badge is never `undefined`.
|
|
1160
|
+
|
|
1161
|
+
Full taxonomy: `references/workflows/rca.md`.
|
|
1162
|
+
|
|
1163
|
+
## Step 8.5a — Build EntityContext (PRD-MP-04 — sub-step BEFORE enricher)
|
|
1164
|
+
|
|
1165
|
+
`entity-context.json` is produced at Step 3.7 by the UniTF reader's INTERNAL CLI
|
|
1166
|
+
transport (`read-unitf.ts --out-entity`). It is consumed DIRECTLY by the enricher
|
|
1167
|
+
via the `--entity-context` flag — there is NO inline `bun -e` injection step. The
|
|
1168
|
+
inline-glue hand-wiring (banned by R-SELF-03-c) that previously rebuilt the
|
|
1169
|
+
FindingsInput envelope here was the root cause of OP-7 (silent `diagnosedEntity`
|
|
1170
|
+
drop) and is removed:
|
|
1171
|
+
|
|
1172
|
+
```bash
|
|
1173
|
+
# entity-context.json was already produced at Step 3.7 via:
|
|
1174
|
+
# run.sh scripts/normalize/read-unitf.ts --in <traces.jsonl> --out-entity /tmp/entity-context.json
|
|
1175
|
+
# Pass it straight to the enricher (Step 8.5) — the enricher's --entity-context
|
|
1176
|
+
# flag takes precedence over findings.entities[0] (override + warn).
|
|
1177
|
+
```
|
|
1178
|
+
|
|
1179
|
+
The enricher call at Step 8.5 therefore includes:
|
|
1180
|
+
|
|
1181
|
+
```
|
|
1182
|
+
--entity-context /tmp/entity-context.json
|
|
1183
|
+
```
|
|
1184
|
+
|
|
1185
|
+
### Entity projection is platform-agnostic (post UniTF flip)
|
|
1186
|
+
|
|
1187
|
+
There is no longer a per-platform extractor branch here. Every source platform
|
|
1188
|
+
(`langfuse-export` · `local-jsonl` · `claude-code` · `codex` · `otel`) was already
|
|
1189
|
+
normalized upstream by `mutagent-cli trace fetch` into UniTF, so a SINGLE call
|
|
1190
|
+
projects the `EntityContext` from each record's `ext.agent` block:
|
|
1191
|
+
|
|
1192
|
+
```
|
|
1193
|
+
run.sh scripts/normalize/read-unitf.ts --in <traces.jsonl> [--manifest <manifest.json>] --out-entity /tmp/entity-context.json
|
|
1194
|
+
```
|
|
1195
|
+
|
|
1196
|
+
**Cross-references:**
|
|
1197
|
+
- Step 3.7 (PRD-MP-01) — the authorized producer of `entity-context.json`
|
|
1198
|
+
- Wave-5 R1.7 — "`diagnosedEntity` must come from the read-unitf EntityContext projection"
|
|
1199
|
+
- PRD-CC-10 — `--entity-context` CLI flag (landed) — preferred over inline inject
|
|
1200
|
+
|
|
1201
|
+
> The operator NEVER hand-fills `diagnosedEntity`. It is derived deterministically
|
|
1202
|
+
> from the UniTF reader's projection (no LLM, no network), produced by a single
|
|
1203
|
+
> run.sh call at Step 3.7.
|
|
1204
|
+
|
|
1205
|
+
## Step 8.5 — Build Render Input (MANDATORY — Wave-5 R1.4/R1.5)
|
|
1206
|
+
|
|
1207
|
+
The renderer is **fail-loud**: it REFUSES (throws) when ≥3 of the 4 internal
|
|
1208
|
+
render shapes (`diagnosedEntity` / `bigStat` / `hourlyHeatmap` / `signalCensus`)
|
|
1209
|
+
are missing (R1 §9.3). The whole Wave-5 regression was the pipeline handing the
|
|
1210
|
+
renderer a starved input that silently degraded to placeholders. **Never call the
|
|
1211
|
+
renderer directly on raw findings.** Run the deterministic enricher first.
|
|
1212
|
+
|
|
1213
|
+
```bash
|
|
1214
|
+
Bash("scripts/cli/run.sh scripts/enrich/build-render-input.ts \
|
|
1215
|
+
--tier0 /tmp/tier0.json \
|
|
1216
|
+
--slice-plan /tmp/slice-plan.json \
|
|
1217
|
+
--findings /tmp/findings.json \
|
|
1218
|
+
--metadata /tmp/trace-metadata.json \
|
|
1219
|
+
--output /tmp/render-input.json \
|
|
1220
|
+
--generated-at <ISO8601> \
|
|
1221
|
+
--signal-ctx /tmp/signal-ctx.json")
|
|
1222
|
+
```
|
|
1223
|
+
|
|
1224
|
+
> **Wave-17 selection wiring (`--signal-ctx`, verified OPTIONAL in the parser).** Pass the
|
|
1225
|
+
> `SignalCensusContext` assembled at **Step 6.5** so deep-read-discovered signals become
|
|
1226
|
+
> first-class, evidence-floored census candidates. **Absent → safe-by-default**: the
|
|
1227
|
+
> enricher runs the legacy Tier-0-only census and discovered signals stay
|
|
1228
|
+
> *suspected-unconfirmed* (never crowned). `--signal-ctx` is additive (Zone-1.5) — existing
|
|
1229
|
+
> callers/tests that omit it are byte-unaffected.
|
|
1230
|
+
> (`--self-diag --skill-entity <path>` remain the self-diagnosis pairing — see Step 12.2.)
|
|
1231
|
+
|
|
1232
|
+
The enricher (`scripts/enrich/build-render-input.ts`) is **deterministic +
|
|
1233
|
+
idempotent** (no network, no LLM, no random; the only clock value is the injected
|
|
1234
|
+
`--generated-at`). It:
|
|
1235
|
+
|
|
1236
|
+
- aggregates `hourlyHeatmap` (24-cell grid) from per-trace `startTime` + `latencyMs`;
|
|
1237
|
+
- computes `bigStat` (latency p50/p95/max, cost, traces, errors);
|
|
1238
|
+
- builds `signalCensus` via the PR-049 5-step reconciled process (failure-validity gate → impact×prevalence → deep-read corroboration → ONE `runMeta.primarySignal`) + `scanFunnel` (4-stage: total → Tier-0 → sample N → deep-read 6/N with honest sample denominator);
|
|
1239
|
+
- normalizes legacy free-text `assumptions` → structured `Assumption[]` (R1.3);
|
|
1240
|
+
- **pulls `diagnosedEntity` FROM the normalizer's `EntityContext`** (R1.7,
|
|
1241
|
+
APPENDIX-A §A.2 — extracted at ingest in Step 4, NOT synthesized here). The
|
|
1242
|
+
orchestrator threads the normalizer's EntityContext through to the enricher;
|
|
1243
|
+
the operator never hand-fills `diagnosedEntity`.
|
|
1244
|
+
|
|
1245
|
+
**Self-diag (PR-022):** when `config.self_diagnostics.enabled`, pass the
|
|
1246
|
+
self-diag option so the enricher forces `isMetaReport:true`, `audience:internal`,
|
|
1247
|
+
a `[INTERNAL]` sessionId prefix, and the skill-typed EntityContext (built from the
|
|
1248
|
+
skill's own SKILL.md + `scripts/` via `buildSkillSelfEntityContext`).
|
|
1249
|
+
|
|
1250
|
+
### F4 — CODIFIED runMeta methodology-widget threading (MANDATORY — not per-run-manual)
|
|
1251
|
+
|
|
1252
|
+
Six Methodology widgets must be threaded into the report. The threading is CODIFIED
|
|
1253
|
+
here (not ad-hoc per run) so it can never silently regress. BEFORE assembling
|
|
1254
|
+
`/tmp/findings.json` for the enricher call above, populate `findings.runMeta`:
|
|
1255
|
+
|
|
1256
|
+
| Widget field | Source | Who computes |
|
|
1257
|
+
|---|---|---|
|
|
1258
|
+
| `selectionRules` | impact×prevalence census scoring | **enricher** (F1, deterministic — do NOT pre-assign) |
|
|
1259
|
+
| `signalSelectionTrace` | the signal-selection decision path | **enricher** (F1, deterministic — do NOT pre-assign) |
|
|
1260
|
+
| `tierBreakdown` | the tier-coverage output / `wave6/*.json` stamp | **orchestrator** — assign from the stamp |
|
|
1261
|
+
| `awarenessSample` | `buildAwarenessSample()` (Step 3.5) | **orchestrator** — assign (F2) |
|
|
1262
|
+
| `blindSpots` | `buildBlindSpots().rows` (Step 3.5) | **orchestrator** — assign (F2) |
|
|
1263
|
+
| per-finding `coverageProof` | the representative sampler (W12-13, Block A) | **already attached** upstream — thread, do NOT redo |
|
|
1264
|
+
|
|
1265
|
+
The enricher PRESERVES every `findings.runMeta` field on its passthrough (the
|
|
1266
|
+
`...findings.runMeta` spread is unconditional — widget fields are never dropped, even
|
|
1267
|
+
on a no-primary run) and ADDS only the two it can derive deterministically
|
|
1268
|
+
(`selectionRules` + `signalSelectionTrace`). The orchestrator-assigned widgets
|
|
1269
|
+
(`tierBreakdown` / `awarenessSample` / `blindSpots`) are upstream/LLM-produced and the
|
|
1270
|
+
enricher NEVER recomputes them.
|
|
1271
|
+
|
|
1272
|
+
```ts
|
|
1273
|
+
// Assign the tier-coverage breakdown from the sampler / wave6 stamp.
|
|
1274
|
+
findings.runMeta.tierBreakdown = tierCoverageFromStamp; // [{ tier, count, color? }]
|
|
1275
|
+
// awarenessSample + blindSpots are assigned at Step 3.5 (see F2 above).
|
|
1276
|
+
// coverageProof is already on each finding (W12-13) — passes through untouched.
|
|
1277
|
+
```
|
|
1278
|
+
|
|
1279
|
+
> Regression proof (TASK-R24-1): run 210635 threaded NOTHING → 0 `<svg>`, widget
|
|
1280
|
+
> fields MISSING in render-input runMeta → generic-fallback Methodology. run 194421
|
|
1281
|
+
> threaded by hand → 1 `<svg>`, widgets rendered. The renderer was correct both times
|
|
1282
|
+
> (`renderTierPie` returns `""` when `tierBreakdown` is absent); the bug was the
|
|
1283
|
+
> missing CODIFIED threading step. This table is that step.
|
|
1284
|
+
|
|
1285
|
+
## Step 8.5b — Dismissal Final-Check (verdict-ledger suppression — semantic)
|
|
1286
|
+
|
|
1287
|
+
> **The negative half of the class-memory loop.** Step 4.5 surfaces APPROVED priors on the
|
|
1288
|
+
> RAW trace batch (before findings exist); Step 8.5b suppresses DISMISSED findings on the
|
|
1289
|
+
> PRODUCED findings (after they exist). The two never race on the same object. Additive +
|
|
1290
|
+
> fail-safe: absent library / no dismissals ⇒ no-op, byte-unchanged output.
|
|
1291
|
+
|
|
1292
|
+
Run this AFTER findings are aggregated (Step 7) and BEFORE the enricher call (Step 8.5),
|
|
1293
|
+
so the enricher can be handed the `dismissalContext`. It is the semantic FINAL-CHECK that
|
|
1294
|
+
hides findings the operator previously ruled *not-an-issue* for this entity.
|
|
1295
|
+
|
|
1296
|
+
1. **Load + fold the valid dismissals** (deterministic, no LLM):
|
|
1297
|
+
`foldValidDismissals(entityName, { entityFingerprint })` (`scripts/library/store.ts`).
|
|
1298
|
+
NEVER-EXPIRE: a dismissal has NO TTL — the ONLY invalidation is an `entityFingerprint`
|
|
1299
|
+
change (the agent's prompt/config changed under it). Empty library ⇒ `[]` ⇒ skip.
|
|
1300
|
+
|
|
1301
|
+
2. **Severity-escalation is handled in the partition, BEFORE the semantic match** — a
|
|
1302
|
+
dismissed-but-now-worse finding (`severity` outranks `severityAtDismissal`,
|
|
1303
|
+
`crit>high>med>info`) is NEVER suppressed regardless of overlap (HARD GUARD). You do
|
|
1304
|
+
NOT run the reasoning check for an escalated (finding × entry) pair.
|
|
1305
|
+
|
|
1306
|
+
3. **Run the PINNED host-runtime reasoning check** (`RC-LLM-PIN` parity — see §W13-C) for
|
|
1307
|
+
each non-escalated (new finding × folded dismissal) pair of the SAME entity. Build the
|
|
1308
|
+
request with `buildSemanticMatchRequest()` (`scripts/enrich/dismissal-match.ts`):
|
|
1309
|
+
`model: claude-sonnet-4-6` (default), `temperature: 0` (PINNED), `cPin` reproducibility
|
|
1310
|
+
tag. The check compares the (what, why, where) failure-mode triples — which are LLM
|
|
1311
|
+
prose that NEVER string-matches run-to-run — and returns `{ overlap, confidence }`. This
|
|
1312
|
+
is a HOST-RUNTIME reasoning step (the orchestrator's own pinned reasoning, or a dispatched
|
|
1313
|
+
host-runtime subagent) — NOT a provider-API call and NOT inside the deterministic enricher.
|
|
1314
|
+
|
|
1315
|
+
4. **Assemble the `VerdictLookup`** keyed by `verdictLookupKey(findingId, entry)` and pass
|
|
1316
|
+
`dismissalContext: { entitySlug, dismissedEntries, verdicts }` to the enricher (Step 8.5).
|
|
1317
|
+
The enricher applies the PURE `partitionByDismissal` — CONSERVATIVE: suppress ONLY on
|
|
1318
|
+
`overlap && confidence === "high"`; anything else (unsure / medium / low / missing) ⇒ the
|
|
1319
|
+
finding SHOWS (fail-safe). Suppressed findings are REMOVED from `renderInput.findings` —
|
|
1320
|
+
there is **NO visible "Suppressed" section**; the verdict ledger is the audit trail.
|
|
1321
|
+
|
|
1322
|
+
> **Determinism boundary.** The enricher stays "no LLM / no network / no random": the
|
|
1323
|
+
> reasoning verdicts are computed HERE (upstream) and injected. The enricher only applies
|
|
1324
|
+
> the deterministic partition fed by those verdicts.
|
|
1325
|
+
|
|
1326
|
+
## Step 8.9 — Pre-Render Completeness Gate (PRD-SO-06 + W9-08)
|
|
1327
|
+
|
|
1328
|
+
Before rendering, run BOTH the Wave-6 checklist AND the completeness-check (W9-08).
|
|
1329
|
+
A missing RunMeta, unfilled template section, or invalid Finding MUST fail loud —
|
|
1330
|
+
never emit partial HTML.
|
|
1331
|
+
|
|
1332
|
+
**W9-08 completeness check** (B4 creates `scripts/validate/completeness-check.ts`):
|
|
1333
|
+
|
|
1334
|
+
```bash
|
|
1335
|
+
# B2-fix (verified against import.meta.main): the default mode takes the
|
|
1336
|
+
# RenderInput as a POSITIONAL argument (argv[0]) — NOT a --render-input flag.
|
|
1337
|
+
Bash("scripts/cli/run.sh scripts/validate/completeness-check.ts /tmp/render-input.json")
|
|
1338
|
+
# → exits 1 with missing-field list if RunMeta fields absent, sections empty,
|
|
1339
|
+
# template placeholders unfilled, or any Finding fails TypeBox validation.
|
|
1340
|
+
# → assert isMetaReport => audience=internal (self-diag invariant)
|
|
1341
|
+
#
|
|
1342
|
+
# The SAME script also exposes a PRE-ENRICHER mode (distinct flag-driven entrypoint)
|
|
1343
|
+
# that gates findings.runMeta BEFORE Step 8.5 — run it after Step 3.5 threading:
|
|
1344
|
+
# scripts/cli/run.sh scripts/validate/completeness-check.ts --pre-enricher \
|
|
1345
|
+
# --findings /tmp/findings.json [--checklist <report-checklist.yaml>]
|
|
1346
|
+
# → exit 0 = all orchestrator-threaded F4 widgets present; exit 1 = unthreaded list.
|
|
1347
|
+
```
|
|
1348
|
+
|
|
1349
|
+
**W13-C (D-6): the completeness gate also asserts the renderer's EXACT dereference
|
|
1350
|
+
contract** — not just RunMeta/section presence. It checks the fields render.ts
|
|
1351
|
+
interpolates without a guard, so a gate-pass GUARANTEES a render-success (closing the
|
|
1352
|
+
"gate passes but render crashes/garbles" gap that produced the D-1 `undefined` badges):
|
|
1353
|
+
- top-level `sessionId` (render does `sessionId.toUpperCase()`);
|
|
1354
|
+
- per-finding `actionable` is a string (render does `actionable.slice()`);
|
|
1355
|
+
- per-finding `failureOrigin.evidence` + `failureOrigin.confidence`;
|
|
1356
|
+
- per-remedy `rank` (number, enricher-derived) + `cost` + `correctness` (`low|medium|high`).
|
|
1357
|
+
All gaps are reported at once (fail-loud with the full missing list).
|
|
1358
|
+
|
|
1359
|
+
**Wave-6 checklist gate** (PRD-SO-06 — verify each mandatory script emitted its stamp):
|
|
1360
|
+
|
|
1361
|
+
```bash
|
|
1362
|
+
Bash("scripts/cli/run.sh scripts/validate/wave6-checklist.ts \
|
|
1363
|
+
--report-dir /tmp/report-dir \
|
|
1364
|
+
[--accept-exemptions <id>...]")
|
|
1365
|
+
# → { ok: boolean, missing: string[], exemptions: string[] }
|
|
1366
|
+
```
|
|
1367
|
+
|
|
1368
|
+
The checker reads `<report-dir>/wave6/<step>.json` for the steps:
|
|
1369
|
+
`parse-brief` · `awareness-sample` · `blind-spots` · `library-match` ·
|
|
1370
|
+
`caps-result` · `deep-read-gate` · `awareness-witness` (W11-05).
|
|
1371
|
+
|
|
1372
|
+
**W11-05 extra rule**: if the `awareness-witness` stamp declares
|
|
1373
|
+
`isFreshRun=true + awarenessRan=false + no exemptionReason`, the checklist
|
|
1374
|
+
HARD-FAILS regardless of `isClientReport`. This is non-bypassable.
|
|
1375
|
+
|
|
1376
|
+
| Report type | Gate behavior |
|
|
1377
|
+
|---|---|
|
|
1378
|
+
| Internal / self-diag | **Warn-only** — log missing steps, continue |
|
|
1379
|
+
| Client (operator paid LLM tokens) | **Hard-fail** — HALT and AskUserQuestion before rendering |
|
|
1380
|
+
|
|
1381
|
+
When a step was skipped under a documented exemption, pass its id via
|
|
1382
|
+
`--accept-exemptions`. The exemption MUST be recorded in `runMeta`:
|
|
1383
|
+
|
|
1384
|
+
```json
|
|
1385
|
+
{
|
|
1386
|
+
"exemptions": [
|
|
1387
|
+
{
|
|
1388
|
+
"stepId": "awareness-sample",
|
|
1389
|
+
"reason": "library prior exists for this entity",
|
|
1390
|
+
"declaredBy": "Step 3.5 conditional skip rule"
|
|
1391
|
+
}
|
|
1392
|
+
]
|
|
1393
|
+
}
|
|
1394
|
+
```
|
|
1395
|
+
|
|
1396
|
+
See `references/workflows/rca.md` Gate Exemption Taxonomy for valid exemption
|
|
1397
|
+
reasons. `writeWave6Stamp` (exported from `scripts/validate/wave6-checklist.ts`)
|
|
1398
|
+
is called by each Wave-6 script at completion to produce its stamp file.
|
|
1399
|
+
|
|
1400
|
+
**W12-02 inline-JS syntax gate** (propose **PR-050** — render-js-syntax.ts).
|
|
1401
|
+
The completeness and Wave-6 gates above validate DATA shape; they cannot catch
|
|
1402
|
+
interactivity-breaking JavaScript. A single asymmetric-escape in an emitted
|
|
1403
|
+
template literal (W12-01: `/\r?\\n/` emitting a literal-CR regex) produces a
|
|
1404
|
+
`SyntaxError` that silently kills the live-preview IIFE — check/uncheck + Copy
|
|
1405
|
+
go dead while every data gate still passes. This gate runs on the RENDERED HTML
|
|
1406
|
+
emitted by **Step 9** and is a HARD blocker before the report is opened/handed
|
|
1407
|
+
back:
|
|
1408
|
+
|
|
1409
|
+
```bash
|
|
1410
|
+
# Run AFTER Step 9 emits the HTML, BEFORE the report is opened or handed off.
|
|
1411
|
+
Bash("scripts/cli/run.sh scripts/validate/render-js-syntax.ts \
|
|
1412
|
+
/tmp/report-dir/report.html")
|
|
1413
|
+
# → parses every executable inline <script> body via new Function (no execution);
|
|
1414
|
+
# exits 1 with the offending script index + snippet on any SyntaxError,
|
|
1415
|
+
# exits 0 when all inline JS parses.
|
|
1416
|
+
```
|
|
1417
|
+
|
|
1418
|
+
Skips non-executable scripts (external `src=`, `application/ld+json`,
|
|
1419
|
+
`text/plain` payload bundles). **Non-bypassable** for all report types — a
|
|
1420
|
+
report whose inline JS does not parse is broken for the operator regardless of
|
|
1421
|
+
audience (the copy-back HITL surface is the skill's primary approval channel,
|
|
1422
|
+
PR-014).
|
|
1423
|
+
|
|
1424
|
+
## Step 9 — Render HTML Report
|
|
1425
|
+
|
|
1426
|
+
Render the **enriched** input from Step 8.5 (NOT raw findings):
|
|
1427
|
+
|
|
1428
|
+
```bash
|
|
1429
|
+
Bash("scripts/cli/run.sh scripts/report/render.ts --findings /tmp/render-input.json
|
|
1430
|
+
--audience <AUDIENCE>
|
|
1431
|
+
--output .mutagent/diagnostics/reports/{sessionId}/report.html")
|
|
1432
|
+
```
|
|
1433
|
+
|
|
1434
|
+
**W13-D — `<AUDIENCE>` resolution (MANDATORY).** Compute the effective audience
|
|
1435
|
+
deterministically via `resolveEffectiveAudience` (`scripts/config/schema.ts`) —
|
|
1436
|
+
precedence, highest first:
|
|
1437
|
+
|
|
1438
|
+
1. an **explicit** operator `--audience` flag for THIS run (the operator typed it), else
|
|
1439
|
+
2. `config.default_audience` (read from `.mutagent/config.yaml`; a fresh
|
|
1440
|
+
`init` writes `client`), else
|
|
1441
|
+
3. the schema fallback `"internal"` (renderer's own argv default — only reached when
|
|
1442
|
+
neither config nor flag is present).
|
|
1443
|
+
|
|
1444
|
+
A fresh init therefore makes the **effective default `client`** (client-stripped
|
|
1445
|
+
report). Pass the resolved value to `--audience`. When the operator gave no explicit
|
|
1446
|
+
flag AND no config exists, OMIT the flag and let the renderer fall back to `internal`.
|
|
1447
|
+
|
|
1448
|
+
> **PR-022 invariant (non-bypassable):** for a self-diagnosis render
|
|
1449
|
+
> (`isMetaReport:true`), the audience is ALWAYS `internal` — it overrides both the
|
|
1450
|
+
> flag and `config.default_audience`. The enricher already forces `audience:internal`
|
|
1451
|
+
> (Step 8.5) and the renderer hard-refuses `--audience client` on a meta report, so a
|
|
1452
|
+
> self-diag run is never client. `resolveEffectiveAudience({ isMetaReport: true, … })`
|
|
1453
|
+
> returns `internal` for this reason — call it with `isMetaReport` set so the
|
|
1454
|
+
> orchestrator never even builds an `--audience client` invocation for self-diag.
|
|
1455
|
+
|
|
1456
|
+
Template defaults to `assets/templates/report.html.tpl` (or `default.html` pre-P4).
|
|
1457
|
+
|
|
1458
|
+
## Step 9.9 — Finalization Gate + Bounded Self-Heal (W14 — MANDATORY)
|
|
1459
|
+
|
|
1460
|
+
> **⚠ MANDATORY — non-bypassable, like the Step 6 deep-read pre-gate and Step 3.7
|
|
1461
|
+
> normalize precondition.** This step runs AFTER Step 9 renders `report.html` and
|
|
1462
|
+
> BEFORE any "report is done / rendered / open it" declaration — including Step 9.5
|
|
1463
|
+
> (auto-open) and Step 10 (HITL handback). The orchestrator MUST NOT open, hand back,
|
|
1464
|
+
> or describe the report as done until this gate is **CRIT-clean**.
|
|
1465
|
+
|
|
1466
|
+
**WHY this exists.** Step 8.9's completeness-check gates the **INPUT** (the
|
|
1467
|
+
`RenderInput` JSON — the fields `render.ts` will dereference). This step gates the
|
|
1468
|
+
**OUTPUT** — the rendered HTML the operator actually reads. It catches the
|
|
1469
|
+
output-only bug class the input gate structurally cannot see: a soft render fallback
|
|
1470
|
+
that shipped (`RANK n/a`, `cost:n/a`, `correct:n/a`), a literal `undefined`/`null`/`NaN`
|
|
1471
|
+
that reached visible prose, a raw-JSON entity prompt (`{"prompt":`), or a
|
|
1472
|
+
`class="internal"` node that survived the client strip (a leak). Deterministic,
|
|
1473
|
+
no LLM, no spend — runs every render (operator decision: DOM/regex/string only).
|
|
1474
|
+
|
|
1475
|
+
**The producer + gate.** Checklist: `scripts/validate/report-checklist.yaml`
|
|
1476
|
+
(one row per section: `require`/`forbid`/`tier`/`okEmpty`/`source`/`heal`). Gate:
|
|
1477
|
+
`scripts/validate/finalize-gate.ts` → `{ pass, gaps[] }`, each gap carrying
|
|
1478
|
+
`section · tier · what · sourceStep · healAction`. CRIT gaps ⇒ `pass=false`
|
|
1479
|
+
(block "done"); WARN gaps ⇒ reported, `pass` stays true (stamp a banner).
|
|
1480
|
+
|
|
1481
|
+
> **Backtrace table:** `references/script-index.md` maps every section → its producer
|
|
1482
|
+
> step → the gate that enforces it. Each checklist row's `source` is the producer to
|
|
1483
|
+
> re-run; each `heal` is the re-run action. Step 9.9's self-heal loop is driven by
|
|
1484
|
+
> these two columns — never by memory.
|
|
1485
|
+
|
|
1486
|
+
### The loop (precise — operator decision 3 + 4)
|
|
1487
|
+
|
|
1488
|
+
```text
|
|
1489
|
+
1. Render → report.html (Step 9)
|
|
1490
|
+
2. Run finalize-gate.ts on the RENDERED HTML:
|
|
1491
|
+
Bash("scripts/cli/run.sh scripts/validate/finalize-gate.ts \
|
|
1492
|
+
--report .mutagent/diagnostics/reports/${SID}/report.html \
|
|
1493
|
+
--audience <effective-audience-from-Step-9>")
|
|
1494
|
+
→ exit 0 = CRIT-clean (may carry WARN gaps); exit 1 = ≥1 CRIT gap (block done).
|
|
1495
|
+
3. IF CRIT gaps: for EACH gap, backtrace via its `sourceStep` (the script-index.md
|
|
1496
|
+
map) → re-run that producer/step → re-build the render input (Step 8.5) →
|
|
1497
|
+
re-render (Step 9) → re-run the gate (Step 9.9).
|
|
1498
|
+
► BOUNDED: at most **2 self-heal rounds**. (round counter starts at 0; after the
|
|
1499
|
+
2nd failed re-check, STOP — do NOT attempt a 3rd.)
|
|
1500
|
+
4. IF still CRIT-failing after the bound → **ESCALATE LOUD** to the operator:
|
|
1501
|
+
surface the unrecoverable section(s) + `sourceStep` + every heal action tried +
|
|
1502
|
+
the residual gap list. Do NOT declare done. Do NOT silently emit the report.
|
|
1503
|
+
(Use AskUserQuestion — the operator decides: accept-with-defect / abort / manual fix.)
|
|
1504
|
+
5. WARN gaps (CRIT-clean) → emit the report WITH a visible
|
|
1505
|
+
"⚠ incomplete: <sections>" banner; report status as "rendered with N gaps."
|
|
1506
|
+
6. ONLY when the gate is CRIT-clean may the orchestrator proceed to Step 9.5
|
|
1507
|
+
(auto-open) / Step 10 (handback) and declare the report done/rendered.
|
|
1508
|
+
```
|
|
1509
|
+
|
|
1510
|
+
**Non-negotiable invariants** (operator decision 3): NEVER infinite-loop (hard cap = 2
|
|
1511
|
+
rounds), NEVER silently emit a CRIT-incomplete report, ALWAYS escalate loud when the
|
|
1512
|
+
bound is exhausted. The bound + escalate path is what makes "self-heal" safe — it is a
|
|
1513
|
+
bounded retry with a loud failure mode, never an unbounded repair loop.
|
|
1514
|
+
|
|
1515
|
+
### Step 9.9.1 — Append AutoMemory on operator feedback (PHASE 3 — run FINALIZE)
|
|
1516
|
+
|
|
1517
|
+
At run FINALIZE (this gate is CRIT-clean, report handed back), IF the operator gives
|
|
1518
|
+
feedback ABOUT the TOOL or a standing preference (not a subject finding), classify it
|
|
1519
|
+
with the `references/memory-format.md` rubric (first-match: reference → user →
|
|
1520
|
+
feedback → project) and persist it:
|
|
1521
|
+
|
|
1522
|
+
```bash
|
|
1523
|
+
Bash("scripts/cli/run.sh scripts/memory/append.ts \
|
|
1524
|
+
<slug> <name> <description> <type> diagnose <YYYY-MM-DD> \"<body>\" --root \"$PROJECT_ROOT\"")
|
|
1525
|
+
# type ∈ reference|user|feedback|project ; date = today (absolute) ; lifecycle=diagnose
|
|
1526
|
+
# (use `general` for a tool-wide standing preference). Dedupe on slug (update + refresh date).
|
|
1527
|
+
```
|
|
1528
|
+
|
|
1529
|
+
- Subject = the TOOL + operator, NOT the diagnosed agent (findings go to the
|
|
1530
|
+
approved-only class-memory library, NOT here).
|
|
1531
|
+
- For `feedback` / `project` bodies, include **Why:** + **How to apply:** lines.
|
|
1532
|
+
- Do NOT save anything already in code / config / git, or ephemeral-to-one-run facts.
|
|
1533
|
+
- The next diagnose run reads this back at Step 3a.1 (lifecycle ∈ { diagnose, general }).
|
|
1534
|
+
|
|
1535
|
+
### Audience resolution
|
|
1536
|
+
|
|
1537
|
+
Pass the SAME effective audience computed at Step 9 (`resolveEffectiveAudience`,
|
|
1538
|
+
see Step 9 W13-D). `--audience client` activates the internal-leak CRIT row
|
|
1539
|
+
(`global-client-no-internal`) + exempts the Methodology sub-section rows (the whole
|
|
1540
|
+
tab is node-stripped on client). `--audience internal` exempts the leak row and
|
|
1541
|
+
arms the `loud-missing-widget` CRIT checks (a widget whose data was never threaded
|
|
1542
|
+
into `runMeta` renders a loud marker — itself a forgotten-data tell).
|
|
1543
|
+
|
|
1544
|
+
### Relationship to the other rendered-HTML gates
|
|
1545
|
+
|
|
1546
|
+
Step 8.9's **render-js-syntax.ts** (W12-02 / PR-050) and this **finalize-gate.ts**
|
|
1547
|
+
both run on the Step-9 RENDERED HTML and both block the open/handback. They are
|
|
1548
|
+
complementary: render-js-syntax validates that inline JS *parses* (interactivity);
|
|
1549
|
+
finalize-gate validates that section *data landed* (completeness). Run render-js-syntax
|
|
1550
|
+
first (a broken IIFE makes the copy-back surface dead), then finalize-gate.
|
|
1551
|
+
|
|
1552
|
+
## Step 9.5 — Auto-Open Rendered Report (MANDATORY)
|
|
1553
|
+
|
|
1554
|
+
> **GATED BY Step 9.9.** Do NOT open until the finalization gate (Step 9.9) is
|
|
1555
|
+
> CRIT-clean. Opening a CRIT-incomplete report violates the "never silently emit"
|
|
1556
|
+
> invariant — the auto-open IS the "report is done, look at it" signal.
|
|
1557
|
+
|
|
1558
|
+
PR-014 declares the HTML report the **primary HITL surface**. After Step 9 writes
|
|
1559
|
+
the HTML, the orchestrator opens it BEFORE prompting for copy-back — the operator
|
|
1560
|
+
should never have to click a file path.
|
|
1561
|
+
|
|
1562
|
+
```bash
|
|
1563
|
+
REPORT_PATH=".mutagent/diagnostics/reports/${SID}/report.html"
|
|
1564
|
+
case "$(uname -s)" in
|
|
1565
|
+
Darwin) Bash("open \"$REPORT_PATH\"") ;;
|
|
1566
|
+
Linux) Bash("xdg-open \"$REPORT_PATH\" 2>/dev/null || printf 'file://%s\\n' \"$PWD/$REPORT_PATH\"") ;;
|
|
1567
|
+
MINGW*|MSYS*|CYGWIN*) Bash("start \"\" \"$REPORT_PATH\"") ;;
|
|
1568
|
+
*) Bash("printf 'file://%s\\n' \"$PWD/$REPORT_PATH\"") ;;
|
|
1569
|
+
esac
|
|
1570
|
+
```
|
|
1571
|
+
|
|
1572
|
+
`$MUTAGENT_DIAG_HEADLESS=1` short-circuits the open and prints the `file://` URL only
|
|
1573
|
+
(SSH-remote / CI / scripted runs).
|
|
1574
|
+
|
|
1575
|
+
## Step 10 — HITL Review Gate (PR-014)
|
|
1576
|
+
|
|
1577
|
+
Emit to operator:
|
|
1578
|
+
|
|
1579
|
+
```
|
|
1580
|
+
Diagnostic report ready at: .mutagent/diagnostics/reports/{session}/report.html
|
|
1581
|
+
|
|
1582
|
+
Open the HTML in your browser, review the gold-standard report (Methodology · Overview · per-finding · Decisions), pick remedies using
|
|
1583
|
+
the "Copy back markdown" buttons, and paste the markdown payload back into this chat.
|
|
1584
|
+
That paste IS the approval signal.
|
|
1585
|
+
```
|
|
1586
|
+
|
|
1587
|
+
**AskUserQuestion does NOT fire at this step.** Wait for operator to paste markdown.
|
|
1588
|
+
|
|
1589
|
+
## Step 11.0 — Report-Only Short-Circuit (PRD-SO-04)
|
|
1590
|
+
|
|
1591
|
+
Check `config.target.platform` before entering the apply flow:
|
|
1592
|
+
|
|
1593
|
+
```typescript
|
|
1594
|
+
if (config.target.platform === 'report-only') {
|
|
1595
|
+
runMeta.applySkipped = { reason: 'config target = report-only' };
|
|
1596
|
+
// persist runMeta, then HALT — do NOT proceed to Step 11
|
|
1597
|
+
return;
|
|
1598
|
+
}
|
|
1599
|
+
```
|
|
1600
|
+
|
|
1601
|
+
When `platform === 'report-only'`:
|
|
1602
|
+
- Skip the AskUserQuestion apply-confirmation gate entirely.
|
|
1603
|
+
- Log `runMeta.applySkipped: { reason: 'config target = report-only' }`.
|
|
1604
|
+
- Halt after Step 10 HITL review — the report is the deliverable.
|
|
1605
|
+
- The Methodology tab Decisions row surfaces `applySkipped`.
|
|
1606
|
+
|
|
1607
|
+
When `platform` is any other value (e.g., `local-agent`, `local-code-construct`,
|
|
1608
|
+
`remote`): fall through to the standard Step 11 parse-and-apply gate.
|
|
1609
|
+
|
|
1610
|
+
## Step 11 — Parse Copy-Back + Apply Gate
|
|
1611
|
+
|
|
1612
|
+
Parse the pasted markdown payload. Extract approved remedy IDs + finding references.
|
|
1613
|
+
Validate each `remedyId` exists in the findings JSON.
|
|
1614
|
+
|
|
1615
|
+
### Multi-target apply — ALWAYS ASK which, then confirm (Item #6)
|
|
1616
|
+
|
|
1617
|
+
**Before** the destructive-action gate below, when **≥2 targets/harnesses are in
|
|
1618
|
+
play** — `global.targets[]` has >1 entry, OR the approved remedies resolve to more
|
|
1619
|
+
than one candidate `--target <root>` — the apply gate MUST first ASK **which target
|
|
1620
|
+
to bind**, then confirm. It **NEVER silently picks a default**, even when one
|
|
1621
|
+
`global.targets[]` entry carries `default: true`. This re-asserts the Step-0 detect
|
|
1622
|
+
selection (Cases 3/5) at the WRITE boundary, because the write is destructive and a
|
|
1623
|
+
wrong-target apply is expensive to unwind.
|
|
1624
|
+
|
|
1625
|
+
```
|
|
1626
|
+
AskUserQuestion({
|
|
1627
|
+
questions: [{
|
|
1628
|
+
header: "Apply target",
|
|
1629
|
+
question: "≥2 targets are installed — which target should these {N} remedies bind to?",
|
|
1630
|
+
options: [ /* one option per candidate target root — NO option pre-marked as default */ ],
|
|
1631
|
+
multiSelect: false,
|
|
1632
|
+
}]
|
|
1633
|
+
})
|
|
1634
|
+
```
|
|
1635
|
+
|
|
1636
|
+
Only after the operator picks does the confirm gate below fire, and the shared CLI
|
|
1637
|
+
is invoked with a SINGLE resolved target (or an explicit `--pick`). With exactly ONE
|
|
1638
|
+
target installed, this ASK is skipped (no spurious prompt) — the binding is already
|
|
1639
|
+
unambiguous. This is an INTERACTIVE prompt the session surfaces (Model-B / PR-ORCH-01
|
|
1640
|
+
— NOT a code-driven decision loop).
|
|
1641
|
+
|
|
1642
|
+
> **DETERMINISTIC BACKSTOP (code):** `mutagent-cli apply` enforces this underneath —
|
|
1643
|
+
> handed ≥2 candidate `--target` values with no explicit `--pick`, it REFUSES
|
|
1644
|
+
> (exit 2, `needs-disambiguation`) instead of binding one
|
|
1645
|
+
> (`@mutagent/tools` src/apply/target-select.ts · `selectApplyTarget`). A silent
|
|
1646
|
+
> default is therefore impossible at BOTH layers: the protocol ASK here and the CLI
|
|
1647
|
+
> guard beneath it.
|
|
1648
|
+
|
|
1649
|
+
Then AskUserQuestion for the final destructive-action gate:
|
|
1650
|
+
|
|
1651
|
+
```
|
|
1652
|
+
AskUserQuestion: "Spawn BG agent on worktree to apply {N} approved remedies?"
|
|
1653
|
+
Options:
|
|
1654
|
+
- Confirm — spawn BG worktree apply (opens PR per target, no changes to your branch)
|
|
1655
|
+
- Dry-run preview first (show exact diffs before committing)
|
|
1656
|
+
- Cancel
|
|
1657
|
+
```
|
|
1658
|
+
|
|
1659
|
+
On confirm (the apply-worker is RETIRED — M9/DC-6): build an ImproveHandover from the
|
|
1660
|
+
approved RemedyPacket and route it as a GATED call to the SHARED `mutagent-cli apply`
|
|
1661
|
+
service (caller b — NOT through the optimize loop). The human gate is HERE (the
|
|
1662
|
+
AskUserQuestion above); the CLI holds no approval logic (DC-1). The flow:
|
|
1663
|
+
|
|
1664
|
+
```
|
|
1665
|
+
# 1. copy-back parsed above → approved remedies + target spec (from config.yaml) + diagnosed_at_hash
|
|
1666
|
+
# 2. Helix builds the HandoverBundle (adl_stage: improve — a frozen enum member) from the RemedyPacket
|
|
1667
|
+
# 3. spawn ai-engineer (the WRITE actor) to produce the amended artifact per subject-kind (#apply):
|
|
1668
|
+
Agent(
|
|
1669
|
+
subagent_type: "ai-engineer", # Helix-provided (from builder); vendored for standalone (M10)
|
|
1670
|
+
run_in_background: true,
|
|
1671
|
+
worktree: true,
|
|
1672
|
+
prompt_includes: [approved_remedies, target_spec, diagnosed_at_hash, apply.kind]
|
|
1673
|
+
)
|
|
1674
|
+
# 4. ai-engineer shells the SHARED transport (stale-check → dry-run → write → audit):
|
|
1675
|
+
# mutagent-cli apply --kind <code-pr|markdown|cloud-deploy|report-only> --target <root> … --dry-run
|
|
1676
|
+
# mutagent-cli apply … --commit # after this gate (opens a PR per local target / new-rev for cloud)
|
|
1677
|
+
# 5. the audit is written to .mutagent/apply/<id>/apply-audit.json (the rollback ledger)
|
|
1678
|
+
```
|
|
1679
|
+
|
|
1680
|
+
The mechanics that used to live in the apply-worker (stale-check · worktree/PR · REST
|
|
1681
|
+
idempotent · dual-emit audit) are now the shared `mutagent-cli apply` adapters
|
|
1682
|
+
(`@mutagent/tools` src/apply — worktree-PR · REST · vendor-CLI). Rollback is
|
|
1683
|
+
non-destructive: `mutagent-cli apply --rollback <apply-audit.json>`. Reference:
|
|
1684
|
+
`@mutagent/tools` apply layer + the ai-engineer `#apply` modes.
|
|
1685
|
+
|
|
1686
|
+
## Step 11.5 — Persist Verdicts to the Unified Ledger (finding-verdict memory)
|
|
1687
|
+
|
|
1688
|
+
The copy-back paste carries BOTH polarities in ONE bundle (the report's live-preview folds
|
|
1689
|
+
them into the same `#lp-body` — no new egress):
|
|
1690
|
+
|
|
1691
|
+
- **Approved remedies** → the existing `## Approved Remedies (handoff)` block (Step 11 above).
|
|
1692
|
+
- **Dismissed findings** → the `## Invalid findings (dismissed — "not an issue")` block, one
|
|
1693
|
+
entry per finding the operator marked *Invalid*, each carrying the hidden
|
|
1694
|
+
`dismiss-payload` JSON (`findingId` + `what/why/where` + `severity`) + an optional `reason`.
|
|
1695
|
+
|
|
1696
|
+
For EACH dismissed entry, call `recordVerdict({ verdict: "dismissed", … }, home)`
|
|
1697
|
+
(`scripts/library/store.ts`) — injecting the run-context provenance the report cannot know:
|
|
1698
|
+
`runId`, `ts` (nowIso), `operatorInvocation` (verbatim brief, D2 parity), and
|
|
1699
|
+
`entityFingerprint` (the SAME value used at Step 8.5b's fold). This lands the dismissal in
|
|
1700
|
+
the per-entity **unified verdict ledger** (`verdict-ledger.json`), which suppresses a
|
|
1701
|
+
semantically-matching finding on the NEXT run (Step 8.5b).
|
|
1702
|
+
|
|
1703
|
+
Approvals are ALSO recorded to the SAME unified ledger: `writeApprovedFinding` mirrors an
|
|
1704
|
+
`"approved"` verdict ADDITIVELY when passed `entityFingerprint` + the failure-mode triple
|
|
1705
|
+
(the approve→pattern promote is byte-unchanged). The ledger is therefore the complete
|
|
1706
|
+
finding-verdict MEMORY — "what I found (approved) vs what was dismissed".
|
|
1707
|
+
|
|
1708
|
+
> **Layer boundary (do NOT cross).** Dismissals are a RUNTIME per-entity verdict — they
|
|
1709
|
+
> live ONLY in the library `verdict-ledger.json`. NEVER auto-write `.meta/learnings.yaml`
|
|
1710
|
+
> (skill-design layer, human-authored). A *pattern* of dismissals that reveals the tool
|
|
1711
|
+
> over-reports is a metalearn observation a human folds into `.meta/` — never automatic.
|
|
1712
|
+
|
|
1713
|
+
## Step 12 — Self-Diagnostics Check [INTERNAL] (PR-022) — REPORT-ONLY
|
|
1714
|
+
|
|
1715
|
+
> **Integration status: WIRED.** This step *is* the "Orchestrator integration:
|
|
1716
|
+
> self-diagnostics check in `orchestrator-protocol.md`" item tracked in
|
|
1717
|
+
> `references/internal/self-diagnostics.md` → §v0.1 Scope. The 2-line stub it
|
|
1718
|
+
> replaced (probe + dispatch with no consumer) is gone; the full report-only
|
|
1719
|
+
> sub-pipeline is described below. (The maintainer of `self-diagnostics.md` may
|
|
1720
|
+
> now tick that scope checkbox — that file is owned there, not here.)
|
|
1721
|
+
|
|
1722
|
+
Self-diagnostics is **skill-maintainer-only, on-demand only, and REPORT-ONLY**:
|
|
1723
|
+
it surfaces a meta-report about the diagnostics workflow itself and STOPS. It
|
|
1724
|
+
**never** dispatches an apply-worker, never opens a PR, never mutates anything.
|
|
1725
|
+
The host runtime being diagnosed is the current Claude Code session (the skill
|
|
1726
|
+
diagnosing ITSELF), so the source platform is always `claude-code`.
|
|
1727
|
+
|
|
1728
|
+
Gate: only runs when `config.yaml: self_diagnostics.enabled == true` (default
|
|
1729
|
+
`false` — explicit opt-in, see `references/internal/self-diagnostics.md`).
|
|
1730
|
+
|
|
1731
|
+
### 12.1 — Probe + dispatch descriptor
|
|
1732
|
+
|
|
1733
|
+
```bash
|
|
1734
|
+
Bash("scripts/cli/run.sh scripts/self-diagnostics/probe.ts") # detect host runtime
|
|
1735
|
+
Bash("scripts/cli/run.sh scripts/self-diagnostics/dispatch.ts") # write pending.json descriptor
|
|
1736
|
+
```
|
|
1737
|
+
|
|
1738
|
+
`probe.ts` resolves the Claude Code session JSONL dir; `dispatch.ts` writes the
|
|
1739
|
+
`SelfDiagnosticsDispatchDescriptor` to
|
|
1740
|
+
`.mutagent/diagnostics/self-diagnostics/pending.json` (gitignored).
|
|
1741
|
+
|
|
1742
|
+
### 12.2 — The SKILL-typed EntityContext (self-diag entity)
|
|
1743
|
+
|
|
1744
|
+
This is a self-diagnosis path, NOT a platform trace read — it does NOT go through
|
|
1745
|
+
`read-unitf.ts`. The **skill-typed** `EntityContext` (`entityType:"skill"`,
|
|
1746
|
+
`codeAccess:true`) is built by `buildSkillSelfEntityContext` (kept in
|
|
1747
|
+
`scripts/normalize/platforms/entity-context.ts` — deterministic, no LLM) from the
|
|
1748
|
+
skill's own `SKILL.md` + session, and is emitted by `dispatch.ts` at Step 12.1:
|
|
1749
|
+
the `SelfDiagnosticsDispatchDescriptor` written to
|
|
1750
|
+
`.mutagent/diagnostics/self-diagnostics/pending.json` already carries it under
|
|
1751
|
+
`descriptor.entityContext`.
|
|
1752
|
+
|
|
1753
|
+
Extract that block into the file the enricher consumes at Step 12.3:
|
|
1754
|
+
|
|
1755
|
+
```bash
|
|
1756
|
+
Bash("scripts/cli/run.sh scripts/self-diagnostics/dispatch.ts") # Step 12.1 — writes pending.json (incl. entityContext)
|
|
1757
|
+
# descriptor.entityContext → /tmp/self-entity-context.json (skill-typed EntityContext)
|
|
1758
|
+
```
|
|
1759
|
+
|
|
1760
|
+
- The self-diag entity comes from `dispatch.ts` (not the deleted per-platform
|
|
1761
|
+
claude-code normalizer — the UniTF flip removed that transport).
|
|
1762
|
+
- `/tmp/self-entity-context.json` is fed to the enricher at Step 12.3 via
|
|
1763
|
+
`--skill-entity`.
|
|
1764
|
+
|
|
1765
|
+
### 12.3 — Enrich with `--self-diag` (flips on the meta-report invariants)
|
|
1766
|
+
|
|
1767
|
+
Pass the Step 12.2 skill-typed entity to the enricher via the **paired**
|
|
1768
|
+
`--self-diag --skill-entity <path>` flags (SD-3). **`--self-diag` is what flips
|
|
1769
|
+
the enricher into meta-report mode** — it forces `isMetaReport:true`,
|
|
1770
|
+
`audience:internal`, an `[INTERNAL]` sessionId prefix, and cluster-grouped layout
|
|
1771
|
+
(see `references/internal/self-diagnostics.md` → "Self-Diagnosis Report Mode").
|
|
1772
|
+
`--skill-entity` supplies the skill-typed `EntityContext` that the self-diag path
|
|
1773
|
+
renders as the diagnosed entity:
|
|
1774
|
+
|
|
1775
|
+
```bash
|
|
1776
|
+
Bash("scripts/cli/run.sh scripts/enrich/build-render-input.ts \
|
|
1777
|
+
--tier0 /tmp/tier0-out.json \
|
|
1778
|
+
--slice-plan /tmp/slice-plan.json \
|
|
1779
|
+
--findings /tmp/findings.json \
|
|
1780
|
+
--metadata /tmp/traces-metadata.json \
|
|
1781
|
+
--self-diag \
|
|
1782
|
+
--skill-entity /tmp/self-entity-context.json \
|
|
1783
|
+
--output /tmp/render-input.json")
|
|
1784
|
+
```
|
|
1785
|
+
|
|
1786
|
+
- `--self-diag` + `--skill-entity` are **paired and fail-loud**: each REQUIRES the
|
|
1787
|
+
other (the enricher exits non-zero if only one is supplied). Neither is useful
|
|
1788
|
+
alone — `--skill-entity` without `--self-diag` loads an entity the enricher
|
|
1789
|
+
would ignore; `--self-diag` without `--skill-entity` has no skill entity to
|
|
1790
|
+
render.
|
|
1791
|
+
- **`--entity-context` is the WRONG flag here** — it supplies a *generic* entity
|
|
1792
|
+
for a normal client diagnosis and does NOT touch the self-diag gate (it never
|
|
1793
|
+
sets `isMetaReport`). The meta-report invariants come from `--self-diag` only.
|
|
1794
|
+
|
|
1795
|
+
### 12.4 — Render the meta-report + STOP
|
|
1796
|
+
|
|
1797
|
+
```bash
|
|
1798
|
+
Bash("scripts/cli/run.sh scripts/report/render.ts /tmp/render-input.json")
|
|
1799
|
+
```
|
|
1800
|
+
|
|
1801
|
+
`render.ts` REFUSES `--audience client` when `isMetaReport:true` (PR-022). The
|
|
1802
|
+
meta-report HTML is the deliverable. **No Step 11 apply gate fires for
|
|
1803
|
+
self-diag** — `target.platform` is effectively `report-only`, so the pipeline
|
|
1804
|
+
HALTS at the rendered report. Maintainer reviews; nothing is applied.
|
|
1805
|
+
|
|
1806
|
+
Reference: `references/internal/self-diagnostics.md`.
|
|
1807
|
+
|
|
1808
|
+
---
|
|
1809
|
+
|
|
1810
|
+
## Script Execution Discipline (R-SELF-03-c — MANDATORY)
|
|
1811
|
+
|
|
1812
|
+
**Never inline Python heredocs or multi-line shell scripts in Bash() calls.**
|
|
1813
|
+
|
|
1814
|
+
Always call a published script via `scripts/cli/run.sh`:
|
|
1815
|
+
|
|
1816
|
+
```bash
|
|
1817
|
+
# CORRECT
|
|
1818
|
+
Bash("scripts/cli/run.sh scripts/normalize/read-unitf.ts --in /tmp/traces.jsonl --out-metadata /tmp/traces-metadata.json")
|
|
1819
|
+
Bash("scripts/cli/run.sh scripts/tier0-scan.ts /tmp/traces.json")
|
|
1820
|
+
|
|
1821
|
+
# BANNED
|
|
1822
|
+
Bash("python3 << 'EOF'\nimport json\n...\nEOF")
|
|
1823
|
+
```
|
|
1824
|
+
|
|
1825
|
+
## Monitor Tool Compliance (R-SELF-12-a)
|
|
1826
|
+
|
|
1827
|
+
**Never** use `Bash("sleep N && cat <file>")`.
|
|
1828
|
+
**Use** Monitor tool with `until` loop:
|
|
1829
|
+
|
|
1830
|
+
```
|
|
1831
|
+
Monitor: until test -f /tmp/findings.json; do sleep 2; done
|
|
1832
|
+
```
|
|
1833
|
+
|
|
1834
|
+
## Branch Hygiene (PR-004)
|
|
1835
|
+
|
|
1836
|
+
Never apply fixes directly to the operator's checked-out branch.
|
|
1837
|
+
All local applies happen in an isolated git worktree (enforced by the shared
|
|
1838
|
+
`mutagent-cli apply` worktree-PR adapter, driven by ai-engineer; the bespoke
|
|
1839
|
+
apply-worker is retired, M9). PRs are the delivery mechanism.
|
|
1840
|
+
|
|
1841
|
+
## Design Principles in Effect
|
|
1842
|
+
|
|
1843
|
+
- PR-001: Tier 0 before LLM (Step 4 above)
|
|
1844
|
+
- PR-002: RCA layer mandatory (Step 8)
|
|
1845
|
+
- PR-004: Branch hygiene (Step 11)
|
|
1846
|
+
- PR-005: Cap-of-5 on parallel analyzers (Step 6)
|
|
1847
|
+
- PR-014: HITL via HTML + markdown copy-back (Step 10)
|
|
1848
|
+
- PR-017: Dynamic-cluster slicing (Step 5)
|
|
1849
|
+
- PR-018: Evidence-grounded RCA (Step 8)
|
|
1850
|
+
- PR-019: Scripts vs agent ops — see `references/operation-inventory.md`
|
|
1851
|
+
- PR-020: Recursive whys to failure origin (Step 8)
|
|
1852
|
+
- PR-022: Self-diagnostics [INTERNAL] (Step 12)
|
|
1853
|
+
- PR-023: Clipboard payloads = self-contained actionable plans (PR-023)
|
|
1854
|
+
- PR-035: Deep-read gate mandatory (Step 6 pre-gate)
|
|
1855
|
+
- PR-037: Library-first match + class memory (Step 4.5); best-effort — empty library proceeds fresh
|
|
1856
|
+
- PR-040: Operator-driven single-arg invocation (Step 3a parse-brief)
|
|
1857
|
+
- PR-043: Methodology decision-logging (Steps 5b · 6 · 8.9 · 11.0)
|
|
1858
|
+
- PR-044: Normalize before analyze — Step 3.7 MANDATORY (Step 3.7)
|
|
1859
|
+
- PR-048: Trace hungriness — tiered deep-read scales with population (Step 6 escalation loop)
|
|
1860
|
+
- W18-context: Grounded diagnosis-context lens built before fan-out; analyzer MANDATORY-PRE-READ (Step 5.7 + Step 6 dispatch). Landed principles: PR-054 (problem-statement-format) + PR-055 (ground-dont-infer) in `.meta/design-principles.md` (proposals archived at `.memory/features/mutagent-diagnostics/PROPOSED/archive/`).
|
|
1861
|
+
- W18-cache: Grounded cache detection — `cacheStatus` field only; absent ⇒ UNKNOWN, never "uncached"; never infer from flat `promptTokens` (Step 6 dispatch cache rule)
|