@mutagent/helix 0.1.0-alpha.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/skills/mutagent-helix/SKILL.md +113 -0
- package/.claude/skills/mutagent-helix/scripts/cli/codex-transcode.ts +143 -0
- package/.claude/skills/mutagent-helix/scripts/cli/doctor.ts +111 -0
- package/.claude/skills/mutagent-helix/scripts/cli/init.ts +382 -0
- package/.claude/skills/mutagent-helix/scripts/cli/run.sh +61 -0
- package/CLAUDE.md +193 -0
- package/LICENSE +201 -0
- package/NOTICE +22 -0
- package/README.md +209 -0
- package/assets/adl-loop.svg +162 -0
- package/assets/system-status.svg +422 -0
- package/bin/mutagent-cli.mjs +9950 -0
- package/bin/mutagent-helix.mjs +78 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/CLAUDE.md +28 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/SKILL.md +138 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/assets/templates/agentspec.yaml.tpl +247 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/references/frameworks/doc-pins.md +53 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/references/principles.md +91 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/references/requirements.yaml +295 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/references/workflows/orchestrator-protocol.md +302 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/cli/doctor.ts +116 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/cli/init.ts +230 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/cli/run.sh +60 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/config/resolve-spec-dir.ts +199 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/contract/agentspec.schema.ts +455 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/setup/detect.ts +67 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/validate/validate-spec.ts +93 -0
- package/mutagent-agentspec/LICENSE +201 -0
- package/mutagent-agentspec/NOTICE +19 -0
- package/mutagent-agentspec/README.md +44 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/SKILL.md +100 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/assets/agents/ai-architect.md +312 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/assets/agents/ai-engineer.md +357 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/assets/templates/build-report.md.tpl +39 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/references/principles.md +31 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/references/workflows/build-protocol.md +61 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/references/workflows/improve-handoff.md +23 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/scripts/cli/run.sh +60 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/scripts/handoff/validate-build-input.ts +72 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/scripts/sync-spec/check-sync-spec.ts +499 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/scripts/verify/spec-impl-coverage.ts +182 -0
- package/mutagent-builder/CLAUDE.md +25 -0
- package/mutagent-builder/README.md +26 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/README.md +192 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/SKILL.md +352 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/agents/ai-architect.md +256 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/agents/ai-engineer.md +344 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/agents/diagnostics-analyzer.md +576 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/agents/discovery.md +425 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/audit.json.tpl +24 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/audit.md.tpl +49 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/config.yaml.tpl +100 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/pr-body.md.tpl +56 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/report.html.tpl +327 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/README.md +27 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/iter-N-handover.md.tpl +125 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/self-diagnosis-contract.v0.1.0.yaml.tpl +234 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/spec.yaml.tpl +152 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/team.yaml.tpl +127 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/wave-N-dashboard.html.tpl +1712 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.1-nl-query.md +59 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.2-tier0-progress.md +58 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.3-analysis-progress.md +44 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.4-html-report.md +61 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.5-rca-detail.md +69 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.6-stale-warning.md +53 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.1-welcome.md +60 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.2-source-platform-picker.md +41 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.3-source-auth.md +75 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.4-target-picker.md +48 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.5-ask-tool.md +43 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.6-config-review.md +57 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.7-filter-defaults.md +52 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.8-first-run.md +59 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/examples/sample-findings.json +92 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/examples/sample-meta.json +47 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/adapter-strategy.md +75 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/config-migration.md +84 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/config.md +159 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/filter-search-matrix.md +66 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/harness-knowledge.md +22 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/memory-format.md +78 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/onboarding-decisions.yaml +187 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/operation-inventory.md +59 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/overview.md +167 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/reference.md +119 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/script-index.md +101 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-adapter-contract.md +93 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/claude-code-transcripts.md +66 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/codex-transcripts.md +78 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/codex.md +145 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/install-paths.md +57 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/langfuse.md +178 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/local-jsonl.md +55 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/openobserve.md +102 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/otel.md +63 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/cloud-rest.md +84 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-claude.md +65 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-cloud-agent-sdk.md +92 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-codex.md +72 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-cursor.md +108 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-mastra.md +77 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-opencode.md +70 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/apply-pr-comment-format.md +82 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/diagnostics.md +35 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/handover-contract.md +247 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/onboarding.md +300 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/orchestrator-protocol.md +1861 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/rca.md +331 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/rendering-anatomy.md +314 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/schedule-prep.md +100 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/scope-model.md +128 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/verification-methodology.md +174 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/aggregate/sort-findings.ts +0 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/awareness/blind-spots.ts +167 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/awareness/llm-sample.ts +681 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/doctor.ts +215 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/init.ts +1013 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/install-agents.ts +350 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/run.sh +61 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/shared-templates.ts +79 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/config/load.ts +392 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/config/resolve-credential.ts +120 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/config/schema.ts +641 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/config/validate.ts +134 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/context/build-diagnosis-context.ts +417 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/context/load-context.ts +105 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/contract/types.ts +166 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/enrich/build-render-input.ts +1964 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/enrich/corpus-severity.ts +172 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/enrich/dismissal-match.ts +270 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/enrich/rank-remedies.ts +75 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/hooks/install-hooks.sh +46 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/hooks/pre-push.sh +78 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/invocation/parse-brief.ts +188 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/library/match.ts +114 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/library/paths.ts +95 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/library/store.ts +535 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/library/types.ts +181 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/lint/template-inline-js.ts +267 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/memory/append.ts +188 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/memory/read.ts +218 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/platforms/entity-context.ts +686 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/read-unitf.ts +267 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/trace.ts +1078 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/unitf-adapter.ts +0 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/unitf-types.ts +213 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/report/persist-selections.ts +174 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/report/render.ts +3254 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/run/diagnose.ts +64 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/run/session.ts +167 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/sample/caps.ts +345 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/sample/deep-read-gate.ts +424 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/sample/representative.ts +584 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/scan/objection.ts +224 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/scan/trajectory.ts +489 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/self-diagnostics/dispatch.ts +135 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/self-diagnostics/probe.ts +170 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/setup/detect.ts +307 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/setup/ensure-cli.ts +315 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/setup/reconfigure.ts +77 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/setup/verify-agents.ts +207 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/slicer.ts +166 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/claude-code.ts +17 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/codex.ts +11 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/langfuse.ts +17 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/local-jsonl.ts +12 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/otel.ts +12 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/index.ts +54 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/registry.ts +45 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/types.ts +86 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/stale-detector.ts +74 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/test/render-playwright.cli.ts +477 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/tier0/claude-code.ts +133 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/tier0/langfuse.ts +143 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/tier0-scan.ts +423 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/completeness-check.ts +436 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/doc-refs.ts +190 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/finalize-gate.ts +417 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/findings-contract.ts +628 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/render-js-syntax.ts +188 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/report-checklist.yaml +217 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/trace-shape.ts +262 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/wave6-checklist.ts +326 -0
- package/mutagent-diagnostics/CLAUDE.md +41 -0
- package/mutagent-diagnostics/LICENSE +201 -0
- package/mutagent-diagnostics/NOTICE +18 -0
- package/mutagent-diagnostics/README.md +343 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/SKILL.md +548 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/agents/audit-executor.md +169 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/agents/dataset-builder.md +160 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/agents/discovery.md +425 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/agents/evaluator.md +1108 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/brand/theme.css +213 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/brand/wordmark.html +9 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/context-flow-lens.md +85 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/data-lens.md +47 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/decision-lens.md +48 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/methodology-critic-lens.md +53 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/trajectory-lens.md +44 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/build-review-interface.md +83 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/edd-loop.md +134 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/error-analysis.md +113 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/eval-audit.md +154 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/eval-stage.md +168 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/generate-synthetic-data.md +81 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/grounded-adjudication.md +221 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/memory-format.md +65 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/methodology.md +201 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/operation-inventory.md +197 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/validate-evaluator.md +125 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/workflows/orchestrator-protocol.md +300 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/write-judge-prompt.md +123 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/behavior-tree.schema.yaml +73 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/dataset.schema.yaml +66 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/edd-change-request.schema.yaml +114 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/eval-matrix.schema.yaml +74 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/flow-graph.schema.yaml +69 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/flow-profile.schema.yaml +49 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/methodology-review.schema.yaml +40 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/scorecard.schema.yaml +85 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/agent-dispatch.ts +0 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/aggregate-discover.ts +543 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/artifact-paths.ts +115 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/assemble-scorecard.ts +172 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/build-dataset.ts +186 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/build-evals.ts +93 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/build-review-ui.ts +393 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/check-method-router.ts +170 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/aggregate.ts +112 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/audit-run.ts +182 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/doctor.ts +211 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/dogfood.ts +133 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/init.ts +601 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/methodology-review.ts +122 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/prep.ts +295 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/profile-subject.ts +165 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/run.sh +56 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/variance-check.ts +105 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/code-eval.ts +248 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/code-quality-verdict.ts +285 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/codegen-evals.ts +173 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cold-start-project.ts +151 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cold-start-sampler.ts +267 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/config/load.ts +343 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/config/schema.ts +380 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/agentspec-evals.ts +85 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/dataset.ts +149 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/eval-engine.ts +118 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/eval-matrix.ts +577 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/eval-types.ts +1074 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/flow-graph.ts +194 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/types.ts +303 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/validation.ts +193 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/derive-dataset.ts +152 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/determine-outcome.ts +219 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/diff-discriminate.ts +160 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/discover-criteria.ts +348 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/edd/change-request.ts +232 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/edd/edd-types.ts +210 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/edd/variance-gate.ts +186 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/emit-completeness.ts +111 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/eval-engine.ts +160 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/evaluate.ts +333 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/flow-graph.ts +230 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/judge-prompt-template.ts +253 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/judge-provider.ts +135 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/lint-grounding.ts +229 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/lint-uniformity.ts +168 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/living-suite.ts +109 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/load-bundle.ts +203 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/load-profile-vocab.ts +64 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/load-profile.ts +106 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/mask.ts +138 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/materialize-dataset.ts +113 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/matrix-judge.ts +750 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/memory/append.ts +215 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/memory/read.ts +168 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/persist-eval-criteria.ts +232 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/prep-tasks.ts +125 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/profile-subject.ts +310 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/publish-report.ts +131 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/read-unitf-traces.ts +81 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/render-build-cards.ts +195 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/render-discover-report.ts +1640 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/render-eval-report.ts +3838 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/render-report.ts +212 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/resolve-credential.ts +110 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/resolve-ref.ts +98 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/result-verify.ts +129 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/route-failures.ts +330 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/run-deterministic.ts +367 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/run-evaluate.ts +719 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/run-judge.ts +155 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/run-pipeline.ts +175 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/sample-traces.ts +210 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/self-audit.ts +387 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/source-map.ts +106 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/subject-profile.ts +134 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/substrate.ts +162 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/sync-eval-criteria.ts +244 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/ui-slots.ts +119 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/unitf-to-evaltrace.ts +284 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/validate-judge.ts +358 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/variance-compare.ts +177 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/subjects/mutagent-diagnostics/behavior-tree.yaml +140 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/subjects/mutagent-diagnostics/eval-matrix.yaml +1270 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/subjects/mutagent-diagnostics/methodology-review.yaml +105 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/workflows/audit.workflow.js +82 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/workflows/data-leak.workflow.js +236 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/workflows/variance.workflow.js +163 -0
- package/mutagent-evaluator/CLAUDE.md +45 -0
- package/mutagent-evaluator/LICENSE +201 -0
- package/mutagent-evaluator/NOTICE +23 -0
- package/mutagent-evaluator/README.md +88 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/SKILL.md +287 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/scripts/contracts/amend-request.ts +503 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/scripts/improve-loop-run.ts +304 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/scripts/loop-state-cli.ts +300 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/scripts/loop-state.ts +440 -0
- package/mutagent-optimize/CLAUDE.md +49 -0
- package/mutagent-optimize/README.md +26 -0
- package/mutagent-orchestrator/CLAUDE.md +102 -0
- package/mutagent-orchestrator/README.md +20 -0
- package/mutagent-orchestrator/assets/agents/discovery.md +425 -0
- package/mutagent-orchestrator/assets/agents/dogfood-monitor.md +266 -0
- package/mutagent-orchestrator/assets/agents/monitor.md +258 -0
- package/mutagent-orchestrator/orchestrator.md +782 -0
- package/mutagent-orchestrator/references/MONITORING.md +209 -0
- package/mutagent-orchestrator/references/config-fields.md +132 -0
- package/mutagent-orchestrator/references/config-migration.md +132 -0
- package/mutagent-orchestrator/references/index-registry.md +92 -0
- package/mutagent-orchestrator/routing.yaml +391 -0
- package/mutagent-orchestrator/scripts/config-schema.ts +1010 -0
- package/mutagent-orchestrator/scripts/dispatch.ts +439 -0
- package/mutagent-orchestrator/scripts/dogfood/extract-feedback.ts +203 -0
- package/mutagent-orchestrator/scripts/dogfood/reconstruct-trajectory.ts +327 -0
- package/mutagent-orchestrator/scripts/dogfood/render-dogfood-report.ts +347 -0
- package/mutagent-orchestrator/scripts/dogfood/resolve-subagents.ts +160 -0
- package/mutagent-orchestrator/scripts/dogfood/run-report.ts +264 -0
- package/mutagent-orchestrator/scripts/dogfood/slack-thread.ts +106 -0
- package/mutagent-orchestrator/scripts/dogfood/types.ts +257 -0
- package/mutagent-orchestrator/scripts/eval-routing.ts +753 -0
- package/mutagent-orchestrator/scripts/gate.ts +219 -0
- package/mutagent-orchestrator/scripts/handover-contract.ts +362 -0
- package/mutagent-orchestrator/scripts/index/build-index.ts +351 -0
- package/mutagent-orchestrator/scripts/index/render-index.ts +110 -0
- package/mutagent-orchestrator/scripts/monitor/slack-notify.ts +111 -0
- package/mutagent-orchestrator/scripts/monitor/triggers.ts +162 -0
- package/mutagent-orchestrator/scripts/onboarding-check.ts +369 -0
- package/mutagent-orchestrator/scripts/render-roster.ts +308 -0
- package/mutagent-orchestrator/scripts/resolve-credential.ts +185 -0
- package/mutagent-orchestrator/scripts/resolve-paths.ts +114 -0
- package/mutagent-orchestrator/scripts/slack/format.ts +63 -0
- package/mutagent-orchestrator/scripts/slack/post.ts +115 -0
- package/mutagent-orchestrator/scripts/sync-index.ts +311 -0
- package/package.json +64 -0
|
@@ -0,0 +1,331 @@
|
|
|
1
|
+
# RCA Layer — Root Cause Analysis Procedure
|
|
2
|
+
|
|
3
|
+
> Renamed from "Translation Layer" (operator iter-6 T6).
|
|
4
|
+
> 3-dimensional WHAT/WHY/WHERE taxonomy + recursive whys until failure origin.
|
|
5
|
+
|
|
6
|
+
## Step 0 Construct scope
|
|
7
|
+
|
|
8
|
+
**I-021 + I-030: Mandatory before any hypothesis or signal enumeration.**
|
|
9
|
+
|
|
10
|
+
Define the scope of the diagnostic run BEFORE generating any hypothesis:
|
|
11
|
+
|
|
12
|
+
1. **What is IN scope**: specify agent IDs, time window, platforms, session range, or `TraceFilter.skillAgentScope` entries.
|
|
13
|
+
2. **What is OUT of scope**: explicitly name what will NOT be analyzed (e.g., dependent services, pre-existing flakiness, traces older than N days).
|
|
14
|
+
3. **Sampling strategy**: if more traces exist than the LLM budget allows, document how the sample was drawn (e.g., "top-N by negative score", "stratified by error rate", "random seed 42"). Populate `RunMeta.samplingStrategy` for the Methodology tab renderer.
|
|
15
|
+
4. **Scope filter expression**: record the filter as `RunMeta.scopeFilter` (freeform, e.g., `"agentId=search-agent, last 7d, hasFeedback=true"`).
|
|
16
|
+
|
|
17
|
+
> **Why this matters**: Scope discipline prevents false attributions. A finding that is "out of scope" for this session is NOT a finding — it belongs to a separate diagnostic run. Without an explicit scope boundary, analysts overgeneralize from a sample.
|
|
18
|
+
|
|
19
|
+
**I-030 Scope/Sampling methodology** — practitioners completing `run-meta.json`:
|
|
20
|
+
- `totalTraces`: total traces in the platform matching the query (pre-sampling)
|
|
21
|
+
- `tier0ScannedCount`: how many were seen by tier-0 static scan
|
|
22
|
+
- `llmReadCount`: how many were sent to LLM analyzers
|
|
23
|
+
- `scopeFilter`: human-readable filter expression (copied from Step 3 NL→TraceFilter translation)
|
|
24
|
+
- `samplingStrategy`: e.g., `"top-20 by negative normalizedScore"` or `"stratified: 5 per error type"`
|
|
25
|
+
- `decisions`: log key choices (model, threshold, excluded platforms, session limits)
|
|
26
|
+
|
|
27
|
+
> **PR-043 — Methodology decision-logging**: every methodology decision made
|
|
28
|
+
> during the run (slicer override, dispatch count, exemption, apply-skip) MUST
|
|
29
|
+
> be recorded in the corresponding `runMeta` append-only field. See
|
|
30
|
+
> `references/principles.md` PR-043 and `references/workflows/orchestrator-protocol.md`
|
|
31
|
+
> Steps 5b · 6 · 8.9 · 11.0 for the full field map. Silent skips are a protocol
|
|
32
|
+
> violation — reproducibility depends on this trail.
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## Step 0 Signal census
|
|
37
|
+
|
|
38
|
+
**I-031 (PR-028 earn-keep + PR-049): Enumerate ALL signals BEFORE generating any hypothesis; select primary by the reconciled process.**
|
|
39
|
+
|
|
40
|
+
After scope is established, run a complete signal census using the PR-049 5-step detect+select process:
|
|
41
|
+
|
|
42
|
+
1. Execute `scripts/tier0-scan.ts` on the scoped trace set.
|
|
43
|
+
2. Record every signal type present: errors, feedback clusters, low-score traces, high-latency traces, API exhaustion, skill-behavior-deviation counts.
|
|
44
|
+
3. **Failure-validity gate** — rule out benign observability artifacts (e.g. `low-tagging-rate`) before scoring. Ruled-out signals appear in the census with a "ruled out — benign" label.
|
|
45
|
+
4. **Impact×prevalence scoring** — score = impact × (matchCount / total). Never use raw frequency alone as the primacy criterion.
|
|
46
|
+
5. **Deep-read corroboration** — after LLM analysis, reconcile with `findings[0].failureOrigin.what`. The LLM-found primary confirms or overrides the static rank. Emit ONE `runMeta.primarySignal` driving census·heatmap·funnel.
|
|
47
|
+
6. Only AFTER the full census + primary selection is documented, proceed to hypothesis generation.
|
|
48
|
+
|
|
49
|
+
> **Principle PR-028**: No diagnosis without signal census first. The census ensures all signal types are considered, even if most are subsequently dismissed.
|
|
50
|
+
>
|
|
51
|
+
> **Principle PR-049**: Primary signal MUST be selected by the reconciled 5-step process (failure-validity gate → impact×prevalence → deep-read corroboration). Never a frequency artifact.
|
|
52
|
+
|
|
53
|
+
**Anti-pattern** (PR-028 + PR-049 violation): "I see lots of `low-tagging-rate` patterns so the problem is X" — `low-tagging-rate` is a benign observability artifact, not a failure WHAT; it is ruled out by step 3 above. Check feedback, score, latency signals first.
|
|
54
|
+
|
|
55
|
+
---
|
|
56
|
+
|
|
57
|
+
## Flow
|
|
58
|
+
|
|
59
|
+
```
|
|
60
|
+
Feedback OR Issue (raw from trace)
|
|
61
|
+
↓
|
|
62
|
+
Step 0: Construct scope (I-021 — what's in/out, sampling strategy, RunMeta fields)
|
|
63
|
+
↓
|
|
64
|
+
Step 0: Signal census (I-031 — enumerate ALL signals before any hypothesis; PR-028)
|
|
65
|
+
↓
|
|
66
|
+
Clustered Feedback + Issues (cross-actor dedup + cluster correlation)
|
|
67
|
+
↓
|
|
68
|
+
Root Cause Analysis
|
|
69
|
+
- 3-dim WHAT/WHY/WHERE + evidences + reference IDs
|
|
70
|
+
- Read-definition discipline (I-022) + Assumption enumeration (I-023)
|
|
71
|
+
↓
|
|
72
|
+
Remedy Chain (ranked by cost × correctness — each = failureOrigin + before/after diff)
|
|
73
|
+
↓
|
|
74
|
+
HTML Report (gold-standard multi-tab presentation — Methodology · Overview · per-finding (one tab per finding) · Decisions; built via the Step 8.5 enricher)
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
## Taxonomy
|
|
78
|
+
|
|
79
|
+
### WHAT (symptom) — 10 categories
|
|
80
|
+
|
|
81
|
+
| Category | Looks like |
|
|
82
|
+
|----------|-----------|
|
|
83
|
+
| `wrong-output` | Agent returned incorrect answer (factually, structurally, or semantically) |
|
|
84
|
+
| `missing-output` | Agent returned empty / refused / gave up |
|
|
85
|
+
| `loop` | Agent repeated the same operation N times without progress |
|
|
86
|
+
| `latency-spike` | Trace took dramatically longer than baseline |
|
|
87
|
+
| `cost-overshoot` | Trace consumed more tokens / dollars than budget |
|
|
88
|
+
| `format-violation` | Output didn't match expected schema (JSON parse fail, wrong shape) |
|
|
89
|
+
| `hallucination` | Agent invented a fact / cited non-existent source |
|
|
90
|
+
| `user-complaint` | Embedded user feedback expressed dissatisfaction |
|
|
91
|
+
| `low-score` | Score signal below threshold (after scale auto-discovery) |
|
|
92
|
+
| `missing-context` | Agent lacked needed context — missing tools, OR tools/data exist but stale, OR empty results |
|
|
93
|
+
|
|
94
|
+
### WHY (root cause type) — 9 categories
|
|
95
|
+
|
|
96
|
+
| Category | Looks like |
|
|
97
|
+
|----------|-----------|
|
|
98
|
+
| `prompt-underspec` | Agent did X because the prompt didn't tell it not to |
|
|
99
|
+
| `prompt-overspec` | Prompt contradicts itself; agent picked wrong arm |
|
|
100
|
+
| `tool-misuse` | Right tool, wrong args / wrong order |
|
|
101
|
+
| `tool-missing` | Needed a tool that wasn't available |
|
|
102
|
+
| `context-overflow` | Truncation / summarization cut required information |
|
|
103
|
+
| `provider-limit` | Model refused / quota / rate limit / safety filter |
|
|
104
|
+
| `data-staleness` | Agent used stale info; needed refresh |
|
|
105
|
+
| `handoff-loss` | Multi-agent flow lost context between handoffs |
|
|
106
|
+
| `dependency-failure` | Upstream service / API / DB failed |
|
|
107
|
+
|
|
108
|
+
### WHERE (origin location) — 8 categories
|
|
109
|
+
|
|
110
|
+
| Category | Looks like |
|
|
111
|
+
|----------|-----------|
|
|
112
|
+
| `system-prompt` | The agent's system prompt (e.g., `.claude/agents/<id>.md:34`) |
|
|
113
|
+
| `tool-definition` | Tool schema / description / parameter spec |
|
|
114
|
+
| `agent-config` | Model selection, temperature, max-tokens, thinkingBudget |
|
|
115
|
+
| `routing-config` | How requests get routed to this agent vs another |
|
|
116
|
+
| `upstream-data` | Input data shape / quality (RAG index, DB row, etc.) |
|
|
117
|
+
| `provider-side` | LLM provider behavior (model deprecated, rate limited, safety-filtered) |
|
|
118
|
+
| `harness-side` | Coding-agent runtime (Claude Code, Codex) — context window, tool injection bugs |
|
|
119
|
+
| `user-input` | Ambiguous or out-of-scope user prompt |
|
|
120
|
+
|
|
121
|
+
## Recursive Why-Chain
|
|
122
|
+
|
|
123
|
+
No fixed depth. Keep asking "why" until failure origin is identified.
|
|
124
|
+
|
|
125
|
+
Rules:
|
|
126
|
+
1. Each `why` entry must have a corresponding `evidence` pointer (trace message index, file:line, code pointer)
|
|
127
|
+
2. `isOrigin: true` marks the deepest causal level — the fix lives here
|
|
128
|
+
3. Never stop at "the agent made an error" — that's a symptom, not an origin
|
|
129
|
+
|
|
130
|
+
Example:
|
|
131
|
+
```
|
|
132
|
+
WHY: Agent re-called search_docs 11x
|
|
133
|
+
evidence: tr_abc msg 4-15
|
|
134
|
+
WHY: Results not deduplicated
|
|
135
|
+
evidence: tool-call outputs identical across calls
|
|
136
|
+
WHY: No dedup instruction in prompt ← ORIGIN
|
|
137
|
+
evidence: .claude/agents/search-agent.md:34
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
## Finding shape
|
|
141
|
+
|
|
142
|
+
```typescript
|
|
143
|
+
{
|
|
144
|
+
findingId: "F-001",
|
|
145
|
+
actionable: "Agent re-called search_docs 11x in trace tr_abc",
|
|
146
|
+
// W18-problem: REQUIRED descriptive problem statement (see "Problem statement" below).
|
|
147
|
+
problem: "Agent re-called search_docs 11× with identical args on 1/40 traces — 11 wasted tool round-trips, no new context gained (tr_abc msgs 4-15).",
|
|
148
|
+
failureOrigin: {
|
|
149
|
+
what: "loop",
|
|
150
|
+
why: "prompt-underspec",
|
|
151
|
+
where: "system-prompt",
|
|
152
|
+
evidence: ".claude/agents/search-agent.md:34 — no dedup directive",
|
|
153
|
+
confidence: "high"
|
|
154
|
+
},
|
|
155
|
+
whyChain: [
|
|
156
|
+
{ why: "Agent re-called search_docs 11x", evidence: "tr_abc msg 4-15" },
|
|
157
|
+
{ why: "Results not deduplicated", evidence: "tool-call outputs identical" },
|
|
158
|
+
{ why: "No dedup instruction in prompt", evidence: "system_prompt:34", isOrigin: true }
|
|
159
|
+
],
|
|
160
|
+
remedies: [
|
|
161
|
+
{
|
|
162
|
+
remedyId: "R-001",
|
|
163
|
+
title: "Add dedup instruction to system prompt",
|
|
164
|
+
failureOrigin: { what: "loop", why: "prompt-underspec", where: "system-prompt" },
|
|
165
|
+
diff: {
|
|
166
|
+
before: "You are a search assistant. Answer questions.",
|
|
167
|
+
after: "You are a search assistant. Answer questions. Deduplicate tool-call results — if a tool returns identical output to a previous call, do not call it again."
|
|
168
|
+
},
|
|
169
|
+
cost: "low",
|
|
170
|
+
correctness: "high",
|
|
171
|
+
rank: 1
|
|
172
|
+
}
|
|
173
|
+
],
|
|
174
|
+
sourceTraceIds: ["tr_abc"],
|
|
175
|
+
referenceIds: { traceId: "tr_abc", sessionId: "s_xyz", findingId: "F-001" }
|
|
176
|
+
}
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
## Problem statement (W18-problem)
|
|
180
|
+
|
|
181
|
+
**Every Finding carries a REQUIRED `problem` field** — the descriptive PRIMARY block of the
|
|
182
|
+
finding panel (rendered at the TOP, before Evidence / why-chain / remedies). It DESCRIBES the
|
|
183
|
+
failure and its measured impact; it is **NOT a task**.
|
|
184
|
+
|
|
185
|
+
**Format** (canonical in `scripts/normalize/trace.ts` `Finding.problem`):
|
|
186
|
+
|
|
187
|
+
```
|
|
188
|
+
<subject> <observed behavior, declarative> — <quantified impact + evidence> [— scope: N/total traces]
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
Describes WHAT is wrong and HOW BAD — not what to do about it. **The fix lives ONLY in
|
|
192
|
+
`remedies`.** A `problem` that LEADS with a bare imperative verb (`make · use · cap · add ·
|
|
193
|
+
reduce · switch · replace · enable · fix · consider · try · avoid · ensure · implement ·
|
|
194
|
+
increase · decrease · move · remove · update · set`) or uses a prescriptive modal (`should` /
|
|
195
|
+
`must` / `needs to`) in its main clause is REJECTED by the W18-gate (`isTaskPhrasedProblem` in
|
|
196
|
+
`scripts/validate/findings-contract.ts` → `RESEND <findingId> with problem(task-phrased)`).
|
|
197
|
+
|
|
198
|
+
**Transform — task phrasing → descriptive phrasing** (generic draft-tool example):
|
|
199
|
+
|
|
200
|
+
| | Statement |
|
|
201
|
+
|---|---|
|
|
202
|
+
| ❌ task-phrased (rejected) | `Make the draft tool faster — use a smaller model.` |
|
|
203
|
+
| ✅ descriptive (accepted) | `The draft tool takes 4.2s p95 — 3.1× the 1.4s session median — on 12/40 traces.` |
|
|
204
|
+
|
|
205
|
+
The transform: strip the imperative verb + the embedded fix; lead with the subject; state the
|
|
206
|
+
observed behavior declaratively; quantify the impact with trace evidence. The remedy ("use a
|
|
207
|
+
smaller model" / batch the calls / …) moves to `remedies` where it belongs. A subject-first,
|
|
208
|
+
past-tense observation is always safe (e.g. "throughput reduced to 12 req/s" leads with the
|
|
209
|
+
subject `throughput`, not a bare verb — never flagged).
|
|
210
|
+
|
|
211
|
+
## Cache / cost detection — read the field, never infer (W18-cache)
|
|
212
|
+
|
|
213
|
+
When a Finding touches prompt-caching, cost, or token volume, cache state comes ONLY from the
|
|
214
|
+
grounded cache-token fields on the trace (`scripts/normalize/trace.ts`): `cacheStatus` ∈
|
|
215
|
+
`hit` | `miss` | `unknown` (derived from `cachedInputTokens` / `cacheCreationTokens`);
|
|
216
|
+
`cacheHitRate` populated ONLY when the cache fields were present.
|
|
217
|
+
|
|
218
|
+
1. `cacheStatus` **absent or `"unknown"`** ⇒ report cache state as **UNKNOWN**, NEVER as
|
|
219
|
+
"uncached". Absence of a cache-token field is NOT evidence of no caching.
|
|
220
|
+
2. **NEVER infer** caching (or its absence) from a flat `promptTokens` value or byte sizes —
|
|
221
|
+
that inference is the motivating miss (caching active ~89%, but a byte-size guess reported
|
|
222
|
+
"uncached → 408M billed tokens").
|
|
223
|
+
3. `miss` (cache fields present, nothing served) is a GROUNDED no-read — distinct from
|
|
224
|
+
`unknown`. Never collapse "we don't know" into "no caching happened".
|
|
225
|
+
|
|
226
|
+
> **Corollary to the Evidence bar (I-032 / PR-018):** a cache/cost claim is observable evidence
|
|
227
|
+
> ONLY when it reads the cache-token field. An inferred uncached assumption is NOT evidence — a
|
|
228
|
+
> `cost-overshoot` Finding built on it is a hypothesis, not a confirmed finding.
|
|
229
|
+
|
|
230
|
+
## Remedy ranking criteria
|
|
231
|
+
|
|
232
|
+
| Axis | Description |
|
|
233
|
+
|------|-------------|
|
|
234
|
+
| cost | low / medium / high (code changes required, risk of regression, operator effort) |
|
|
235
|
+
| correctness | low / medium / high (confidence this fixes the failure origin) |
|
|
236
|
+
| rank | 1 = highest priority (low cost + high correctness) |
|
|
237
|
+
|
|
238
|
+
Always provide at least one remedy per finding. Prefer low-cost high-correctness options at rank 1.
|
|
239
|
+
|
|
240
|
+
## Read-definition discipline
|
|
241
|
+
|
|
242
|
+
**I-022: Every Finding MUST cite the source that defines the deviation.**
|
|
243
|
+
|
|
244
|
+
Before marking any Finding as confirmed:
|
|
245
|
+
1. **Locate the authoritative definition**: the rule, spec, or contract that the observed behavior violates. This is a file path + line number (e.g., `references/workflows/rca.md:34`) or an explicit design principle (e.g., PR-028).
|
|
246
|
+
2. **Quote or cite the relevant passage**: don't paraphrase from memory — read the actual source file.
|
|
247
|
+
3. **Tag unverified claims**: if you believe a rule exists but haven't read the source yet, mark the Finding with tag `hypothesis-pending-source` and do NOT finalize it.
|
|
248
|
+
|
|
249
|
+
**Anti-patterns** (I-022 violations):
|
|
250
|
+
- "The agent should have done X" — without citing where X is specified
|
|
251
|
+
- "According to PR-028, ..." — without reading `references/principles.md` to confirm the principle's text
|
|
252
|
+
- Citing training-data recollections instead of reading the actual repo file
|
|
253
|
+
|
|
254
|
+
> **Why**: Misremembered or stale rules are a common source of false findings. The read-definition discipline forces evidence from the current codebase, not from memory.
|
|
255
|
+
|
|
256
|
+
---
|
|
257
|
+
|
|
258
|
+
## Assumption enumeration
|
|
259
|
+
|
|
260
|
+
**I-023 (PR-024 + PR-030 earn-keep): Enumerate all assumptions before finalizing any Finding.**
|
|
261
|
+
|
|
262
|
+
Before writing `Finding.assumptions[]`, enumerate:
|
|
263
|
+
1. **Domain facts assumed true** but not verified from trace evidence (e.g., "the tool was available", "rate limits were not hit during this window").
|
|
264
|
+
2. **Preconditions assumed met** (e.g., "upstream data was fresh", "the agent had the correct model").
|
|
265
|
+
3. **Causal claims**: every link in the why-chain that is inferred (not directly observed) should be listed as an assumption until evidence is found.
|
|
266
|
+
|
|
267
|
+
**Format**: each entry in `Finding.assumptions[]` is a structured `Assumption` object — `{ text, status, basis }` (canonical type in `scripts/normalize/trace.ts`). `text` = the assumption as a single declarative sentence; `status` ∈ `"verified" | "unverified" | "hypothesis-pending"` (drives the renderer pill); `basis` = the evidence the status rests on, or the source still required.
|
|
268
|
+
|
|
269
|
+
**Examples**:
|
|
270
|
+
```json
|
|
271
|
+
"assumptions": [
|
|
272
|
+
{ "text": "Tool timeout is the primary cause of empty results", "status": "hypothesis-pending", "basis": "search.ts timeout config not yet read" },
|
|
273
|
+
{ "text": "No existing retry-cap exists in the tool definition", "status": "unverified", "basis": "tool definition not inspected" },
|
|
274
|
+
{ "text": "The session ran in production (not staging)", "status": "verified", "basis": "env=prod in trace metadata" }
|
|
275
|
+
]
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
> **Minimum bar**: the assumptions array MUST NOT be empty on any confirmed Finding. An empty array means either (a) the analyst didn't enumerate, or (b) the finding has zero unverified claims — document the latter explicitly with a single entry `{ "text": "all claims directly evidenced", "status": "verified", "basis": "every why-chain link observed in the trace" }`.
|
|
279
|
+
|
|
280
|
+
---
|
|
281
|
+
|
|
282
|
+
## Assumption explicitness
|
|
283
|
+
|
|
284
|
+
**I-023 (PR-030): Making assumptions explicit is load-bearing for remedy correctness.**
|
|
285
|
+
|
|
286
|
+
The assumptions array is not a formality — it is the primary mechanism for catching overconfident findings. Concrete rules:
|
|
287
|
+
|
|
288
|
+
1. **No assumption → no diagnosis**: If you can't enumerate any assumptions for a Finding, you either have a trivially-obvious bug (rare) or you haven't thought hard enough about what you're asserting.
|
|
289
|
+
2. **hypothesis-pending-source items block finalization**: A Finding tagged with `hypothesis-pending-source` entries is a DRAFT, not a confirmed finding. It cannot be promoted to the approved remedy set.
|
|
290
|
+
3. **Reviewers check the assumptions array first**: When reviewing a Finding, the first question is "are the assumptions warranted?" not "is the remedy correct?". A correct remedy built on a false assumption is worse than no remedy.
|
|
291
|
+
|
|
292
|
+
---
|
|
293
|
+
|
|
294
|
+
## Evidence bar
|
|
295
|
+
|
|
296
|
+
**I-032 (PR-018): Every Finding MUST carry observable evidence before diagnosis.**
|
|
297
|
+
|
|
298
|
+
PR-018 (Evidence-Grounded RCA) defines the minimum evidence bar:
|
|
299
|
+
1. **Observable in the trace**: the evidence must be a specific message index, tool call output, file:line pointer, or metric value — NOT an inference from absence.
|
|
300
|
+
2. **Cited in `failureOrigin.evidence`**: the `evidence` field is MANDATORY on every `FailureOrigin`. Empty evidence string = invalid Finding.
|
|
301
|
+
3. **Each why-chain entry must cite evidence**: `WhyChainEntry.evidence` is required. "because that's how it works" is not evidence.
|
|
302
|
+
|
|
303
|
+
**Anti-patterns** (PR-018 violations):
|
|
304
|
+
- `"evidence": "agent clearly looped"` ← "clearly" is a judgment, not a pointer
|
|
305
|
+
- `"evidence": ""` ← empty (invalid)
|
|
306
|
+
- `"evidence": "expected behavior from training"` ← memory, not observable trace data
|
|
307
|
+
|
|
308
|
+
> **Corollary to I-031 Signal Census**: Evidence must be gathered FROM the trace batch, not invented from domain intuition. If you can't find the evidence in the signal census output, the Finding is a hypothesis, not a confirmed finding — keep the `hypothesis-pending-source` tag.
|
|
309
|
+
|
|
310
|
+
---
|
|
311
|
+
|
|
312
|
+
## Known failure patterns (Tier 1 fast-path)
|
|
313
|
+
|
|
314
|
+
Patterns detected by `scripts/tier0-scan.ts` or Tier 1 of the analyzer:
|
|
315
|
+
|
|
316
|
+
| Pattern ID | Name | Detection | WHAT mapping |
|
|
317
|
+
|------------|------|-----------|-------------|
|
|
318
|
+
| P-001 | error-spike | > 20% error rate in trace batch | `wrong-output` |
|
|
319
|
+
| P-002 | latency-spike | > 10% traces with P99 > 10s | `latency-spike` |
|
|
320
|
+
| P-003 | feedback-cluster | > 5% feedback-bearing traces | `user-complaint` / `low-score` |
|
|
321
|
+
| P-004 | tool-loop | same tool called ≥3× with identical args | `loop` |
|
|
322
|
+
| P-005 | empty-output | agent output = empty / null / refusal | `missing-output` |
|
|
323
|
+
| P-006 | format-fail | JSON.parse of structured output fails | `format-violation` |
|
|
324
|
+
| P-007 | token-cap | total_tokens near model limit | `cost-overshoot` |
|
|
325
|
+
|
|
326
|
+
> **PR-048 cross-ref (W9-09 — Trace Hungriness):** the pattern library above accumulates
|
|
327
|
+
> confirmed patterns per escalation batch (Step 6 loop). Each rung appends newly confirmed
|
|
328
|
+
> patterns from that batch's LLM reads so the library grows with each tier. A run's
|
|
329
|
+
> detectable-pattern set after tier 1000 is materially richer than after tier 100.
|
|
330
|
+
> See `references/principles.md` PR-048 and `references/workflows/orchestrator-protocol.md`
|
|
331
|
+
> Step 6 escalation loop.
|
|
@@ -0,0 +1,314 @@
|
|
|
1
|
+
# Rendering Anatomy — mutagent-diagnostics Report
|
|
2
|
+
|
|
3
|
+
> **PRD-CC-12** · Canonical anatomy doc for the per-finding panel, per-remedy card,
|
|
4
|
+
> decisions panel, and live-preview. Authoritative reading for anyone extending the renderer.
|
|
5
|
+
>
|
|
6
|
+
> Visual contract: open the F-DESIGN-EXAMPLE Issue Card in the live design-contract report
|
|
7
|
+
> and read this doc alongside it.
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
## Overview
|
|
12
|
+
|
|
13
|
+
The gold-standard report is a multi-tab HTML document rendered by
|
|
14
|
+
`scripts/report/render.ts` from a fully-enriched `RenderInput` (produced by
|
|
15
|
+
`scripts/enrich/build-render-input.ts`). The template is
|
|
16
|
+
`assets/templates/report.html.tpl`.
|
|
17
|
+
|
|
18
|
+
The renderer is **fail-loud**: it throws (non-zero exit) when 3 or more of the 4
|
|
19
|
+
internal render shapes (`diagnosedEntity` / `bigStat` / `hourlyHeatmap` / `signalCensus`)
|
|
20
|
+
are absent. Never call the renderer on raw findings — always run the enricher first.
|
|
21
|
+
|
|
22
|
+
Tab layout:
|
|
23
|
+
|
|
24
|
+
| Tab ID | Name | Visibility |
|
|
25
|
+
|--------|------|-----------|
|
|
26
|
+
| `t0` | Methodology [INTERNAL] | NODE-STRIPPED on `--audience client` |
|
|
27
|
+
| `t1` | Overview | Always visible |
|
|
28
|
+
| `t2..tN+1` | F-NNN (one per finding) | Always visible |
|
|
29
|
+
| `tdecisions` | Decisions | Always visible |
|
|
30
|
+
|
|
31
|
+
---
|
|
32
|
+
|
|
33
|
+
## §1 — Overview Tab (`t1`)
|
|
34
|
+
|
|
35
|
+
### 1.1 Entity Card
|
|
36
|
+
|
|
37
|
+
Source: `RenderInput.diagnosedEntity` (type `EntityContext` from `scripts/normalize/trace.ts`).
|
|
38
|
+
|
|
39
|
+
| Field | Display | Notes |
|
|
40
|
+
|-------|---------|-------|
|
|
41
|
+
| `name` | Heading | Required. Renderer WARNS when absent on non-meta reports. |
|
|
42
|
+
| `entityType` | Chip | `agent` / `skill` / `service` / `unknown` |
|
|
43
|
+
| `model` | Chip | e.g., `claude-haiku-3.5` |
|
|
44
|
+
| `codeAccess` | Boolean pill | `true` = direct edit access; `false` = hypothesis-pending on remedy targets |
|
|
45
|
+
| `applyTarget` | Dashed row | Symbolic path to agent definition |
|
|
46
|
+
| `systemPrompt` | Always-collapsed `<details>` | PII — explicit click to expand regardless of size |
|
|
47
|
+
| `toolInventory` | Expandable list | Per-tool stats (call count, error rate); > 1 KB collapses |
|
|
48
|
+
| `inputSample` | Expandable | Sample user inputs |
|
|
49
|
+
|
|
50
|
+
### 1.2 Big-Stat Row (6 tiles)
|
|
51
|
+
|
|
52
|
+
Source: `RenderInput.bigStat`. Six tiles laid out in a responsive grid:
|
|
53
|
+
|
|
54
|
+
| Tile | Field | Color threshold |
|
|
55
|
+
|------|-------|----------------|
|
|
56
|
+
| Latency p50 | `latencyP50Ms` | green < 500ms, yellow < 2000ms, red >= 2000ms |
|
|
57
|
+
| Latency p95 | `latencyP95Ms` | same thresholds |
|
|
58
|
+
| Latency max | `latencyMaxMs` | same thresholds |
|
|
59
|
+
| Cost / window | `totalCostUsd` | muted <= $10, yellow > $10, red > $50 |
|
|
60
|
+
| Traces | `traceCount` | always muted |
|
|
61
|
+
| Error rate | `errorRate` | green < 5%, yellow < 20%, red >= 20% |
|
|
62
|
+
|
|
63
|
+
All CSS uses existing brand vars: `var(--p)` (purple), `var(--c)` (cyan), `var(--g)` (green),
|
|
64
|
+
`var(--y)` (yellow), `var(--r)` (red). Zero new hex colors.
|
|
65
|
+
|
|
66
|
+
### 1.3 24h Latency Heatmap
|
|
67
|
+
|
|
68
|
+
Source: `RenderInput.hourlyHeatmap` (24-entry array, one per UTC hour).
|
|
69
|
+
Color encodes avg latency; number in each cell = trace count.
|
|
70
|
+
|
|
71
|
+
### 1.4 Findings Summary Table
|
|
72
|
+
|
|
73
|
+
Lists all findings with severity, title, WHAT/WHY/WHERE taxonomy, and a link to the finding
|
|
74
|
+
tab.
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## §2 — Finding Panel (`t2..tN+1`)
|
|
79
|
+
|
|
80
|
+
Each finding occupies its own tab. The panel renders the following blocks in order:
|
|
81
|
+
|
|
82
|
+
### 2.1 Finding Header
|
|
83
|
+
|
|
84
|
+
```
|
|
85
|
+
[SEVERITY BADGE] [Finding title — story-led sentence]
|
|
86
|
+
WHAT: <chip> WHY: <chip> WHERE: <chip> APPLY: <chip> AUDIENCE: <chip>
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
- Severity badge: `CRIT` (red), `HIGH` (orange), `MED` (yellow), `INFO` (muted).
|
|
90
|
+
- AUDIENCE chip: `PRODUCT` / `META` / `CORE` — NODE-STRIPPED on `--audience client` when META/CORE.
|
|
91
|
+
|
|
92
|
+
### 2.2 Feedback Block (`feedbackSources[]`)
|
|
93
|
+
|
|
94
|
+
**Position:** between Problem and Evidence. Only rendered when `finding.feedbackSources` is
|
|
95
|
+
non-empty.
|
|
96
|
+
|
|
97
|
+
**Layout:** a bordered card with header "Feedback grounding" (NO emojis — D9) followed by
|
|
98
|
+
one row per source entry.
|
|
99
|
+
|
|
100
|
+
| Source type | CSS class | Color |
|
|
101
|
+
|-------------|-----------|-------|
|
|
102
|
+
| `chat` | `.fb-chat` | `var(--c)` (cyan border + left stripe) |
|
|
103
|
+
| `trace-score` | `.fb-score` | `var(--y)` (yellow border + left stripe) |
|
|
104
|
+
| `external` | `.fb-external` | `var(--p)` (purple border + left stripe) |
|
|
105
|
+
|
|
106
|
+
Label text (exact — D9 forbids emojis):
|
|
107
|
+
- `chat` → "Operator chat"
|
|
108
|
+
- `trace-score` → "Langfuse trace score"
|
|
109
|
+
- `external` → "External feedback platform"
|
|
110
|
+
|
|
111
|
+
### 2.3 Problem Statement
|
|
112
|
+
|
|
113
|
+
Free text. The finding's `problem` field. Rendered as a prose paragraph.
|
|
114
|
+
|
|
115
|
+
### 2.4 Evidence Block
|
|
116
|
+
|
|
117
|
+
Source: `finding.evidence` (array of evidence entries). Rendered as a list of evidence
|
|
118
|
+
bullets. When empty, the block is hidden.
|
|
119
|
+
|
|
120
|
+
### 2.5 Why-Chain
|
|
121
|
+
|
|
122
|
+
Source: `finding.whyChain` (ordered causal chain). Rendered as a numbered list.
|
|
123
|
+
The deepest step carries `isOrigin: true` — rendered with a distinct "root cause" marker.
|
|
124
|
+
|
|
125
|
+
### 2.6 Assumptions Block
|
|
126
|
+
|
|
127
|
+
Source: `finding.assumptions` (array). Rendered as a pill list:
|
|
128
|
+
|
|
129
|
+
| Pill type | When |
|
|
130
|
+
|-----------|------|
|
|
131
|
+
| `verified` (green) | Assumption confirmed by trace evidence |
|
|
132
|
+
| `unverified` (yellow) | Plausible but no direct evidence |
|
|
133
|
+
| `hypothesis-pending` (red) | Requires further investigation — always present on `codeAccess: false` entities |
|
|
134
|
+
|
|
135
|
+
On `codeAccess: false` entities: a disclaimer block is prepended:
|
|
136
|
+
"Code access not available — conclusions are hypothesis-pending until source review."
|
|
137
|
+
|
|
138
|
+
### 2.7 Coverage Proof Widget
|
|
139
|
+
|
|
140
|
+
Source: `finding.coverageProof` (type `CoverageProof` from `scripts/normalize/trace.ts`).
|
|
141
|
+
|
|
142
|
+
Layout: confidence badge + 4 per-dimension rows.
|
|
143
|
+
|
|
144
|
+
```
|
|
145
|
+
[HIGH / MEDIUM / LOW confidence badge]
|
|
146
|
+
latency: coveredBuckets / populationBuckets (coveragePct%)
|
|
147
|
+
score: coveredBuckets / populationBuckets (coveragePct%)
|
|
148
|
+
temporal: coveredBuckets / populationBuckets (coveragePct%)
|
|
149
|
+
tool-trajectory: coveredBuckets / populationBuckets (coveragePct%)
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
Low confidence (< 70%): shows a yellow caveat banner. Never blocks render.
|
|
153
|
+
This widget is WARN-only — low confidence is surfaced, not suppressed.
|
|
154
|
+
|
|
155
|
+
---
|
|
156
|
+
|
|
157
|
+
## §3 — Remedy Card
|
|
158
|
+
|
|
159
|
+
Each finding has one or more remedies. Each remedy is a `.remedy-card` element.
|
|
160
|
+
|
|
161
|
+
### 3.1 Remedy Header
|
|
162
|
+
|
|
163
|
+
```
|
|
164
|
+
[TOP-RIGHT CHECKBOX .remedy-cb data-id="{remedyId}"]
|
|
165
|
+
[★ RANK BADGE] Remedy title
|
|
166
|
+
Meta strip: rank · applyTarget · cost · correctness · targetClass · changeType [pills]
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
The checkbox is the ONLY selection mechanism. No per-remedy Copy buttons (D2 — there is
|
|
170
|
+
ONE global Copy button on the Decisions tab).
|
|
171
|
+
|
|
172
|
+
### 3.2 Two-Rationale Blocks (D1 — ALWAYS VISIBLE)
|
|
173
|
+
|
|
174
|
+
```
|
|
175
|
+
[PURPLE block] Why this remedy
|
|
176
|
+
<rationale text>
|
|
177
|
+
|
|
178
|
+
[CYAN block] Why this works
|
|
179
|
+
<whyWorks text>
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
Both blocks are ALWAYS visible. Neither is collapsible. If either field is absent on the
|
|
183
|
+
finding, the renderer falls back to a placeholder ("Rationale not provided" / "Mechanism
|
|
184
|
+
not provided") — never hides the block.
|
|
185
|
+
|
|
186
|
+
### 3.3 Apply Target Row
|
|
187
|
+
|
|
188
|
+
Source: `remedy.applyTarget`. Rendered as a dashed-border row below the rationale blocks:
|
|
189
|
+
|
|
190
|
+
```
|
|
191
|
+
Apply target: <applyTarget> [targetClass pill] [changeType pill]
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
Only rendered when `applyTarget` is present.
|
|
195
|
+
|
|
196
|
+
### 3.4 Diff Grid (D3 — ALWAYS VISIBLE)
|
|
197
|
+
|
|
198
|
+
Two-column layout: **Before** (left) and **After** (right). Always visible — no collapsibles.
|
|
199
|
+
When `remedy.diff` is absent the columns render as empty placeholders.
|
|
200
|
+
|
|
201
|
+
### 3.5 Apply Plan + Instructions (D4 — side-by-side, ALWAYS VISIBLE)
|
|
202
|
+
|
|
203
|
+
A 2-column grid below the diff:
|
|
204
|
+
|
|
205
|
+
| Column | Source |
|
|
206
|
+
|--------|--------|
|
|
207
|
+
| Apply plan | `remedy.plan` (ordered steps) |
|
|
208
|
+
| Apply instructions | `remedy.applyInstructions[]` (exact commands / diffs) |
|
|
209
|
+
|
|
210
|
+
Both columns are always visible.
|
|
211
|
+
|
|
212
|
+
### 3.6 Feedback Textarea (D7)
|
|
213
|
+
|
|
214
|
+
```
|
|
215
|
+
<textarea class="remedy-notes" placeholder="Notes / override..."></textarea>
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
Free-text override box per remedy. Content is merged into the master Copy bundle.
|
|
219
|
+
|
|
220
|
+
---
|
|
221
|
+
|
|
222
|
+
## §4 — Decisions Tab (`tdecisions`)
|
|
223
|
+
|
|
224
|
+
### 4.1 Live Preview (`#lp-body`)
|
|
225
|
+
|
|
226
|
+
```
|
|
227
|
+
<textarea id="lp-body" readonly></textarea>
|
|
228
|
+
```
|
|
229
|
+
|
|
230
|
+
Auto-updates on every checkbox toggle and every keystroke in any `.remedy-notes` or general
|
|
231
|
+
feedback box. The content is the master plan markdown bundle aggregated from:
|
|
232
|
+
- All checked remedies (full remedy markdown)
|
|
233
|
+
- All `.remedy-notes` content for checked remedies
|
|
234
|
+
- General feedback textarea content
|
|
235
|
+
|
|
236
|
+
### 4.2 Meta Row (`#lp-meta`)
|
|
237
|
+
|
|
238
|
+
Shows: `<N> remedies · <M> chars` (updates live).
|
|
239
|
+
|
|
240
|
+
### 4.3 Global Copy Button (D2 — ONE sticky button)
|
|
241
|
+
|
|
242
|
+
```
|
|
243
|
+
<button id="copy-decisions">Copy to Clipboard</button>
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
ONE button only. Reads `#lp-body.value` and writes to clipboard. No modal, no preview
|
|
247
|
+
button, no second master button. Sticky at the bottom of the Decisions tab.
|
|
248
|
+
|
|
249
|
+
### 4.4 General Feedback Textarea
|
|
250
|
+
|
|
251
|
+
Free-text box for overall feedback (not remedy-specific). Merged into the Copy bundle.
|
|
252
|
+
|
|
253
|
+
### 4.5 Decisions Sync (D10)
|
|
254
|
+
|
|
255
|
+
The Decisions Panel content is mirrored inside the design view AND on the Decisions tab.
|
|
256
|
+
State is synced via JS (general feedback textarea + changelog).
|
|
257
|
+
|
|
258
|
+
---
|
|
259
|
+
|
|
260
|
+
## §5 — Methodology Tab (`t0`) [INTERNAL]
|
|
261
|
+
|
|
262
|
+
Visible only on internal reports (`--audience internal` or default). NODE-STRIPPED on
|
|
263
|
+
`--audience client` — not display-hidden, actually removed from the DOM so operator
|
|
264
|
+
metadata never reaches end-user reports.
|
|
265
|
+
|
|
266
|
+
Contents:
|
|
267
|
+
- Mermaid sequence diagram (orchestrator → scripts → analyzers, with timestamps).
|
|
268
|
+
- Graded decision log (all `runMeta.decisions[]` entries with step, choice, rationale, timestamp).
|
|
269
|
+
- Signal census (signals detected, their types, and Tier-0 coverage).
|
|
270
|
+
- Wave-6 methodology checklist stamp results.
|
|
271
|
+
|
|
272
|
+
---
|
|
273
|
+
|
|
274
|
+
## §6 — Self-Diagnosis Report Mode
|
|
275
|
+
|
|
276
|
+
When `isMetaReport: true` (produced by `scripts/self-diagnostics/dispatch.ts`):
|
|
277
|
+
|
|
278
|
+
- A forced `SELF-DIAGNOSIS` banner is prepended to every finding tab.
|
|
279
|
+
- The entity card shows `entityType: skill`, `codeAccess: true`, and the skill's `applyTarget`.
|
|
280
|
+
- The Decisions tab Decisions row includes a header: "Subject: skill source (skill maintainer mode)".
|
|
281
|
+
- `--audience client` is REFUSED — self-diag is always internal.
|
|
282
|
+
|
|
283
|
+
---
|
|
284
|
+
|
|
285
|
+
## §7 — CSS and Brand Variables
|
|
286
|
+
|
|
287
|
+
All new CSS uses ONLY these existing brand vars:
|
|
288
|
+
|
|
289
|
+
| Var | Color | Usage |
|
|
290
|
+
|-----|-------|-------|
|
|
291
|
+
| `var(--p)` | Purple | `rationale` block, external feedback border, `META` audience badge |
|
|
292
|
+
| `var(--c)` | Cyan | `whyWorks` block, chat feedback border |
|
|
293
|
+
| `var(--g)` | Green | `verified` assumption pill, recommended remedy glow |
|
|
294
|
+
| `var(--y)` | Yellow | `unverified` pill, score feedback border, low-coverage caveat |
|
|
295
|
+
| `var(--r)` | Red | `hypothesis-pending` pill, refused-gate banner |
|
|
296
|
+
| Surface/border tokens | Neutral | Card backgrounds, dividers |
|
|
297
|
+
|
|
298
|
+
Zero new hex colors. Zero emoji on structural labels (D9).
|
|
299
|
+
|
|
300
|
+
---
|
|
301
|
+
|
|
302
|
+
## §8 — Extending the Renderer
|
|
303
|
+
|
|
304
|
+
When adding a new block to the finding panel or remedy card:
|
|
305
|
+
|
|
306
|
+
1. Add the field to the appropriate type in `scripts/normalize/trace.ts` (optional for
|
|
307
|
+
backward-compat).
|
|
308
|
+
2. Add a render function in `scripts/report/render.ts` following the existing
|
|
309
|
+
`renderFeedbackBlock` / `renderAssumptionsBlock` pattern.
|
|
310
|
+
3. Use brand vars only — no new hex colors.
|
|
311
|
+
4. Update the enricher (`scripts/enrich/build-render-input.ts`) if the block needs
|
|
312
|
+
aggregated data.
|
|
313
|
+
5. Add a test in `scripts/report/render.test.ts` covering the with/without fallback cases.
|
|
314
|
+
6. Update this doc (§2 or §3) with the new block's position and source fields.
|