@mutagent/helix 0.1.0-alpha.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/skills/mutagent-helix/SKILL.md +113 -0
- package/.claude/skills/mutagent-helix/scripts/cli/codex-transcode.ts +143 -0
- package/.claude/skills/mutagent-helix/scripts/cli/doctor.ts +111 -0
- package/.claude/skills/mutagent-helix/scripts/cli/init.ts +382 -0
- package/.claude/skills/mutagent-helix/scripts/cli/run.sh +61 -0
- package/CLAUDE.md +193 -0
- package/LICENSE +201 -0
- package/NOTICE +22 -0
- package/README.md +209 -0
- package/assets/adl-loop.svg +162 -0
- package/assets/system-status.svg +422 -0
- package/bin/mutagent-cli.mjs +9950 -0
- package/bin/mutagent-helix.mjs +78 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/CLAUDE.md +28 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/SKILL.md +138 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/assets/templates/agentspec.yaml.tpl +247 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/references/frameworks/doc-pins.md +53 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/references/principles.md +91 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/references/requirements.yaml +295 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/references/workflows/orchestrator-protocol.md +302 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/cli/doctor.ts +116 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/cli/init.ts +230 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/cli/run.sh +60 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/config/resolve-spec-dir.ts +199 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/contract/agentspec.schema.ts +455 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/setup/detect.ts +67 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/validate/validate-spec.ts +93 -0
- package/mutagent-agentspec/LICENSE +201 -0
- package/mutagent-agentspec/NOTICE +19 -0
- package/mutagent-agentspec/README.md +44 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/SKILL.md +100 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/assets/agents/ai-architect.md +312 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/assets/agents/ai-engineer.md +357 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/assets/templates/build-report.md.tpl +39 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/references/principles.md +31 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/references/workflows/build-protocol.md +61 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/references/workflows/improve-handoff.md +23 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/scripts/cli/run.sh +60 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/scripts/handoff/validate-build-input.ts +72 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/scripts/sync-spec/check-sync-spec.ts +499 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/scripts/verify/spec-impl-coverage.ts +182 -0
- package/mutagent-builder/CLAUDE.md +25 -0
- package/mutagent-builder/README.md +26 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/README.md +192 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/SKILL.md +352 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/agents/ai-architect.md +256 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/agents/ai-engineer.md +344 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/agents/diagnostics-analyzer.md +576 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/agents/discovery.md +425 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/audit.json.tpl +24 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/audit.md.tpl +49 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/config.yaml.tpl +100 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/pr-body.md.tpl +56 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/report.html.tpl +327 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/README.md +27 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/iter-N-handover.md.tpl +125 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/self-diagnosis-contract.v0.1.0.yaml.tpl +234 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/spec.yaml.tpl +152 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/team.yaml.tpl +127 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/wave-N-dashboard.html.tpl +1712 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.1-nl-query.md +59 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.2-tier0-progress.md +58 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.3-analysis-progress.md +44 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.4-html-report.md +61 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.5-rca-detail.md +69 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.6-stale-warning.md +53 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.1-welcome.md +60 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.2-source-platform-picker.md +41 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.3-source-auth.md +75 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.4-target-picker.md +48 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.5-ask-tool.md +43 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.6-config-review.md +57 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.7-filter-defaults.md +52 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.8-first-run.md +59 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/examples/sample-findings.json +92 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/examples/sample-meta.json +47 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/adapter-strategy.md +75 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/config-migration.md +84 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/config.md +159 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/filter-search-matrix.md +66 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/harness-knowledge.md +22 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/memory-format.md +78 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/onboarding-decisions.yaml +187 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/operation-inventory.md +59 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/overview.md +167 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/reference.md +119 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/script-index.md +101 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-adapter-contract.md +93 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/claude-code-transcripts.md +66 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/codex-transcripts.md +78 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/codex.md +145 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/install-paths.md +57 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/langfuse.md +178 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/local-jsonl.md +55 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/openobserve.md +102 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/otel.md +63 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/cloud-rest.md +84 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-claude.md +65 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-cloud-agent-sdk.md +92 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-codex.md +72 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-cursor.md +108 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-mastra.md +77 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-opencode.md +70 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/apply-pr-comment-format.md +82 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/diagnostics.md +35 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/handover-contract.md +247 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/onboarding.md +300 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/orchestrator-protocol.md +1861 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/rca.md +331 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/rendering-anatomy.md +314 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/schedule-prep.md +100 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/scope-model.md +128 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/verification-methodology.md +174 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/aggregate/sort-findings.ts +0 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/awareness/blind-spots.ts +167 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/awareness/llm-sample.ts +681 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/doctor.ts +215 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/init.ts +1013 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/install-agents.ts +350 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/run.sh +61 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/shared-templates.ts +79 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/config/load.ts +392 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/config/resolve-credential.ts +120 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/config/schema.ts +641 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/config/validate.ts +134 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/context/build-diagnosis-context.ts +417 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/context/load-context.ts +105 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/contract/types.ts +166 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/enrich/build-render-input.ts +1964 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/enrich/corpus-severity.ts +172 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/enrich/dismissal-match.ts +270 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/enrich/rank-remedies.ts +75 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/hooks/install-hooks.sh +46 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/hooks/pre-push.sh +78 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/invocation/parse-brief.ts +188 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/library/match.ts +114 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/library/paths.ts +95 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/library/store.ts +535 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/library/types.ts +181 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/lint/template-inline-js.ts +267 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/memory/append.ts +188 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/memory/read.ts +218 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/platforms/entity-context.ts +686 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/read-unitf.ts +267 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/trace.ts +1078 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/unitf-adapter.ts +0 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/unitf-types.ts +213 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/report/persist-selections.ts +174 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/report/render.ts +3254 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/run/diagnose.ts +64 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/run/session.ts +167 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/sample/caps.ts +345 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/sample/deep-read-gate.ts +424 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/sample/representative.ts +584 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/scan/objection.ts +224 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/scan/trajectory.ts +489 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/self-diagnostics/dispatch.ts +135 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/self-diagnostics/probe.ts +170 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/setup/detect.ts +307 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/setup/ensure-cli.ts +315 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/setup/reconfigure.ts +77 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/setup/verify-agents.ts +207 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/slicer.ts +166 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/claude-code.ts +17 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/codex.ts +11 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/langfuse.ts +17 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/local-jsonl.ts +12 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/otel.ts +12 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/index.ts +54 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/registry.ts +45 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/types.ts +86 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/stale-detector.ts +74 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/test/render-playwright.cli.ts +477 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/tier0/claude-code.ts +133 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/tier0/langfuse.ts +143 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/tier0-scan.ts +423 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/completeness-check.ts +436 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/doc-refs.ts +190 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/finalize-gate.ts +417 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/findings-contract.ts +628 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/render-js-syntax.ts +188 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/report-checklist.yaml +217 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/trace-shape.ts +262 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/wave6-checklist.ts +326 -0
- package/mutagent-diagnostics/CLAUDE.md +41 -0
- package/mutagent-diagnostics/LICENSE +201 -0
- package/mutagent-diagnostics/NOTICE +18 -0
- package/mutagent-diagnostics/README.md +343 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/SKILL.md +548 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/agents/audit-executor.md +169 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/agents/dataset-builder.md +160 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/agents/discovery.md +425 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/agents/evaluator.md +1108 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/brand/theme.css +213 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/brand/wordmark.html +9 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/context-flow-lens.md +85 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/data-lens.md +47 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/decision-lens.md +48 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/methodology-critic-lens.md +53 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/trajectory-lens.md +44 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/build-review-interface.md +83 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/edd-loop.md +134 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/error-analysis.md +113 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/eval-audit.md +154 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/eval-stage.md +168 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/generate-synthetic-data.md +81 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/grounded-adjudication.md +221 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/memory-format.md +65 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/methodology.md +201 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/operation-inventory.md +197 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/validate-evaluator.md +125 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/workflows/orchestrator-protocol.md +300 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/write-judge-prompt.md +123 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/behavior-tree.schema.yaml +73 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/dataset.schema.yaml +66 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/edd-change-request.schema.yaml +114 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/eval-matrix.schema.yaml +74 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/flow-graph.schema.yaml +69 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/flow-profile.schema.yaml +49 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/methodology-review.schema.yaml +40 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/scorecard.schema.yaml +85 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/agent-dispatch.ts +0 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/aggregate-discover.ts +543 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/artifact-paths.ts +115 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/assemble-scorecard.ts +172 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/build-dataset.ts +186 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/build-evals.ts +93 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/build-review-ui.ts +393 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/check-method-router.ts +170 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/aggregate.ts +112 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/audit-run.ts +182 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/doctor.ts +211 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/dogfood.ts +133 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/init.ts +601 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/methodology-review.ts +122 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/prep.ts +295 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/profile-subject.ts +165 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/run.sh +56 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/variance-check.ts +105 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/code-eval.ts +248 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/code-quality-verdict.ts +285 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/codegen-evals.ts +173 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cold-start-project.ts +151 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cold-start-sampler.ts +267 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/config/load.ts +343 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/config/schema.ts +380 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/agentspec-evals.ts +85 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/dataset.ts +149 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/eval-engine.ts +118 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/eval-matrix.ts +577 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/eval-types.ts +1074 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/flow-graph.ts +194 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/types.ts +303 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/validation.ts +193 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/derive-dataset.ts +152 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/determine-outcome.ts +219 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/diff-discriminate.ts +160 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/discover-criteria.ts +348 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/edd/change-request.ts +232 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/edd/edd-types.ts +210 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/edd/variance-gate.ts +186 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/emit-completeness.ts +111 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/eval-engine.ts +160 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/evaluate.ts +333 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/flow-graph.ts +230 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/judge-prompt-template.ts +253 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/judge-provider.ts +135 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/lint-grounding.ts +229 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/lint-uniformity.ts +168 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/living-suite.ts +109 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/load-bundle.ts +203 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/load-profile-vocab.ts +64 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/load-profile.ts +106 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/mask.ts +138 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/materialize-dataset.ts +113 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/matrix-judge.ts +750 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/memory/append.ts +215 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/memory/read.ts +168 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/persist-eval-criteria.ts +232 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/prep-tasks.ts +125 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/profile-subject.ts +310 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/publish-report.ts +131 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/read-unitf-traces.ts +81 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/render-build-cards.ts +195 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/render-discover-report.ts +1640 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/render-eval-report.ts +3838 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/render-report.ts +212 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/resolve-credential.ts +110 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/resolve-ref.ts +98 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/result-verify.ts +129 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/route-failures.ts +330 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/run-deterministic.ts +367 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/run-evaluate.ts +719 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/run-judge.ts +155 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/run-pipeline.ts +175 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/sample-traces.ts +210 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/self-audit.ts +387 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/source-map.ts +106 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/subject-profile.ts +134 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/substrate.ts +162 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/sync-eval-criteria.ts +244 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/ui-slots.ts +119 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/unitf-to-evaltrace.ts +284 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/validate-judge.ts +358 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/variance-compare.ts +177 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/subjects/mutagent-diagnostics/behavior-tree.yaml +140 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/subjects/mutagent-diagnostics/eval-matrix.yaml +1270 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/subjects/mutagent-diagnostics/methodology-review.yaml +105 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/workflows/audit.workflow.js +82 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/workflows/data-leak.workflow.js +236 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/workflows/variance.workflow.js +163 -0
- package/mutagent-evaluator/CLAUDE.md +45 -0
- package/mutagent-evaluator/LICENSE +201 -0
- package/mutagent-evaluator/NOTICE +23 -0
- package/mutagent-evaluator/README.md +88 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/SKILL.md +287 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/scripts/contracts/amend-request.ts +503 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/scripts/improve-loop-run.ts +304 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/scripts/loop-state-cli.ts +300 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/scripts/loop-state.ts +440 -0
- package/mutagent-optimize/CLAUDE.md +49 -0
- package/mutagent-optimize/README.md +26 -0
- package/mutagent-orchestrator/CLAUDE.md +102 -0
- package/mutagent-orchestrator/README.md +20 -0
- package/mutagent-orchestrator/assets/agents/discovery.md +425 -0
- package/mutagent-orchestrator/assets/agents/dogfood-monitor.md +266 -0
- package/mutagent-orchestrator/assets/agents/monitor.md +258 -0
- package/mutagent-orchestrator/orchestrator.md +782 -0
- package/mutagent-orchestrator/references/MONITORING.md +209 -0
- package/mutagent-orchestrator/references/config-fields.md +132 -0
- package/mutagent-orchestrator/references/config-migration.md +132 -0
- package/mutagent-orchestrator/references/index-registry.md +92 -0
- package/mutagent-orchestrator/routing.yaml +391 -0
- package/mutagent-orchestrator/scripts/config-schema.ts +1010 -0
- package/mutagent-orchestrator/scripts/dispatch.ts +439 -0
- package/mutagent-orchestrator/scripts/dogfood/extract-feedback.ts +203 -0
- package/mutagent-orchestrator/scripts/dogfood/reconstruct-trajectory.ts +327 -0
- package/mutagent-orchestrator/scripts/dogfood/render-dogfood-report.ts +347 -0
- package/mutagent-orchestrator/scripts/dogfood/resolve-subagents.ts +160 -0
- package/mutagent-orchestrator/scripts/dogfood/run-report.ts +264 -0
- package/mutagent-orchestrator/scripts/dogfood/slack-thread.ts +106 -0
- package/mutagent-orchestrator/scripts/dogfood/types.ts +257 -0
- package/mutagent-orchestrator/scripts/eval-routing.ts +753 -0
- package/mutagent-orchestrator/scripts/gate.ts +219 -0
- package/mutagent-orchestrator/scripts/handover-contract.ts +362 -0
- package/mutagent-orchestrator/scripts/index/build-index.ts +351 -0
- package/mutagent-orchestrator/scripts/index/render-index.ts +110 -0
- package/mutagent-orchestrator/scripts/monitor/slack-notify.ts +111 -0
- package/mutagent-orchestrator/scripts/monitor/triggers.ts +162 -0
- package/mutagent-orchestrator/scripts/onboarding-check.ts +369 -0
- package/mutagent-orchestrator/scripts/render-roster.ts +308 -0
- package/mutagent-orchestrator/scripts/resolve-credential.ts +185 -0
- package/mutagent-orchestrator/scripts/resolve-paths.ts +114 -0
- package/mutagent-orchestrator/scripts/slack/format.ts +63 -0
- package/mutagent-orchestrator/scripts/slack/post.ts +115 -0
- package/mutagent-orchestrator/scripts/sync-index.ts +311 -0
- package/package.json +64 -0
package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/filter-search-matrix.md
ADDED
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
# Filter/Search Coverage Matrix
|
|
2
|
+
|
|
3
|
+
> Master Matrix — per-platform support for each filter/search dimension.
|
|
4
|
+
> Ships with the skill. Updated by PRs when platform support changes.
|
|
5
|
+
> For unknown platforms: runtime CLI docs lookup (see adapter-strategy.md Q6).
|
|
6
|
+
|
|
7
|
+
## Dimension definitions
|
|
8
|
+
|
|
9
|
+
| Dimension | What it filters/searches |
|
|
10
|
+
|-----------|-------------------------|
|
|
11
|
+
| by agent ID | Traces from a specific agent identifier |
|
|
12
|
+
| by session ID | A specific session/trace ID |
|
|
13
|
+
| by time window | Traces in a time range |
|
|
14
|
+
| by score | Traces with score above/below threshold |
|
|
15
|
+
| has feedback | Traces with any attached feedback |
|
|
16
|
+
| has error | Traces containing error events |
|
|
17
|
+
| latency P99 | Traces with high latency |
|
|
18
|
+
| by route / trace name | Traces with a specific operation name |
|
|
19
|
+
| by tag | Traces with specific labels/tags |
|
|
20
|
+
| by skill | Traces where a specific skill was triggered |
|
|
21
|
+
| Search: full-text | Content-level search in trace messages |
|
|
22
|
+
| Search: semantic | Semantic similarity search |
|
|
23
|
+
|
|
24
|
+
---
|
|
25
|
+
|
|
26
|
+
## Coverage Matrix
|
|
27
|
+
|
|
28
|
+
| Dimension | Langfuse | OTel | local-jsonl | claude-code | codex |
|
|
29
|
+
|-----------|----------|------|-------------|-------------|-------|
|
|
30
|
+
| by agent ID | `--agent-id` CLI flag | `span.attr[gen_ai.agent.id]` | post-filter on `agentId` field | filename pattern or content grep | grep `agent_type` field in rollout JSONL |
|
|
31
|
+
| by session ID | `--session-id` CLI flag | `trace_id` | post-filter on `sessionId` field | filename = session-id | filename = session-id in `~/.codex/sessions/` |
|
|
32
|
+
| by time window | `--from` / `--to` CLI flags | span `startTime` / `endTime` | post-filter on `startTime` timestamp | file `mtime` (`find -newer`, `-newer`) | file `mtime` in `~/.codex/sessions/` |
|
|
33
|
+
| by score | `--score-below` / `--score-above` flags | partial — depends on emitter | if `score` field present | limited — only `/feedback` thumbs (discrete boolean) | limited — check rollout for custom score field |
|
|
34
|
+
| has feedback | `--has-feedback` flag | partial — depends on emitter | if `hasFeedback` field present | limited — check for `/feedback` approval events | via `approval` events in rollout JSONL |
|
|
35
|
+
| has error | `--has-error` flag | `span.status.code == 2 (ERROR)` | post-filter on `hasError` field | grep `isError: true` in tool_result events | grep `error` events in rollout JSONL |
|
|
36
|
+
| latency P99 | native aggregation | compute from span `duration` | compute from `startTime`/`endTime` | derive from event timestamps | derive from event timestamps in rollout |
|
|
37
|
+
| by route / trace name | `--name` flag | span `name` field | partial — if `route` field present | n/a (single-route per session) | n/a (single-thread) |
|
|
38
|
+
| by tag | `--tag` flag | `span.attr[tags]` | partial — if `tags` field present | n/a | n/a |
|
|
39
|
+
| by skill | via tag/metadata (`--tag skill:<name>`) | via `span.attr[skill]` | grep `skill` metadata field | grep `Skill` tool calls in JSONL | grep skill invocations in rollout events |
|
|
40
|
+
| Search: full-text | partial — via SDK filter, not CLI | no — post-fetch grep required | grep over jsonl lines | grep over session jsonl lines | grep over rollout jsonl lines |
|
|
41
|
+
| Search: semantic | no | no | no | no | no |
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## Lookup strategy
|
|
46
|
+
|
|
47
|
+
### Known platforms (Langfuse, OTel, local-jsonl, claude-code, codex)
|
|
48
|
+
|
|
49
|
+
Use the Master Matrix above. Reference doc for each platform has concrete CLI examples.
|
|
50
|
+
|
|
51
|
+
### Unknown / custom platforms
|
|
52
|
+
|
|
53
|
+
Agent does **runtime CLI docs lookup**:
|
|
54
|
+
1. `Bash("<cli> --help")` — read help text
|
|
55
|
+
2. `WebFetch(upstream-docs-url)` — fetch from the URL in `references/source-platforms/<name>.md`
|
|
56
|
+
3. Reason over help text → map dimensions to CLI flags
|
|
57
|
+
|
|
58
|
+
If mapping is ambiguous, surface to operator via AskUserQuestion.
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
## Notes
|
|
63
|
+
|
|
64
|
+
- **Semantic search**: not supported on any platform in v0.1. Post-v0.1 consideration.
|
|
65
|
+
- **Codex score support**: Codex doesn't ship a feedback primitive natively. Check `codex-rs/rollout/src/lib.rs` for custom score fields if the operator's Codex build includes them.
|
|
66
|
+
- **Claude Code score**: limited to `/feedback` approval (thumbs-up/down boolean). Scale type = `boolean`. Negative = `false`/`"down"`.
|
package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/harness-knowledge.md
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# Platform Knowledge Table
|
|
2
|
+
|
|
3
|
+
> Expandable table. Static info ships with the skill.
|
|
4
|
+
> Runtime details (latest CLI flags, current API endpoints) recovered via the linked upstream docs.
|
|
5
|
+
> Add new harnesses by submitting a PR to this file.
|
|
6
|
+
|
|
7
|
+
| Harness | Transcript path (static) | ASK primitive | Schedule primitive | Upstream docs |
|
|
8
|
+
|---------|--------------------------|---------------|-------------------|---------------|
|
|
9
|
+
| **Claude Code** | `~/.claude/projects/<encoded-path>/<session-id>.jsonl` (plaintext JSONL; 30-day default retention via `cleanupPeriodDays`) | `AskUserQuestion` tool — structured multi-select, `preview` attr for code/diff comparisons. Native. | Native `/loop` + `CronCreate` + `ScheduleWakeup` tools. | [code.claude.com/docs](https://code.claude.com/docs/en/) |
|
|
10
|
+
| **Codex CLI** | `~/.codex/sessions/<session>.jsonl` (active) + `~/.codex/archived_sessions/<session>.jsonl` (archived). Config: `~/.codex/config.toml`. Constants `SESSIONS_SUBDIR="sessions"` + `ARCHIVED_SESSIONS_SUBDIR="archived_sessions"` in `codex-rs/rollout/src/lib.rs`. Default-on; suppress with `codex exec --ephemeral`. | Approval prompts via TUI (Windows toast fallback). No structured-picker tool primitive yet — chat-based multi-choice. | External cron + `codex exec` for non-interactive runs. No native scheduler. | [developers.openai.com/codex](https://developers.openai.com/codex/cli) · [codex-rs/rollout source](https://github.com/openai/codex/tree/main/codex-rs/rollout) |
|
|
11
|
+
| **Cursor** | Local SQLite under VSCode-style workspace storage (e.g. `~/Library/Application Support/Cursor/User/workspaceStorage/<hash>/state.vscdb` on macOS). Verify at runtime via probe. | Chat-based confirmation. No public structured-picker tool surfaced. | Background Agents (2025+) — autonomous task primitive; verify scheduling support at runtime. | [cursor.com/docs](https://cursor.com/docs) |
|
|
12
|
+
| **OpenCode** | Path TBD — runtime probe at onboarding. | Chat-based (Plan mode + general prompting). | None documented — use OS cron. | [opencode.ai/docs](https://opencode.ai/docs) |
|
|
13
|
+
| *Add more harnesses here* | *operator adds rows over time* | | | |
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## Notes
|
|
18
|
+
|
|
19
|
+
- **Encoding of Claude Code project paths**: Claude Code encodes the absolute project path into the directory name under `~/.claude/projects/`. The encoding is an internal detail — `cli/init.ts` probes by trying known encodings and falls back to listing all project dirs.
|
|
20
|
+
- **Codex paths confirmed iter-8**: path confirmed from `codex-rs/rollout/src/lib.rs` source constants. If paths change in a future Codex release, update this table and the Codex adapter in `@mutagent/tools` (`mutagent-cli trace fetch --platform codex` — the skill no longer owns a Codex normalizer post-UniTF-flip).
|
|
21
|
+
- **Semantic search**: not supported on any harness in v0.1.
|
|
22
|
+
- **Schedule recovery**: platform scheduling specifics (exact API, cron syntax, daemon setup) are recovered at runtime by the agent via the upstream docs links above. This table stores only static facts.
|
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
# AutoMemory Format — project-level operator-feedback + tool-lessons
|
|
2
|
+
|
|
3
|
+
> **Standard (mirrored across ADL skills — parity).** The subject of AutoMemory is
|
|
4
|
+
> the **TOOL + operator preferences**, NOT the diagnosed agent (that is the
|
|
5
|
+
> gitignored class-memory library). ONE project-level store for the whole lifecycle.
|
|
6
|
+
> Fixed convention + standardized format; NOT a config field.
|
|
7
|
+
>
|
|
8
|
+
> Read/write helpers: `scripts/memory/read.ts` + `scripts/memory/append.ts`.
|
|
9
|
+
|
|
10
|
+
## Location + shape
|
|
11
|
+
|
|
12
|
+
```
|
|
13
|
+
.mutagent/memory/ ← project-level, COMMITTED
|
|
14
|
+
MEMORY.md ← index: `- [Title](<slug>.md) — <hook>` one line/entry
|
|
15
|
+
<slug>.md ← one FACT per file
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
Each `<slug>.md` entry — Claude-Code AutoMemory format (verbatim):
|
|
19
|
+
|
|
20
|
+
```markdown
|
|
21
|
+
---
|
|
22
|
+
name: <short-kebab-case-slug>
|
|
23
|
+
description: <one-line summary — used to decide relevance during recall>
|
|
24
|
+
metadata:
|
|
25
|
+
type: user | feedback | project | reference # Claude Code's four types
|
|
26
|
+
lifecycle: spec|build|evaluate|diagnose|improve|general # ADL extension
|
|
27
|
+
created: YYYY-MM-DD # REQUIRED · every memory is DATED (stale-pruning)
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
<the fact. For feedback/project: **Why:** + **How to apply:** lines. Link with [[their-name]].>
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
- **Every entry DATED** (`metadata.created`). The date is an ABSOLUTE `YYYY-MM-DD`
|
|
34
|
+
(injected `now` for determinism in `appendMemory`).
|
|
35
|
+
- `MEMORY.md` is an **index only** — one `- [Title](<slug>.md) — <hook>` line per
|
|
36
|
+
entry, sorted by slug. The `<hook>` is the entry's `description`.
|
|
37
|
+
- **Dedupe on append**: an existing slug is UPDATED in place and its `created` date
|
|
38
|
+
REFRESHED (never a duplicate file).
|
|
39
|
+
|
|
40
|
+
## Feedback → type CLASSIFICATION rubric (verbose)
|
|
41
|
+
|
|
42
|
+
Decision order — **first match wins**:
|
|
43
|
+
|
|
44
|
+
| # | Feedback about… | → type | Signals | Body |
|
|
45
|
+
|---|---|---|---|---|
|
|
46
|
+
| 1 | external resource | `reference` | URL · dashboard · ticket · dataset path | pointer + what it's for |
|
|
47
|
+
| 2 | who the operator is | `user` | "I'm the lead" · standing preference | the trait |
|
|
48
|
+
| 3 | how the tool should behave | `feedback` | "stop surfacing X" · "too verbose" · "next time…" | fact + **Why:** + **How to apply:** |
|
|
49
|
+
| 4 | ongoing work/goals/constraints | `project` | "migrating to…" · "don't touch Z until…" | fact + **Why:** + **How to apply:** |
|
|
50
|
+
|
|
51
|
+
**Do NOT save:**
|
|
52
|
+
- Anything already in code / config / git (it is already the source of truth).
|
|
53
|
+
- Ephemeral facts relevant to only ONE run.
|
|
54
|
+
- Subject FINDINGS about the diagnosed agent (those go to the class-memory library
|
|
55
|
+
`.mutagent/diagnostics/library/`, gated by operator approval — NOT here).
|
|
56
|
+
|
|
57
|
+
## Lifecycle filter at recall
|
|
58
|
+
|
|
59
|
+
At run START (parse-brief / Step 3a) the diagnostics run loads memory FILTERED to
|
|
60
|
+
`lifecycle ∈ { diagnose, general }` (`readDiagnoseMemory`) — the diagnostics stage's
|
|
61
|
+
own lessons plus the always-relevant `general` facts. Entries tagged for other
|
|
62
|
+
stages (`spec` / `build` / `evaluate` / `improve`) are skipped for a diagnose run.
|
|
63
|
+
|
|
64
|
+
## When to append (write trigger)
|
|
65
|
+
|
|
66
|
+
At run FINALIZE (finalize-gate / Step 9.9), when the operator gives feedback ABOUT
|
|
67
|
+
the tool or a standing preference, classify it with the rubric above and call
|
|
68
|
+
`appendMemory({ slug, name, description, type, lifecycle, body, now })`. The default
|
|
69
|
+
`lifecycle` for diagnostics-run feedback is `diagnose` (use `general` for a
|
|
70
|
+
tool-wide standing preference).
|
|
71
|
+
|
|
72
|
+
## Memory subsystems (NOT competing — different subjects)
|
|
73
|
+
|
|
74
|
+
| Subsystem | Subject | Content | Write trigger | Location |
|
|
75
|
+
|---|---|---|---|---|
|
|
76
|
+
| **Class-memory library** | diagnosed entity | journal + regex patterns + deep-read ledger | operator-APPROVED findings only | `.mutagent/diagnostics/library/…` · gitignored |
|
|
77
|
+
| **feedback_sources** | diagnosed entity | external signals → enrich findings | read-only, per-run | opt-in config |
|
|
78
|
+
| **AutoMemory** (this doc) | the tool + operator | usage-feedback + lessons (Claude format, dated, classified) | operator feedback | `.mutagent/memory/` · committed |
|
|
@@ -0,0 +1,187 @@
|
|
|
1
|
+
# references/onboarding-decisions.yaml
|
|
2
|
+
# R-SELF-07-a: Structured option lists for each onboarding decision phase.
|
|
3
|
+
# Used by the parent session (inline orchestrator) when presenting choices to the operator.
|
|
4
|
+
#
|
|
5
|
+
# Each decision has:
|
|
6
|
+
# id — machine ID for the choice
|
|
7
|
+
# label — human-readable label shown to operator
|
|
8
|
+
# value — written to config.yaml when chosen
|
|
9
|
+
# platform_doc — (optional) references/source-platforms/{value}.md path to load
|
|
10
|
+
# note — (optional) rendered below the option if selected
|
|
11
|
+
#
|
|
12
|
+
# R-SELF-07-b: Every option list MUST end with the Custom escape hatch entry
|
|
13
|
+
# { label: 'Custom — describe what you have', value: '__custom__' }
|
|
14
|
+
# so the operator is never blocked on an unsupported platform.
|
|
15
|
+
|
|
16
|
+
version: "1.0"
|
|
17
|
+
|
|
18
|
+
decisions:
|
|
19
|
+
|
|
20
|
+
# Phase 1 — Source platform
|
|
21
|
+
source_platform:
|
|
22
|
+
question: "Where do your agent traces live?"
|
|
23
|
+
key: source_platform
|
|
24
|
+
options:
|
|
25
|
+
- id: langfuse
|
|
26
|
+
label: "Langfuse (cloud or self-hosted)"
|
|
27
|
+
value: langfuse
|
|
28
|
+
platform_doc: "references/source-platforms/langfuse.md"
|
|
29
|
+
- id: otel
|
|
30
|
+
label: "OpenTelemetry-compliant endpoint"
|
|
31
|
+
value: otel
|
|
32
|
+
platform_doc: "references/source-platforms/otel.md"
|
|
33
|
+
- id: local-jsonl
|
|
34
|
+
label: "Local trace file (.jsonl / .ndjson)"
|
|
35
|
+
value: local-jsonl
|
|
36
|
+
platform_doc: "references/source-platforms/local-jsonl.md"
|
|
37
|
+
- id: claude-code
|
|
38
|
+
label: "Claude Code local transcripts (~/.claude/projects/…/*.jsonl)"
|
|
39
|
+
value: claude-code
|
|
40
|
+
platform_doc: "references/source-platforms/claude-code.md"
|
|
41
|
+
- id: codex
|
|
42
|
+
label: "Codex local transcripts (~/.codex/sessions/*.jsonl)"
|
|
43
|
+
value: codex
|
|
44
|
+
platform_doc: "references/source-platforms/codex.md"
|
|
45
|
+
- id: __custom__
|
|
46
|
+
label: "Custom — describe what you have"
|
|
47
|
+
value: __custom__
|
|
48
|
+
note: "Describe your trace format. Orchestrator will attempt a general-purpose fallback and guide credential setup."
|
|
49
|
+
|
|
50
|
+
# Phase 4 — Target platform (where agent definitions live)
|
|
51
|
+
target_platform:
|
|
52
|
+
question: "Where do your agent definitions live?"
|
|
53
|
+
key: target_platform
|
|
54
|
+
options:
|
|
55
|
+
- id: local-claude
|
|
56
|
+
label: "Claude Code agents (.claude/agents/*.md)"
|
|
57
|
+
value: local-claude
|
|
58
|
+
platform_doc: "references/target-platforms/local-claude.md"
|
|
59
|
+
- id: local-codex
|
|
60
|
+
label: "Codex agents (.codex/agents/*.md)"
|
|
61
|
+
value: local-codex
|
|
62
|
+
platform_doc: "references/target-platforms/local-codex.md"
|
|
63
|
+
- id: local-cursor
|
|
64
|
+
label: "Cursor agents (cursor-equivalent dir)"
|
|
65
|
+
value: local-cursor
|
|
66
|
+
platform_doc: "references/target-platforms/local-cursor.md"
|
|
67
|
+
- id: local-opencode
|
|
68
|
+
label: "OpenCode agents (opencode-equivalent dir)"
|
|
69
|
+
value: local-opencode
|
|
70
|
+
platform_doc: "references/target-platforms/local-opencode.md"
|
|
71
|
+
- id: mastra
|
|
72
|
+
label: "Mastra code construct (source code: new Agent({...}))"
|
|
73
|
+
value: mastra
|
|
74
|
+
platform_doc: "references/target-platforms/mastra.md"
|
|
75
|
+
- id: anthropic-cloud
|
|
76
|
+
label: "Anthropic Cloud Agent SDK (source code)"
|
|
77
|
+
value: anthropic-cloud
|
|
78
|
+
platform_doc: "references/target-platforms/anthropic-cloud.md"
|
|
79
|
+
- id: rest-api
|
|
80
|
+
label: "Cloud REST API (HTTP GET/PUT)"
|
|
81
|
+
value: rest-api
|
|
82
|
+
platform_doc: "references/target-platforms/rest-api.md"
|
|
83
|
+
- id: __custom__
|
|
84
|
+
label: "Custom — describe where your agents live"
|
|
85
|
+
value: __custom__
|
|
86
|
+
note: "Describe your agent storage format. Orchestrator will attempt the general-purpose code-construct apply path."
|
|
87
|
+
|
|
88
|
+
# Phase 5b agent install scope
|
|
89
|
+
agent_install_scope:
|
|
90
|
+
question: "Install diagnostics-* agents: project-scope or user-global?"
|
|
91
|
+
key: agent_install_scope
|
|
92
|
+
options:
|
|
93
|
+
- id: project
|
|
94
|
+
label: "Project-scope (recommended — visible in repo's .claude/agents/)"
|
|
95
|
+
value: project
|
|
96
|
+
note: "Writes to $PROJECT_ROOT/.claude/agents/ — committed, visible to all contributors."
|
|
97
|
+
- id: user
|
|
98
|
+
label: "User-global (~/.claude/agents/ — available in all projects)"
|
|
99
|
+
value: user
|
|
100
|
+
note: "Writes to ~/.claude/agents/ — not in repo, convenient for personal setups."
|
|
101
|
+
- id: __custom__
|
|
102
|
+
label: "Custom path — specify manually"
|
|
103
|
+
value: __custom__
|
|
104
|
+
note: "You will be prompted for the exact destination directory."
|
|
105
|
+
|
|
106
|
+
# Phase 5c dry-spawn verification offer
|
|
107
|
+
agent_dry_spawn:
|
|
108
|
+
question: "Agents verified. Want to dry-spawn each with a no-op probe?"
|
|
109
|
+
key: agent_dry_spawn
|
|
110
|
+
options:
|
|
111
|
+
- id: yes
|
|
112
|
+
label: "Yes — probe all 3 (safe: no trace data read, no writes)"
|
|
113
|
+
value: yes
|
|
114
|
+
note: "Dispatches each agent with prompt: 'echo: probe' and confirms they activate."
|
|
115
|
+
- id: no
|
|
116
|
+
label: "No — skip, I trust the verification"
|
|
117
|
+
value: no
|
|
118
|
+
- id: __custom__
|
|
119
|
+
label: "Custom — probe only specific agents"
|
|
120
|
+
value: __custom__
|
|
121
|
+
note: "Specify which agents to probe by name."
|
|
122
|
+
|
|
123
|
+
# Phase 6 — Trigger rules
|
|
124
|
+
# ⏸ DISABLED for v0.1: do NOT ask this question. v0.1 is always on-demand.
|
|
125
|
+
# Auto-write `schedule.mode: on-demand` without prompting. Re-enable (flip
|
|
126
|
+
# disabled: false) when scheduling lands post-v0.1. Config field is KEPT.
|
|
127
|
+
trigger_mode:
|
|
128
|
+
disabled: true
|
|
129
|
+
default: on-demand
|
|
130
|
+
question: "When should diagnostics run?"
|
|
131
|
+
key: schedule.mode
|
|
132
|
+
options:
|
|
133
|
+
- id: on-demand
|
|
134
|
+
label: "On-demand (I invoke it manually) — recommended for v0.1"
|
|
135
|
+
value: on-demand
|
|
136
|
+
- id: scheduled
|
|
137
|
+
label: "I'll set up scheduling later (structure prepared)"
|
|
138
|
+
value: on-demand
|
|
139
|
+
note: "Config will contain schedule.mode: on-demand. Edit references/workflows/schedule-prep.md to wire scheduling post-v0.1."
|
|
140
|
+
- id: __custom__
|
|
141
|
+
label: "Custom — describe your trigger"
|
|
142
|
+
value: __custom__
|
|
143
|
+
note: "Orchestrator will translate your description into a trigger rule. May require schedule.mode: scheduled."
|
|
144
|
+
|
|
145
|
+
# Phase 6 — Trigger condition
|
|
146
|
+
trigger_condition:
|
|
147
|
+
question: "Add a trigger condition? (optional)"
|
|
148
|
+
key: trigger_condition
|
|
149
|
+
options:
|
|
150
|
+
- id: score-threshold
|
|
151
|
+
label: "Diagnose when score < threshold (threshold auto-discovered)"
|
|
152
|
+
value: score-threshold
|
|
153
|
+
- id: error-rate
|
|
154
|
+
label: "Diagnose when error rate > 20%"
|
|
155
|
+
value: error-rate
|
|
156
|
+
- id: none
|
|
157
|
+
label: "No trigger condition — manual invocation only"
|
|
158
|
+
value: none
|
|
159
|
+
- id: __custom__
|
|
160
|
+
label: "Custom filter — describe in natural language"
|
|
161
|
+
value: __custom__
|
|
162
|
+
note: "Orchestrator will translate your description into a TraceFilter."
|
|
163
|
+
|
|
164
|
+
# Phase 8 — First-run pick
|
|
165
|
+
first_run:
|
|
166
|
+
question: "Setup complete! What would you like to diagnose first?"
|
|
167
|
+
key: first_run
|
|
168
|
+
options:
|
|
169
|
+
- id: last-24h
|
|
170
|
+
label: "Diagnose traces from the last 24 hours"
|
|
171
|
+
value: last-24h
|
|
172
|
+
- id: most-recent-failing
|
|
173
|
+
label: "Diagnose your most recently failing agent"
|
|
174
|
+
value: most-recent-failing
|
|
175
|
+
- id: negative-feedback-week
|
|
176
|
+
label: "Diagnose all sessions with negative feedback this week"
|
|
177
|
+
value: negative-feedback-week
|
|
178
|
+
- id: high-latency
|
|
179
|
+
label: "Diagnose all high-latency sessions"
|
|
180
|
+
value: high-latency
|
|
181
|
+
- id: manual
|
|
182
|
+
label: "I'll invoke manually later"
|
|
183
|
+
value: manual
|
|
184
|
+
- id: __custom__
|
|
185
|
+
label: "Custom — describe what you want to diagnose"
|
|
186
|
+
value: __custom__
|
|
187
|
+
note: "Orchestrator will translate your description into a TraceFilter."
|
package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/operation-inventory.md
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# Operation Inventory — Type A/B/C Classification
|
|
2
|
+
|
|
3
|
+
> Per PR-019. Every operation is exactly one type. No overlap.
|
|
4
|
+
> Used as the audit surface for code review of new contributions.
|
|
5
|
+
|
|
6
|
+
## Classification Rules
|
|
7
|
+
|
|
8
|
+
- **Type A — Pure Script**: deterministic logic, structured I/O. Written in TypeScript, invoked via `Bash("scripts/cli/run.sh scripts/<name>.ts ...")`. Zero LLM calls. Testable in isolation.
|
|
9
|
+
- **Type B — Agent Operation**: uses agent's native tools (Bash for external CLI, Agent for sub-dispatch, AskUserQuestion/equivalent for HITL, Read/Edit/Write for files). LLM reasoning. Cannot be unit tested in isolation.
|
|
10
|
+
- **Type C — Hybrid**: agent invokes a Type A script, passes structured input, reads structured output. Script provides deterministic shape; agent decides when/how to invoke it.
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## Type A — Pure Scripts (9 scripts, 2 self-diagnostics)
|
|
15
|
+
|
|
16
|
+
| Op | File | Why script |
|
|
17
|
+
|----|------|-----------|
|
|
18
|
+
| Tier 0 static scan | `scripts/tier0-scan.ts` | Deterministic pattern matching + signal counts |
|
|
19
|
+
| Trace slicing | `scripts/slicer.ts` | Deterministic math + cap-of-5 clustering |
|
|
20
|
+
| Config load | `scripts/config/load.ts` | YAML parse + env lookup |
|
|
21
|
+
| Config schema | `scripts/config/schema.ts` | TypeBox type definitions |
|
|
22
|
+
| Config validation | `scripts/config/validate.ts` | Schema validation + typed errors |
|
|
23
|
+
| Trace normalization | `scripts/normalize/platforms/*.ts` | Platform JSON → canonical shape |
|
|
24
|
+
| Setup detection | `scripts/setup/detect.ts` | File reads + state checks |
|
|
25
|
+
| Stale-target hash compare | `scripts/stale-detector.ts` | Pure hash compare |
|
|
26
|
+
| HTML report render | `scripts/report/render.ts` | Template + data → HTML |
|
|
27
|
+
| [INTERNAL] Host probe | `scripts/self-diagnostics/probe.ts` | Env + fs probing |
|
|
28
|
+
| [INTERNAL] Self-dispatch | `scripts/self-diagnostics/dispatch.ts` | Descriptor file write |
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## Type B — Agent Operations (12 items)
|
|
33
|
+
|
|
34
|
+
| Op | Agent action |
|
|
35
|
+
|----|-------------|
|
|
36
|
+
| Sub-agent dispatch | `Agent({subagent_type: 'diagnostics-analyzer', run_in_background: true})` |
|
|
37
|
+
| Trace fetch from source | `Bash(<cli> traces list ...)` per per-platform reference doc |
|
|
38
|
+
| Score-scale probe | `Bash(<cli> scores list --json)` + LLM classification |
|
|
39
|
+
| NL → TraceFilter translation | LLM reasoning over operator query |
|
|
40
|
+
| Cross-analyzer dedup | LLM reasoning over collected findings |
|
|
41
|
+
| RCA (3-dim WHAT/WHY/WHERE) | LLM reasoning per finding |
|
|
42
|
+
| Recursive whys | LLM reasoning (PR-020) |
|
|
43
|
+
| Remedy ranking | LLM judgment (cost × correctness) |
|
|
44
|
+
| HITL approval gate | `AskUserQuestion` / chat-fallback |
|
|
45
|
+
| BG-worktree apply | `Bash(git worktree + commit + push + gh pr create)` |
|
|
46
|
+
| REST apply | `Bash(curl GET + PUT)` with idempotency-key |
|
|
47
|
+
| CLI install check | `Bash(which <cli>)` + prompt (PR-021) |
|
|
48
|
+
|
|
49
|
+
---
|
|
50
|
+
|
|
51
|
+
## Type C — Hybrid (orchestrator invoking scripts)
|
|
52
|
+
|
|
53
|
+
| Op | Pattern |
|
|
54
|
+
|----|---------|
|
|
55
|
+
| Orchestrator invoking tier0 | Agent calls `Bash(scripts/cli/run.sh scripts/tier0-scan.ts ...)` → reads structured JSON output |
|
|
56
|
+
| Orchestrator invoking slicer | Agent calls `Bash(scripts/cli/run.sh scripts/slicer.ts ...)` → reads slice plan JSON |
|
|
57
|
+
| Apply-worker invoking stale-detector | Agent calls `Bash(scripts/cli/run.sh scripts/stale-detector.ts ...)` → reads freshness result |
|
|
58
|
+
| Orchestrator invoking report render | Agent calls `Bash(scripts/cli/run.sh scripts/report/render.ts ...)` → reads report.html path |
|
|
59
|
+
| Init invoking setup/detect | Agent calls `Bash(scripts/cli/run.sh scripts/setup/detect.ts)` → routes based on SetupState |
|
|
@@ -0,0 +1,167 @@
|
|
|
1
|
+
# mutagent-diagnostics — Overview
|
|
2
|
+
|
|
3
|
+
> **PRD-SO-01** · First reading for new users. This is the primer; `SKILL.md` is the
|
|
4
|
+
> manifest. Load `references/reference.md` next for the full architecture + dependency graph.
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## What this skill does
|
|
9
|
+
|
|
10
|
+
`mutagent-diagnostics` is a diagnostics-on-tap skill for AI agents. Given access to your
|
|
11
|
+
agent traces (from Langfuse, OpenTelemetry, local JSONL, Claude Code transcripts, or Codex
|
|
12
|
+
transcripts), the skill runs a full RCA cycle and produces a gold-standard HTML report
|
|
13
|
+
covering every detected failure with ranked remedies, a two-rationale remedy card, a feedback
|
|
14
|
+
block grounded in operator chat and trace scores, and a copy-back markdown bundle you can
|
|
15
|
+
hand straight to an apply agent.
|
|
16
|
+
|
|
17
|
+
The skill operates as an inline orchestrator running inside your coding-agent session. It
|
|
18
|
+
dispatches up to five parallel Analyzer sub-agents, aggregates their structured findings,
|
|
19
|
+
enriches them deterministically, and renders the result. No external server, no subscription,
|
|
20
|
+
no separate process — the skill IS the orchestrator.
|
|
21
|
+
|
|
22
|
+
On first invocation the skill detects whether a config exists. If not, it walks you through
|
|
23
|
+
an eight-phase onboarding: source platform, target platform, ASK tool, schedule mode, and
|
|
24
|
+
trigger rules. Subsequent invocations go straight to the diagnostic pipeline.
|
|
25
|
+
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
## When to use it
|
|
29
|
+
|
|
30
|
+
| Trigger | Example | Outcome |
|
|
31
|
+
|---------|---------|---------|
|
|
32
|
+
| A user reported an agent failure | "my sample-email-agent returned wrong data" | Full RCA with WHAT/WHY/WHERE taxonomy, ranked remedies |
|
|
33
|
+
| Trace score dropped | Langfuse score fell below 0.6 | Cost-scoped analysis; coverage-proofed findings |
|
|
34
|
+
| You shipped a change and want a pre-release check | Before merging a prompt update | Structured findings vs declared success criteria (contract mode) |
|
|
35
|
+
| You suspect methodology drift in your diagnostics skill | Skill maintainer running self-diag | `[INTERNAL]` META-audience report; `--self-diag` flag |
|
|
36
|
+
| You want a neutral survey of the last 24h | No specific complaint | Window-based slicer; exploratory open-ended report |
|
|
37
|
+
|
|
38
|
+
**When NOT to use it:**
|
|
39
|
+
- Real-time alerting — the skill produces a point-in-time report, not a streaming monitor.
|
|
40
|
+
- Replacing your observability platform — the skill reads FROM your platform, not instead of it.
|
|
41
|
+
- Scheduling automated runs — v0.1 is on-demand only; see `references/workflows/schedule-prep.md`.
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## Quick-start (5-minute onboarding)
|
|
46
|
+
|
|
47
|
+
### Step 1 — Install (one-time)
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
pnpx @mutagent/diagnostics init
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
This detects your runtime, copies required templates, and launches the onboarding wizard.
|
|
54
|
+
|
|
55
|
+
### Step 2 — Run the wizard
|
|
56
|
+
|
|
57
|
+
The onboarding wizard asks four questions:
|
|
58
|
+
|
|
59
|
+
1. **Source platform** — where your traces live (Langfuse / OTel / local JSONL / Claude Code / Codex).
|
|
60
|
+
2. **Target platform** — where your agent definitions live (e.g., `.claude/agents/` for Claude Code agents; `cloud-rest` for remote agents; `report-only` to skip apply entirely).
|
|
61
|
+
3. **ASK tool** — auto-detected from your runtime.
|
|
62
|
+
4. **Trigger rules** — which traces qualify for diagnosis (e.g., `score_below: 0.6`).
|
|
63
|
+
|
|
64
|
+
Config is saved to `<project>/.mutagent/config.yaml`. Secrets (API keys) live in
|
|
65
|
+
`<project>/.mutagentrc` (gitignored).
|
|
66
|
+
|
|
67
|
+
### Step 3 — Diagnose
|
|
68
|
+
|
|
69
|
+
In your coding-agent chat:
|
|
70
|
+
|
|
71
|
+
```
|
|
72
|
+
/mutagent-diagnostics
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Or with a brief:
|
|
76
|
+
|
|
77
|
+
```
|
|
78
|
+
/mutagent-diagnostics "diagnose sample-email-agent focus on cost loops last 24h"
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
The brief is parsed into `{ agent, timeWindow, focus, residual }` by
|
|
82
|
+
`scripts/invocation/parse-brief.ts`. A `focus` directive activates Guided mode (the Guided
|
|
83
|
+
tab replaces Overview). No focus means a neutral survey.
|
|
84
|
+
|
|
85
|
+
### Step 4 — Review the report
|
|
86
|
+
|
|
87
|
+
The skill opens an HTML report in your browser. The report has four tab types:
|
|
88
|
+
|
|
89
|
+
- **Methodology [INTERNAL]** — how the diagnosis was run (orchestrator trace, decisions log).
|
|
90
|
+
- **Overview** — entity card, 6-tile big-stat row, 24h latency heatmap, findings table.
|
|
91
|
+
- **F-NNN tabs** — one per finding: severity badge, taxonomy chips, problem, evidence,
|
|
92
|
+
why-chain, assumptions, coverage proof, ranked remedies.
|
|
93
|
+
- **Decisions** — live-preview markdown bundle; check remedies, add notes, click Copy.
|
|
94
|
+
|
|
95
|
+
### Step 5 — Apply
|
|
96
|
+
|
|
97
|
+
Paste the copied markdown bundle into your coding-agent chat. The apply worker spawns a
|
|
98
|
+
background agent on an isolated worktree, makes the changes, and opens a PR.
|
|
99
|
+
|
|
100
|
+
---
|
|
101
|
+
|
|
102
|
+
## Anatomy of a diagnostic report
|
|
103
|
+
|
|
104
|
+
For the full panel-by-panel anatomy — every block in the issue card, every element in the
|
|
105
|
+
remedy card, the live-preview mechanics — see:
|
|
106
|
+
|
|
107
|
+
```
|
|
108
|
+
references/workflows/rendering-anatomy.md
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
Key elements per finding panel:
|
|
112
|
+
|
|
113
|
+
| Block | Location | Notes |
|
|
114
|
+
|-------|----------|-------|
|
|
115
|
+
| Severity badge | Top-left | CRIT / HIGH / MED / INFO |
|
|
116
|
+
| Taxonomy chips | Below title | WHAT · WHY · WHERE · APPLY · AUDIENCE |
|
|
117
|
+
| Feedback block | Between Problem and Evidence | Color-coded by source (chat=cyan, score=yellow, external=purple). Only appears when `feedbackSources[]` is populated. |
|
|
118
|
+
| Why-chain | Below evidence | Causal steps; deepest marked `isOrigin: true` |
|
|
119
|
+
| Assumptions | Below why-chain | Verified / unverified / hypothesis-pending pills |
|
|
120
|
+
| Coverage proof | Below assumptions | 4-dim widget (latency, score, temporal, tool-trajectory); low confidence shows yellow banner |
|
|
121
|
+
| Remedy card | Per remedy | Two-rationale blocks (purple Why + cyan WhyWorks); diff grid; apply plan + instructions; notes textarea |
|
|
122
|
+
|
|
123
|
+
---
|
|
124
|
+
|
|
125
|
+
## What the skill does NOT do
|
|
126
|
+
|
|
127
|
+
- **Does not monitor continuously.** It runs on-demand or on a schedule you configure.
|
|
128
|
+
- **Does not auto-apply without approval.** Every apply goes through a HITL copy-back gate.
|
|
129
|
+
- **Does not hallucinate entity data.** The entity card is populated deterministically from
|
|
130
|
+
trace metadata and your config — no LLM is involved in entity extraction.
|
|
131
|
+
- **Does not reveal PII.** The system prompt field in the entity card is always collapsed
|
|
132
|
+
(explicit click to view) regardless of size.
|
|
133
|
+
- **Does not run self-diagnostics automatically.** `self_diagnostics.enabled` defaults to
|
|
134
|
+
false; enabling it is a skill-maintainer opt-in.
|
|
135
|
+
- **Does not cross-reference other skills.** Each skill is a sealed unit.
|
|
136
|
+
|
|
137
|
+
---
|
|
138
|
+
|
|
139
|
+
## Glossary
|
|
140
|
+
|
|
141
|
+
| Term | Definition |
|
|
142
|
+
|------|------------|
|
|
143
|
+
| **Tier-0** | Static pattern scan (`scripts/tier0-scan.ts`) — runs BEFORE any LLM call to bound token cost. Measures cheap signals (latency distribution, error counts, score thresholds). |
|
|
144
|
+
| **RCA** | Root Cause Analysis — the 3-dimensional `(WHAT, WHY, WHERE)` failure taxonomy produced by the Analyzer agents. See `references/workflows/rca.md`. |
|
|
145
|
+
| **EntityContext** | Auto-extracted agent descriptor: name, model, system prompt, tool inventory with per-tool stats, input sample. Populated deterministically by the source-platform normalizer — no LLM. |
|
|
146
|
+
| **applyTarget** | The symbolic reference to the agent definition the remedy should be applied to (e.g., `.claude/agents/search-agent.md` for a local Claude agent). Declared by the Analyzer; rendered in the remedy meta strip. |
|
|
147
|
+
| **rationale** | The comparative remedy rationale block — WHY this remedy over alternatives (purple). Always visible, never collapsible. |
|
|
148
|
+
| **whyWorks** | The causal mechanism block — HOW the fix closes the failure (cyan). Always visible, never collapsible. |
|
|
149
|
+
| **coverageProof** | 4-dimensional sampling audit: for each of latency, score, temporal, and tool-trajectory, what fraction of the population buckets did the sample cover? Confidence mapped to high/med/low via 90/70 thresholds. |
|
|
150
|
+
| **Wave-6 methodology** | The diagnostic methodology layer (R2.1–R2.6 + D1/D2): mandatory LLM deep-read gate, awareness mini-sample, representative 4-bucket sampler, class-memory library, blind-spots taxonomy, and coverage proof. |
|
|
151
|
+
| **report-only** | A `target.platform` value that skips the apply gate entirely. Useful for read-only environments or when you want a report without committing to any changes. |
|
|
152
|
+
| **feedbackSources** | Structured list of operator feedback linked to a finding: chat messages (`chat`), Langfuse trace scores (`trace-score`), or external feedback platform entries (`external`). |
|
|
153
|
+
|
|
154
|
+
---
|
|
155
|
+
|
|
156
|
+
## Further reading (progressive disclosure)
|
|
157
|
+
|
|
158
|
+
| Doc | When to read |
|
|
159
|
+
|-----|-------------|
|
|
160
|
+
| `references/reference.md` | Full architecture DAG + complete TOC |
|
|
161
|
+
| `references/workflows/orchestrator-protocol.md` | The step-by-step diagnostic run procedure |
|
|
162
|
+
| `references/workflows/onboarding.md` | Onboarding phase-by-phase detail |
|
|
163
|
+
| `references/workflows/rca.md` | RCA taxonomy + finding shape |
|
|
164
|
+
| `references/workflows/rendering-anatomy.md` | Canonical per-finding + per-remedy anatomy |
|
|
165
|
+
| `references/principles.md` | 53 design principles (PR-001..PR-053) — the audit surface |
|
|
166
|
+
| `references/config.md` | Full config schema with doc strings and examples |
|
|
167
|
+
| `SKILL.md` | The manifest: §0 setup detection, §3 architecture, §4 bill of materials |
|