@mutagent/helix 0.1.0-alpha.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/skills/mutagent-helix/SKILL.md +113 -0
- package/.claude/skills/mutagent-helix/scripts/cli/codex-transcode.ts +143 -0
- package/.claude/skills/mutagent-helix/scripts/cli/doctor.ts +111 -0
- package/.claude/skills/mutagent-helix/scripts/cli/init.ts +382 -0
- package/.claude/skills/mutagent-helix/scripts/cli/run.sh +61 -0
- package/CLAUDE.md +193 -0
- package/LICENSE +201 -0
- package/NOTICE +22 -0
- package/README.md +209 -0
- package/assets/adl-loop.svg +162 -0
- package/assets/system-status.svg +422 -0
- package/bin/mutagent-cli.mjs +9950 -0
- package/bin/mutagent-helix.mjs +78 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/CLAUDE.md +28 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/SKILL.md +138 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/assets/templates/agentspec.yaml.tpl +247 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/references/frameworks/doc-pins.md +53 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/references/principles.md +91 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/references/requirements.yaml +295 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/references/workflows/orchestrator-protocol.md +302 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/cli/doctor.ts +116 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/cli/init.ts +230 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/cli/run.sh +60 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/config/resolve-spec-dir.ts +199 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/contract/agentspec.schema.ts +455 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/setup/detect.ts +67 -0
- package/mutagent-agentspec/.claude/skills/mutagent-agentspec/scripts/validate/validate-spec.ts +93 -0
- package/mutagent-agentspec/LICENSE +201 -0
- package/mutagent-agentspec/NOTICE +19 -0
- package/mutagent-agentspec/README.md +44 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/SKILL.md +100 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/assets/agents/ai-architect.md +312 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/assets/agents/ai-engineer.md +357 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/assets/templates/build-report.md.tpl +39 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/references/principles.md +31 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/references/workflows/build-protocol.md +61 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/references/workflows/improve-handoff.md +23 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/scripts/cli/run.sh +60 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/scripts/handoff/validate-build-input.ts +72 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/scripts/sync-spec/check-sync-spec.ts +499 -0
- package/mutagent-builder/.claude/skills/mutagent-builder/scripts/verify/spec-impl-coverage.ts +182 -0
- package/mutagent-builder/CLAUDE.md +25 -0
- package/mutagent-builder/README.md +26 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/README.md +192 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/SKILL.md +352 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/agents/ai-architect.md +256 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/agents/ai-engineer.md +344 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/agents/diagnostics-analyzer.md +576 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/agents/discovery.md +425 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/audit.json.tpl +24 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/audit.md.tpl +49 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/config.yaml.tpl +100 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/pr-body.md.tpl +56 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/report.html.tpl +327 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/README.md +27 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/iter-N-handover.md.tpl +125 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/self-diagnosis-contract.v0.1.0.yaml.tpl +234 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/spec.yaml.tpl +152 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/team.yaml.tpl +127 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/templates/shared/wave-N-dashboard.html.tpl +1712 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.1-nl-query.md +59 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.2-tier0-progress.md +58 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.3-analysis-progress.md +44 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.4-html-report.md +61 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.5-rca-detail.md +69 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/diagnostics/WF-2.6-stale-warning.md +53 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.1-welcome.md +60 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.2-source-platform-picker.md +41 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.3-source-auth.md +75 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.4-target-picker.md +48 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.5-ask-tool.md +43 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.6-config-review.md +57 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.7-filter-defaults.md +52 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/assets/wireframes/onboarding/WF-1.8-first-run.md +59 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/examples/sample-findings.json +92 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/examples/sample-meta.json +47 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/adapter-strategy.md +75 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/config-migration.md +84 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/config.md +159 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/filter-search-matrix.md +66 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/harness-knowledge.md +22 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/memory-format.md +78 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/onboarding-decisions.yaml +187 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/operation-inventory.md +59 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/overview.md +167 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/reference.md +119 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/script-index.md +101 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-adapter-contract.md +93 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/claude-code-transcripts.md +66 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/codex-transcripts.md +78 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/codex.md +145 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/install-paths.md +57 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/langfuse.md +178 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/local-jsonl.md +55 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/openobserve.md +102 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/source-platforms/otel.md +63 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/cloud-rest.md +84 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-claude.md +65 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-cloud-agent-sdk.md +92 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-codex.md +72 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-cursor.md +108 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-mastra.md +77 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/target-platforms/local-opencode.md +70 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/apply-pr-comment-format.md +82 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/diagnostics.md +35 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/handover-contract.md +247 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/onboarding.md +300 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/orchestrator-protocol.md +1861 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/rca.md +331 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/rendering-anatomy.md +314 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/schedule-prep.md +100 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/scope-model.md +128 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/references/workflows/verification-methodology.md +174 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/aggregate/sort-findings.ts +0 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/awareness/blind-spots.ts +167 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/awareness/llm-sample.ts +681 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/doctor.ts +215 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/init.ts +1013 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/install-agents.ts +350 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/run.sh +61 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/cli/shared-templates.ts +79 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/config/load.ts +392 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/config/resolve-credential.ts +120 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/config/schema.ts +641 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/config/validate.ts +134 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/context/build-diagnosis-context.ts +417 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/context/load-context.ts +105 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/contract/types.ts +166 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/enrich/build-render-input.ts +1964 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/enrich/corpus-severity.ts +172 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/enrich/dismissal-match.ts +270 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/enrich/rank-remedies.ts +75 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/hooks/install-hooks.sh +46 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/hooks/pre-push.sh +78 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/invocation/parse-brief.ts +188 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/library/match.ts +114 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/library/paths.ts +95 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/library/store.ts +535 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/library/types.ts +181 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/lint/template-inline-js.ts +267 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/memory/append.ts +188 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/memory/read.ts +218 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/platforms/entity-context.ts +686 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/read-unitf.ts +267 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/trace.ts +1078 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/unitf-adapter.ts +0 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/normalize/unitf-types.ts +213 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/report/persist-selections.ts +174 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/report/render.ts +3254 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/run/diagnose.ts +64 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/run/session.ts +167 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/sample/caps.ts +345 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/sample/deep-read-gate.ts +424 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/sample/representative.ts +584 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/scan/objection.ts +224 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/scan/trajectory.ts +489 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/self-diagnostics/dispatch.ts +135 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/self-diagnostics/probe.ts +170 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/setup/detect.ts +307 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/setup/ensure-cli.ts +315 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/setup/reconfigure.ts +77 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/setup/verify-agents.ts +207 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/slicer.ts +166 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/claude-code.ts +17 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/codex.ts +11 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/langfuse.ts +17 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/local-jsonl.ts +12 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/adapters/otel.ts +12 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/index.ts +54 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/registry.ts +45 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/source/types.ts +86 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/stale-detector.ts +74 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/test/render-playwright.cli.ts +477 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/tier0/claude-code.ts +133 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/tier0/langfuse.ts +143 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/tier0-scan.ts +423 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/completeness-check.ts +436 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/doc-refs.ts +190 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/finalize-gate.ts +417 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/findings-contract.ts +628 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/render-js-syntax.ts +188 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/report-checklist.yaml +217 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/trace-shape.ts +262 -0
- package/mutagent-diagnostics/.claude/skills/mutagent-diagnostics/scripts/validate/wave6-checklist.ts +326 -0
- package/mutagent-diagnostics/CLAUDE.md +41 -0
- package/mutagent-diagnostics/LICENSE +201 -0
- package/mutagent-diagnostics/NOTICE +18 -0
- package/mutagent-diagnostics/README.md +343 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/SKILL.md +548 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/agents/audit-executor.md +169 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/agents/dataset-builder.md +160 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/agents/discovery.md +425 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/agents/evaluator.md +1108 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/brand/theme.css +213 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/assets/brand/wordmark.html +9 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/context-flow-lens.md +85 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/data-lens.md +47 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/decision-lens.md +48 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/methodology-critic-lens.md +53 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/lenses/trajectory-lens.md +44 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/build-review-interface.md +83 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/edd-loop.md +134 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/error-analysis.md +113 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/eval-audit.md +154 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/eval-stage.md +168 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/generate-synthetic-data.md +81 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/grounded-adjudication.md +221 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/memory-format.md +65 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/methodology.md +201 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/operation-inventory.md +197 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/validate-evaluator.md +125 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/workflows/orchestrator-protocol.md +300 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/references/write-judge-prompt.md +123 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/behavior-tree.schema.yaml +73 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/dataset.schema.yaml +66 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/edd-change-request.schema.yaml +114 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/eval-matrix.schema.yaml +74 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/flow-graph.schema.yaml +69 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/flow-profile.schema.yaml +49 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/methodology-review.schema.yaml +40 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/schemas/scorecard.schema.yaml +85 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/agent-dispatch.ts +0 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/aggregate-discover.ts +543 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/artifact-paths.ts +115 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/assemble-scorecard.ts +172 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/build-dataset.ts +186 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/build-evals.ts +93 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/build-review-ui.ts +393 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/check-method-router.ts +170 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/aggregate.ts +112 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/audit-run.ts +182 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/doctor.ts +211 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/dogfood.ts +133 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/init.ts +601 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/methodology-review.ts +122 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/prep.ts +295 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/profile-subject.ts +165 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/run.sh +56 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cli/variance-check.ts +105 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/code-eval.ts +248 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/code-quality-verdict.ts +285 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/codegen-evals.ts +173 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cold-start-project.ts +151 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/cold-start-sampler.ts +267 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/config/load.ts +343 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/config/schema.ts +380 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/agentspec-evals.ts +85 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/dataset.ts +149 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/eval-engine.ts +118 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/eval-matrix.ts +577 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/eval-types.ts +1074 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/flow-graph.ts +194 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/types.ts +303 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/contracts/validation.ts +193 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/derive-dataset.ts +152 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/determine-outcome.ts +219 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/diff-discriminate.ts +160 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/discover-criteria.ts +348 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/edd/change-request.ts +232 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/edd/edd-types.ts +210 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/edd/variance-gate.ts +186 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/emit-completeness.ts +111 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/eval-engine.ts +160 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/evaluate.ts +333 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/flow-graph.ts +230 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/judge-prompt-template.ts +253 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/judge-provider.ts +135 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/lint-grounding.ts +229 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/lint-uniformity.ts +168 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/living-suite.ts +109 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/load-bundle.ts +203 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/load-profile-vocab.ts +64 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/load-profile.ts +106 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/mask.ts +138 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/materialize-dataset.ts +113 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/matrix-judge.ts +750 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/memory/append.ts +215 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/memory/read.ts +168 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/persist-eval-criteria.ts +232 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/prep-tasks.ts +125 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/profile-subject.ts +310 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/publish-report.ts +131 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/read-unitf-traces.ts +81 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/render-build-cards.ts +195 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/render-discover-report.ts +1640 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/render-eval-report.ts +3838 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/render-report.ts +212 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/resolve-credential.ts +110 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/resolve-ref.ts +98 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/result-verify.ts +129 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/route-failures.ts +330 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/run-deterministic.ts +367 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/run-evaluate.ts +719 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/run-judge.ts +155 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/run-pipeline.ts +175 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/sample-traces.ts +210 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/self-audit.ts +387 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/source-map.ts +106 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/subject-profile.ts +134 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/substrate.ts +162 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/sync-eval-criteria.ts +244 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/ui-slots.ts +119 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/unitf-to-evaltrace.ts +284 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/validate-judge.ts +358 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/scripts/variance-compare.ts +177 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/subjects/mutagent-diagnostics/behavior-tree.yaml +140 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/subjects/mutagent-diagnostics/eval-matrix.yaml +1270 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/subjects/mutagent-diagnostics/methodology-review.yaml +105 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/workflows/audit.workflow.js +82 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/workflows/data-leak.workflow.js +236 -0
- package/mutagent-evaluator/.claude/skills/mutagent-evaluator/workflows/variance.workflow.js +163 -0
- package/mutagent-evaluator/CLAUDE.md +45 -0
- package/mutagent-evaluator/LICENSE +201 -0
- package/mutagent-evaluator/NOTICE +23 -0
- package/mutagent-evaluator/README.md +88 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/SKILL.md +287 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/scripts/contracts/amend-request.ts +503 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/scripts/improve-loop-run.ts +304 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/scripts/loop-state-cli.ts +300 -0
- package/mutagent-optimize/.claude/skills/mutagent-optimize/scripts/loop-state.ts +440 -0
- package/mutagent-optimize/CLAUDE.md +49 -0
- package/mutagent-optimize/README.md +26 -0
- package/mutagent-orchestrator/CLAUDE.md +102 -0
- package/mutagent-orchestrator/README.md +20 -0
- package/mutagent-orchestrator/assets/agents/discovery.md +425 -0
- package/mutagent-orchestrator/assets/agents/dogfood-monitor.md +266 -0
- package/mutagent-orchestrator/assets/agents/monitor.md +258 -0
- package/mutagent-orchestrator/orchestrator.md +782 -0
- package/mutagent-orchestrator/references/MONITORING.md +209 -0
- package/mutagent-orchestrator/references/config-fields.md +132 -0
- package/mutagent-orchestrator/references/config-migration.md +132 -0
- package/mutagent-orchestrator/references/index-registry.md +92 -0
- package/mutagent-orchestrator/routing.yaml +391 -0
- package/mutagent-orchestrator/scripts/config-schema.ts +1010 -0
- package/mutagent-orchestrator/scripts/dispatch.ts +439 -0
- package/mutagent-orchestrator/scripts/dogfood/extract-feedback.ts +203 -0
- package/mutagent-orchestrator/scripts/dogfood/reconstruct-trajectory.ts +327 -0
- package/mutagent-orchestrator/scripts/dogfood/render-dogfood-report.ts +347 -0
- package/mutagent-orchestrator/scripts/dogfood/resolve-subagents.ts +160 -0
- package/mutagent-orchestrator/scripts/dogfood/run-report.ts +264 -0
- package/mutagent-orchestrator/scripts/dogfood/slack-thread.ts +106 -0
- package/mutagent-orchestrator/scripts/dogfood/types.ts +257 -0
- package/mutagent-orchestrator/scripts/eval-routing.ts +753 -0
- package/mutagent-orchestrator/scripts/gate.ts +219 -0
- package/mutagent-orchestrator/scripts/handover-contract.ts +362 -0
- package/mutagent-orchestrator/scripts/index/build-index.ts +351 -0
- package/mutagent-orchestrator/scripts/index/render-index.ts +110 -0
- package/mutagent-orchestrator/scripts/monitor/slack-notify.ts +111 -0
- package/mutagent-orchestrator/scripts/monitor/triggers.ts +162 -0
- package/mutagent-orchestrator/scripts/onboarding-check.ts +369 -0
- package/mutagent-orchestrator/scripts/render-roster.ts +308 -0
- package/mutagent-orchestrator/scripts/resolve-credential.ts +185 -0
- package/mutagent-orchestrator/scripts/resolve-paths.ts +114 -0
- package/mutagent-orchestrator/scripts/slack/format.ts +63 -0
- package/mutagent-orchestrator/scripts/slack/post.ts +115 -0
- package/mutagent-orchestrator/scripts/sync-index.ts +311 -0
- package/package.json +64 -0
|
@@ -0,0 +1,440 @@
|
|
|
1
|
+
// ---------------------------------------------------------------------------
|
|
2
|
+
// mutagent-optimize — the improve-loop STATE MODEL (the deterministic core the
|
|
3
|
+
// improve-loop-run.ts driver advances). Pure data + pure transitions: no clock,
|
|
4
|
+
// no random, no IO — every time-like input (elapsed budget, iteration) is passed
|
|
5
|
+
// IN, so the same (state, inputs) always yields the same next state + terminator.
|
|
6
|
+
//
|
|
7
|
+
// The loop is a BOUNDED, goal-legal FSM (Build → Verify → Eval → Gate →
|
|
8
|
+
// [Diagnose ↺]). It may only run if it declares an observable goal + hard
|
|
9
|
+
// termination gates (max-iters · wallclock budget · no-improvement) — else it
|
|
10
|
+
// refuses (see assertGoalLegal). An iteration counts as improvement ONLY on a
|
|
11
|
+
// strict, above-noise gain that does not regress variance (the variance-gate).
|
|
12
|
+
// ---------------------------------------------------------------------------
|
|
13
|
+
|
|
14
|
+
/** The FSM phases — S0..S5 + the terminal state. */
|
|
15
|
+
export const LoopPhase = {
|
|
16
|
+
Init: "S0-init",
|
|
17
|
+
Build: "S1-build", // ai-engineer #apply — WRITE (worktree-scoped)
|
|
18
|
+
Verify: "S2-verify", // ai-architect PROCEED|STEER|ABORT
|
|
19
|
+
Eval: "S3-eval", // evaluator re-eval swing — JUDGE
|
|
20
|
+
Gate: "S4-gate", // GATE verdict PASS/FAIL
|
|
21
|
+
Diagnose: "S5-diagnose", // analyzer RCA on new failures → back to S1
|
|
22
|
+
Terminal: "terminal",
|
|
23
|
+
} as const;
|
|
24
|
+
export type LoopPhaseValue = (typeof LoopPhase)[keyof typeof LoopPhase];
|
|
25
|
+
|
|
26
|
+
/** How a bounded loop legally STOPS (goal-legal terminators). */
|
|
27
|
+
export const Terminator = {
|
|
28
|
+
/** the observable goal was met (eval-pass · criterion · delta) */
|
|
29
|
+
Converged: "converged",
|
|
30
|
+
/** the eval delta flat-lined below the noise floor for N turns */
|
|
31
|
+
NoImprovement: "no-improvement",
|
|
32
|
+
/** hit the max-iters ceiling */
|
|
33
|
+
MaxIters: "max-iters",
|
|
34
|
+
/** hit the wallclock budget */
|
|
35
|
+
Budget: "budget",
|
|
36
|
+
/** ai-architect returned ABORT on an applied remedy */
|
|
37
|
+
Aborted: "aborted",
|
|
38
|
+
} as const;
|
|
39
|
+
export type TerminatorValue = (typeof Terminator)[keyof typeof Terminator];
|
|
40
|
+
|
|
41
|
+
/** The observable goal a loop must declare to run at all. */
|
|
42
|
+
export type Goal =
|
|
43
|
+
| { kind: "eval-pass" } // GATE = PASS (0 critical/high fail)
|
|
44
|
+
| { kind: "criterion"; id: string } // a named criterion flips to pass
|
|
45
|
+
| { kind: "delta"; min: number } // eval score improves by ≥ min
|
|
46
|
+
// CODE-TARGET (Wave-2 W2I1). Converged = BOTH a hard deterministic gate (the code
|
|
47
|
+
// subject's OWN test suite is green, `IterationRecord.testsGreen === true`) AND the
|
|
48
|
+
// code-quality evaluator verdict passes (`gate === PASS`). NEITHER alone converges:
|
|
49
|
+
// a green test suite with a failing quality verdict keeps looping, and a passing
|
|
50
|
+
// quality verdict on a red test suite keeps looping. The session supplies both
|
|
51
|
+
// signals per round (S1 build → test-green; S3/S4 → the quality gate) — see the
|
|
52
|
+
// mutagent-optimize SKILL §2·5 code-target conduct branch.
|
|
53
|
+
| { kind: "code-quality" }
|
|
54
|
+
// NATURAL-LANGUAGE goal (Wave-2 W2I10; OT-4 widened to a criterion SET). A free-text
|
|
55
|
+
// goal the operator typed (`*optimize <subject> --goal "<free text>"`). It carries the
|
|
56
|
+
// raw `text` for provenance/audit, and is ILLEGAL until it is FROZEN — the loop needs an
|
|
57
|
+
// OBSERVABLE, unchanging yardstick, so a free-text intent is not runnable on its own
|
|
58
|
+
// (assertGoalLegal rejects `resolved === undefined` / an empty set, exactly as it rejects
|
|
59
|
+
// an unbounded config). The INTERPRETATION of the text is AGENT reasoning done by the
|
|
60
|
+
// top-level session at loop entry (interpret → confirm-once → freeze; SKILL §2·5), NOT a
|
|
61
|
+
// code loop — a real NL intent usually decomposes into a COMPOSITION of binary criteria /
|
|
62
|
+
// measurable-metric criteria, so the frozen form is a SET `resolved.criterionIds[]` (a
|
|
63
|
+
// single-criterion intent is just a 1-element set). Once frozen, those ids name the
|
|
64
|
+
// concrete criteria the loop runs against, and `goalMet` treats it like an AND of
|
|
65
|
+
// `criterion` goals — met only when ALL listed ids pass (each a strict binary check).
|
|
66
|
+
// A headless run (no human to confirm-freeze) therefore refuses at the goal-legal
|
|
67
|
+
// gate — an unfrozen NL goal never runs unbounded.
|
|
68
|
+
| { kind: "nl"; text: string; resolved?: { criterionIds: string[] } };
|
|
69
|
+
|
|
70
|
+
/**
|
|
71
|
+
* The subject a loop is optimizing, by KIND (Wave-2 W2I10). Governs which goals are
|
|
72
|
+
* even applicable — see `assertGoalAllowedForSubject`:
|
|
73
|
+
* - `code` — actual source: the ONLY kind with a test suite + code to gate, so
|
|
74
|
+
* the `code-quality` BOTH-gate applies here and ONLY here.
|
|
75
|
+
* - `agent` — a markdown / platform agent (Claude Code `.md`, Codex, prompt-defined):
|
|
76
|
+
* evals only; there is no code to test, so `code-quality` NEVER fires.
|
|
77
|
+
* - `skill` — a prompt/markdown-defined skill: evals only (same reason as `agent`).
|
|
78
|
+
* - `platform` — a hosted/reconstruct-spec subject (non-code markdown): evals only.
|
|
79
|
+
* The `code-quality` NAME is retained deliberately (operator-confirmed): it is accurate
|
|
80
|
+
* precisely because it applies only to real code.
|
|
81
|
+
*
|
|
82
|
+
* `SUBJECT_KINDS` is the SINGLE SOURCE OF TRUTH for the legal values — the `SubjectKind`
|
|
83
|
+
* type is derived from it, and `assertSubjectKind` validates raw CLI input against it (so a
|
|
84
|
+
* bogus `--subject-kind` fails LOUD instead of casting through silently). This is the
|
|
85
|
+
* optimize-loop goal-legality vocabulary: the inter-stage handover / config `TargetSubject`
|
|
86
|
+
* canon is the narrower `agent | skill | code`; `platform` is optimize-only (an evals-only
|
|
87
|
+
* non-code subject, same goal-class as agent/skill — it just never has a code test suite).
|
|
88
|
+
*/
|
|
89
|
+
export const SUBJECT_KINDS = ["code", "agent", "skill", "platform"] as const;
|
|
90
|
+
export type SubjectKind = (typeof SUBJECT_KINDS)[number];
|
|
91
|
+
|
|
92
|
+
/** Type-guard: is `v` a recognized SubjectKind? */
|
|
93
|
+
export function isSubjectKind(v: string): v is SubjectKind {
|
|
94
|
+
return (SUBJECT_KINDS as readonly string[]).includes(v);
|
|
95
|
+
}
|
|
96
|
+
|
|
97
|
+
/**
|
|
98
|
+
* Assert `v` is a recognized SubjectKind — throws (fail-closed) on an unrecognized value so a
|
|
99
|
+
* bogus `--subject-kind` is rejected loudly rather than silently accepted. Used by the CLI
|
|
100
|
+
* `assert-goal-legal --subject-kind` path.
|
|
101
|
+
*/
|
|
102
|
+
export function assertSubjectKind(v: string): asserts v is SubjectKind {
|
|
103
|
+
if (!isSubjectKind(v)) {
|
|
104
|
+
throw new Error(
|
|
105
|
+
`unknown subject-kind '${v}' — expected one of: ${SUBJECT_KINDS.join(" · ")}`,
|
|
106
|
+
);
|
|
107
|
+
}
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
/** The loop's hard bounds + variance policy (the termination contract). */
|
|
111
|
+
export interface LoopConfig {
|
|
112
|
+
goal: Goal;
|
|
113
|
+
maxIters: number;
|
|
114
|
+
budgetMs: number;
|
|
115
|
+
/** consecutive non-improving turns before a no-improvement stop. */
|
|
116
|
+
noImprovementStreak: number;
|
|
117
|
+
/** the calibrated noise floor a delta must clear to count as improvement. */
|
|
118
|
+
noiseFloor: number;
|
|
119
|
+
/**
|
|
120
|
+
* The subject KIND the loop is optimizing (OPTIONAL). When present, `assertGoalLegal`
|
|
121
|
+
* additionally enforces the subject-kind → goal legality (e.g. a `code-quality` goal is
|
|
122
|
+
* illegal on a non-code subject) — so a HEADLESS `runImproveLoop` enforces the same guard
|
|
123
|
+
* the interactive CLI enforces via its `--subject-kind` flag, without the flag. ADDITIVE:
|
|
124
|
+
* absent ⇒ the loop skips the subject-kind gate (pre-existing behavior, unchanged).
|
|
125
|
+
*/
|
|
126
|
+
subjectKind?: SubjectKind;
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
/** The ai-architect verdict on an applied remedy. */
|
|
130
|
+
export const ArchitectVerdict = {
|
|
131
|
+
Proceed: "PROCEED",
|
|
132
|
+
Steer: "STEER",
|
|
133
|
+
Abort: "ABORT",
|
|
134
|
+
} as const;
|
|
135
|
+
export type ArchitectVerdictValue =
|
|
136
|
+
(typeof ArchitectVerdict)[keyof typeof ArchitectVerdict];
|
|
137
|
+
|
|
138
|
+
/** The GATE verdict from the re-eval. */
|
|
139
|
+
export const GateVerdict = { Pass: "PASS", Fail: "FAIL" } as const;
|
|
140
|
+
export type GateVerdictValue = (typeof GateVerdict)[keyof typeof GateVerdict];
|
|
141
|
+
|
|
142
|
+
/** One iteration's recorded outcome (persisted in loop-state.json). */
|
|
143
|
+
export interface IterationRecord {
|
|
144
|
+
iteration: number;
|
|
145
|
+
verify: ArchitectVerdictValue;
|
|
146
|
+
gate: GateVerdictValue;
|
|
147
|
+
/** the eval score this turn (higher = better). */
|
|
148
|
+
score: number;
|
|
149
|
+
/** delta vs the previous kept score. */
|
|
150
|
+
delta: number;
|
|
151
|
+
/** whether this turn counted as an above-noise, variance-safe improvement. */
|
|
152
|
+
improved: boolean;
|
|
153
|
+
/**
|
|
154
|
+
* CODE-TARGET (Wave-2 W2I1) — the hard deterministic test-suite gate result for
|
|
155
|
+
* a `code` subject this turn: `true` iff the subject's OWN test suite ran green.
|
|
156
|
+
* OPTIONAL + additive: undefined for non-code targets (they omit it, unchanged).
|
|
157
|
+
* The `code-quality` goal requires this to be strictly `true` (an undefined /
|
|
158
|
+
* absent signal never converges — safe default). It is a SEPARATE axis from
|
|
159
|
+
* `gate` (the quality verdict) so both signals stay first-class + auditable in
|
|
160
|
+
* loop-state.json; `goalMet` does the AND.
|
|
161
|
+
*/
|
|
162
|
+
testsGreen?: boolean;
|
|
163
|
+
}
|
|
164
|
+
|
|
165
|
+
/** The loop-state.json cursor. */
|
|
166
|
+
export interface LoopState {
|
|
167
|
+
iteration: number;
|
|
168
|
+
phase: LoopPhaseValue;
|
|
169
|
+
/** the score of the last KEPT (improving) iteration — the convergence anchor. */
|
|
170
|
+
bestScore: number;
|
|
171
|
+
/** running count of consecutive non-improving turns. */
|
|
172
|
+
noImprovementCount: number;
|
|
173
|
+
/** cumulative wallclock spent (ms), injected by the driver. */
|
|
174
|
+
budgetSpentMs: number;
|
|
175
|
+
history: IterationRecord[];
|
|
176
|
+
terminator: TerminatorValue | null;
|
|
177
|
+
}
|
|
178
|
+
|
|
179
|
+
/** A loop is illegal (refuses to start) unless it declares real bounds. */
|
|
180
|
+
export function assertGoalLegal(config: LoopConfig): void {
|
|
181
|
+
const problems: string[] = [];
|
|
182
|
+
if (!config.goal) problems.push("no observable goal declared");
|
|
183
|
+
if (!(config.maxIters > 0)) problems.push("maxIters must be > 0");
|
|
184
|
+
if (!(config.budgetMs > 0)) problems.push("budgetMs must be > 0");
|
|
185
|
+
if (!(config.noImprovementStreak > 0)) problems.push("noImprovementStreak must be > 0");
|
|
186
|
+
if (config.noiseFloor < 0) problems.push("noiseFloor must be >= 0");
|
|
187
|
+
// NATURAL-LANGUAGE goal (Wave-2 W2I10): an UNRESOLVED NL goal is NOT an observable,
|
|
188
|
+
// bounded yardstick — it is a free-text intent, not yet a measurable criterion. It is
|
|
189
|
+
// illegal exactly like an unbounded config, so the loop (interactive OR headless)
|
|
190
|
+
// refuses to run it. A headless run has no human to confirm-freeze, so this gate is
|
|
191
|
+
// precisely what makes it fall back / refuse rather than spin on an un-yardsticked goal.
|
|
192
|
+
if (
|
|
193
|
+
config.goal &&
|
|
194
|
+
config.goal.kind === "nl" &&
|
|
195
|
+
(config.goal.resolved === undefined || config.goal.resolved.criterionIds.length === 0)
|
|
196
|
+
) {
|
|
197
|
+
problems.push(
|
|
198
|
+
'natural-language goal is unresolved — freeze it to a measurable criterion set first ' +
|
|
199
|
+
'(interpret → confirm-once → freeze; SKILL §2·5). A free-text goal is not a runnable yardstick.',
|
|
200
|
+
);
|
|
201
|
+
}
|
|
202
|
+
if (problems.length > 0) {
|
|
203
|
+
throw new Error(`optimize loop is not goal-legal — refusing to run: ${problems.join("; ")}`);
|
|
204
|
+
}
|
|
205
|
+
// SUBJECT-KIND gate (Wave-2 follow-up): when the config names the subject kind, enforce
|
|
206
|
+
// that the goal is applicable to it (a `code-quality` goal is illegal on a non-code
|
|
207
|
+
// subject). Folding this into assertGoalLegal — the ONE gate both the interactive CLI and
|
|
208
|
+
// the headless `runImproveLoop` already call — is what lets a HEADLESS run enforce the
|
|
209
|
+
// guard (no `--subject-kind` flag to pass). Throws with assertGoalAllowedForSubject's own
|
|
210
|
+
// clear message. Absent subjectKind ⇒ skipped (unchanged).
|
|
211
|
+
if (config.subjectKind !== undefined) {
|
|
212
|
+
assertGoalAllowedForSubject(config.goal, config.subjectKind);
|
|
213
|
+
}
|
|
214
|
+
}
|
|
215
|
+
|
|
216
|
+
/**
|
|
217
|
+
* SUBJECT-KIND → GOAL legality (Wave-2 W2I10). The `code-quality` BOTH-gate is only
|
|
218
|
+
* meaningful for a `code` subject (it ANDs a real test suite with a quality verdict);
|
|
219
|
+
* a non-code subject (agent · skill · platform) has no code to test, so `code-quality`
|
|
220
|
+
* NEVER fires there — it must use an eval-based goal (`eval-pass` · `criterion` · `delta`
|
|
221
|
+
* · a frozen NL criterion). All eval-based goals are valid for ANY subject-kind (code
|
|
222
|
+
* included), so this gate constrains ONLY `code-quality`. Throws with a clear message.
|
|
223
|
+
*/
|
|
224
|
+
export function assertGoalAllowedForSubject(goal: Goal, subjectKind: SubjectKind): void {
|
|
225
|
+
if (goal.kind === "code-quality" && subjectKind !== "code") {
|
|
226
|
+
throw new Error(
|
|
227
|
+
`goal 'code-quality' is only valid for a 'code' subject — a '${subjectKind}' subject has ` +
|
|
228
|
+
'no test suite / code to gate; use an eval-based goal (eval-pass · criterion · delta · a frozen NL criterion)',
|
|
229
|
+
);
|
|
230
|
+
}
|
|
231
|
+
}
|
|
232
|
+
|
|
233
|
+
/**
|
|
234
|
+
* Parse the `--goal` argument (Wave-2 W2I10) into a `Goal`. Recognizes the four
|
|
235
|
+
* STRUCTURED shapes and treats anything else as a NATURAL-LANGUAGE goal (free text) —
|
|
236
|
+
* the catch-all is deliberate: an NL goal is illegal until frozen (see assertGoalLegal),
|
|
237
|
+
* so a typo'd structured goal cannot silently run; it surfaces at the confirm-freeze step.
|
|
238
|
+
* eval-pass | code-quality | criterion:<id> | delta:<n> | "<any free text>"
|
|
239
|
+
* PURE: no IO, no clock. Throws (fail-loud) on a malformed structured shape or empty input.
|
|
240
|
+
*/
|
|
241
|
+
export function parseGoal(raw: string): Goal {
|
|
242
|
+
const s = raw.trim();
|
|
243
|
+
if (!s) throw new Error("goal is empty — pass eval-pass · criterion:<id> · delta:<n> · code-quality · a free-text goal");
|
|
244
|
+
if (s === "eval-pass") return { kind: "eval-pass" };
|
|
245
|
+
if (s === "code-quality") return { kind: "code-quality" };
|
|
246
|
+
if (s.startsWith("criterion:")) {
|
|
247
|
+
const id = s.slice("criterion:".length).trim();
|
|
248
|
+
if (!id) throw new Error('criterion goal needs an id — "criterion:<id>"');
|
|
249
|
+
return { kind: "criterion", id };
|
|
250
|
+
}
|
|
251
|
+
if (s.startsWith("delta:")) {
|
|
252
|
+
// Guard the EMPTY payload FIRST: `Number("") === 0` (and `Number.isFinite(0) === true`),
|
|
253
|
+
// so without this an empty `delta:` would silently parse to `{kind:"delta",min:0}` — a
|
|
254
|
+
// min of 0 converges on ANY above-noise gain, which is not what an operator who typed a
|
|
255
|
+
// bare `delta:` meant. Fail loud instead (a missing threshold is a malformed goal).
|
|
256
|
+
const payload = s.slice("delta:".length).trim();
|
|
257
|
+
if (!payload) throw new Error('delta goal needs a number — "delta:<n>" (got an empty payload)');
|
|
258
|
+
const min = Number(payload);
|
|
259
|
+
if (!Number.isFinite(min)) throw new Error('delta goal needs a finite number — "delta:<n>"');
|
|
260
|
+
return { kind: "delta", min };
|
|
261
|
+
}
|
|
262
|
+
// Everything else is a free-text (natural-language) goal — UNRESOLVED until frozen.
|
|
263
|
+
return { kind: "nl", text: s };
|
|
264
|
+
}
|
|
265
|
+
|
|
266
|
+
/**
|
|
267
|
+
* FREEZE a natural-language goal (Wave-2 W2I10; OT-4 widened to a criterion SET) to a
|
|
268
|
+
* concrete, measurable criterion COMPOSITION — the step that makes it goal-legal. The
|
|
269
|
+
* session INTERPRETS the free text into one or more binary / measurable-metric criteria
|
|
270
|
+
* (agent reasoning at entry) and CONFIRMS once with the operator; this pure transform
|
|
271
|
+
* records that frozen decision on the goal (keeping the original `text` for audit). After
|
|
272
|
+
* freezing, `goalMet` runs it as an AND of `criterion` goals — met only when ALL listed
|
|
273
|
+
* ids pass. Accepts either a single id (a 1-element set — back-compat) or a list, and
|
|
274
|
+
* splits comma-separated ids so the CLI can pass `--criterion a,b`. Throws if the goal is
|
|
275
|
+
* not an NL goal, or if no criterion id is supplied.
|
|
276
|
+
*/
|
|
277
|
+
export function freezeNlGoal(goal: Goal, criterionIds: string | string[]): Goal {
|
|
278
|
+
if (goal.kind !== "nl") {
|
|
279
|
+
throw new Error(`freezeNlGoal: goal is '${goal.kind}', not a natural-language goal — nothing to freeze`);
|
|
280
|
+
}
|
|
281
|
+
const ids = (Array.isArray(criterionIds) ? criterionIds : [criterionIds])
|
|
282
|
+
.flatMap((c) => c.split(","))
|
|
283
|
+
.map((s) => s.trim())
|
|
284
|
+
.filter(Boolean);
|
|
285
|
+
if (ids.length === 0) throw new Error("freezeNlGoal: a criterion id is required to freeze the NL goal");
|
|
286
|
+
return { kind: "nl", text: goal.text, resolved: { criterionIds: ids } };
|
|
287
|
+
}
|
|
288
|
+
|
|
289
|
+
/** The variance-gate: an above-noise gain that does NOT regress variance. */
|
|
290
|
+
export function isImprovement(delta: number, noiseFloor: number, varianceRegressed: boolean): boolean {
|
|
291
|
+
return delta > noiseFloor && !varianceRegressed;
|
|
292
|
+
}
|
|
293
|
+
|
|
294
|
+
/** Whether the observable goal is met this turn. */
|
|
295
|
+
export function goalMet(goal: Goal, rec: IterationRecord, criteriaPassed: Set<string>): boolean {
|
|
296
|
+
switch (goal.kind) {
|
|
297
|
+
case "eval-pass":
|
|
298
|
+
return rec.gate === GateVerdict.Pass;
|
|
299
|
+
case "criterion":
|
|
300
|
+
return criteriaPassed.has(goal.id);
|
|
301
|
+
case "delta":
|
|
302
|
+
return rec.improved && rec.delta >= goal.min;
|
|
303
|
+
case "code-quality":
|
|
304
|
+
// BOTH-gate (Wave-2 W2I1): test-green AND quality-verdict PASS. `=== true`
|
|
305
|
+
// makes an undefined/absent test signal fail-safe (never a false converge).
|
|
306
|
+
return rec.testsGreen === true && rec.gate === GateVerdict.Pass;
|
|
307
|
+
case "nl":
|
|
308
|
+
// NATURAL-LANGUAGE goal (Wave-2 W2I10; OT-4 widened to a criterion SET). Only a
|
|
309
|
+
// FROZEN NL goal is ever met — an unresolved / empty-set one can't reach here
|
|
310
|
+
// (assertGoalLegal refuses to run it), and if it somehow does, it is never met
|
|
311
|
+
// (fail-safe: the empty-set guard stops `[].every` from vacuously converging).
|
|
312
|
+
// Frozen ⇒ an AND of `criterion` goals: met only when EVERY resolved id is in the
|
|
313
|
+
// passed set (each id is a strict binary check / measurable-metric criterion).
|
|
314
|
+
return (
|
|
315
|
+
goal.resolved !== undefined &&
|
|
316
|
+
goal.resolved.criterionIds.length > 0 &&
|
|
317
|
+
goal.resolved.criterionIds.every((id) => criteriaPassed.has(id))
|
|
318
|
+
);
|
|
319
|
+
default: {
|
|
320
|
+
const _never: never = goal;
|
|
321
|
+
throw new Error(`unknown goal kind ${JSON.stringify(_never)}`);
|
|
322
|
+
}
|
|
323
|
+
}
|
|
324
|
+
}
|
|
325
|
+
|
|
326
|
+
/**
|
|
327
|
+
* Compute the terminator (or null to continue) AFTER an iteration is recorded.
|
|
328
|
+
* Ordering is deliberate: an ABORT halts immediately; then goal (success) wins
|
|
329
|
+
* over the bounded stops so a converging final turn is reported as Converged.
|
|
330
|
+
*/
|
|
331
|
+
export function checkTerminators(
|
|
332
|
+
state: LoopState,
|
|
333
|
+
config: LoopConfig,
|
|
334
|
+
lastVerify: ArchitectVerdictValue,
|
|
335
|
+
criteriaPassed: Set<string>,
|
|
336
|
+
): TerminatorValue | null {
|
|
337
|
+
const last = state.history[state.history.length - 1];
|
|
338
|
+
if (lastVerify === ArchitectVerdict.Abort) return Terminator.Aborted;
|
|
339
|
+
if (last && goalMet(config.goal, last, criteriaPassed)) return Terminator.Converged;
|
|
340
|
+
if (state.noImprovementCount >= config.noImprovementStreak) return Terminator.NoImprovement;
|
|
341
|
+
if (state.iteration >= config.maxIters) return Terminator.MaxIters;
|
|
342
|
+
if (state.budgetSpentMs >= config.budgetMs) return Terminator.Budget;
|
|
343
|
+
return null;
|
|
344
|
+
}
|
|
345
|
+
|
|
346
|
+
/** The next phase from the current one given the two in-loop gate results. */
|
|
347
|
+
export function nextPhase(
|
|
348
|
+
phase: LoopPhaseValue,
|
|
349
|
+
verify?: ArchitectVerdictValue,
|
|
350
|
+
gate?: GateVerdictValue,
|
|
351
|
+
): LoopPhaseValue {
|
|
352
|
+
switch (phase) {
|
|
353
|
+
case LoopPhase.Init:
|
|
354
|
+
return LoopPhase.Build;
|
|
355
|
+
case LoopPhase.Build:
|
|
356
|
+
return LoopPhase.Verify;
|
|
357
|
+
case LoopPhase.Verify:
|
|
358
|
+
// ABORT ends the turn; STEER re-builds; PROCEED advances to eval.
|
|
359
|
+
if (verify === ArchitectVerdict.Abort) return LoopPhase.Terminal;
|
|
360
|
+
if (verify === ArchitectVerdict.Steer) return LoopPhase.Build;
|
|
361
|
+
return LoopPhase.Eval;
|
|
362
|
+
case LoopPhase.Eval:
|
|
363
|
+
return LoopPhase.Gate;
|
|
364
|
+
case LoopPhase.Gate:
|
|
365
|
+
// PASS → the turn may converge (driver checks terminators); FAIL → diagnose.
|
|
366
|
+
return gate === GateVerdict.Pass ? LoopPhase.Terminal : LoopPhase.Diagnose;
|
|
367
|
+
case LoopPhase.Diagnose:
|
|
368
|
+
return LoopPhase.Build; // RCA remedy → next Build turn
|
|
369
|
+
case LoopPhase.Terminal:
|
|
370
|
+
return LoopPhase.Terminal;
|
|
371
|
+
default: {
|
|
372
|
+
const _never: never = phase;
|
|
373
|
+
throw new Error(`unknown phase ${JSON.stringify(_never)}`);
|
|
374
|
+
}
|
|
375
|
+
}
|
|
376
|
+
}
|
|
377
|
+
|
|
378
|
+
/** A fresh loop-state cursor. */
|
|
379
|
+
export function initLoopState(): LoopState {
|
|
380
|
+
return {
|
|
381
|
+
iteration: 0,
|
|
382
|
+
phase: LoopPhase.Init,
|
|
383
|
+
bestScore: Number.NEGATIVE_INFINITY,
|
|
384
|
+
noImprovementCount: 0,
|
|
385
|
+
budgetSpentMs: 0,
|
|
386
|
+
history: [],
|
|
387
|
+
terminator: null,
|
|
388
|
+
};
|
|
389
|
+
}
|
|
390
|
+
|
|
391
|
+
/**
|
|
392
|
+
* Record one completed iteration into the cursor (pure — returns a new state).
|
|
393
|
+
* Applies the variance-gate to decide improvement + advance/reset the streak.
|
|
394
|
+
*/
|
|
395
|
+
export function recordIteration(
|
|
396
|
+
state: LoopState,
|
|
397
|
+
config: LoopConfig,
|
|
398
|
+
input: {
|
|
399
|
+
verify: ArchitectVerdictValue;
|
|
400
|
+
gate: GateVerdictValue;
|
|
401
|
+
score: number;
|
|
402
|
+
varianceRegressed: boolean;
|
|
403
|
+
budgetSpentMs: number;
|
|
404
|
+
/**
|
|
405
|
+
* CODE-TARGET (Wave-2 W2I1) — the deterministic test-suite gate for a `code`
|
|
406
|
+
* subject this turn. OPTIONAL: omit for non-code targets (recorded as
|
|
407
|
+
* undefined, no behavior change). Threaded verbatim into the record so the
|
|
408
|
+
* `code-quality` goal's BOTH-gate can AND it with the quality verdict.
|
|
409
|
+
*/
|
|
410
|
+
testsGreen?: boolean;
|
|
411
|
+
},
|
|
412
|
+
): LoopState {
|
|
413
|
+
const iteration = state.iteration + 1;
|
|
414
|
+
const prevBest = state.bestScore;
|
|
415
|
+
const delta = prevBest === Number.NEGATIVE_INFINITY ? input.score : input.score - prevBest;
|
|
416
|
+
const improved =
|
|
417
|
+
prevBest === Number.NEGATIVE_INFINITY
|
|
418
|
+
? input.score > config.noiseFloor // first turn: any real signal
|
|
419
|
+
: isImprovement(delta, config.noiseFloor, input.varianceRegressed);
|
|
420
|
+
const rec: IterationRecord = {
|
|
421
|
+
iteration,
|
|
422
|
+
verify: input.verify,
|
|
423
|
+
gate: input.gate,
|
|
424
|
+
score: input.score,
|
|
425
|
+
delta,
|
|
426
|
+
improved,
|
|
427
|
+
// Only stamp the field when the session supplied it — a non-code turn keeps the
|
|
428
|
+
// record shape identical to pre-W2I1 (no `testsGreen` key ⇒ byte-identical).
|
|
429
|
+
...(input.testsGreen === undefined ? {} : { testsGreen: input.testsGreen }),
|
|
430
|
+
};
|
|
431
|
+
return {
|
|
432
|
+
iteration,
|
|
433
|
+
phase: LoopPhase.Gate,
|
|
434
|
+
bestScore: improved ? input.score : prevBest,
|
|
435
|
+
noImprovementCount: improved ? 0 : state.noImprovementCount + 1,
|
|
436
|
+
budgetSpentMs: input.budgetSpentMs,
|
|
437
|
+
history: [...state.history, rec],
|
|
438
|
+
terminator: null,
|
|
439
|
+
};
|
|
440
|
+
}
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# mutagent-optimize/ — skill navigation map
|
|
2
|
+
|
|
3
|
+
> Lean map of THIS skill's internals. Repo-wide conventions live in `../CLAUDE.md`.
|
|
4
|
+
> **DESIGN / scaffolded** — the loop-state FSM + the shared apply transport landed; the
|
|
5
|
+
> improve-loop driver + crew wiring are the remaining build. Not yet tagged `optimize-v*`.
|
|
6
|
+
|
|
7
|
+
**What it is:** the ADL ⑤ IMPROVE stage — the closed, bounded eval-driven optimize loop.
|
|
8
|
+
A sub-orchestration skill: the parent owns the loop FSM + gates and dispatches a borrowed
|
|
9
|
+
crew (ai-engineer · ai-architect · evaluator · diagnostics-analyzer). Writes nothing itself.
|
|
10
|
+
|
|
11
|
+
## Layout
|
|
12
|
+
|
|
13
|
+
```
|
|
14
|
+
.claude/skills/mutagent-optimize/
|
|
15
|
+
SKILL.md the stage contract — ONE command *optimize; §2·5 = the runtime conduct procedure
|
|
16
|
+
.meta/
|
|
17
|
+
design-principles.md OP-PR-001..004 (the constitution)
|
|
18
|
+
prd.yaml goals · key_problems · design_choices · verification · REQ registry
|
|
19
|
+
scripts/
|
|
20
|
+
loop-state.ts the PURE FSM decision model (bounded · goal-legal · variance-aware)
|
|
21
|
+
loop-state-cli.ts one-shot CLI over loop-state.ts — the conducting SESSION shells this between Task dispatches
|
|
22
|
+
improve-loop-run.ts the unit-test harness + future HEADLESS verification driver (NOT the interactive runtime)
|
|
23
|
+
tests/
|
|
24
|
+
loop-state.test.ts the FSM unit suite
|
|
25
|
+
loop-state-cli.test.ts the decision-CLI suite
|
|
26
|
+
improve-loop-run.test.ts the harness suite
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
## Two runtimes — do NOT conflate (the load-bearing distinction)
|
|
30
|
+
|
|
31
|
+
The same FSM (`loop-state.ts`) is driven two ways:
|
|
32
|
+
|
|
33
|
+
- **Model B · INTERACTIVE `*optimize` (the operator runtime).** The TOP-LEVEL SESSION conducts
|
|
34
|
+
the loop (only the session can Task-dispatch sub-agents — a sub-agent can't; a TS `while`
|
|
35
|
+
awaiting dispatches is the wrong shape). The session follows **SKILL.md §2·5**: confirm once →
|
|
36
|
+
per round Task-dispatch ai-engineer → ai-architect → evaluator, then shell `loop-state-cli.ts`
|
|
37
|
+
to record + decide (terminator / next-phase), and on a terminator raise the single apply-gate.
|
|
38
|
+
There is deliberately **no** production `CrewBindings` object — the "crew binding" IS the §2·5
|
|
39
|
+
procedure. This mirrors diagnostics running its `orchestrator-protocol` inline in the parent.
|
|
40
|
+
- **Model A · HEADLESS verification (`improve-loop-run.ts`).** A deterministic child-spawn driver
|
|
41
|
+
for unit tests + future SimuLatte/CI regression — NOT a live operator loop. The injected
|
|
42
|
+
`CrewBindings` seam is a TEST/verification abstraction only.
|
|
43
|
+
|
|
44
|
+
## Boundaries
|
|
45
|
+
|
|
46
|
+
- **Apply is a call OUT** to the shared `mutagent-cli apply` (`@mutagent/tools`) — NOT a
|
|
47
|
+
subcommand. optimize gates the human decision at the skill layer, then shells the transport.
|
|
48
|
+
- **Delegation** — WRITE → ② builder (ai-engineer) · JUDGE → ③ evaluator · RCA → ④ diagnostics.
|
|
49
|
+
- **Sealed-sibling** — the borrowed crew arrives via a Helix HandoverBundle; no source imports.
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
# @mutagent/optimize
|
|
2
|
+
|
|
3
|
+
The **ADL ⑤ IMPROVE** stage — the closed, bounded, eval-driven optimize loop
|
|
4
|
+
(`Build → Eval → Diagnose → Improve ↻`).
|
|
5
|
+
|
|
6
|
+
One command, `*optimize`: confirm once at entry → run the bounded auto-loop →
|
|
7
|
+
one human apply-gate at convergence. It is a sub-orchestration skill — the parent
|
|
8
|
+
session owns the loop FSM + gates and dispatches a **borrowed** crew (ai-engineer
|
|
9
|
+
WRITE · ai-architect VERIFY · evaluator JUDGE · diagnostics-analyzer RCA). It writes
|
|
10
|
+
nothing itself: the WRITE delegates to ② builder, the JUDGE to ③ evaluator, the RCA
|
|
11
|
+
to ④ diagnostics, and the TRANSPORT to the shared `mutagent-cli apply` service.
|
|
12
|
+
|
|
13
|
+
```
|
|
14
|
+
*optimize <subject> [--goal eval-pass|criterion:<id>|delta:<n>] [--max-iters N] [--budget 30m] [--dry-run] [--html]
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
- **Bounded + goal-legal** — the loop refuses to run without an observable goal + hard
|
|
18
|
+
gates (max-iters · budget · no-improvement). Variance-gate: only an above-noise,
|
|
19
|
+
variance-safe gain counts as improvement.
|
|
20
|
+
- **Apply is a call OUT** — at the promotion gate optimize collects the human approval,
|
|
21
|
+
then shells `mutagent-cli apply --dry-run` → `--commit` (the CLI holds no approval
|
|
22
|
+
logic; every commit emits an apply-audit). Non-destructive + revertible.
|
|
23
|
+
- **One owner** — `*improve` resolves here; it replaces the prior three surfaces
|
|
24
|
+
(builder route · diagnostics apply-worker · evaluator EDD mode).
|
|
25
|
+
|
|
26
|
+
See `.claude/skills/mutagent-optimize/SKILL.md` for the full contract.
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
# mutagent-orchestrator/
|
|
2
|
+
|
|
3
|
+
> **Status: IN CONSTRUCTION** (`feat/adl-system-v1`). This directory frames the
|
|
4
|
+
> orchestrator's role and contract, and now carries its executable logic: the
|
|
5
|
+
> build package + the `*sync` topology indexer + the HandoverBundle contract +
|
|
6
|
+
> the config schema/onboarding-check + the **dispatch + execution-gate engine**.
|
|
7
|
+
> The only remaining declare-only piece is the live cross-skill RUNTIME invocation
|
|
8
|
+
> (step 3 of `execution-flow` in orchestrator.md).
|
|
9
|
+
|
|
10
|
+
## What this is
|
|
11
|
+
|
|
12
|
+
`mutagent-orchestrator` is the **composition layer** for the three standalone skills in
|
|
13
|
+
`mutagent-system/`. It drives a skill (or agent) through the **Agentic Development
|
|
14
|
+
Lifecycle (ADL)** — taking it from *idea → built → evaluated → diagnosed → improved* —
|
|
15
|
+
by composing the existing skills as ADL stages. It owns no domain logic of its own; each
|
|
16
|
+
stage is delegated to the skill that already does that job.
|
|
17
|
+
|
|
18
|
+
## The ADL stages (skill composition)
|
|
19
|
+
|
|
20
|
+
| ADL stage | Skill | Role |
|
|
21
|
+
|---|---|---|
|
|
22
|
+
| **SPEC** | `mutagent-agentspec` | Author + validate `agentspec.yaml`; `*sync-spec` reconciles the spec against a target (delegating the code-read to builder's `ai-architect #sync-spec`). |
|
|
23
|
+
| **BUILD** | `mutagent-builder` | Implement a validated `agentspec.yaml` into the target; `ai-architect` runs `#sync-spec` build-internally when code drifts ahead mid-build. |
|
|
24
|
+
| **AUDIT / verify** | `mutagent-evaluator` | Evaluate the built skill against declared success criteria; emit a verdict. |
|
|
25
|
+
| **DIAGNOSE** | `mutagent-diagnostics` | Pull evidence from traces, root-cause failures, surface ranked remedies. |
|
|
26
|
+
| **IMPROVE** | (loop back to BUILD) | Feed approved agentspec-backed remedies back into `mutagent-builder` for the next iteration. |
|
|
27
|
+
|
|
28
|
+
```
|
|
29
|
+
idea
|
|
30
|
+
│
|
|
31
|
+
BUILD ───────────────┐ (mutagent-builder)
|
|
32
|
+
│ │
|
|
33
|
+
▼ │
|
|
34
|
+
AUDIT/verify │ (mutagent-evaluator)
|
|
35
|
+
│ │
|
|
36
|
+
▼ │
|
|
37
|
+
DIAGNOSE │ (mutagent-diagnostics)
|
|
38
|
+
│ │
|
|
39
|
+
▼ │
|
|
40
|
+
IMPROVE ─────────────┘ (loop back to BUILD until goal met)
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
## Design stance (skeleton)
|
|
44
|
+
|
|
45
|
+
- **Composes, never re-implements.** Each stage is a thin dispatch into an already-shipping
|
|
46
|
+
skill. The orchestrator owns sequencing, gate adjudication, and state hand-off between
|
|
47
|
+
stages — not the work inside any stage.
|
|
48
|
+
- **Standalone + symbiosis preserved.** Skills do not reference each other in source
|
|
49
|
+
(see `../CLAUDE.md` → Standalone + Symbiosis principle). The orchestrator is the *only*
|
|
50
|
+
place that knows about all three; it talks to each via its public skill contract.
|
|
51
|
+
- **Goal-driven loop with a termination contract.** Like the skills it composes, the ADL
|
|
52
|
+
loop must declare an observable terminator (goal met / verdict PASS) — never open-ended.
|
|
53
|
+
|
|
54
|
+
## Present now
|
|
55
|
+
|
|
56
|
+
- `package.json` + `tsconfig*.json` + `bunfig.toml` + `eslint.config.js` — the build package.
|
|
57
|
+
- `scripts/sync-index.ts` — the `*sync` deterministic topology indexer (+ `tests/`). Also scans
|
|
58
|
+
`assets/agents/*.md` and indexes each as a `system_agent` subject (kind `skill | agent |
|
|
59
|
+
system_agent`; `system_agent` folds to `SubjectKind.Agent` on the frozen bundle).
|
|
60
|
+
- `assets/agents/` — the orchestrator's first `assets/`: cross-stage **system-agent** defs (minimal
|
|
61
|
+
frontmatter + a body ```yaml``` operating block, per the agent-def conformance check). Today:
|
|
62
|
+
`discovery.md` (`class: system_agent`) — the trace-Discovery cell dispatched by `*discover`
|
|
63
|
+
(route_target "discovery"; `*collect-traces` alias), INTENT-BRANCHED on `TraceQuery.operationIntent`
|
|
64
|
+
(`fetch` generic collect/list/monitor · `evals`/`dataset` purposed fetch→select→classify curate).
|
|
65
|
+
- `scripts/handover-contract.ts` — the TypeBox HandoverBundle contract blueprint
|
|
66
|
+
(schema + `validateHandoverBundle` + `makeHandoverBundle`, frozen v0.1.0) (+ `tests/`).
|
|
67
|
+
- `scripts/config-schema.ts` — the TypeBox `MutagentConfigSchema` for `.mutagent/config.yaml`
|
|
68
|
+
(closed object, frozen `config_version` **0.2.0** — `global` (providers·workspace·models{default,`judge_model`}·brand·`context[]`·**`sources[]`**·**`targets[]`**, each source/target carries an optional `default` selector + `credential_ref: string|{env,path}`) + `lifecycle.{agentspec,builder,evaluator,diagnostics}` (per-skill, opaque-passthrough) + per-stage `triggers` (ship DISABLED) · `validateConfig` · `loadConfig` over an injected YAML path with legacy (pre-0.2.0) detection → `migration-required`) (+ `tests/` + `tests/fixtures/config/`).
|
|
69
|
+
- `scripts/onboarding-check.ts` — the pure stage-aware `checkOnboardingComplete(config, activeStages, runCtx)`
|
|
70
|
+
completion-check: a scoped **global base floor** + **source/target/provider** floors only for the invoked stage, with source/target bound **by role** from the `global` catalog — `resolveSourceByRole`/`resolveTargetByRole`: one ⇒ resolved · one `default:true` among many ⇒ resolved · many + no default ⇒ `needs-selection` (non-blocking; operator picks at run time) · `>1 default` ⇒ `multiple-defaults` (config-error) (+ `tests/`).
|
|
71
|
+
- `scripts/dispatch.ts` — the O6 dispatch engine: pure `resolveDispatch(command, ctx)` →
|
|
72
|
+
a DispatchDescriptor (dispatch | local | unknown). Resolves the target subject from the
|
|
73
|
+
injected `*sync` topology, maps the *command → a ROUTING AdlStage (the two-enum split),
|
|
74
|
+
and BUILDS the HandoverBundle via `makeHandoverBundle`. `lookupRoute` exports the shared
|
|
75
|
+
route table (+ `tests/`).
|
|
76
|
+
- `scripts/gate.ts` — the O8 execution gate: pure `gateExecution(command, config, ctx)` →
|
|
77
|
+
`{ allowed, blockers }`. Two floors — onboarding (calls `checkOnboardingComplete` for
|
|
78
|
+
source stages; never tightens the schema) + approval (gated commands need
|
|
79
|
+
`approval_granted`) (+ `tests/`).
|
|
80
|
+
- `scripts/eval-routing.ts` — the reusable LLM-DRIVEN routing eval (`*eval-routing`):
|
|
81
|
+
the DI harness `runRoutingEval(dataset, routerFn, ctx)` (pure, stub-router in the gate;
|
|
82
|
+
categorical exact-match grader + per-command + in-dist + held-out cohorts) + the pure
|
|
83
|
+
`buildRoutingPrompt`/`parseRoutedCommand`/`loadDataset`/`loadRoutingContext` + the REAL
|
|
84
|
+
on-demand `routeViaLLM` (mirrors the repo's `@langchain/google-genai` call, temp 0;
|
|
85
|
+
model intent sacred — no silent swap). Closes pred3 the operator's way. (+ `tests/` +
|
|
86
|
+
`tests/fixtures/routing-eval/dataset.yaml` — labeled in-dist + held-out + out-of-domain).
|
|
87
|
+
- `orchestrator.md` + `routing.yaml` — the agent definition (incl. the `execution-flow`
|
|
88
|
+
resolve→gate→emit contract, step 3 declare-only · the `routing-eval` `*eval-routing`
|
|
89
|
+
capability) + NL-intent routing map.
|
|
90
|
+
|
|
91
|
+
## Not yet present (follow-up)
|
|
92
|
+
|
|
93
|
+
- `SKILL.md` — this orchestrator FOLDER has none by design; it is the dev-internal composition
|
|
94
|
+
layer. The orchestrator's PUBLISHED surface is **`@mutagent/helix`** (the ADL bundle, rooted at
|
|
95
|
+
`mutagent-system/`): it wraps `orchestrator.md` + `routing.yaml` as the conductor skill at
|
|
96
|
+
`mutagent-system/.claude/skills/mutagent-helix/SKILL.md` and ships the lifecycle skills alongside.
|
|
97
|
+
See `mutagent-system/RELEASING.md`.
|
|
98
|
+
- The live cross-skill RUNTIME invocation (step 3 of `execution-flow`) — handing the gated
|
|
99
|
+
HandoverBundle to a real sibling skill (Skill-tool / sub-agent spawn). The deterministic
|
|
100
|
+
engine (resolve → gate → emit-descriptor) has landed; the live sibling call is declare-only.
|
|
101
|
+
- The session state model + termination-gate machinery (the ADL loop's durable state + the
|
|
102
|
+
goal-met / verdict-PASS terminator) land in a follow-up.
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
# mutagent-orchestrator
|
|
2
|
+
|
|
3
|
+
> **Scaffold.** Construction lands in a follow-up.
|
|
4
|
+
|
|
5
|
+
`mutagent-orchestrator` composes the `mutagent-system` lifecycle skills into the **Agentic
|
|
6
|
+
Development Lifecycle (ADL)** — driving a skill or agent from idea to a built, evaluated,
|
|
7
|
+
diagnosed, and continuously-improved unit. It owns sequencing and stage hand-off only;
|
|
8
|
+
each stage's work is delegated to the skill that already does that job.
|
|
9
|
+
|
|
10
|
+
## ADL stage map
|
|
11
|
+
|
|
12
|
+
| Stage | Skill | Does |
|
|
13
|
+
|---|---|---|
|
|
14
|
+
| **SPEC** | `mutagent-agentspec` | Guided interview → portable, validated `agentspec.yaml` (the Definition). |
|
|
15
|
+
| **BUILD** | `mutagent-builder` | Implement a validated `agentspec.yaml` into the chosen target/harness. |
|
|
16
|
+
| **AUDIT / verify** | `mutagent-evaluator` | Evaluate against success criteria → verdict. |
|
|
17
|
+
| **DIAGNOSE** | `mutagent-diagnostics` | Trace-grounded root-cause + ranked remedies. |
|
|
18
|
+
| **IMPROVE** | `mutagent-optimize` | Session-conducted, bounded eval-driven optimize loop (Build → Eval → Diagnose → Improve ↻); ONE gated apply at convergence via the shared `mutagent-cli apply`. |
|
|
19
|
+
|
|
20
|
+
See `CLAUDE.md` in this directory for the full framing.
|