@proflandrigan/shards 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +475 -0
- package/package.json +37 -0
- package/src/agents/academic.md +276 -0
- package/src/agents/ai-engineer.md +377 -0
- package/src/agents/analytics-engineer.md +364 -0
- package/src/agents/applied-ml-scientist.md +410 -0
- package/src/agents/backend-engineer.md +255 -0
- package/src/agents/bi-engineer.md +333 -0
- package/src/agents/data-analyst.md +343 -0
- package/src/agents/data-engineer.md +260 -0
- package/src/agents/data-modeller.md +386 -0
- package/src/agents/data-scientist.md +366 -0
- package/src/agents/deep-learning-engineer.md +389 -0
- package/src/agents/ml-engineer.md +424 -0
- package/src/agents/mlops-engineer.md +339 -0
- package/src/agents/researcher.md +187 -0
- package/src/agents/specific_instructions/academic/critical_review.md +263 -0
- package/src/agents/specific_instructions/academic/report.md +113 -0
- package/src/agents/specific_instructions/ai_engineer/advise.md +162 -0
- package/src/agents/specific_instructions/ai_engineer/bi_engineer_handoff.md +86 -0
- package/src/agents/specific_instructions/ai_engineer/experiment.md +471 -0
- package/src/agents/specific_instructions/ai_engineer/experiment_ui_mode.md +44 -0
- package/src/agents/specific_instructions/ai_engineer/phases/index.md +45 -0
- package/src/agents/specific_instructions/ai_engineer/phases/phase-1.md +55 -0
- package/src/agents/specific_instructions/ai_engineer/phases/phase-2.md +86 -0
- package/src/agents/specific_instructions/ai_engineer/phases/phase-3.md +96 -0
- package/src/agents/specific_instructions/ai_engineer/phases/phase-4.md +138 -0
- package/src/agents/specific_instructions/ai_engineer/phases/phase-5.md +157 -0
- package/src/agents/specific_instructions/ai_engineer/phases/phase-6.md +196 -0
- package/src/agents/specific_instructions/ai_engineer/phases/phase-7.md +313 -0
- package/src/agents/specific_instructions/ai_engineer/phases.md +1011 -0
- package/src/agents/specific_instructions/ai_engineer/prompt_lab.md +161 -0
- package/src/agents/specific_instructions/ai_engineer/prompt_lab_ui_mode.md +28 -0
- package/src/agents/specific_instructions/ai_engineer/research.md +393 -0
- package/src/agents/specific_instructions/ai_engineer/research_ui_mode.md +66 -0
- package/src/agents/specific_instructions/ai_engineer/review.md +159 -0
- package/src/agents/specific_instructions/ai_engineer/validation_checklist.md +182 -0
- package/src/agents/specific_instructions/analytics_engineer/advise.md +155 -0
- package/src/agents/specific_instructions/analytics_engineer/bi_engineer_handoff.md +91 -0
- package/src/agents/specific_instructions/analytics_engineer/data_analyst_handoff.md +84 -0
- package/src/agents/specific_instructions/analytics_engineer/deep_phases.md +818 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/index.md +24 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-1.md +77 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-2.md +106 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-3.md +93 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-4.md +79 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-5.md +61 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-6.md +45 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-7.md +235 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-8.md +221 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_quick/index.md +19 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_quick/phase-1.md +47 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_quick/phase-2.md +78 -0
- package/src/agents/specific_instructions/analytics_engineer/quick_phases.md +112 -0
- package/src/agents/specific_instructions/analytics_engineer/review.md +167 -0
- package/src/agents/specific_instructions/analytics_engineer/service_mode.md +369 -0
- package/src/agents/specific_instructions/analytics_engineer/ui_mode.md +45 -0
- package/src/agents/specific_instructions/analytics_engineer/update.md +162 -0
- package/src/agents/specific_instructions/analytics_engineer/validation_checklist.md +121 -0
- package/src/agents/specific_instructions/applied_ml_scientist/advise.md +143 -0
- package/src/agents/specific_instructions/applied_ml_scientist/phases/index.md +21 -0
- package/src/agents/specific_instructions/applied_ml_scientist/phases/phase-1.md +51 -0
- package/src/agents/specific_instructions/applied_ml_scientist/phases/phase-2.md +66 -0
- package/src/agents/specific_instructions/applied_ml_scientist/phases/phase-3.md +113 -0
- package/src/agents/specific_instructions/applied_ml_scientist/phases/phase-4.md +104 -0
- package/src/agents/specific_instructions/applied_ml_scientist/phases/phase-5.md +156 -0
- package/src/agents/specific_instructions/applied_ml_scientist/phases.md +428 -0
- package/src/agents/specific_instructions/applied_ml_scientist/research.md +379 -0
- package/src/agents/specific_instructions/applied_ml_scientist/review.md +142 -0
- package/src/agents/specific_instructions/applied_ml_scientist/validation_checklist.md +136 -0
- package/src/agents/specific_instructions/backend_engineer/clean.md +149 -0
- package/src/agents/specific_instructions/backend_engineer/review.md +91 -0
- package/src/agents/specific_instructions/backend_engineer/review_checklist.md +54 -0
- package/src/agents/specific_instructions/backend_engineer/service_mode.md +67 -0
- package/src/agents/specific_instructions/bi_engineer/advise.md +137 -0
- package/src/agents/specific_instructions/bi_engineer/data_analyst_handoff.md +77 -0
- package/src/agents/specific_instructions/bi_engineer/incoming_handoff.md +45 -0
- package/src/agents/specific_instructions/bi_engineer/phases/index.md +20 -0
- package/src/agents/specific_instructions/bi_engineer/phases/phase-1.md +164 -0
- package/src/agents/specific_instructions/bi_engineer/phases/phase-2.md +92 -0
- package/src/agents/specific_instructions/bi_engineer/phases/phase-3.md +121 -0
- package/src/agents/specific_instructions/bi_engineer/phases/phase-4.md +106 -0
- package/src/agents/specific_instructions/bi_engineer/phases.md +451 -0
- package/src/agents/specific_instructions/bi_engineer/review.md +166 -0
- package/src/agents/specific_instructions/bi_engineer/update.md +147 -0
- package/src/agents/specific_instructions/bi_engineer/validation_checklist.md +124 -0
- package/src/agents/specific_instructions/data_analyst/advise.md +138 -0
- package/src/agents/specific_instructions/data_analyst/explain.md +221 -0
- package/src/agents/specific_instructions/data_analyst/incoming_handoff.md +40 -0
- package/src/agents/specific_instructions/data_analyst/phases/index.md +20 -0
- package/src/agents/specific_instructions/data_analyst/phases/phase-1.md +159 -0
- package/src/agents/specific_instructions/data_analyst/phases/phase-2.md +112 -0
- package/src/agents/specific_instructions/data_analyst/phases/phase-3.md +265 -0
- package/src/agents/specific_instructions/data_analyst/phases/phase-4.md +100 -0
- package/src/agents/specific_instructions/data_analyst/phases.md +501 -0
- package/src/agents/specific_instructions/data_analyst/review.md +138 -0
- package/src/agents/specific_instructions/data_analyst/ui_mode.md +26 -0
- package/src/agents/specific_instructions/data_analyst/update.md +144 -0
- package/src/agents/specific_instructions/data_analyst/validation_checklist.md +95 -0
- package/src/agents/specific_instructions/data_engineer/advise.md +137 -0
- package/src/agents/specific_instructions/data_engineer/phases.md +466 -0
- package/src/agents/specific_instructions/data_engineer/phases_deep/index.md +23 -0
- package/src/agents/specific_instructions/data_engineer/phases_deep/phase-1.md +49 -0
- package/src/agents/specific_instructions/data_engineer/phases_deep/phase-2.md +93 -0
- package/src/agents/specific_instructions/data_engineer/phases_deep/phase-3.md +55 -0
- package/src/agents/specific_instructions/data_engineer/phases_deep/phase-4.md +48 -0
- package/src/agents/specific_instructions/data_engineer/phases_deep/phase-5.md +40 -0
- package/src/agents/specific_instructions/data_engineer/phases_deep/phase-6.md +102 -0
- package/src/agents/specific_instructions/data_engineer/phases_deep/phase-7.md +87 -0
- package/src/agents/specific_instructions/data_engineer/phases_quick/index.md +19 -0
- package/src/agents/specific_instructions/data_engineer/phases_quick/phase-1.md +45 -0
- package/src/agents/specific_instructions/data_engineer/phases_quick/phase-2.md +54 -0
- package/src/agents/specific_instructions/data_engineer/review.md +135 -0
- package/src/agents/specific_instructions/data_engineer/validation_checklist.md +136 -0
- package/src/agents/specific_instructions/data_modeller/advise.md +137 -0
- package/src/agents/specific_instructions/data_modeller/phases.md +581 -0
- package/src/agents/specific_instructions/data_modeller/phases_deep/index.md +23 -0
- package/src/agents/specific_instructions/data_modeller/phases_deep/phase-1.md +52 -0
- package/src/agents/specific_instructions/data_modeller/phases_deep/phase-2.md +113 -0
- package/src/agents/specific_instructions/data_modeller/phases_deep/phase-3.md +47 -0
- package/src/agents/specific_instructions/data_modeller/phases_deep/phase-4.md +51 -0
- package/src/agents/specific_instructions/data_modeller/phases_deep/phase-5.md +45 -0
- package/src/agents/specific_instructions/data_modeller/phases_deep/phase-6.md +105 -0
- package/src/agents/specific_instructions/data_modeller/phases_deep/phase-7.md +136 -0
- package/src/agents/specific_instructions/data_modeller/phases_quick/index.md +19 -0
- package/src/agents/specific_instructions/data_modeller/phases_quick/phase-1.md +47 -0
- package/src/agents/specific_instructions/data_modeller/phases_quick/phase-2.md +65 -0
- package/src/agents/specific_instructions/data_modeller/review.md +141 -0
- package/src/agents/specific_instructions/data_modeller/service_mode.md +218 -0
- package/src/agents/specific_instructions/data_modeller/validation_checklist.md +125 -0
- package/src/agents/specific_instructions/data_scientist/advise.md +158 -0
- package/src/agents/specific_instructions/data_scientist/bi_engineer_handoff.md +63 -0
- package/src/agents/specific_instructions/data_scientist/experiment.md +482 -0
- package/src/agents/specific_instructions/data_scientist/experiment_ui_mode.md +44 -0
- package/src/agents/specific_instructions/data_scientist/explain.md +247 -0
- package/src/agents/specific_instructions/data_scientist/greenfield_data.md +35 -0
- package/src/agents/specific_instructions/data_scientist/ml_engineer_handoff.md +52 -0
- package/src/agents/specific_instructions/data_scientist/notebook_walkthrough.md +76 -0
- package/src/agents/specific_instructions/data_scientist/phases/index.md +24 -0
- package/src/agents/specific_instructions/data_scientist/phases/phase-1.md +45 -0
- package/src/agents/specific_instructions/data_scientist/phases/phase-2.md +67 -0
- package/src/agents/specific_instructions/data_scientist/phases/phase-3.md +89 -0
- package/src/agents/specific_instructions/data_scientist/phases/phase-4.md +143 -0
- package/src/agents/specific_instructions/data_scientist/phases/phase-5.md +71 -0
- package/src/agents/specific_instructions/data_scientist/phases/phase-6.md +239 -0
- package/src/agents/specific_instructions/data_scientist/phases/phase-7.md +207 -0
- package/src/agents/specific_instructions/data_scientist/phases.md +651 -0
- package/src/agents/specific_instructions/data_scientist/research.md +345 -0
- package/src/agents/specific_instructions/data_scientist/research_ui_mode.md +52 -0
- package/src/agents/specific_instructions/data_scientist/review.md +136 -0
- package/src/agents/specific_instructions/data_scientist/service_mode.md +247 -0
- package/src/agents/specific_instructions/data_scientist/validation_checklist.md +183 -0
- package/src/agents/specific_instructions/deep_learning_engineer/advise.md +145 -0
- package/src/agents/specific_instructions/deep_learning_engineer/phases/index.md +21 -0
- package/src/agents/specific_instructions/deep_learning_engineer/phases/phase-1.md +74 -0
- package/src/agents/specific_instructions/deep_learning_engineer/phases/phase-2.md +98 -0
- package/src/agents/specific_instructions/deep_learning_engineer/phases/phase-3.md +76 -0
- package/src/agents/specific_instructions/deep_learning_engineer/phases/phase-4.md +128 -0
- package/src/agents/specific_instructions/deep_learning_engineer/phases/phase-5.md +292 -0
- package/src/agents/specific_instructions/deep_learning_engineer/phases.md +567 -0
- package/src/agents/specific_instructions/deep_learning_engineer/research.md +389 -0
- package/src/agents/specific_instructions/deep_learning_engineer/review.md +155 -0
- package/src/agents/specific_instructions/deep_learning_engineer/validation_checklist.md +147 -0
- package/src/agents/specific_instructions/ml_engineer/advise.md +174 -0
- package/src/agents/specific_instructions/ml_engineer/bi_engineer_handoff.md +71 -0
- package/src/agents/specific_instructions/ml_engineer/experiment.md +474 -0
- package/src/agents/specific_instructions/ml_engineer/experiment_ui_mode.md +44 -0
- package/src/agents/specific_instructions/ml_engineer/notebook_walkthrough.md +75 -0
- package/src/agents/specific_instructions/ml_engineer/phases/index.md +25 -0
- package/src/agents/specific_instructions/ml_engineer/phases/phase-1.md +49 -0
- package/src/agents/specific_instructions/ml_engineer/phases/phase-2.md +75 -0
- package/src/agents/specific_instructions/ml_engineer/phases/phase-3.md +124 -0
- package/src/agents/specific_instructions/ml_engineer/phases/phase-4.md +279 -0
- package/src/agents/specific_instructions/ml_engineer/phases/phase-5.md +160 -0
- package/src/agents/specific_instructions/ml_engineer/phases/phase-6-5.md +170 -0
- package/src/agents/specific_instructions/ml_engineer/phases/phase-6.md +295 -0
- package/src/agents/specific_instructions/ml_engineer/phases/phase-7.md +337 -0
- package/src/agents/specific_instructions/ml_engineer/phases.md +1068 -0
- package/src/agents/specific_instructions/ml_engineer/research.md +437 -0
- package/src/agents/specific_instructions/ml_engineer/research_ui_mode.md +71 -0
- package/src/agents/specific_instructions/ml_engineer/review.md +187 -0
- package/src/agents/specific_instructions/ml_engineer/service_mode.md +273 -0
- package/src/agents/specific_instructions/ml_engineer/validation_checklist.md +185 -0
- package/src/agents/specific_instructions/mlops_engineer/advise.md +139 -0
- package/src/agents/specific_instructions/mlops_engineer/phases/index.md +23 -0
- package/src/agents/specific_instructions/mlops_engineer/phases/phase-1.md +52 -0
- package/src/agents/specific_instructions/mlops_engineer/phases/phase-2.md +86 -0
- package/src/agents/specific_instructions/mlops_engineer/phases/phase-3.md +105 -0
- package/src/agents/specific_instructions/mlops_engineer/phases/phase-4.md +128 -0
- package/src/agents/specific_instructions/mlops_engineer/phases/phase-5.md +106 -0
- package/src/agents/specific_instructions/mlops_engineer/phases/phase-6.md +128 -0
- package/src/agents/specific_instructions/mlops_engineer/phases/phase-7.md +144 -0
- package/src/agents/specific_instructions/mlops_engineer/phases.md +671 -0
- package/src/agents/specific_instructions/mlops_engineer/review.md +164 -0
- package/src/agents/specific_instructions/mlops_engineer/service_mode.md +81 -0
- package/src/agents/specific_instructions/mlops_engineer/validation_checklist.md +151 -0
- package/src/agents/specific_instructions/researcher/critical_review.md +292 -0
- package/src/agents/specific_instructions/researcher/review_checklist.md +67 -0
- package/src/agents/specific_instructions/researcher/service_mode.md +224 -0
- package/src/agents/specific_instructions/shared/auto_verify_mode.md +141 -0
- package/src/agents/specific_instructions/shared/autonomous_research.md +1289 -0
- package/src/agents/specific_instructions/shared/behavioral_rules.md +36 -0
- package/src/agents/specific_instructions/shared/diverge_protocol.md +387 -0
- package/src/agents/specific_instructions/shared/engineering_guidelines.md +136 -0
- package/src/agents/specific_instructions/shared/experiment_versioning.md +184 -0
- package/src/agents/specific_instructions/shared/goal_mode.md +187 -0
- package/src/agents/specific_instructions/shared/incremental_testing.md +139 -0
- package/src/agents/specific_instructions/shared/intent_discovery.md +223 -0
- package/src/agents/specific_instructions/shared/join_path_protocol.md +168 -0
- package/src/agents/specific_instructions/shared/knowledge_checkpoint.md +83 -0
- package/src/agents/specific_instructions/shared/knowledge_harvest.md +220 -0
- package/src/agents/specific_instructions/shared/knowledge_retrieval.md +100 -0
- package/src/agents/specific_instructions/shared/notebook_walkthrough_protocol.md +367 -0
- package/src/agents/specific_instructions/shared/reviewer_verdict_protocol.md +74 -0
- package/src/agents/specific_instructions/shared/swarm_protocol.md +97 -0
- package/src/agents/specific_instructions/shared/validation_protocol.md +139 -0
- package/src/agents/specific_instructions/syn/arbiter.md +140 -0
- package/src/agents/specific_instructions/syn/brainstorm.md +550 -0
- package/src/agents/specific_instructions/syn/code_review.md +232 -0
- package/src/agents/specific_instructions/syn/diff.md +239 -0
- package/src/agents/specific_instructions/syn/final_review.md +65 -0
- package/src/agents/specific_instructions/syn/fixer.md +240 -0
- package/src/agents/specific_instructions/syn/free_form.md +130 -0
- package/src/agents/specific_instructions/syn/knowledge.md +468 -0
- package/src/agents/specific_instructions/syn/notebook_walkthrough.md +78 -0
- package/src/agents/specific_instructions/syn/panel_review.md +634 -0
- package/src/agents/specific_instructions/syn/pm.md +453 -0
- package/src/agents/specific_instructions/syn/pr_review.md +255 -0
- package/src/agents/specific_instructions/syn/slides.md +417 -0
- package/src/agents/syn.md +729 -0
- package/src/commands/academic.md +41 -0
- package/src/commands/ai-engineer.md +45 -0
- package/src/commands/analytics-engineer.md +48 -0
- package/src/commands/applied-ml-scientist.md +45 -0
- package/src/commands/backend-engineer.md +35 -0
- package/src/commands/bi-engineer.md +40 -0
- package/src/commands/brainstorm.md +24 -0
- package/src/commands/data-analyst.md +38 -0
- package/src/commands/data-engineer.md +37 -0
- package/src/commands/data-modeller.md +38 -0
- package/src/commands/data-scientist.md +38 -0
- package/src/commands/deep-learning-engineer.md +47 -0
- package/src/commands/end.md +49 -0
- package/src/commands/knowledge.md +24 -0
- package/src/commands/ml-engineer.md +42 -0
- package/src/commands/mlops-engineer.md +47 -0
- package/src/commands/notebook-walkthrough.md +58 -0
- package/src/commands/researcher.md +40 -0
- package/src/commands/resume.md +57 -0
- package/src/commands/review-pr.md +26 -0
- package/src/commands/shards-guide.md +41 -0
- package/src/commands/shards-ui.md +32 -0
- package/src/commands/shards.md +41 -0
- package/src/docs/01-getting-started/concepts.md +109 -0
- package/src/docs/01-getting-started/first-session.md +79 -0
- package/src/docs/01-getting-started/install.md +61 -0
- package/src/docs/02-agents/academic.md +71 -0
- package/src/docs/02-agents/ai-engineer.md +78 -0
- package/src/docs/02-agents/analytics-engineer.md +58 -0
- package/src/docs/02-agents/applied-ml-scientist.md +59 -0
- package/src/docs/02-agents/backend-engineer.md +58 -0
- package/src/docs/02-agents/bi-engineer.md +65 -0
- package/src/docs/02-agents/data-analyst.md +67 -0
- package/src/docs/02-agents/data-engineer.md +57 -0
- package/src/docs/02-agents/data-modeller.md +51 -0
- package/src/docs/02-agents/data-scientist.md +78 -0
- package/src/docs/02-agents/deep-learning-engineer.md +64 -0
- package/src/docs/02-agents/ml-engineer.md +80 -0
- package/src/docs/02-agents/mlops-engineer.md +59 -0
- package/src/docs/02-agents/overview.md +62 -0
- package/src/docs/02-agents/researcher.md +73 -0
- package/src/docs/02-agents/syn.md +88 -0
- package/src/docs/03-protocols/auto-verify.md +82 -0
- package/src/docs/03-protocols/autonomous-research.md +59 -0
- package/src/docs/03-protocols/behavioral-rules.md +35 -0
- package/src/docs/03-protocols/diverge.md +50 -0
- package/src/docs/03-protocols/engineering-guidelines.md +56 -0
- package/src/docs/03-protocols/experiment-versioning.md +38 -0
- package/src/docs/03-protocols/gate-pattern.md +65 -0
- package/src/docs/03-protocols/incremental-testing.md +68 -0
- package/src/docs/03-protocols/join-path.md +46 -0
- package/src/docs/03-protocols/knowledge-ledger.md +70 -0
- package/src/docs/03-protocols/reviewer-verdicts.md +39 -0
- package/src/docs/03-protocols/swarm.md +40 -0
- package/src/docs/03-protocols/validation.md +174 -0
- package/src/docs/04-ui/activity-bar.md +70 -0
- package/src/docs/04-ui/chat-pane.md +80 -0
- package/src/docs/04-ui/code-intel.md +62 -0
- package/src/docs/04-ui/file-editing.md +61 -0
- package/src/docs/04-ui/git.md +54 -0
- package/src/docs/04-ui/keybindings.md +79 -0
- package/src/docs/04-ui/knowledge-map.md +76 -0
- package/src/docs/04-ui/overview.md +93 -0
- package/src/docs/04-ui/panels.md +49 -0
- package/src/docs/04-ui/pinboard-selection.md +66 -0
- package/src/docs/04-ui/quick-open-palette.md +56 -0
- package/src/docs/04-ui/sessions.md +81 -0
- package/src/docs/04-ui/settings-permissions.md +56 -0
- package/src/docs/05-commands/reference.md +59 -0
- package/src/docs/06-outputs/directory-map.md +116 -0
- package/src/docs/07-workflows/ai-eval-first.md +57 -0
- package/src/docs/07-workflows/deep-study-to-production.md +76 -0
- package/src/docs/07-workflows/diverge-exploration.md +77 -0
- package/src/docs/07-workflows/quick-analysis.md +45 -0
- package/src/docs/08-integrations/claude-code-auto-mode.md +191 -0
- package/src/docs/08-integrations/google-slides.md +175 -0
- package/src/docs/README.md +30 -0
- package/src/docs/manifest.json +108 -0
- package/src/templates/analysis-template.md +20 -0
- package/src/templates/branch-report.md +46 -0
- package/src/templates/diff-report.md +88 -0
- package/src/templates/knowledge-index.md +7 -0
- package/src/templates/model-card-schema.json +186 -0
- package/src/templates/model-card-schema.md +88 -0
- package/src/templates/model-card.md +124 -0
- package/src/templates/project-plan.md +47 -0
- package/src/templates/project-specs.md +81 -0
- package/src/templates/report-template.md +43 -0
- package/src/templates/study-template.md +25 -0
- package/src/ui/cc-readonly.js +181 -0
- package/src/ui/chat-session.js +466 -0
- package/src/ui/css/base.css +136 -0
- package/src/ui/css/brainstorm.css +525 -0
- package/src/ui/css/chat.css +1405 -0
- package/src/ui/css/editor.css +546 -0
- package/src/ui/css/eval-dashboard.css +157 -0
- package/src/ui/css/experiment.css +237 -0
- package/src/ui/css/guide.css +186 -0
- package/src/ui/css/knowledge-map.css +383 -0
- package/src/ui/css/layout.css +431 -0
- package/src/ui/css/model-card.css +161 -0
- package/src/ui/css/notebook-walkthrough.css +271 -0
- package/src/ui/css/pr-review.css +403 -0
- package/src/ui/css/prompt-lab.css +325 -0
- package/src/ui/css/sessions.css +258 -0
- package/src/ui/css/sidebar.css +661 -0
- package/src/ui/css/terminal.css +113 -0
- package/src/ui/css/theme-light.css +542 -0
- package/src/ui/index.html +389 -0
- package/src/ui/js/agents.js +32 -0
- package/src/ui/js/bookmarks.js +230 -0
- package/src/ui/js/chat.js +1776 -0
- package/src/ui/js/code-intel.js +328 -0
- package/src/ui/js/command-palette.js +142 -0
- package/src/ui/js/events.js +591 -0
- package/src/ui/js/explorer.js +317 -0
- package/src/ui/js/file-view.js +477 -0
- package/src/ui/js/git.js +536 -0
- package/src/ui/js/guide.js +198 -0
- package/src/ui/js/hud.js +75 -0
- package/src/ui/js/init.js +351 -0
- package/src/ui/js/knowledge-map.js +906 -0
- package/src/ui/js/markdown.js +114 -0
- package/src/ui/js/monaco.js +164 -0
- package/src/ui/js/notebook-walkthrough.js +272 -0
- package/src/ui/js/notebook.js +448 -0
- package/src/ui/js/panels.js +2681 -0
- package/src/ui/js/pinboard.js +186 -0
- package/src/ui/js/quick-open.js +164 -0
- package/src/ui/js/selection-context.js +131 -0
- package/src/ui/js/sessions.js +256 -0
- package/src/ui/js/settings.js +476 -0
- package/src/ui/js/split-view.js +82 -0
- package/src/ui/js/state.js +343 -0
- package/src/ui/js/table.js +161 -0
- package/src/ui/js/tabs.js +284 -0
- package/src/ui/js/tabular.js +125 -0
- package/src/ui/js/terminal.js +354 -0
- package/src/ui/js/timeline.js +137 -0
- package/src/ui/js/utils.js +293 -0
- package/src/ui/notebook-kernel.py +790 -0
- package/src/ui/open-browser.js +55 -0
- package/src/ui/permission-pattern.js +42 -0
- package/src/ui/relay.js +513 -0
- package/src/ui/server.js +3072 -0
- package/src/ui/session-index.js +225 -0
- package/src/ui/shards_icon.png +0 -0
- package/src/ui/spawn-server.js +41 -0
- package/src/ui/symbol-index.js +813 -0
- package/src/ui/ui-push.js +177 -0
- package/tools/gate-hook/VALIDATION_SPEC.md +273 -0
- package/tools/gate-hook/__tests__/auto-verify.test.js +343 -0
- package/tools/gate-hook/auto-allowlist.js +179 -0
- package/tools/gate-hook/auto-state.js +68 -0
- package/tools/gate-hook/classify.js +21 -0
- package/tools/gate-hook/log.js +57 -0
- package/tools/gate-hook/parser.js +205 -0
- package/tools/gate-hook/sql-guard.js +230 -0
- package/tools/gate-hook/state.js +170 -0
- package/tools/gate-hook/sweep.js +139 -0
- package/tools/gate-hook/transcript.js +45 -0
- package/tools/gate-hook/validation.js +321 -0
- package/tools/gate-hook.js +475 -0
- package/tools/install.js +914 -0
- package/tools/shards-gates.js +311 -0
- package/tools/shards-sessions.js +261 -0
- package/tools/shards-ui.js +377 -0
|
@@ -0,0 +1,1289 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: autonomous-research-protocol
|
|
3
|
+
description: >
|
|
4
|
+
Shared autonomous research (AR) mode protocol. Defines the self-steering
|
|
5
|
+
research loop inspired by autoresearch: budget-bounded, adaptive hypothesis
|
|
6
|
+
generation, auto-keep/revert based on metric movement, steering document
|
|
7
|
+
the user can edit mid-loop, and convergence detection. Referenced by AR-capable
|
|
8
|
+
agents (ML Engineer, AI Engineer, Data Scientist, Applied ML Scientist, Deep
|
|
9
|
+
Learning Engineer). Section H defines how AR composes with DIVERGE for
|
|
10
|
+
fan-out of parallel approach families.
|
|
11
|
+
type: reference
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
# Autonomous Research (AR) Protocol
|
|
15
|
+
|
|
16
|
+
This protocol governs `[AR]` mode — a self-steering research loop that runs a
|
|
17
|
+
bounded number of iterations against a single primary metric and auto-decides
|
|
18
|
+
whether to keep or revert each change. It complements `[EX]` (human-planned
|
|
19
|
+
fixed-N experiments) rather than replacing it.
|
|
20
|
+
|
|
21
|
+
**When to use `[AR]`:**
|
|
22
|
+
- You have a metric and a budget and want the agent to push as far as it will go.
|
|
23
|
+
- You cannot enumerate the specific experiments ahead of time.
|
|
24
|
+
- You want hypotheses generated adaptively based on accumulated results.
|
|
25
|
+
|
|
26
|
+
**When NOT to use `[AR]`:**
|
|
27
|
+
- You have 2-3 specific things to try — use `[EX]` instead.
|
|
28
|
+
- You have not identified a primary metric — AR requires a single north-star.
|
|
29
|
+
- The changes are not revertable via git — AR requires versioning for safety.
|
|
30
|
+
|
|
31
|
+
---
|
|
32
|
+
|
|
33
|
+
## Section A — Research Brief Initialization
|
|
34
|
+
|
|
35
|
+
Run this section at the start of Phase 1 (Research Brief), after Phase 0 setup
|
|
36
|
+
has been gated and confirmed.
|
|
37
|
+
|
|
38
|
+
### A.1 Knowledge retrieval
|
|
39
|
+
|
|
40
|
+
Before writing the brief, read
|
|
41
|
+
`.claude/agents/specific_instructions/shared/knowledge_retrieval.md` in full and
|
|
42
|
+
follow the AR entry point (Step 2 under AR Phase 0 / pre-brief context).
|
|
43
|
+
|
|
44
|
+
Match criteria most useful for AR:
|
|
45
|
+
- **Metric** — has a prior AR or study measured this metric on similar data?
|
|
46
|
+
- **Domain** — dataset, entity, or business area match.
|
|
47
|
+
- **Approach family** — have prior runs explored the same model family (e.g.,
|
|
48
|
+
gradient boosting, transformer, prompt-chain)?
|
|
49
|
+
|
|
50
|
+
Surface relevant ledger entries (saturation points, known-leaky features,
|
|
51
|
+
architectural dead ends) to the user alongside the draft brief. Prior runs may
|
|
52
|
+
warn you off hypotheses that look promising but are already-tested dead ends.
|
|
53
|
+
|
|
54
|
+
### A.2 Determine preset
|
|
55
|
+
|
|
56
|
+
Confirm with the user which preset is active (established at Phase 0):
|
|
57
|
+
|
|
58
|
+
- **`interactive`** (default) — budget=10, reviewer cadence=3, no required cost
|
|
59
|
+
ceiling. User is nearby and expects an engaged session. Closest in character
|
|
60
|
+
to a beefed-up `[EX]`.
|
|
61
|
+
- **`overnight`** — budget=100, reviewer cadence=10, cost ceiling required.
|
|
62
|
+
User is away; agent runs long. Interrupt mechanism for unattended runs
|
|
63
|
+
is the Steering Notes section of `research_brief.md` — the agent re-reads
|
|
64
|
+
it every iteration (see B.1) and honors directives written there
|
|
65
|
+
(including "STOP" / "PAUSE"). Micro-gates exist as an opt-in feature but
|
|
66
|
+
currently block the loop (no auto-close); see B.12.
|
|
67
|
+
- **`custom`** — user has overridden one or more preset defaults; record the
|
|
68
|
+
overrides.
|
|
69
|
+
|
|
70
|
+
Presets are hints. All parameters remain user-overridable.
|
|
71
|
+
|
|
72
|
+
### A.3 Write `experiments/research_brief.md`
|
|
73
|
+
|
|
74
|
+
Write the brief using `templates/research-brief.md` as the source template.
|
|
75
|
+
Populate every `{{PLACEHOLDER}}`. The brief is the steering document — the user
|
|
76
|
+
can edit it between iterations and the agent re-reads it every iteration.
|
|
77
|
+
|
|
78
|
+
Required sections:
|
|
79
|
+
- **Objective** — one paragraph on what this AR run is trying to achieve.
|
|
80
|
+
- **Primary metric** — name, direction (maximize/minimize), baseline value, source
|
|
81
|
+
of baseline, target value (optional).
|
|
82
|
+
- **Constraints** — iteration budget, per-iteration time limit, max consecutive
|
|
83
|
+
regressions, metric degradation floor, cost ceiling (tokens and/or dollars;
|
|
84
|
+
optional for interactive, required for overnight).
|
|
85
|
+
- **Scope** — explicit list of mutable files/directories vs. immutable
|
|
86
|
+
files/directories. Anything not listed as mutable is immutable by default.
|
|
87
|
+
- **Preset** — `interactive` | `overnight` | `custom`.
|
|
88
|
+
- **Steering Notes** — user-editable section. Empty at brief creation; the user
|
|
89
|
+
may add notes like "prioritize feature engineering over hyperparameter tuning"
|
|
90
|
+
or "avoid transformer-family — too slow at serve time" at any point.
|
|
91
|
+
- **Research Log** — append-only log. One line per iteration with outcome.
|
|
92
|
+
|
|
93
|
+
### A.4 Write `experiments/results.json`
|
|
94
|
+
|
|
95
|
+
Initialize `results.json` with the AR-extended schema (see Section F for the
|
|
96
|
+
full schema). Set `mode: "autonomous-research"` and `preset: <chosen preset>`.
|
|
97
|
+
|
|
98
|
+
### A.5 Announce behavioral exception
|
|
99
|
+
|
|
100
|
+
Before the gate, tell the user:
|
|
101
|
+
|
|
102
|
+
> "Facilitate, don't generate" is suspended for Phase 2 of this AR session. I
|
|
103
|
+
> will autonomously generate hypotheses, implement changes, and auto-keep or
|
|
104
|
+
> auto-revert each iteration based on the primary metric. Phase 0, Phase 1, and
|
|
105
|
+
> Phase 3 remain gated. You can steer the loop at any time by editing
|
|
106
|
+
> `experiments/research_brief.md` — I re-read it every iteration.
|
|
107
|
+
|
|
108
|
+
### A.6 Optional `/goal` activation
|
|
109
|
+
|
|
110
|
+
After announcing the behavioral exception and before the per-agent Phase 1
|
|
111
|
+
gate, propose an opt-in `/goal` to drive the Phase 2 loop autonomously without
|
|
112
|
+
per-turn prompts.
|
|
113
|
+
|
|
114
|
+
Read `.claude/agents/specific_instructions/shared/goal_mode.md` in full and
|
|
115
|
+
follow the **AR condition template**. Compose a candidate condition using the
|
|
116
|
+
Phase 0 parameters (primary metric, target, iteration budget, metric floor)
|
|
117
|
+
and include it as a copy-paste block in the per-agent Phase 1 message,
|
|
118
|
+
immediately before the gate fence.
|
|
119
|
+
|
|
120
|
+
Tell the user the activation is optional:
|
|
121
|
+
- **With `/goal`:** the loop drives itself turn-by-turn without per-iteration
|
|
122
|
+
prompts; the evaluator decides when to stop based on the inline iteration
|
|
123
|
+
summaries the agent prints (see B.4 + B.8 transcript discipline below).
|
|
124
|
+
- **Without `/goal`:** the loop still runs autonomously per §B, and §E
|
|
125
|
+
convergence + §G safety rails still terminate it. Per-turn prompts may
|
|
126
|
+
appear depending on the user's permission settings.
|
|
127
|
+
|
|
128
|
+
Either way, §E convergence and §G safety rails are unchanged. If `/goal` is
|
|
129
|
+
unavailable (old Code version, hooks disabled, command rejected), accept that
|
|
130
|
+
and proceed — the existing stop logic still terminates the loop.
|
|
131
|
+
|
|
132
|
+
If the user activates `/goal`, the per-iteration **inline summary echo** in
|
|
133
|
+
§B.4 and §B.8 below is mandatory (otherwise the evaluator has nothing to read).
|
|
134
|
+
If the user skips `/goal`, the inline echoes are still recommended — they keep
|
|
135
|
+
the run readable in the transcript.
|
|
136
|
+
|
|
137
|
+
---
|
|
138
|
+
|
|
139
|
+
## Section B — The Research Loop
|
|
140
|
+
|
|
141
|
+
Phase 2 executes this loop until a stop condition in Section E fires or the user
|
|
142
|
+
interrupts. **No intermediate gates between iterations by default** — the
|
|
143
|
+
autonomous loop is the whole point. The optional micro-gate (Section B.12) is
|
|
144
|
+
opt-in and only on the overnight preset.
|
|
145
|
+
|
|
146
|
+
At the start of Phase 2, run `git rev-parse HEAD` (via the `experiment_versioning.md`
|
|
147
|
+
Section A detection) to capture the pre-loop commit. Set
|
|
148
|
+
`results.json.lastGreenCommit` to this SHA — this is the baseline you revert to
|
|
149
|
+
when an iteration regresses. It is always updated by the agent, never searched
|
|
150
|
+
via `git log`.
|
|
151
|
+
|
|
152
|
+
### Per-iteration steps
|
|
153
|
+
|
|
154
|
+
For iteration N:
|
|
155
|
+
|
|
156
|
+
#### B.1 Re-read steering
|
|
157
|
+
|
|
158
|
+
Read `experiments/research_brief.md` in full. Look for changes to the Steering
|
|
159
|
+
Notes section since the previous iteration. If the user has added new notes,
|
|
160
|
+
incorporate them into the next hypothesis.
|
|
161
|
+
|
|
162
|
+
**Honor halt directives.** If the Steering Notes contain any of the following
|
|
163
|
+
(case-insensitive, as a standalone directive or clearly flagged as such):
|
|
164
|
+
`STOP`, `PAUSE`, `HALT`, or `stop the loop` — halt the loop immediately,
|
|
165
|
+
record `results.json.convergence.reason = "user-interrupt"`, and proceed to
|
|
166
|
+
Phase 3 as if any stop condition had fired. This is the primary interrupt
|
|
167
|
+
mechanism for unattended overnight runs.
|
|
168
|
+
|
|
169
|
+
#### B.2 Windowed history read
|
|
170
|
+
|
|
171
|
+
To keep per-iteration context cost bounded:
|
|
172
|
+
|
|
173
|
+
- At iteration N ≤ 10: read the full `results.json` (all prior entries).
|
|
174
|
+
- At iteration N > 10: read only the **last 10 experiment entries** from
|
|
175
|
+
`results.json`, plus `experiments/history_summary.md` which you maintain and
|
|
176
|
+
compress every 10 iterations.
|
|
177
|
+
|
|
178
|
+
Writing `history_summary.md`: at iterations 11, 21, 31, ..., consolidate
|
|
179
|
+
iterations 1..N-10 into a short summary (approach families tried, what worked,
|
|
180
|
+
what didn't, metric trajectory). Overwrite the file. At iteration 11 it
|
|
181
|
+
summarizes 1-10; at 21 it summarizes 1-20; and so on.
|
|
182
|
+
|
|
183
|
+
#### B.3 Generate next hypothesis
|
|
184
|
+
|
|
185
|
+
Synthesize the next hypothesis from:
|
|
186
|
+
- The Steering Notes section of the research brief
|
|
187
|
+
- Recent iteration results (windowed history)
|
|
188
|
+
- Any reviewer verdicts with `REDIRECT` or `CONTINUE + suggestion`
|
|
189
|
+
- Your agent-specific experiment categories (see per-agent `research.md`)
|
|
190
|
+
|
|
191
|
+
The hypothesis must be **one specific, testable change**. Avoid bundling
|
|
192
|
+
multiple changes in a single iteration.
|
|
193
|
+
|
|
194
|
+
Record `hypothesisSource` in the results entry:
|
|
195
|
+
- `"adaptive"` — generated by agent from accumulated evidence
|
|
196
|
+
- `"steering"` — taken directly from user's Steering Notes
|
|
197
|
+
- `"reviewer-suggested"` — from a reviewer's `REDIRECT` or suggestion
|
|
198
|
+
|
|
199
|
+
#### B.4 Announce
|
|
200
|
+
|
|
201
|
+
Print inline: `[AR] Iteration N: <one-line hypothesis>`
|
|
202
|
+
|
|
203
|
+
This is the **start-of-iteration** echo. The matching **end-of-iteration** echo
|
|
204
|
+
is required at B.8 below — it's what the `/goal` evaluator (if active) reads
|
|
205
|
+
to decide whether the loop is done.
|
|
206
|
+
|
|
207
|
+
#### B.5 Implement
|
|
208
|
+
|
|
209
|
+
Make the changes. **Scope enforcement is hard**: every Edit/Write target path
|
|
210
|
+
must be in the mutable set from the research brief. Before writing, verify the
|
|
211
|
+
target path matches one of the mutable patterns. If a change would require
|
|
212
|
+
touching an immutable file, halt the iteration and record a scope-violation
|
|
213
|
+
stop per Section G.
|
|
214
|
+
|
|
215
|
+
Keep changes minimal and isolated to the hypothesis. No drive-by cleanups.
|
|
216
|
+
|
|
217
|
+
#### B.6 Evaluate
|
|
218
|
+
|
|
219
|
+
Run the evaluation protocol for this specialist (full train/eval, proxy, etc.)
|
|
220
|
+
and measure the primary metric. Record both primary and secondary metrics.
|
|
221
|
+
|
|
222
|
+
**Proxy vs full-eval rules (Section E also governs):**
|
|
223
|
+
- Record `evalType: "full" | "proxy"` in the iteration entry.
|
|
224
|
+
- Full evaluation must run at least every M iterations (M=5 interactive, M=10
|
|
225
|
+
overnight) and always on the final iteration before Phase 3.
|
|
226
|
+
- A proxy below the metric floor triggers an automatic full re-evaluation on
|
|
227
|
+
the same iteration before any revert decision is made (see Section G.2).
|
|
228
|
+
|
|
229
|
+
#### B.7 Auto-keep/revert decision (Section C)
|
|
230
|
+
|
|
231
|
+
Classify the result as GREEN / RED / YELLOW per Section C and execute the
|
|
232
|
+
corresponding action (keep, revert, or conditional-keep).
|
|
233
|
+
|
|
234
|
+
#### B.8 Record results
|
|
235
|
+
|
|
236
|
+
Write `experiments/experiment_<N>_<name>.md` using the template below. Append
|
|
237
|
+
the iteration entry to `results.json.experiments[]`. Append a one-line entry to
|
|
238
|
+
the Research Log in `research_brief.md`:
|
|
239
|
+
|
|
240
|
+
```
|
|
241
|
+
- **Iter N** (<green|red|yellow>, <kept|reverted>): <hypothesis> — <metric> <before> → <after>
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
**Experiment file template:**
|
|
245
|
+
|
|
246
|
+
```markdown
|
|
247
|
+
# AR Iteration N: <Name>
|
|
248
|
+
|
|
249
|
+
- **Date:** <date>
|
|
250
|
+
- **Agent:** <agent name>
|
|
251
|
+
- **Iteration:** N of <budget>
|
|
252
|
+
- **Primary metric:** <metric name>
|
|
253
|
+
- **Hypothesis source:** <adaptive | steering | reviewer-suggested>
|
|
254
|
+
- **Eval type:** <full | proxy>
|
|
255
|
+
|
|
256
|
+
## Hypothesis
|
|
257
|
+
<what you expected and why>
|
|
258
|
+
|
|
259
|
+
## Changes Made
|
|
260
|
+
<precise description of files touched and what changed>
|
|
261
|
+
|
|
262
|
+
## Metrics
|
|
263
|
+
| Metric | Before | After | Delta |
|
|
264
|
+
|--------|--------|-------|-------|
|
|
265
|
+
| **<primary metric>** | **<value>** | **<value>** | **<+/->** |
|
|
266
|
+
| <secondary metric> | <value> | <value> | <+/-> |
|
|
267
|
+
|
|
268
|
+
## Auto-Decision
|
|
269
|
+
<GREEN | RED | YELLOW> — <one-sentence reasoning>
|
|
270
|
+
|
|
271
|
+
## Action Taken
|
|
272
|
+
<Kept | Reverted | Kept with next-steps>
|
|
273
|
+
|
|
274
|
+
## Next Steps (populated for kept YELLOWs)
|
|
275
|
+
<what hypothesis this opens up>
|
|
276
|
+
|
|
277
|
+
## Reviewer Note (populated when reviewer consulted)
|
|
278
|
+
<verdict and key points from Section D>
|
|
279
|
+
|
|
280
|
+
## Self-Assessment (populated when reviewer not consulted this iteration)
|
|
281
|
+
<one-line honest read: does this result feel real? methodology concerns?>
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
**Inline iteration summary (required when `/goal` is active; recommended
|
|
285
|
+
otherwise).** After writing the iteration file and updating `results.json`,
|
|
286
|
+
print the per-iteration summary inline in the same assistant turn. The `/goal`
|
|
287
|
+
evaluator (per `goal_mode.md`) can only read the transcript, not files —
|
|
288
|
+
without this echo the goal can never resolve. Use this exact format:
|
|
289
|
+
|
|
290
|
+
```
|
|
291
|
+
[AR] Iteration N complete.
|
|
292
|
+
Primary metric: <metric_name> <before> → <after> (delta: <+/->)
|
|
293
|
+
Auto-decision: <GREEN | RED | YELLOW> — <one-sentence reason>
|
|
294
|
+
Action: <Kept | Reverted | Kept with next-steps>
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
When a §E stop condition fires later in this iteration (at B.11) **or a §G
|
|
298
|
+
safety rail halts the loop**, additionally print on the same turn:
|
|
299
|
+
|
|
300
|
+
```
|
|
301
|
+
Convergence detected: <plateau | diminishing-returns | budget-exhausted | cost-ceiling | consecutive-failures | metric-floor-breach | user-interrupt | reviewer-pause | scope-violation | error-limit | timeout-limit>
|
|
302
|
+
```
|
|
303
|
+
|
|
304
|
+
The enum here matches §E.7's canonical list — §E stop conditions plus §G safety-rail halts.
|
|
305
|
+
|
|
306
|
+
These echoes carry the same content already written to `results.json` and the
|
|
307
|
+
per-iteration markdown — the inline copy makes the run legible in the
|
|
308
|
+
transcript and is the substrate the `/goal` condition templates anchor to.
|
|
309
|
+
|
|
310
|
+
#### B.9 Git checkpoint
|
|
311
|
+
|
|
312
|
+
Follow **Section B** of `.claude/agents/specific_instructions/shared/experiment_versioning.md`
|
|
313
|
+
with these AR-specific conventions:
|
|
314
|
+
|
|
315
|
+
- **Tag prefix:** `research/<project_name>/<N>-<name>` (not `exp/<...>`).
|
|
316
|
+
- **Annotated tag message** includes the auto-decision color:
|
|
317
|
+
```
|
|
318
|
+
Iteration N (<green|red|yellow>): <name> | <metric>: <before> -> <after> (delta: <delta>)
|
|
319
|
+
```
|
|
320
|
+
- The color suffix lives only in the annotated tag message and in
|
|
321
|
+
`results.json.experiments[N].autoDecision`, never in the tag name itself.
|
|
322
|
+
This matches `[EX]`'s `exp/<project>/<N>-<name>` convention and keeps
|
|
323
|
+
`git tag -l "research/<project>/*"` globs clean.
|
|
324
|
+
|
|
325
|
+
If versioning mode is `none`, skip this step silently (per
|
|
326
|
+
`experiment_versioning.md` Section B).
|
|
327
|
+
|
|
328
|
+
#### B.10 Reviewer consultation if cadence hit
|
|
329
|
+
|
|
330
|
+
Apply Section D cadence rules. If a reviewer is consulted, record their verdict
|
|
331
|
+
per Section D.
|
|
332
|
+
|
|
333
|
+
#### B.11 Convergence check
|
|
334
|
+
|
|
335
|
+
Run Section E stop-condition checks. If any condition fires, break the loop and
|
|
336
|
+
proceed to Phase 3.
|
|
337
|
+
|
|
338
|
+
#### B.12 Cost accounting + micro-gate (overnight only)
|
|
339
|
+
|
|
340
|
+
Update `results.json.costAccounting`:
|
|
341
|
+
- `tokensIn` and `tokensOut` — approximate counts from this iteration
|
|
342
|
+
- `dollarsSpent` — running total
|
|
343
|
+
- `reviewerTasksSpawned` — increment if a reviewer Task was called
|
|
344
|
+
|
|
345
|
+
Check against the cost ceiling (Section G.5). If at 50% or 80% of ceiling emit
|
|
346
|
+
a soft warning; at 100% hard-stop the loop.
|
|
347
|
+
|
|
348
|
+
**Micro-gate (opt-in, overnight preset):** micro-gates are designed as a
|
|
349
|
+
user-interrupt window emitted every K iterations (default K=5). They are
|
|
350
|
+
currently **disabled by default** because the gate hook does not yet
|
|
351
|
+
support auto-close — a micro-gate blocks the loop until the user confirms,
|
|
352
|
+
which defeats the overnight preset's purpose.
|
|
353
|
+
|
|
354
|
+
**Default behavior:** do NOT emit micro-gates during Phase 2 unless the user
|
|
355
|
+
has explicitly opted in at Phase 0 with the understanding that the loop will
|
|
356
|
+
block at each micro-gate.
|
|
357
|
+
|
|
358
|
+
**If the user opts in,** emit a `::GATE::` fence with `kind=confirm` every K
|
|
359
|
+
iterations:
|
|
360
|
+
|
|
361
|
+
```
|
|
362
|
+
::GATE:: id=specific-instructions-shared-autonomous-research-micro-<project>-<N> phase=2 kind=confirm
|
|
363
|
+
Iteration N complete. Reply "continue" to proceed, or give steering notes.
|
|
364
|
+
::ENDGATE::
|
|
365
|
+
```
|
|
366
|
+
|
|
367
|
+
Note the ID includes the iteration number so each micro-gate has a unique ID
|
|
368
|
+
per the harness's "never duplicate gate ids" rule.
|
|
369
|
+
|
|
370
|
+
**Alternative for unattended overnight runs:** rely on the user editing
|
|
371
|
+
`research_brief.md` Steering Notes mid-run as the interrupt mechanism
|
|
372
|
+
(re-read every iteration — see B.1). No gate, no block. The user can pause
|
|
373
|
+
the run anytime by writing "STOP" or similar directive into Steering Notes,
|
|
374
|
+
which the agent reads at the next iteration and halts on.
|
|
375
|
+
|
|
376
|
+
#### B.13 Continue or stop
|
|
377
|
+
|
|
378
|
+
If no stop condition fired and iteration budget remains, proceed to iteration
|
|
379
|
+
N+1. Otherwise exit to Phase 3.
|
|
380
|
+
|
|
381
|
+
---
|
|
382
|
+
|
|
383
|
+
## Section C — Auto-Keep/Revert Protocol
|
|
384
|
+
|
|
385
|
+
### C.1 Classification
|
|
386
|
+
|
|
387
|
+
Classify each iteration result by comparing the primary metric to the
|
|
388
|
+
**comparison reference** — not to the original baseline. The comparison
|
|
389
|
+
reference is the "after" value from the most recent GREEN iteration, or the
|
|
390
|
+
original baseline (`results.json.baseline.value`) if no GREEN has occurred
|
|
391
|
+
yet. This ensures each iteration is measured against the current best state,
|
|
392
|
+
not against the original starting point (which would make every small
|
|
393
|
+
improvement look GREEN and break convergence detection).
|
|
394
|
+
|
|
395
|
+
- **GREEN (improved):** Primary metric moved in the configured direction
|
|
396
|
+
**from the comparison reference** by more than `epsilon` (default: 1% of
|
|
397
|
+
baseline, configurable per project).
|
|
398
|
+
- **RED (regressed):** Primary metric moved in the opposite direction from
|
|
399
|
+
the comparison reference by more than `epsilon`.
|
|
400
|
+
- **YELLOW (neutral):** Primary metric moved by less than `epsilon` in either
|
|
401
|
+
direction from the comparison reference.
|
|
402
|
+
|
|
403
|
+
The original `results.json.baseline.value` **never changes** after Phase 0 —
|
|
404
|
+
it is used at Phase 3 for computing net delta and for the epsilon default.
|
|
405
|
+
|
|
406
|
+
### C.2 Actions
|
|
407
|
+
|
|
408
|
+
**GREEN:**
|
|
409
|
+
1. Keep the changes (no revert).
|
|
410
|
+
2. Update `results.json.lastGreenCommit` to the commit SHA of this iteration's
|
|
411
|
+
checkpoint. The "after" value from this iteration becomes the comparison
|
|
412
|
+
reference for the next iteration's classification (derived on-demand from
|
|
413
|
+
`results.json.experiments[lastGreenIndex].metrics.outcome.after`).
|
|
414
|
+
3. Reset `consecutiveRegressions` to 0.
|
|
415
|
+
4. Reset `consecutiveYellowKept` to 0.
|
|
416
|
+
5. Annotate the git tag message with `color=green`.
|
|
417
|
+
6. **Do NOT modify `results.json.baseline.value`** — that remains the original
|
|
418
|
+
baseline. The comparison reference is tracked via `lastGreenCommit` (and
|
|
419
|
+
its iteration's `metrics.outcome.after`).
|
|
420
|
+
|
|
421
|
+
**RED:**
|
|
422
|
+
1. Revert per C.3 (Revert Mechanism).
|
|
423
|
+
2. Increment `consecutiveRegressions`.
|
|
424
|
+
3. Record the failure in the experiment file — we learn from REDs too.
|
|
425
|
+
4. Annotate the git tag message with `color=red`.
|
|
426
|
+
|
|
427
|
+
**YELLOW:**
|
|
428
|
+
Default behavior: **revert**. Exceptions (keep) when either is true:
|
|
429
|
+
- (a) The change **reduces complexity**: net LOC decrease, dependency removal,
|
|
430
|
+
dead code deletion, simpler architecture.
|
|
431
|
+
- (b) The change **opens a new hypothesis space** that is explicitly documented
|
|
432
|
+
in the iteration's `nextSteps` field — the YELLOW is a stepping stone.
|
|
433
|
+
|
|
434
|
+
If kept:
|
|
435
|
+
1. Increment `consecutiveYellowKept`.
|
|
436
|
+
2. Record `autoDecision: "yellow"`, `reverted: false`, populate `nextSteps`.
|
|
437
|
+
3. Annotate the git tag with `color=yellow`.
|
|
438
|
+
4. **Three-consecutive-YELLOW-kept limit:** if `consecutiveYellowKept` reaches
|
|
439
|
+
3, the next iteration — even if GREEN — must be followed by a force-revert
|
|
440
|
+
to `lastGreenCommit` before proceeding. This prevents quiet drift where a
|
|
441
|
+
series of "small simplifications" erodes the GREEN state unnoticed. Reset
|
|
442
|
+
`consecutiveYellowKept` to 0 after the force-revert.
|
|
443
|
+
|
|
444
|
+
If reverted:
|
|
445
|
+
1. Revert per C.3.
|
|
446
|
+
2. Record `autoDecision: "yellow"`, `reverted: true`.
|
|
447
|
+
|
|
448
|
+
### C.3 Revert mechanism
|
|
449
|
+
|
|
450
|
+
Reverts are **file-scoped**, never `git reset --hard`, never `git clean -f`.
|
|
451
|
+
|
|
452
|
+
1. Read `results.json.lastGreenCommit` — the target state.
|
|
453
|
+
2. Compute the set of mutable paths modified since `lastGreenCommit`:
|
|
454
|
+
```bash
|
|
455
|
+
git diff --name-only <lastGreenCommit>..HEAD -- <mutable_path_1> <mutable_path_2> ...
|
|
456
|
+
```
|
|
457
|
+
3. For each path in that set, run:
|
|
458
|
+
```bash
|
|
459
|
+
git checkout <lastGreenCommit> -- <path>
|
|
460
|
+
```
|
|
461
|
+
4. Defensive scope check: if any file outside the mutable list shows as dirty
|
|
462
|
+
after the revert, **halt the loop with a safety-rail stop** (Section G.1 —
|
|
463
|
+
scope violation during the iteration). Do not auto-clean. Alert the user.
|
|
464
|
+
5. Create a revert commit to keep history linear and the next iteration's
|
|
465
|
+
`git add` clean:
|
|
466
|
+
```bash
|
|
467
|
+
git commit -m "research: revert iter <N> (<red|yellow>) — <metric> <before> -> <after>"
|
|
468
|
+
git tag -a "research/<project>/<N>-revert" -m "Reverted iteration <N>: <reason>"
|
|
469
|
+
```
|
|
470
|
+
6. Record `reverted: true` in the iteration's `results.json` entry.
|
|
471
|
+
|
|
472
|
+
### C.4 Retroactive revert (reviewer-initiated)
|
|
473
|
+
|
|
474
|
+
If a reviewer consultation at iteration N returns verdict `RETRO_REVERT` with a
|
|
475
|
+
target iteration M < N (e.g., reviewer identifies that iteration M's GREEN was
|
|
476
|
+
based on data leakage or a methodology flaw), execute:
|
|
477
|
+
|
|
478
|
+
1. Mark `results.json.experiments[M].retroInvalidated = true` and set
|
|
479
|
+
`retroInvalidationReason` to the reviewer's explanation.
|
|
480
|
+
2. **Prompt the user via a micro-gate** — retro-reverts are destructive enough
|
|
481
|
+
to warrant explicit confirmation even in overnight mode:
|
|
482
|
+
```
|
|
483
|
+
::GATE:: id=specific-instructions-shared-autonomous-research-retro-<project>-<N> phase=2 kind=confirm
|
|
484
|
+
Reviewer flagged iteration M's GREEN as invalid (<reason>). Rolling back to
|
|
485
|
+
iteration M-1's state will discard iterations M..<current>. Confirm?
|
|
486
|
+
::ENDGATE::
|
|
487
|
+
```
|
|
488
|
+
3. On confirm: revert all commits after iteration M-1's `lastGreenCommit`
|
|
489
|
+
using the file-level checkout protocol in C.3, expanded to cover the full
|
|
490
|
+
range (M..HEAD).
|
|
491
|
+
4. Update `results.json.lastGreenCommit` to iteration M-1's commit SHA.
|
|
492
|
+
5. Resume the loop from the reverted baseline. **Iteration counter keeps
|
|
493
|
+
incrementing** — no re-use of numbers. The next iteration is N+1, not M+1.
|
|
494
|
+
|
|
495
|
+
### C.5 Per-iteration validation (Fixer-level)
|
|
496
|
+
|
|
497
|
+
Each **kept** iteration (GREEN or kept-YELLOW) must produce a lightweight validation block following `shared/validation_protocol.md`. Reverted iterations (RED or reverted-YELLOW) do not — the change is gone, validation is moot.
|
|
498
|
+
|
|
499
|
+
The block is **not** written to `project-specs.md`. Instead, it is added to the iteration's entry in `experiments/results.json` under a new `validation` field:
|
|
500
|
+
|
|
501
|
+
```json
|
|
502
|
+
{
|
|
503
|
+
"iteration": 7,
|
|
504
|
+
"autoDecision": "green",
|
|
505
|
+
"reverted": false,
|
|
506
|
+
"metrics": { ... },
|
|
507
|
+
"validation": {
|
|
508
|
+
"track": "quick",
|
|
509
|
+
"mode": "experiment",
|
|
510
|
+
"checklist": "<agent_name>/validation_checklist.md",
|
|
511
|
+
"checks": [
|
|
512
|
+
{ "check": "<id>", "observed": "<value>", "passFail": "✓", "notes": "" }
|
|
513
|
+
],
|
|
514
|
+
"artifacts": ["results/iter7_metrics.json"],
|
|
515
|
+
"summary": "<one sentence: what this iteration changed, what the smoke-check confirmed>"
|
|
516
|
+
}
|
|
517
|
+
}
|
|
518
|
+
```
|
|
519
|
+
|
|
520
|
+
Constraints:
|
|
521
|
+
- Use the Fixer-level subset from the relevant checklist (e.g., ML `ML-12 + ML-06 diff`; AI `AI-02 + AI-03 diff`; DS `DS-11 + DS-12`).
|
|
522
|
+
- Minimum: headline metric value, at-most-one component test or smoke-check result, one-sentence summary.
|
|
523
|
+
- No `## Validation` section is written to `project-specs.md` during the AR loop — the Phase 3 research summary (Section I) consolidates validation across kept iterations.
|
|
524
|
+
|
|
525
|
+
The gate hook does **not** enforce this per-iteration block because AR iterations do not emit gate fences. Discipline is on the AR loop itself; the Phase 3 consolidation is where structural enforcement resumes.
|
|
526
|
+
|
|
527
|
+
---
|
|
528
|
+
|
|
529
|
+
## Section D — Reviewer Consultation Cadence
|
|
530
|
+
|
|
531
|
+
### D.1 When to consult
|
|
532
|
+
|
|
533
|
+
Always consult the reviewer on:
|
|
534
|
+
1. **First iteration** — establish methodology sanity early.
|
|
535
|
+
2. **Every K-th iteration** where K is the configured cadence (default: 3
|
|
536
|
+
interactive, 10 overnight).
|
|
537
|
+
3. **After notable improvements** where the delta exceeds T% (default T=5% of
|
|
538
|
+
baseline).
|
|
539
|
+
4. **Before stopping** on consecutive regression limit (Section E.5) — reviewer
|
|
540
|
+
may save the run.
|
|
541
|
+
5. **When Steering Notes change** — a user-steered pivot benefits from a
|
|
542
|
+
reviewer checkpoint.
|
|
543
|
+
|
|
544
|
+
Between consultations, record a one-line self-assessment in the iteration file
|
|
545
|
+
(the `## Self-Assessment` section). Honest: does this result feel real?
|
|
546
|
+
Methodology concerns? Don't wait for the reviewer to notice what you already
|
|
547
|
+
suspect.
|
|
548
|
+
|
|
549
|
+
### D.2 Reviewer verdict schema
|
|
550
|
+
|
|
551
|
+
Reviewers return one of four verdicts (in addition to the standard
|
|
552
|
+
APPROVED/NEEDS REVISION/BLOCKED structure from `reviewer_verdict_protocol.md`):
|
|
553
|
+
|
|
554
|
+
- **`CONTINUE`** — proceed as planned. Reviewer may include optional
|
|
555
|
+
suggestions for the next hypothesis.
|
|
556
|
+
- **`REDIRECT`** — proceed, but pivot approach. Reviewer supplies a new
|
|
557
|
+
hypothesis direction. Record `hypothesisSource: "reviewer-suggested"` on the
|
|
558
|
+
next iteration.
|
|
559
|
+
- **`PAUSE`** — stop the loop and escalate to the user. Reviewer supplies the
|
|
560
|
+
concern. The loop halts immediately and Phase 3 begins.
|
|
561
|
+
- **`RETRO_REVERT`** — a prior GREEN is invalid. Reviewer specifies the target
|
|
562
|
+
iteration. Execute Section C.4.
|
|
563
|
+
|
|
564
|
+
### D.3 Dual-reviewer agents
|
|
565
|
+
|
|
566
|
+
Applied ML Scientist (DL Engineer + Researcher) and Deep Learning Engineer
|
|
567
|
+
(Applied ML Scientist + Researcher) have two reviewers. Consult them
|
|
568
|
+
**sequentially**, not in parallel:
|
|
569
|
+
|
|
570
|
+
1. **Technical reviewer first** (DL Engineer for Applied ML Scientist; Applied
|
|
571
|
+
ML Scientist for Deep Learning Engineer) — architecture, implementation,
|
|
572
|
+
feasibility.
|
|
573
|
+
2. **Methodology reviewer second** (Researcher) — gets the technical
|
|
574
|
+
reviewer's verdict as input so they can comment on the combined picture.
|
|
575
|
+
|
|
576
|
+
Dual-reviewer cost is **2× per cadence hit** and must be reflected in the cost
|
|
577
|
+
ceiling (Section G.5). If cost ceiling pressure is high, consider raising K
|
|
578
|
+
rather than dropping one reviewer — both perspectives matter.
|
|
579
|
+
|
|
580
|
+
### D.4 Consultation call format
|
|
581
|
+
|
|
582
|
+
Call the reviewer via Task:
|
|
583
|
+
|
|
584
|
+
```
|
|
585
|
+
Task(
|
|
586
|
+
subagent_type="<reviewer>",
|
|
587
|
+
description="AR iteration N review",
|
|
588
|
+
prompt="""
|
|
589
|
+
You are being consulted mid-AR-loop to review this iteration's result and the
|
|
590
|
+
run's trajectory so far.
|
|
591
|
+
|
|
592
|
+
**Project context:** <1-paragraph summary from project-specs.md>
|
|
593
|
+
**Primary metric:** <metric name>
|
|
594
|
+
**Baseline:** <value>
|
|
595
|
+
**Current iteration:** N of <budget>
|
|
596
|
+
**Hypothesis:** <this iteration's hypothesis>
|
|
597
|
+
**Changes made:** <what was touched>
|
|
598
|
+
|
|
599
|
+
**Metrics (before -> after):**
|
|
600
|
+
| Metric | Before | After | Delta |
|
|
601
|
+
|--------|--------|-------|-------|
|
|
602
|
+
<rows>
|
|
603
|
+
|
|
604
|
+
**Auto-decision:** <green | red | yellow>
|
|
605
|
+
**Trajectory so far:** <one-paragraph summary of accumulated results — best
|
|
606
|
+
iteration, average delta, any patterns>
|
|
607
|
+
|
|
608
|
+
Please respond with:
|
|
609
|
+
1. **Verdict:** CONTINUE | REDIRECT | PAUSE | RETRO_REVERT
|
|
610
|
+
2. **Reasoning:** why
|
|
611
|
+
3. **If REDIRECT:** suggested next hypothesis
|
|
612
|
+
4. **If RETRO_REVERT:** which iteration number is invalid and why
|
|
613
|
+
5. **If PAUSE:** what concern the user needs to see
|
|
614
|
+
|
|
615
|
+
Keep it concise and actionable.
|
|
616
|
+
"""
|
|
617
|
+
)
|
|
618
|
+
```
|
|
619
|
+
|
|
620
|
+
Apply the standard reviewer verdict protocol afterward (`reviewer_verdict_protocol.md`).
|
|
621
|
+
|
|
622
|
+
---
|
|
623
|
+
|
|
624
|
+
## Section E — Convergence Detection
|
|
625
|
+
|
|
626
|
+
Convergence checks **do not fire until after an activation floor** of
|
|
627
|
+
`max(10, budget * 0.2)` iterations. Early in a hard problem it is normal to see
|
|
628
|
+
5+ consecutive REDs before the first GREEN — firing PLATEAU at iteration 5 is
|
|
629
|
+
a false positive. The floor prevents premature convergence.
|
|
630
|
+
|
|
631
|
+
**Exception:** METRIC FLOOR BREACH (E.6) is always live — emergency stops do
|
|
632
|
+
not wait.
|
|
633
|
+
|
|
634
|
+
Stop when ANY of the following is true:
|
|
635
|
+
|
|
636
|
+
### E.1 PLATEAU
|
|
637
|
+
|
|
638
|
+
No GREEN result in W consecutive iterations (default W=5) **AND** DIMINISHING
|
|
639
|
+
RETURNS (E.2) is also true. **Require both signals** — single-signal PLATEAU
|
|
640
|
+
produces too many false positives on hard problems.
|
|
641
|
+
|
|
642
|
+
### E.2 DIMINISHING RETURNS
|
|
643
|
+
|
|
644
|
+
Average improvement rate over the last W GREEN results is less than threshold
|
|
645
|
+
(default: 0.1% of baseline per GREEN iteration). Measured as
|
|
646
|
+
`mean(delta_over_last_W_greens) < threshold`.
|
|
647
|
+
|
|
648
|
+
### E.3 BUDGET EXHAUSTED
|
|
649
|
+
|
|
650
|
+
Iteration count reaches the configured budget.
|
|
651
|
+
|
|
652
|
+
### E.4 COST CEILING
|
|
653
|
+
|
|
654
|
+
Token count or dollars-spent reaches the ceiling from Section A.3. Hard stop —
|
|
655
|
+
no further iterations. See Section G.5 for the warning-and-stop schedule.
|
|
656
|
+
|
|
657
|
+
### E.5 CONSECUTIVE FAILURES
|
|
658
|
+
|
|
659
|
+
`consecutiveRegressions` reaches K (default K=3). Before stopping, consult the
|
|
660
|
+
reviewer per Section D.1 item 4 — the reviewer may return `REDIRECT` and save
|
|
661
|
+
the run.
|
|
662
|
+
|
|
663
|
+
### E.6 METRIC FLOOR BREACH
|
|
664
|
+
|
|
665
|
+
Primary metric falls below the configured floor (if set) on a **full
|
|
666
|
+
evaluation**. Always live — ignores the activation floor. Immediately:
|
|
667
|
+
1. Revert per C.3.
|
|
668
|
+
2. Halt the loop.
|
|
669
|
+
3. Record the breach in `results.json.convergence.reason` as `metric-floor-breach`.
|
|
670
|
+
|
|
671
|
+
A proxy evaluation below floor does NOT trigger E.6 directly. Instead:
|
|
672
|
+
1. Automatically re-run a full evaluation on the same iteration.
|
|
673
|
+
2. If the full evaluation is also below floor, trigger E.6.
|
|
674
|
+
3. If the full evaluation is above floor, the proxy was noisy; continue
|
|
675
|
+
normally but mark this iteration for future attention.
|
|
676
|
+
|
|
677
|
+
### E.7 Convergence record
|
|
678
|
+
|
|
679
|
+
When any stop condition fires (Section E) **or any safety rail halts the loop
|
|
680
|
+
(Section G)**, write to `results.json.convergence`:
|
|
681
|
+
|
|
682
|
+
```json
|
|
683
|
+
"convergence": {
|
|
684
|
+
"detected": true,
|
|
685
|
+
"reason": "plateau | diminishing-returns | budget-exhausted | cost-ceiling | consecutive-failures | metric-floor-breach | user-interrupt | reviewer-pause | scope-violation | error-limit | timeout-limit",
|
|
686
|
+
"iterationsSinceLastGreen": <N>,
|
|
687
|
+
"consecutiveRegressions": <N>,
|
|
688
|
+
"consecutiveYellowKept": <N>,
|
|
689
|
+
"avgImprovementRate": <number>,
|
|
690
|
+
"activationFloorReached": <bool>,
|
|
691
|
+
"finalIteration": <N>
|
|
692
|
+
}
|
|
693
|
+
```
|
|
694
|
+
|
|
695
|
+
The reason enum:
|
|
696
|
+
- Section E stops: `plateau`, `diminishing-returns`, `budget-exhausted`,
|
|
697
|
+
`cost-ceiling`, `consecutive-failures`, `metric-floor-breach`,
|
|
698
|
+
`user-interrupt`, `reviewer-pause`.
|
|
699
|
+
- Section G safety-rail halts: `scope-violation` (G.1), `error-limit` (G.3 —
|
|
700
|
+
three consecutive crashes), `timeout-limit` (G.4 — three consecutive
|
|
701
|
+
timeouts which itself triggers G.3's error-limit path — record
|
|
702
|
+
`timeout-limit` as the more specific reason).
|
|
703
|
+
|
|
704
|
+
---
|
|
705
|
+
|
|
706
|
+
## Section F — results.json Schema Extensions
|
|
707
|
+
|
|
708
|
+
The AR schema extends the `[EX]` results.json schema. All existing fields
|
|
709
|
+
(`projectName`, `agent`, `outcomeMetric`, `baseline`, `experiments[]`,
|
|
710
|
+
`versioningMode`, `checkpoint`, etc.) remain the same. AR adds these fields.
|
|
711
|
+
|
|
712
|
+
### F.1 Top-level additive fields
|
|
713
|
+
|
|
714
|
+
```json
|
|
715
|
+
{
|
|
716
|
+
"mode": "autonomous-research",
|
|
717
|
+
"preset": "interactive" | "overnight" | "custom",
|
|
718
|
+
"steeringDocument": "experiments/research_brief.md",
|
|
719
|
+
"constraints": {
|
|
720
|
+
"iterationBudget": <N>,
|
|
721
|
+
"perIterationTimeLimit": "<duration or null>",
|
|
722
|
+
"maxConsecutiveRegressions": <N>,
|
|
723
|
+
"metricDegradationFloor": <number or null>,
|
|
724
|
+
"reviewerCadence": <N>,
|
|
725
|
+
"plateauWindow": <N>,
|
|
726
|
+
"diminishingReturnsThreshold": <number>,
|
|
727
|
+
"epsilon": <number>,
|
|
728
|
+
"costCeiling": {
|
|
729
|
+
"tokens": <N or null>,
|
|
730
|
+
"dollars": <number or null>
|
|
731
|
+
},
|
|
732
|
+
"fullEvalCadence": <N>,
|
|
733
|
+
"microGateEveryK": <N or null>,
|
|
734
|
+
"microGateAutoCloseSeconds": <N or null>
|
|
735
|
+
},
|
|
736
|
+
"convergence": {
|
|
737
|
+
"detected": <bool>,
|
|
738
|
+
"reason": "<stop reason or null>",
|
|
739
|
+
"iterationsSinceLastGreen": <N>,
|
|
740
|
+
"consecutiveRegressions": <N>,
|
|
741
|
+
"consecutiveYellowKept": <N>,
|
|
742
|
+
"avgImprovementRate": <number>,
|
|
743
|
+
"activationFloorReached": <bool>,
|
|
744
|
+
"finalIteration": <N or null>
|
|
745
|
+
},
|
|
746
|
+
"lastGreenCommit": "<sha or null>",
|
|
747
|
+
"costAccounting": {
|
|
748
|
+
"tokensIn": <N>,
|
|
749
|
+
"tokensOut": <N>,
|
|
750
|
+
"dollarsSpent": <number>,
|
|
751
|
+
"reviewerTasksSpawned": <N>
|
|
752
|
+
}
|
|
753
|
+
}
|
|
754
|
+
```
|
|
755
|
+
|
|
756
|
+
### F.2 Per-experiment additive fields
|
|
757
|
+
|
|
758
|
+
Each entry in `experiments[]` gains:
|
|
759
|
+
|
|
760
|
+
```json
|
|
761
|
+
{
|
|
762
|
+
"autoDecision": "green" | "red" | "yellow",
|
|
763
|
+
"reverted": <bool>,
|
|
764
|
+
"retroInvalidated": <bool>,
|
|
765
|
+
"retroInvalidationReason": "<string or null>",
|
|
766
|
+
"hypothesisSource": "adaptive" | "steering" | "reviewer-suggested",
|
|
767
|
+
"evalType": "full" | "proxy",
|
|
768
|
+
"reviewerVerdict": "CONTINUE" | "REDIRECT" | "PAUSE" | "RETRO_REVERT" | null,
|
|
769
|
+
"nextSteps": "<string, populated for kept YELLOWs>"
|
|
770
|
+
}
|
|
771
|
+
```
|
|
772
|
+
|
|
773
|
+
### F.3 Optional branch context (fan-out only)
|
|
774
|
+
|
|
775
|
+
Populated only in branches spawned by the Section H fan-out protocol:
|
|
776
|
+
|
|
777
|
+
```json
|
|
778
|
+
"branchContext": {
|
|
779
|
+
"branchSlug": "<slug>",
|
|
780
|
+
"approachConstraint": "<one-line description>",
|
|
781
|
+
"parentProjectDir": "<path>",
|
|
782
|
+
"gitStrategy": "branch-local" | "lockfile" | "no-vcs"
|
|
783
|
+
}
|
|
784
|
+
```
|
|
785
|
+
|
|
786
|
+
### F.4 Backward compatibility
|
|
787
|
+
|
|
788
|
+
- An `[EX]` `results.json` has `mode: "experiment"` (or no `mode` field in
|
|
789
|
+
pre-AR installations — treat absence as `"experiment"`).
|
|
790
|
+
- An AR `results.json` has `mode: "autonomous-research"`.
|
|
791
|
+
- The UI dashboard must detect `mode` and render accordingly (AR-specific
|
|
792
|
+
panel) OR gracefully degrade to the experiment view if no AR renderer is
|
|
793
|
+
available.
|
|
794
|
+
|
|
795
|
+
### F.5 UI compatibility
|
|
796
|
+
|
|
797
|
+
The Shards UI dashboard reads `results.json` for live updates. Before shipping
|
|
798
|
+
AR, verify that the dashboard renderers tolerate `mode: "autonomous-research"`.
|
|
799
|
+
The dashboard should:
|
|
800
|
+
- Show iteration budget (not just plannedCount).
|
|
801
|
+
- Show per-iteration color coding (green/red/yellow).
|
|
802
|
+
- Show `lastGreenCommit` and revert history.
|
|
803
|
+
- Show cost accounting running totals.
|
|
804
|
+
- Surface convergence reason when the run ends.
|
|
805
|
+
|
|
806
|
+
If no AR-specific renderer is available, the experiment dashboard must render
|
|
807
|
+
AR runs by falling back to `iterationBudget` where `plannedCount` is absent,
|
|
808
|
+
and displaying auto-decision colors in the outcome column.
|
|
809
|
+
|
|
810
|
+
---
|
|
811
|
+
|
|
812
|
+
## Section G — Safety Rails
|
|
813
|
+
|
|
814
|
+
### G.1 Scope enforcement
|
|
815
|
+
|
|
816
|
+
Every file-modifying tool call in Phase 2 (`Edit`, `Write`, `NotebookEdit`)
|
|
817
|
+
verifies the target path is in the mutable set from the research brief. If a
|
|
818
|
+
path is not mutable:
|
|
819
|
+
1. Halt the iteration immediately.
|
|
820
|
+
2. Record `results.json.convergence.reason = "scope-violation"`.
|
|
821
|
+
3. Alert the user with the attempted path and the reason.
|
|
822
|
+
|
|
823
|
+
Scope is **path-based, not content-based**. Globs in the mutable list are
|
|
824
|
+
supported (e.g., `training/**/*.py`). An immutable-by-default policy applies:
|
|
825
|
+
anything not explicitly mutable is immutable.
|
|
826
|
+
|
|
827
|
+
**Bash commands** that modify files (e.g., `rm`, `mv`, `cp` into immutable
|
|
828
|
+
paths, `sed -i`, tooling that regenerates configs) also fall under this rule.
|
|
829
|
+
If an iteration's plan requires a shell command that touches immutable paths,
|
|
830
|
+
halt before running it and escalate to the user. The safe practice is to
|
|
831
|
+
avoid file-modifying shell commands in the loop entirely — use `Edit` / `Write`
|
|
832
|
+
/ `NotebookEdit` which are checkable ahead of time.
|
|
833
|
+
|
|
834
|
+
### G.2 Metric floor breach
|
|
835
|
+
|
|
836
|
+
Covered in E.6. Full-eval-only trigger; proxy breaches force a full re-eval.
|
|
837
|
+
|
|
838
|
+
### G.3 Error limit
|
|
839
|
+
|
|
840
|
+
3 consecutive iteration crashes (uncaught exception during implementation or
|
|
841
|
+
evaluation) → stop. The codebase may be broken. Record
|
|
842
|
+
`convergence.reason = "error-limit"`. Revert to `lastGreenCommit`.
|
|
843
|
+
|
|
844
|
+
### G.4 Time budget
|
|
845
|
+
|
|
846
|
+
Per-iteration timeout. If exceeded: abandon the iteration, revert per C.3,
|
|
847
|
+
record `"timeout"` in the iteration file, and do NOT increment the iteration
|
|
848
|
+
counter for retry — count it as a normal iteration that happened to fail. If
|
|
849
|
+
3 consecutive timeouts occur, trigger G.3 (error limit).
|
|
850
|
+
|
|
851
|
+
### G.5 Cost ceiling
|
|
852
|
+
|
|
853
|
+
Hard stop at 100% of the ceiling from Section A.3. **Soft warnings** at 50%
|
|
854
|
+
and 80% printed inline:
|
|
855
|
+
|
|
856
|
+
```
|
|
857
|
+
[AR] Cost warning: 50% of ceiling reached (<metric>: <spent> / <ceiling>).
|
|
858
|
+
Continuing. You can edit research_brief.md to adjust the ceiling.
|
|
859
|
+
```
|
|
860
|
+
|
|
861
|
+
Reviewer Tasks and proxy evaluations **both count against the ceiling**.
|
|
862
|
+
|
|
863
|
+
Required for `overnight` preset; optional for `interactive` (warn but do not
|
|
864
|
+
force if user chose to omit).
|
|
865
|
+
|
|
866
|
+
### G.6 No destructive operations
|
|
867
|
+
|
|
868
|
+
**Never:**
|
|
869
|
+
- Delete data files (training data, eval sets, raw inputs).
|
|
870
|
+
- Drop database tables or views.
|
|
871
|
+
- Remove checkpoints, model artifacts, or logs.
|
|
872
|
+
- Run `git reset --hard`, `git clean -f`, or `git push --force`.
|
|
873
|
+
- Remove git tags or refs.
|
|
874
|
+
|
|
875
|
+
File-level `git checkout <sha> -- <path>` is the only allowed reverting operation.
|
|
876
|
+
|
|
877
|
+
### G.7 Git safety
|
|
878
|
+
|
|
879
|
+
- File-scoped checkout only.
|
|
880
|
+
- Never hard reset.
|
|
881
|
+
- Never force push.
|
|
882
|
+
- Never `git clean -f` or `git clean -fd`.
|
|
883
|
+
- Tag creation is append-only — never delete or force-overwrite tags.
|
|
884
|
+
|
|
885
|
+
### G.8 Interrupt handling
|
|
886
|
+
|
|
887
|
+
If the agent detects it was interrupted mid-iteration (on resume, finds a
|
|
888
|
+
commit without a matching `results.json` entry, or vice versa):
|
|
889
|
+
1. Record the partial state as a dropped iteration in the research log.
|
|
890
|
+
2. Resume from the last clean `lastGreenCommit`.
|
|
891
|
+
3. Do not auto-clean any orphan files. Present the situation to the user
|
|
892
|
+
before continuing.
|
|
893
|
+
|
|
894
|
+
---
|
|
895
|
+
|
|
896
|
+
## Section H — Fan-Out Composition with DIVERGE
|
|
897
|
+
|
|
898
|
+
This section defines how AR composes with the DIVERGE protocol to support
|
|
899
|
+
parallel exploration of multiple approach families. Solo AR remains the default
|
|
900
|
+
and is the depth primitive. Fan-out is the breadth primitive.
|
|
901
|
+
|
|
902
|
+
### H.1 When to propose fan-out
|
|
903
|
+
|
|
904
|
+
Fan-out can be initiated by either:
|
|
905
|
+
|
|
906
|
+
- **The specialist** at its `[AR]` Phase 1 gate, when writing the research
|
|
907
|
+
brief surfaces 2-3 viable approach families that satisfy DIVERGE
|
|
908
|
+
preconditions (see `diverge_protocol.md` Section A).
|
|
909
|
+
- **Syn** during triage or brainstorm, when the user's request is broad
|
|
910
|
+
("improve metric X, try everything") and Syn identifies distinct approach
|
|
911
|
+
families warranting parallel exploration, possibly across different
|
|
912
|
+
specialist types.
|
|
913
|
+
|
|
914
|
+
Fan-out preconditions (inherited from DIVERGE):
|
|
915
|
+
- 2-3 mutually exclusive approaches that are genuinely viable.
|
|
916
|
+
- No single approach is clearly superior.
|
|
917
|
+
- The approaches are fundamentally different (not hyperparameter variations —
|
|
918
|
+
that is what the AR loop itself already does).
|
|
919
|
+
|
|
920
|
+
### H.2 DIVERGE proposal at AR Phase 1
|
|
921
|
+
|
|
922
|
+
If fan-out is warranted, propose DIVERGE before the research brief gate.
|
|
923
|
+
Follow `diverge_protocol.md` Section B (proposal format + gate), with these
|
|
924
|
+
AR-specific adjustments:
|
|
925
|
+
|
|
926
|
+
- **Gate ID namespace:** use
|
|
927
|
+
`specific-instructions-shared-diverge-protocol-ar-<project>` for the AR
|
|
928
|
+
proposal gate. This avoids collision with planning-phase DIVERGE gate IDs.
|
|
929
|
+
- **Branch slugs:** reflect approach families, not specialists. Example slugs
|
|
930
|
+
for an ML Engineer fan-out: `ml-xgboost`, `ml-neural-net`, `ml-linear-baseline`.
|
|
931
|
+
- **Proposal context:** include the AR budget, primary metric, and per-branch
|
|
932
|
+
budget split (each branch gets the full budget; the total AR budget is
|
|
933
|
+
effectively multiplied by K branches — confirm with the user that cost
|
|
934
|
+
implications are accepted).
|
|
935
|
+
|
|
936
|
+
The user confirms one of:
|
|
937
|
+
- (a) Solo AR with the current brief (no fan-out).
|
|
938
|
+
- (b) Fan-out with the proposed K approach families.
|
|
939
|
+
|
|
940
|
+
### H.3 Branch spawning
|
|
941
|
+
|
|
942
|
+
If fan-out confirmed, spawn K parallel Task calls in a single message (not
|
|
943
|
+
sequentially). Each Task is:
|
|
944
|
+
|
|
945
|
+
```python
|
|
946
|
+
Task(
|
|
947
|
+
subagent_type="<specialist>",
|
|
948
|
+
description="AR fan-out branch: <branch-slug>",
|
|
949
|
+
prompt="""
|
|
950
|
+
You are in BRANCH + AR MODE — an isolated Time-Travel branch running the
|
|
951
|
+
autonomous research protocol.
|
|
952
|
+
|
|
953
|
+
**Branch slug:** <branch-slug>
|
|
954
|
+
**Branch directory:** <project_dir>/.shards/branches/<branch-slug>/
|
|
955
|
+
**Approach constraint:** <one-paragraph description of the approach family
|
|
956
|
+
this branch must stay within — e.g., "tree-based methods only; no neural
|
|
957
|
+
networks; no linear models">
|
|
958
|
+
**Git strategy:** <branch-local | lockfile | no-vcs — see Section H.5>
|
|
959
|
+
|
|
960
|
+
## Project Context (from completed planning phases)
|
|
961
|
+
|
|
962
|
+
<Insert full text of completed project-specs.md phases — Phase 0 through any
|
|
963
|
+
prior planning phase. Do NOT include the DIVERGE section or brainstorm
|
|
964
|
+
transcripts.>
|
|
965
|
+
|
|
966
|
+
## AR Configuration (inherited from parent's Phase 0)
|
|
967
|
+
|
|
968
|
+
- **Primary metric:** <metric> (direction: <max | min>)
|
|
969
|
+
- **Baseline:** <value>
|
|
970
|
+
- **Preset:** <interactive | overnight | custom>
|
|
971
|
+
- **Iteration budget (this branch):** <N>
|
|
972
|
+
- **Mutable scope:** <list>
|
|
973
|
+
- **Immutable scope:** <list>
|
|
974
|
+
- **Cost ceiling (this branch):** <value or null>
|
|
975
|
+
- **Reviewer cadence:** <K>
|
|
976
|
+
- **Steering documents:**
|
|
977
|
+
- Global: `<project_dir>/experiments/research_brief.md`
|
|
978
|
+
- This branch (optional): `<branch_dir>/experiments/research_brief.md`
|
|
979
|
+
|
|
980
|
+
## Your Task
|
|
981
|
+
|
|
982
|
+
Execute Sections A-G of the shared AR protocol autonomously inside this branch
|
|
983
|
+
directory. **Do not emit `::GATE::` fences** — sub-agents invoked via Task must
|
|
984
|
+
not emit gates (harness rule). Phase 0 is already set up by this spawn prompt.
|
|
985
|
+
Phase 1 writes the branch-local research_brief.md without a gate. Phase 2 runs
|
|
986
|
+
gate-free as designed. Phase 3 is truncated to writing `branch-report.md` only
|
|
987
|
+
(no user-facing summary gate).
|
|
988
|
+
|
|
989
|
+
Produce:
|
|
990
|
+
1. Branch-local AR artifacts: `experiments/research_brief.md`,
|
|
991
|
+
`experiments/results.json` (with `branchContext` populated), per-iteration
|
|
992
|
+
files.
|
|
993
|
+
2. A final `<branch_dir>/branch-report.md` summarizing the branch's best state
|
|
994
|
+
per the DIVERGE branch-report template (`diverge_protocol.md` Section E).
|
|
995
|
+
3. Git commits on the chosen git strategy (H.5).
|
|
996
|
+
"""
|
|
997
|
+
)
|
|
998
|
+
```
|
|
999
|
+
|
|
1000
|
+
### H.4 Per-branch file layout
|
|
1001
|
+
|
|
1002
|
+
```
|
|
1003
|
+
<project_dir>/.shards/branches/<branch-slug>/
|
|
1004
|
+
experiments/
|
|
1005
|
+
research_brief.md # per-branch steering (optional override of global)
|
|
1006
|
+
results.json # per-branch AR results with branchContext set
|
|
1007
|
+
history_summary.md # per-branch windowed history (if N > 10)
|
|
1008
|
+
experiment_1_<name>.md
|
|
1009
|
+
experiment_2_<name>.md
|
|
1010
|
+
...
|
|
1011
|
+
branch-report.md # final summary for arbiter
|
|
1012
|
+
```
|
|
1013
|
+
|
|
1014
|
+
The main project dir keeps:
|
|
1015
|
+
- The global `experiments/research_brief.md` (applies to all branches unless
|
|
1016
|
+
overridden at the branch level).
|
|
1017
|
+
- The canonical `project-specs.md` with the DIVERGE section written per
|
|
1018
|
+
`diverge_protocol.md` Section D.
|
|
1019
|
+
|
|
1020
|
+
### H.5 Concurrent git strategy
|
|
1021
|
+
|
|
1022
|
+
When multiple branches run in parallel, all committing to the same repo, naive
|
|
1023
|
+
`experiment_versioning.md` Section B calls will race on the index lock, collide
|
|
1024
|
+
on tag names, and interleave history. Each branch MUST pick one strategy at
|
|
1025
|
+
spawn time and record it in `results.json.branchContext.gitStrategy`:
|
|
1026
|
+
|
|
1027
|
+
**`branch-local` (default for AR fan-out):**
|
|
1028
|
+
- Before Phase 2, branch runs `git checkout -b ar/<branch-slug>`.
|
|
1029
|
+
- All commits on its own git branch. No cross-branch collisions.
|
|
1030
|
+
- Tag namespace: `research/<project>/<branch-slug>/<N>-<name>`.
|
|
1031
|
+
- At promotion (Section H.9), `git merge --squash ar/<winner>` onto the main
|
|
1032
|
+
working branch. Losing branches remain as refs for reference — deletion
|
|
1033
|
+
requires explicit user confirmation.
|
|
1034
|
+
|
|
1035
|
+
**`lockfile`:**
|
|
1036
|
+
- Branches share the main working ref.
|
|
1037
|
+
- All git operations serialize via a file lock at
|
|
1038
|
+
`.shards/branches/.git-lock` (create with `O_EXCL`, retry-with-backoff on
|
|
1039
|
+
contention — default 5 retries, 100ms-1s exponential backoff).
|
|
1040
|
+
- Acceptable for K ≤ 2; degrades under contention.
|
|
1041
|
+
- Tag namespace: `research/<project>/<branch-slug>/<N>-<name>` (includes slug
|
|
1042
|
+
to prevent collisions even on shared ref).
|
|
1043
|
+
|
|
1044
|
+
**`no-vcs`:**
|
|
1045
|
+
- Branches disable Section B checkpoint calls entirely.
|
|
1046
|
+
- Lineage relies solely on per-iteration markdown + `results.json`.
|
|
1047
|
+
- A single consolidation commit is made at promotion time.
|
|
1048
|
+
- Fallback only — use when git is unavailable or intentionally disabled.
|
|
1049
|
+
|
|
1050
|
+
Solo AR uses the main working ref directly — no branch-local ref required.
|
|
1051
|
+
|
|
1052
|
+
### H.6 Steering in fan-out
|
|
1053
|
+
|
|
1054
|
+
Every iteration, each branch re-reads:
|
|
1055
|
+
1. The **global** `research_brief.md` at the project root (applies to all
|
|
1056
|
+
branches).
|
|
1057
|
+
2. Its **branch-level** steering notes at
|
|
1058
|
+
`<branch_dir>/experiments/research_brief.md` if present.
|
|
1059
|
+
|
|
1060
|
+
The user can edit either during the run:
|
|
1061
|
+
- Edit the global brief to steer all branches simultaneously.
|
|
1062
|
+
- Edit a branch-level brief to redirect one branch without affecting others.
|
|
1063
|
+
|
|
1064
|
+
### H.7 Branch convergence
|
|
1065
|
+
|
|
1066
|
+
Each branch runs Section E convergence checks independently inside its own AR
|
|
1067
|
+
loop. A branch terminates when it converges, exhausts its budget, or breaches
|
|
1068
|
+
a safety rail. **Branches do not wait for each other.** The parent specialist
|
|
1069
|
+
(or Syn) waits for all branch Tasks to return before arbitration.
|
|
1070
|
+
|
|
1071
|
+
### H.8 Branch gate policy
|
|
1072
|
+
|
|
1073
|
+
Branches are Task-invoked sub-agents. Per `syn/arbiter.md` and the harness
|
|
1074
|
+
sub-agent rule, sub-agents **must not emit `::GATE::` fences**. All
|
|
1075
|
+
human-facing gates live in the parent's Phase 3 (arbitration + consolidated
|
|
1076
|
+
summary).
|
|
1077
|
+
|
|
1078
|
+
- Phase 0: inherited from the parent's spawn prompt. No gate.
|
|
1079
|
+
- Phase 1: write the branch-local research_brief.md. No gate.
|
|
1080
|
+
- Phase 2: gate-free loop as designed.
|
|
1081
|
+
- Phase 3: truncated to writing `branch-report.md`. No gate.
|
|
1082
|
+
|
|
1083
|
+
### H.9 Arbitration
|
|
1084
|
+
|
|
1085
|
+
After all branches complete, the initiating specialist (or Syn) invokes Syn
|
|
1086
|
+
Arbiter Mode per `diverge_protocol.md` Section F, passing the list of
|
|
1087
|
+
`branch-report.md` paths. The arbiter produces
|
|
1088
|
+
`.shards/branches/leaderboard.md`.
|
|
1089
|
+
|
|
1090
|
+
The parent specialist presents the leaderboard to the user, who selects the
|
|
1091
|
+
winner.
|
|
1092
|
+
|
|
1093
|
+
### H.10 Knowledge harvest in fan-out
|
|
1094
|
+
|
|
1095
|
+
**Losing branches do NOT run `knowledge_harvest.md` independently.** That would
|
|
1096
|
+
flood the ledger with duplicate or conflicting candidates across branches
|
|
1097
|
+
exploring similar territory.
|
|
1098
|
+
|
|
1099
|
+
- Only the **winning branch** contributes to harvest.
|
|
1100
|
+
- The parent specialist runs harvest as part of its consolidated Phase 3
|
|
1101
|
+
**after promotion**.
|
|
1102
|
+
- Harvest candidates come from:
|
|
1103
|
+
1. The winning branch's artifacts (brief, iteration files, results.json).
|
|
1104
|
+
2. Cross-branch patterns that Syn Arbiter flagged in the leaderboard (e.g.,
|
|
1105
|
+
"three of four branches hit the same data leakage issue" — a pattern
|
|
1106
|
+
worth harvesting even though no individual branch would have flagged it).
|
|
1107
|
+
|
|
1108
|
+
### H.11 Promotion
|
|
1109
|
+
|
|
1110
|
+
Follow `diverge_protocol.md` Section G with these extensions for AR git
|
|
1111
|
+
strategies:
|
|
1112
|
+
|
|
1113
|
+
- **`branch-local`:** `git merge --squash ar/<winner>` onto the main working
|
|
1114
|
+
ref. Copy branch-dir artifacts (brief, results.json, iteration files,
|
|
1115
|
+
history_summary.md) to `<project_dir>/experiments/`. Tag the merge with
|
|
1116
|
+
`research/<project>/converged/<winner>`.
|
|
1117
|
+
- **`lockfile`:** artifacts are already on the main ref under the
|
|
1118
|
+
`<branch-slug>/` tag prefix. Copy branch-dir artifacts to
|
|
1119
|
+
`<project_dir>/experiments/`. Tag the consolidation:
|
|
1120
|
+
`research/<project>/converged/<winner>`.
|
|
1121
|
+
- **`no-vcs`:** make the single consolidation commit now
|
|
1122
|
+
(`research: converge <winner>`). Copy artifacts. Tag
|
|
1123
|
+
`research/<project>/converged/<winner>`.
|
|
1124
|
+
|
|
1125
|
+
Losing branches are preserved under `.shards/branches/` for reference.
|
|
1126
|
+
|
|
1127
|
+
---
|
|
1128
|
+
|
|
1129
|
+
## Section I — Phase 3 (Research Summary)
|
|
1130
|
+
|
|
1131
|
+
Run this section after the loop exits (any E condition, user interrupt, or
|
|
1132
|
+
reviewer PAUSE).
|
|
1133
|
+
|
|
1134
|
+
### I.1 Finalize results.json
|
|
1135
|
+
|
|
1136
|
+
- Set `"status": "complete"`.
|
|
1137
|
+
- Set `convergence.detected = true` and populate the full convergence object.
|
|
1138
|
+
- Set final cost accounting totals.
|
|
1139
|
+
- Set final metric value and net delta from baseline.
|
|
1140
|
+
|
|
1141
|
+
### I.2 Write `experiments/research_summary.md`
|
|
1142
|
+
|
|
1143
|
+
Factual synthesis — no opinions in this file:
|
|
1144
|
+
|
|
1145
|
+
```markdown
|
|
1146
|
+
# AR Research Summary: <Project Name>
|
|
1147
|
+
|
|
1148
|
+
- **Date:** <date>
|
|
1149
|
+
- **Agent:** <agent name>
|
|
1150
|
+
- **Brief:** `experiments/research_brief.md`
|
|
1151
|
+
- **Primary metric:** <metric name>
|
|
1152
|
+
- **Preset:** <interactive | overnight | custom>
|
|
1153
|
+
|
|
1154
|
+
## Baseline vs Final
|
|
1155
|
+
- **Baseline:** <value> (from <source>)
|
|
1156
|
+
- **Final:** <value>
|
|
1157
|
+
- **Net delta:** <+/->
|
|
1158
|
+
- **Target reached:** Yes | No | N/A (no target set)
|
|
1159
|
+
|
|
1160
|
+
## Convergence
|
|
1161
|
+
- **Detected:** Yes | No
|
|
1162
|
+
- **Reason:** <plateau | diminishing-returns | budget-exhausted | cost-ceiling | consecutive-failures | metric-floor-breach | user-interrupt | reviewer-pause | scope-violation | error-limit | timeout-limit>
|
|
1163
|
+
- **Iterations completed:** N of <budget>
|
|
1164
|
+
- **Activation floor reached:** Yes | No
|
|
1165
|
+
|
|
1166
|
+
## Keep/Revert Breakdown
|
|
1167
|
+
| Color | Count | Kept | Reverted |
|
|
1168
|
+
|-------|-------|------|----------|
|
|
1169
|
+
| Green | <N> | <N> | 0 |
|
|
1170
|
+
| Red | <N> | 0 | <N> |
|
|
1171
|
+
| Yellow | <N> | <N> | <N> |
|
|
1172
|
+
|
|
1173
|
+
## Evaluation Type Breakdown
|
|
1174
|
+
- **Full evaluations:** <N>
|
|
1175
|
+
- **Proxy evaluations:** <N>
|
|
1176
|
+
- **Proxy→full re-runs triggered:** <N>
|
|
1177
|
+
|
|
1178
|
+
## Cost Accounting
|
|
1179
|
+
- **Tokens in:** <N>
|
|
1180
|
+
- **Tokens out:** <N>
|
|
1181
|
+
- **Dollars spent:** <value>
|
|
1182
|
+
- **Reviewer Tasks spawned:** <N>
|
|
1183
|
+
- **Cost ceiling:** <value or "none set">
|
|
1184
|
+
|
|
1185
|
+
## Iteration Timeline
|
|
1186
|
+
| # | Hypothesis | Source | Eval | Color | Kept | Delta | Reviewer |
|
|
1187
|
+
|---|-----------|--------|------|-------|------|-------|----------|
|
|
1188
|
+
| 1 | ... | adaptive | full | green | yes | +0.02 | CONTINUE |
|
|
1189
|
+
| 2 | ... | steering | proxy | yellow | yes | 0 | - |
|
|
1190
|
+
| ... |
|
|
1191
|
+
|
|
1192
|
+
## Patterns
|
|
1193
|
+
<factual observations across iterations — what clusters of hypotheses worked,
|
|
1194
|
+
which failed for the same reason>
|
|
1195
|
+
|
|
1196
|
+
## Current State
|
|
1197
|
+
<what is currently checked in — the state that would be preserved if the user
|
|
1198
|
+
adopts this run>
|
|
1199
|
+
```
|
|
1200
|
+
|
|
1201
|
+
### I.3 Write `experiments/research_recommendations.md`
|
|
1202
|
+
|
|
1203
|
+
Opinionated agent voice:
|
|
1204
|
+
|
|
1205
|
+
```markdown
|
|
1206
|
+
# AR Research Recommendations: <Project Name>
|
|
1207
|
+
|
|
1208
|
+
- **Date:** <date>
|
|
1209
|
+
- **Agent:** <agent name>
|
|
1210
|
+
- **Iterations run:** N
|
|
1211
|
+
- **Primary metric:** <metric>
|
|
1212
|
+
- **Baseline → Final:** <value> → <value> (<delta>)
|
|
1213
|
+
|
|
1214
|
+
## What I Tried
|
|
1215
|
+
<narrative of the run — what I chased, why, and how the search evolved>
|
|
1216
|
+
|
|
1217
|
+
## What Worked
|
|
1218
|
+
<GREEN iterations with your read on why>
|
|
1219
|
+
|
|
1220
|
+
## What Didn't Work
|
|
1221
|
+
<RED and reverted YELLOW iterations with your interpretation>
|
|
1222
|
+
|
|
1223
|
+
## Surprises
|
|
1224
|
+
<anything you did not expect — either pleasantly or otherwise>
|
|
1225
|
+
|
|
1226
|
+
## My Recommendation
|
|
1227
|
+
<the clearest path forward — what to adopt, what to discard, what to try next
|
|
1228
|
+
if the user wants to keep going. Opinionated.>
|
|
1229
|
+
|
|
1230
|
+
## If I Could Run Another Budget
|
|
1231
|
+
<your top 3 hypotheses for a next run, prioritized>
|
|
1232
|
+
```
|
|
1233
|
+
|
|
1234
|
+
### I.4 Update project-specs.md
|
|
1235
|
+
|
|
1236
|
+
Append / update the `## Autonomous Research` section in `project-specs.md`:
|
|
1237
|
+
|
|
1238
|
+
```markdown
|
|
1239
|
+
## Autonomous Research
|
|
1240
|
+
- **Status:** Complete
|
|
1241
|
+
- **Preset:** <preset>
|
|
1242
|
+
- **Iterations:** <N> of <budget>
|
|
1243
|
+
- **Primary metric:** <metric>: <baseline> → <final> (<delta>)
|
|
1244
|
+
- **Convergence reason:** <reason>
|
|
1245
|
+
- **Cost spent:** <tokens / dollars>
|
|
1246
|
+
- **Brief:** `experiments/research_brief.md`
|
|
1247
|
+
- **Summary:** `experiments/research_summary.md`
|
|
1248
|
+
- **Recommendations:** `experiments/research_recommendations.md`
|
|
1249
|
+
```
|
|
1250
|
+
|
|
1251
|
+
Also write a consolidated `## Validation` section to `project-specs.md` per `shared/validation_protocol.md`. The Phase 3 gate is validation-eligible (the AR run produces a durable set of kept iterations / final artifact), and the gate hook enforces the schema at Phase 3.
|
|
1252
|
+
|
|
1253
|
+
Consolidation rules:
|
|
1254
|
+
- **Track:** `deep` (Phase 3 is the deep gate; per-iteration blocks were already Fixer-level)
|
|
1255
|
+
- **Mode:** `research`
|
|
1256
|
+
- **Evidence:** pull the headline metric per kept iteration from `results.json.experiments[*].validation` and aggregate. Expected format: one row per check ID from the agent's Fixer subset, with Observed = "N iterations × check, M passes, K n/a" or the final-iteration value for diff-style checks.
|
|
1257
|
+
- **Artifacts:** `experiments/results.json`, `experiments/research_summary.md`, the final-iteration checkpoint commit SHA, any per-iteration artifacts referenced in `validation.artifacts`
|
|
1258
|
+
- **Downstream Impact:** consumers of the final artifact (services, marts, reports) — same analysis the agent would do in a normal deep-track phase
|
|
1259
|
+
- **Summary:** two to four sentences on what was validated across the kept iterations, what residual risk exists, and what would need a fuller validation pass if this is productionized
|
|
1260
|
+
|
|
1261
|
+
This section is machine-readable by the gate hook when Phase 3 emits its gate with `validates=<agent>`. Per-agent `research.md` files are responsible for including that attribute on their Phase 3 gate fence.
|
|
1262
|
+
|
|
1263
|
+
### I.5 Knowledge harvest
|
|
1264
|
+
|
|
1265
|
+
Run `.claude/agents/specific_instructions/shared/knowledge_harvest.md` per its
|
|
1266
|
+
protocol. In fan-out contexts, only the initiating parent runs harvest after
|
|
1267
|
+
promotion (see Section H.10).
|
|
1268
|
+
|
|
1269
|
+
### I.6 Present to user (GATE)
|
|
1270
|
+
|
|
1271
|
+
Read both the summary and recommendations back to the user.
|
|
1272
|
+
|
|
1273
|
+
**The Phase 3 gate is owned by the per-agent `research.md` file**, not by this
|
|
1274
|
+
shared protocol. Each agent emits its own Phase 3 gate with its
|
|
1275
|
+
agent-specific ID (e.g., `specific-instructions-ml-engineer-research-phase3`
|
|
1276
|
+
for ML Engineer, `specific-instructions-ai-engineer-research-phase3` for AI
|
|
1277
|
+
Engineer, etc.). Do **not** emit a shared-protocol Phase 3 gate here — doing
|
|
1278
|
+
so would duplicate the gate or orphan the ID namespace.
|
|
1279
|
+
|
|
1280
|
+
The typical gate body is:
|
|
1281
|
+
|
|
1282
|
+
> Ask the user:
|
|
1283
|
+
> - What do you want to adopt from this AR run?
|
|
1284
|
+
> - Do you want to run another budget (fresh AR session)?
|
|
1285
|
+
> - Or should we stop here?
|
|
1286
|
+
|
|
1287
|
+
Wait for the user's decision before taking any further action. Per-agent
|
|
1288
|
+
`research.md` files define the exact gate line (see each agent's Phase 3
|
|
1289
|
+
section).
|