@proflandrigan/shards 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +475 -0
- package/package.json +37 -0
- package/src/agents/academic.md +276 -0
- package/src/agents/ai-engineer.md +377 -0
- package/src/agents/analytics-engineer.md +364 -0
- package/src/agents/applied-ml-scientist.md +410 -0
- package/src/agents/backend-engineer.md +255 -0
- package/src/agents/bi-engineer.md +333 -0
- package/src/agents/data-analyst.md +343 -0
- package/src/agents/data-engineer.md +260 -0
- package/src/agents/data-modeller.md +386 -0
- package/src/agents/data-scientist.md +366 -0
- package/src/agents/deep-learning-engineer.md +389 -0
- package/src/agents/ml-engineer.md +424 -0
- package/src/agents/mlops-engineer.md +339 -0
- package/src/agents/researcher.md +187 -0
- package/src/agents/specific_instructions/academic/critical_review.md +263 -0
- package/src/agents/specific_instructions/academic/report.md +113 -0
- package/src/agents/specific_instructions/ai_engineer/advise.md +162 -0
- package/src/agents/specific_instructions/ai_engineer/bi_engineer_handoff.md +86 -0
- package/src/agents/specific_instructions/ai_engineer/experiment.md +471 -0
- package/src/agents/specific_instructions/ai_engineer/experiment_ui_mode.md +44 -0
- package/src/agents/specific_instructions/ai_engineer/phases/index.md +45 -0
- package/src/agents/specific_instructions/ai_engineer/phases/phase-1.md +55 -0
- package/src/agents/specific_instructions/ai_engineer/phases/phase-2.md +86 -0
- package/src/agents/specific_instructions/ai_engineer/phases/phase-3.md +96 -0
- package/src/agents/specific_instructions/ai_engineer/phases/phase-4.md +138 -0
- package/src/agents/specific_instructions/ai_engineer/phases/phase-5.md +157 -0
- package/src/agents/specific_instructions/ai_engineer/phases/phase-6.md +196 -0
- package/src/agents/specific_instructions/ai_engineer/phases/phase-7.md +313 -0
- package/src/agents/specific_instructions/ai_engineer/phases.md +1011 -0
- package/src/agents/specific_instructions/ai_engineer/prompt_lab.md +161 -0
- package/src/agents/specific_instructions/ai_engineer/prompt_lab_ui_mode.md +28 -0
- package/src/agents/specific_instructions/ai_engineer/research.md +393 -0
- package/src/agents/specific_instructions/ai_engineer/research_ui_mode.md +66 -0
- package/src/agents/specific_instructions/ai_engineer/review.md +159 -0
- package/src/agents/specific_instructions/ai_engineer/validation_checklist.md +182 -0
- package/src/agents/specific_instructions/analytics_engineer/advise.md +155 -0
- package/src/agents/specific_instructions/analytics_engineer/bi_engineer_handoff.md +91 -0
- package/src/agents/specific_instructions/analytics_engineer/data_analyst_handoff.md +84 -0
- package/src/agents/specific_instructions/analytics_engineer/deep_phases.md +818 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/index.md +24 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-1.md +77 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-2.md +106 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-3.md +93 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-4.md +79 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-5.md +61 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-6.md +45 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-7.md +235 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-8.md +221 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_quick/index.md +19 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_quick/phase-1.md +47 -0
- package/src/agents/specific_instructions/analytics_engineer/phases_quick/phase-2.md +78 -0
- package/src/agents/specific_instructions/analytics_engineer/quick_phases.md +112 -0
- package/src/agents/specific_instructions/analytics_engineer/review.md +167 -0
- package/src/agents/specific_instructions/analytics_engineer/service_mode.md +369 -0
- package/src/agents/specific_instructions/analytics_engineer/ui_mode.md +45 -0
- package/src/agents/specific_instructions/analytics_engineer/update.md +162 -0
- package/src/agents/specific_instructions/analytics_engineer/validation_checklist.md +121 -0
- package/src/agents/specific_instructions/applied_ml_scientist/advise.md +143 -0
- package/src/agents/specific_instructions/applied_ml_scientist/phases/index.md +21 -0
- package/src/agents/specific_instructions/applied_ml_scientist/phases/phase-1.md +51 -0
- package/src/agents/specific_instructions/applied_ml_scientist/phases/phase-2.md +66 -0
- package/src/agents/specific_instructions/applied_ml_scientist/phases/phase-3.md +113 -0
- package/src/agents/specific_instructions/applied_ml_scientist/phases/phase-4.md +104 -0
- package/src/agents/specific_instructions/applied_ml_scientist/phases/phase-5.md +156 -0
- package/src/agents/specific_instructions/applied_ml_scientist/phases.md +428 -0
- package/src/agents/specific_instructions/applied_ml_scientist/research.md +379 -0
- package/src/agents/specific_instructions/applied_ml_scientist/review.md +142 -0
- package/src/agents/specific_instructions/applied_ml_scientist/validation_checklist.md +136 -0
- package/src/agents/specific_instructions/backend_engineer/clean.md +149 -0
- package/src/agents/specific_instructions/backend_engineer/review.md +91 -0
- package/src/agents/specific_instructions/backend_engineer/review_checklist.md +54 -0
- package/src/agents/specific_instructions/backend_engineer/service_mode.md +67 -0
- package/src/agents/specific_instructions/bi_engineer/advise.md +137 -0
- package/src/agents/specific_instructions/bi_engineer/data_analyst_handoff.md +77 -0
- package/src/agents/specific_instructions/bi_engineer/incoming_handoff.md +45 -0
- package/src/agents/specific_instructions/bi_engineer/phases/index.md +20 -0
- package/src/agents/specific_instructions/bi_engineer/phases/phase-1.md +164 -0
- package/src/agents/specific_instructions/bi_engineer/phases/phase-2.md +92 -0
- package/src/agents/specific_instructions/bi_engineer/phases/phase-3.md +121 -0
- package/src/agents/specific_instructions/bi_engineer/phases/phase-4.md +106 -0
- package/src/agents/specific_instructions/bi_engineer/phases.md +451 -0
- package/src/agents/specific_instructions/bi_engineer/review.md +166 -0
- package/src/agents/specific_instructions/bi_engineer/update.md +147 -0
- package/src/agents/specific_instructions/bi_engineer/validation_checklist.md +124 -0
- package/src/agents/specific_instructions/data_analyst/advise.md +138 -0
- package/src/agents/specific_instructions/data_analyst/explain.md +221 -0
- package/src/agents/specific_instructions/data_analyst/incoming_handoff.md +40 -0
- package/src/agents/specific_instructions/data_analyst/phases/index.md +20 -0
- package/src/agents/specific_instructions/data_analyst/phases/phase-1.md +159 -0
- package/src/agents/specific_instructions/data_analyst/phases/phase-2.md +112 -0
- package/src/agents/specific_instructions/data_analyst/phases/phase-3.md +265 -0
- package/src/agents/specific_instructions/data_analyst/phases/phase-4.md +100 -0
- package/src/agents/specific_instructions/data_analyst/phases.md +501 -0
- package/src/agents/specific_instructions/data_analyst/review.md +138 -0
- package/src/agents/specific_instructions/data_analyst/ui_mode.md +26 -0
- package/src/agents/specific_instructions/data_analyst/update.md +144 -0
- package/src/agents/specific_instructions/data_analyst/validation_checklist.md +95 -0
- package/src/agents/specific_instructions/data_engineer/advise.md +137 -0
- package/src/agents/specific_instructions/data_engineer/phases.md +466 -0
- package/src/agents/specific_instructions/data_engineer/phases_deep/index.md +23 -0
- package/src/agents/specific_instructions/data_engineer/phases_deep/phase-1.md +49 -0
- package/src/agents/specific_instructions/data_engineer/phases_deep/phase-2.md +93 -0
- package/src/agents/specific_instructions/data_engineer/phases_deep/phase-3.md +55 -0
- package/src/agents/specific_instructions/data_engineer/phases_deep/phase-4.md +48 -0
- package/src/agents/specific_instructions/data_engineer/phases_deep/phase-5.md +40 -0
- package/src/agents/specific_instructions/data_engineer/phases_deep/phase-6.md +102 -0
- package/src/agents/specific_instructions/data_engineer/phases_deep/phase-7.md +87 -0
- package/src/agents/specific_instructions/data_engineer/phases_quick/index.md +19 -0
- package/src/agents/specific_instructions/data_engineer/phases_quick/phase-1.md +45 -0
- package/src/agents/specific_instructions/data_engineer/phases_quick/phase-2.md +54 -0
- package/src/agents/specific_instructions/data_engineer/review.md +135 -0
- package/src/agents/specific_instructions/data_engineer/validation_checklist.md +136 -0
- package/src/agents/specific_instructions/data_modeller/advise.md +137 -0
- package/src/agents/specific_instructions/data_modeller/phases.md +581 -0
- package/src/agents/specific_instructions/data_modeller/phases_deep/index.md +23 -0
- package/src/agents/specific_instructions/data_modeller/phases_deep/phase-1.md +52 -0
- package/src/agents/specific_instructions/data_modeller/phases_deep/phase-2.md +113 -0
- package/src/agents/specific_instructions/data_modeller/phases_deep/phase-3.md +47 -0
- package/src/agents/specific_instructions/data_modeller/phases_deep/phase-4.md +51 -0
- package/src/agents/specific_instructions/data_modeller/phases_deep/phase-5.md +45 -0
- package/src/agents/specific_instructions/data_modeller/phases_deep/phase-6.md +105 -0
- package/src/agents/specific_instructions/data_modeller/phases_deep/phase-7.md +136 -0
- package/src/agents/specific_instructions/data_modeller/phases_quick/index.md +19 -0
- package/src/agents/specific_instructions/data_modeller/phases_quick/phase-1.md +47 -0
- package/src/agents/specific_instructions/data_modeller/phases_quick/phase-2.md +65 -0
- package/src/agents/specific_instructions/data_modeller/review.md +141 -0
- package/src/agents/specific_instructions/data_modeller/service_mode.md +218 -0
- package/src/agents/specific_instructions/data_modeller/validation_checklist.md +125 -0
- package/src/agents/specific_instructions/data_scientist/advise.md +158 -0
- package/src/agents/specific_instructions/data_scientist/bi_engineer_handoff.md +63 -0
- package/src/agents/specific_instructions/data_scientist/experiment.md +482 -0
- package/src/agents/specific_instructions/data_scientist/experiment_ui_mode.md +44 -0
- package/src/agents/specific_instructions/data_scientist/explain.md +247 -0
- package/src/agents/specific_instructions/data_scientist/greenfield_data.md +35 -0
- package/src/agents/specific_instructions/data_scientist/ml_engineer_handoff.md +52 -0
- package/src/agents/specific_instructions/data_scientist/notebook_walkthrough.md +76 -0
- package/src/agents/specific_instructions/data_scientist/phases/index.md +24 -0
- package/src/agents/specific_instructions/data_scientist/phases/phase-1.md +45 -0
- package/src/agents/specific_instructions/data_scientist/phases/phase-2.md +67 -0
- package/src/agents/specific_instructions/data_scientist/phases/phase-3.md +89 -0
- package/src/agents/specific_instructions/data_scientist/phases/phase-4.md +143 -0
- package/src/agents/specific_instructions/data_scientist/phases/phase-5.md +71 -0
- package/src/agents/specific_instructions/data_scientist/phases/phase-6.md +239 -0
- package/src/agents/specific_instructions/data_scientist/phases/phase-7.md +207 -0
- package/src/agents/specific_instructions/data_scientist/phases.md +651 -0
- package/src/agents/specific_instructions/data_scientist/research.md +345 -0
- package/src/agents/specific_instructions/data_scientist/research_ui_mode.md +52 -0
- package/src/agents/specific_instructions/data_scientist/review.md +136 -0
- package/src/agents/specific_instructions/data_scientist/service_mode.md +247 -0
- package/src/agents/specific_instructions/data_scientist/validation_checklist.md +183 -0
- package/src/agents/specific_instructions/deep_learning_engineer/advise.md +145 -0
- package/src/agents/specific_instructions/deep_learning_engineer/phases/index.md +21 -0
- package/src/agents/specific_instructions/deep_learning_engineer/phases/phase-1.md +74 -0
- package/src/agents/specific_instructions/deep_learning_engineer/phases/phase-2.md +98 -0
- package/src/agents/specific_instructions/deep_learning_engineer/phases/phase-3.md +76 -0
- package/src/agents/specific_instructions/deep_learning_engineer/phases/phase-4.md +128 -0
- package/src/agents/specific_instructions/deep_learning_engineer/phases/phase-5.md +292 -0
- package/src/agents/specific_instructions/deep_learning_engineer/phases.md +567 -0
- package/src/agents/specific_instructions/deep_learning_engineer/research.md +389 -0
- package/src/agents/specific_instructions/deep_learning_engineer/review.md +155 -0
- package/src/agents/specific_instructions/deep_learning_engineer/validation_checklist.md +147 -0
- package/src/agents/specific_instructions/ml_engineer/advise.md +174 -0
- package/src/agents/specific_instructions/ml_engineer/bi_engineer_handoff.md +71 -0
- package/src/agents/specific_instructions/ml_engineer/experiment.md +474 -0
- package/src/agents/specific_instructions/ml_engineer/experiment_ui_mode.md +44 -0
- package/src/agents/specific_instructions/ml_engineer/notebook_walkthrough.md +75 -0
- package/src/agents/specific_instructions/ml_engineer/phases/index.md +25 -0
- package/src/agents/specific_instructions/ml_engineer/phases/phase-1.md +49 -0
- package/src/agents/specific_instructions/ml_engineer/phases/phase-2.md +75 -0
- package/src/agents/specific_instructions/ml_engineer/phases/phase-3.md +124 -0
- package/src/agents/specific_instructions/ml_engineer/phases/phase-4.md +279 -0
- package/src/agents/specific_instructions/ml_engineer/phases/phase-5.md +160 -0
- package/src/agents/specific_instructions/ml_engineer/phases/phase-6-5.md +170 -0
- package/src/agents/specific_instructions/ml_engineer/phases/phase-6.md +295 -0
- package/src/agents/specific_instructions/ml_engineer/phases/phase-7.md +337 -0
- package/src/agents/specific_instructions/ml_engineer/phases.md +1068 -0
- package/src/agents/specific_instructions/ml_engineer/research.md +437 -0
- package/src/agents/specific_instructions/ml_engineer/research_ui_mode.md +71 -0
- package/src/agents/specific_instructions/ml_engineer/review.md +187 -0
- package/src/agents/specific_instructions/ml_engineer/service_mode.md +273 -0
- package/src/agents/specific_instructions/ml_engineer/validation_checklist.md +185 -0
- package/src/agents/specific_instructions/mlops_engineer/advise.md +139 -0
- package/src/agents/specific_instructions/mlops_engineer/phases/index.md +23 -0
- package/src/agents/specific_instructions/mlops_engineer/phases/phase-1.md +52 -0
- package/src/agents/specific_instructions/mlops_engineer/phases/phase-2.md +86 -0
- package/src/agents/specific_instructions/mlops_engineer/phases/phase-3.md +105 -0
- package/src/agents/specific_instructions/mlops_engineer/phases/phase-4.md +128 -0
- package/src/agents/specific_instructions/mlops_engineer/phases/phase-5.md +106 -0
- package/src/agents/specific_instructions/mlops_engineer/phases/phase-6.md +128 -0
- package/src/agents/specific_instructions/mlops_engineer/phases/phase-7.md +144 -0
- package/src/agents/specific_instructions/mlops_engineer/phases.md +671 -0
- package/src/agents/specific_instructions/mlops_engineer/review.md +164 -0
- package/src/agents/specific_instructions/mlops_engineer/service_mode.md +81 -0
- package/src/agents/specific_instructions/mlops_engineer/validation_checklist.md +151 -0
- package/src/agents/specific_instructions/researcher/critical_review.md +292 -0
- package/src/agents/specific_instructions/researcher/review_checklist.md +67 -0
- package/src/agents/specific_instructions/researcher/service_mode.md +224 -0
- package/src/agents/specific_instructions/shared/auto_verify_mode.md +141 -0
- package/src/agents/specific_instructions/shared/autonomous_research.md +1289 -0
- package/src/agents/specific_instructions/shared/behavioral_rules.md +36 -0
- package/src/agents/specific_instructions/shared/diverge_protocol.md +387 -0
- package/src/agents/specific_instructions/shared/engineering_guidelines.md +136 -0
- package/src/agents/specific_instructions/shared/experiment_versioning.md +184 -0
- package/src/agents/specific_instructions/shared/goal_mode.md +187 -0
- package/src/agents/specific_instructions/shared/incremental_testing.md +139 -0
- package/src/agents/specific_instructions/shared/intent_discovery.md +223 -0
- package/src/agents/specific_instructions/shared/join_path_protocol.md +168 -0
- package/src/agents/specific_instructions/shared/knowledge_checkpoint.md +83 -0
- package/src/agents/specific_instructions/shared/knowledge_harvest.md +220 -0
- package/src/agents/specific_instructions/shared/knowledge_retrieval.md +100 -0
- package/src/agents/specific_instructions/shared/notebook_walkthrough_protocol.md +367 -0
- package/src/agents/specific_instructions/shared/reviewer_verdict_protocol.md +74 -0
- package/src/agents/specific_instructions/shared/swarm_protocol.md +97 -0
- package/src/agents/specific_instructions/shared/validation_protocol.md +139 -0
- package/src/agents/specific_instructions/syn/arbiter.md +140 -0
- package/src/agents/specific_instructions/syn/brainstorm.md +550 -0
- package/src/agents/specific_instructions/syn/code_review.md +232 -0
- package/src/agents/specific_instructions/syn/diff.md +239 -0
- package/src/agents/specific_instructions/syn/final_review.md +65 -0
- package/src/agents/specific_instructions/syn/fixer.md +240 -0
- package/src/agents/specific_instructions/syn/free_form.md +130 -0
- package/src/agents/specific_instructions/syn/knowledge.md +468 -0
- package/src/agents/specific_instructions/syn/notebook_walkthrough.md +78 -0
- package/src/agents/specific_instructions/syn/panel_review.md +634 -0
- package/src/agents/specific_instructions/syn/pm.md +453 -0
- package/src/agents/specific_instructions/syn/pr_review.md +255 -0
- package/src/agents/specific_instructions/syn/slides.md +417 -0
- package/src/agents/syn.md +729 -0
- package/src/commands/academic.md +41 -0
- package/src/commands/ai-engineer.md +45 -0
- package/src/commands/analytics-engineer.md +48 -0
- package/src/commands/applied-ml-scientist.md +45 -0
- package/src/commands/backend-engineer.md +35 -0
- package/src/commands/bi-engineer.md +40 -0
- package/src/commands/brainstorm.md +24 -0
- package/src/commands/data-analyst.md +38 -0
- package/src/commands/data-engineer.md +37 -0
- package/src/commands/data-modeller.md +38 -0
- package/src/commands/data-scientist.md +38 -0
- package/src/commands/deep-learning-engineer.md +47 -0
- package/src/commands/end.md +49 -0
- package/src/commands/knowledge.md +24 -0
- package/src/commands/ml-engineer.md +42 -0
- package/src/commands/mlops-engineer.md +47 -0
- package/src/commands/notebook-walkthrough.md +58 -0
- package/src/commands/researcher.md +40 -0
- package/src/commands/resume.md +57 -0
- package/src/commands/review-pr.md +26 -0
- package/src/commands/shards-guide.md +41 -0
- package/src/commands/shards-ui.md +32 -0
- package/src/commands/shards.md +41 -0
- package/src/docs/01-getting-started/concepts.md +109 -0
- package/src/docs/01-getting-started/first-session.md +79 -0
- package/src/docs/01-getting-started/install.md +61 -0
- package/src/docs/02-agents/academic.md +71 -0
- package/src/docs/02-agents/ai-engineer.md +78 -0
- package/src/docs/02-agents/analytics-engineer.md +58 -0
- package/src/docs/02-agents/applied-ml-scientist.md +59 -0
- package/src/docs/02-agents/backend-engineer.md +58 -0
- package/src/docs/02-agents/bi-engineer.md +65 -0
- package/src/docs/02-agents/data-analyst.md +67 -0
- package/src/docs/02-agents/data-engineer.md +57 -0
- package/src/docs/02-agents/data-modeller.md +51 -0
- package/src/docs/02-agents/data-scientist.md +78 -0
- package/src/docs/02-agents/deep-learning-engineer.md +64 -0
- package/src/docs/02-agents/ml-engineer.md +80 -0
- package/src/docs/02-agents/mlops-engineer.md +59 -0
- package/src/docs/02-agents/overview.md +62 -0
- package/src/docs/02-agents/researcher.md +73 -0
- package/src/docs/02-agents/syn.md +88 -0
- package/src/docs/03-protocols/auto-verify.md +82 -0
- package/src/docs/03-protocols/autonomous-research.md +59 -0
- package/src/docs/03-protocols/behavioral-rules.md +35 -0
- package/src/docs/03-protocols/diverge.md +50 -0
- package/src/docs/03-protocols/engineering-guidelines.md +56 -0
- package/src/docs/03-protocols/experiment-versioning.md +38 -0
- package/src/docs/03-protocols/gate-pattern.md +65 -0
- package/src/docs/03-protocols/incremental-testing.md +68 -0
- package/src/docs/03-protocols/join-path.md +46 -0
- package/src/docs/03-protocols/knowledge-ledger.md +70 -0
- package/src/docs/03-protocols/reviewer-verdicts.md +39 -0
- package/src/docs/03-protocols/swarm.md +40 -0
- package/src/docs/03-protocols/validation.md +174 -0
- package/src/docs/04-ui/activity-bar.md +70 -0
- package/src/docs/04-ui/chat-pane.md +80 -0
- package/src/docs/04-ui/code-intel.md +62 -0
- package/src/docs/04-ui/file-editing.md +61 -0
- package/src/docs/04-ui/git.md +54 -0
- package/src/docs/04-ui/keybindings.md +79 -0
- package/src/docs/04-ui/knowledge-map.md +76 -0
- package/src/docs/04-ui/overview.md +93 -0
- package/src/docs/04-ui/panels.md +49 -0
- package/src/docs/04-ui/pinboard-selection.md +66 -0
- package/src/docs/04-ui/quick-open-palette.md +56 -0
- package/src/docs/04-ui/sessions.md +81 -0
- package/src/docs/04-ui/settings-permissions.md +56 -0
- package/src/docs/05-commands/reference.md +59 -0
- package/src/docs/06-outputs/directory-map.md +116 -0
- package/src/docs/07-workflows/ai-eval-first.md +57 -0
- package/src/docs/07-workflows/deep-study-to-production.md +76 -0
- package/src/docs/07-workflows/diverge-exploration.md +77 -0
- package/src/docs/07-workflows/quick-analysis.md +45 -0
- package/src/docs/08-integrations/claude-code-auto-mode.md +191 -0
- package/src/docs/08-integrations/google-slides.md +175 -0
- package/src/docs/README.md +30 -0
- package/src/docs/manifest.json +108 -0
- package/src/templates/analysis-template.md +20 -0
- package/src/templates/branch-report.md +46 -0
- package/src/templates/diff-report.md +88 -0
- package/src/templates/knowledge-index.md +7 -0
- package/src/templates/model-card-schema.json +186 -0
- package/src/templates/model-card-schema.md +88 -0
- package/src/templates/model-card.md +124 -0
- package/src/templates/project-plan.md +47 -0
- package/src/templates/project-specs.md +81 -0
- package/src/templates/report-template.md +43 -0
- package/src/templates/study-template.md +25 -0
- package/src/ui/cc-readonly.js +181 -0
- package/src/ui/chat-session.js +466 -0
- package/src/ui/css/base.css +136 -0
- package/src/ui/css/brainstorm.css +525 -0
- package/src/ui/css/chat.css +1405 -0
- package/src/ui/css/editor.css +546 -0
- package/src/ui/css/eval-dashboard.css +157 -0
- package/src/ui/css/experiment.css +237 -0
- package/src/ui/css/guide.css +186 -0
- package/src/ui/css/knowledge-map.css +383 -0
- package/src/ui/css/layout.css +431 -0
- package/src/ui/css/model-card.css +161 -0
- package/src/ui/css/notebook-walkthrough.css +271 -0
- package/src/ui/css/pr-review.css +403 -0
- package/src/ui/css/prompt-lab.css +325 -0
- package/src/ui/css/sessions.css +258 -0
- package/src/ui/css/sidebar.css +661 -0
- package/src/ui/css/terminal.css +113 -0
- package/src/ui/css/theme-light.css +542 -0
- package/src/ui/index.html +389 -0
- package/src/ui/js/agents.js +32 -0
- package/src/ui/js/bookmarks.js +230 -0
- package/src/ui/js/chat.js +1776 -0
- package/src/ui/js/code-intel.js +328 -0
- package/src/ui/js/command-palette.js +142 -0
- package/src/ui/js/events.js +591 -0
- package/src/ui/js/explorer.js +317 -0
- package/src/ui/js/file-view.js +477 -0
- package/src/ui/js/git.js +536 -0
- package/src/ui/js/guide.js +198 -0
- package/src/ui/js/hud.js +75 -0
- package/src/ui/js/init.js +351 -0
- package/src/ui/js/knowledge-map.js +906 -0
- package/src/ui/js/markdown.js +114 -0
- package/src/ui/js/monaco.js +164 -0
- package/src/ui/js/notebook-walkthrough.js +272 -0
- package/src/ui/js/notebook.js +448 -0
- package/src/ui/js/panels.js +2681 -0
- package/src/ui/js/pinboard.js +186 -0
- package/src/ui/js/quick-open.js +164 -0
- package/src/ui/js/selection-context.js +131 -0
- package/src/ui/js/sessions.js +256 -0
- package/src/ui/js/settings.js +476 -0
- package/src/ui/js/split-view.js +82 -0
- package/src/ui/js/state.js +343 -0
- package/src/ui/js/table.js +161 -0
- package/src/ui/js/tabs.js +284 -0
- package/src/ui/js/tabular.js +125 -0
- package/src/ui/js/terminal.js +354 -0
- package/src/ui/js/timeline.js +137 -0
- package/src/ui/js/utils.js +293 -0
- package/src/ui/notebook-kernel.py +790 -0
- package/src/ui/open-browser.js +55 -0
- package/src/ui/permission-pattern.js +42 -0
- package/src/ui/relay.js +513 -0
- package/src/ui/server.js +3072 -0
- package/src/ui/session-index.js +225 -0
- package/src/ui/shards_icon.png +0 -0
- package/src/ui/spawn-server.js +41 -0
- package/src/ui/symbol-index.js +813 -0
- package/src/ui/ui-push.js +177 -0
- package/tools/gate-hook/VALIDATION_SPEC.md +273 -0
- package/tools/gate-hook/__tests__/auto-verify.test.js +343 -0
- package/tools/gate-hook/auto-allowlist.js +179 -0
- package/tools/gate-hook/auto-state.js +68 -0
- package/tools/gate-hook/classify.js +21 -0
- package/tools/gate-hook/log.js +57 -0
- package/tools/gate-hook/parser.js +205 -0
- package/tools/gate-hook/sql-guard.js +230 -0
- package/tools/gate-hook/state.js +170 -0
- package/tools/gate-hook/sweep.js +139 -0
- package/tools/gate-hook/transcript.js +45 -0
- package/tools/gate-hook/validation.js +321 -0
- package/tools/gate-hook.js +475 -0
- package/tools/install.js +914 -0
- package/tools/shards-gates.js +311 -0
- package/tools/shards-sessions.js +261 -0
- package/tools/shards-ui.js +377 -0
|
@@ -0,0 +1,161 @@
|
|
|
1
|
+
# Prompt Laboratory — AI Engineer
|
|
2
|
+
|
|
3
|
+
Interactive prompt editing, evaluation, and versioning via the Shards UI.
|
|
4
|
+
|
|
5
|
+
The Prompt Laboratory gives the user a browser-based scratchpad for prompt iteration.
|
|
6
|
+
You remain the AI Engineer — same persona, same rigor, same skepticism about whether
|
|
7
|
+
AI is even needed. But now the user drives edits directly in a Monaco editor, and you
|
|
8
|
+
handle the evaluation and sync mechanics behind the scenes.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## Setup
|
|
13
|
+
|
|
14
|
+
1. Ask the user for the **project name** (or detect from context).
|
|
15
|
+
2. Locate the project directory and `project-specs.md`.
|
|
16
|
+
3. Scan the `prompts/` directory (or the directory specified in project-specs.md) for
|
|
17
|
+
all prompt files. Parse each file's metadata header:
|
|
18
|
+
```
|
|
19
|
+
# Prompt: <name>
|
|
20
|
+
# Version: <version>
|
|
21
|
+
# Purpose: <purpose>
|
|
22
|
+
# Model: <model>
|
|
23
|
+
# Evaluation score: <metric: value>
|
|
24
|
+
```
|
|
25
|
+
4. Locate the evaluation script and test set (from project-specs.md or by scanning for
|
|
26
|
+
`eval/`, `evaluation/`, `tests/` directories).
|
|
27
|
+
5. Build the initial `prompt-lab.json` in the project directory:
|
|
28
|
+
|
|
29
|
+
```json
|
|
30
|
+
{
|
|
31
|
+
"projectName": "<project_name>",
|
|
32
|
+
"projectDir": "<absolute_path>",
|
|
33
|
+
"promptsDir": "<relative_path_to_prompts>",
|
|
34
|
+
"prompts": [
|
|
35
|
+
{
|
|
36
|
+
"name": "<name from header>",
|
|
37
|
+
"filename": "<filename>",
|
|
38
|
+
"currentVersion": "<version from header>",
|
|
39
|
+
"model": "<model from header>",
|
|
40
|
+
"purpose": "<purpose from header>",
|
|
41
|
+
"versions": [
|
|
42
|
+
{
|
|
43
|
+
"version": "<version>",
|
|
44
|
+
"timestamp": "<file modified time or header date>",
|
|
45
|
+
"evaluationScore": { "<metric>": <value> },
|
|
46
|
+
"source": "manual"
|
|
47
|
+
}
|
|
48
|
+
]
|
|
49
|
+
}
|
|
50
|
+
],
|
|
51
|
+
"activePrompt": "<first prompt name>",
|
|
52
|
+
"editedContent": null,
|
|
53
|
+
"originalContent": "<content of first prompt file>",
|
|
54
|
+
"testRuns": [],
|
|
55
|
+
"syncHistory": [],
|
|
56
|
+
"status": "idle"
|
|
57
|
+
}
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
6. Push the Prompt Lab panel to the Shards UI. Read
|
|
61
|
+
`.claude/agents/specific_instructions/ai_engineer/prompt_lab_ui_mode.md` for the
|
|
62
|
+
push command.
|
|
63
|
+
7. Tell the user the Prompt Lab is ready. Explain the three controls:
|
|
64
|
+
- **Diff** — toggle side-by-side diff between original and edited prompt
|
|
65
|
+
- **Run Test** — evaluate the edited prompt against the project's test set
|
|
66
|
+
- **Sync to Project** — write the edited prompt to disk with a new version, update
|
|
67
|
+
project-specs.md, and commit
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
## Handling `[PROMPT-LAB]` Messages
|
|
72
|
+
|
|
73
|
+
The Shards UI sends structured messages to you via the chat session. Recognize them by
|
|
74
|
+
the `[PROMPT-LAB]` prefix.
|
|
75
|
+
|
|
76
|
+
### Run Evaluation
|
|
77
|
+
|
|
78
|
+
Pattern: `[PROMPT-LAB] Run evaluation for prompt "<name>" with content:`
|
|
79
|
+
|
|
80
|
+
Steps:
|
|
81
|
+
1. Set `prompt-lab.json` status to `"testing"`.
|
|
82
|
+
2. Extract the prompt content from between the `---` fences.
|
|
83
|
+
3. Write the content to a temporary file (e.g., `/tmp/pl-<name>-test.md`).
|
|
84
|
+
4. Run the project's evaluation script against the temporary prompt file.
|
|
85
|
+
- If no evaluation script exists, explain this to the user in chat and set status
|
|
86
|
+
back to `"idle"`.
|
|
87
|
+
5. Collect metrics from the evaluation output.
|
|
88
|
+
6. If the evaluation produces sample outputs, collect up to 5 representative samples.
|
|
89
|
+
7. Append a new entry to `prompt-lab.json` `testRuns`:
|
|
90
|
+
```json
|
|
91
|
+
{
|
|
92
|
+
"id": "run-<N>",
|
|
93
|
+
"promptName": "<name>",
|
|
94
|
+
"timestamp": "<ISO-8601>",
|
|
95
|
+
"status": "complete",
|
|
96
|
+
"metrics": { "<metric>": <value>, ... },
|
|
97
|
+
"sampleOutputs": [
|
|
98
|
+
{ "input": "<input>", "output": "<output>", "score": <score> }
|
|
99
|
+
],
|
|
100
|
+
"error": null
|
|
101
|
+
}
|
|
102
|
+
```
|
|
103
|
+
8. Set `prompt-lab.json` status back to `"idle"`.
|
|
104
|
+
9. Summarize results briefly in chat (one sentence with the key metric delta).
|
|
105
|
+
|
|
106
|
+
If evaluation fails:
|
|
107
|
+
- Write a `testRuns` entry with `"status": "error"` and `"error": "<message>"`.
|
|
108
|
+
- Set status back to `"idle"`.
|
|
109
|
+
- Explain the failure in chat.
|
|
110
|
+
|
|
111
|
+
### Sync to Project
|
|
112
|
+
|
|
113
|
+
Pattern: `[PROMPT-LAB] Sync prompt "<name>" to project.`
|
|
114
|
+
|
|
115
|
+
Steps:
|
|
116
|
+
1. Set `prompt-lab.json` status to `"syncing"`.
|
|
117
|
+
2. Extract the prompt content from between the `---` fences.
|
|
118
|
+
3. Read the current prompt file to determine the current version number.
|
|
119
|
+
4. Increment the version (e.g., `1.2` → `1.3`, or `2.0` → `2.1`).
|
|
120
|
+
5. Update the metadata header in the content:
|
|
121
|
+
- Set `# Version:` to the new version
|
|
122
|
+
- Set `# Date:` to today's date
|
|
123
|
+
- If the latest test run has metrics, update `# Evaluation score:`
|
|
124
|
+
6. Write the file to `prompts/<filename>`.
|
|
125
|
+
7. Update `project-specs.md`:
|
|
126
|
+
- In the Phase 6 section, update the prompt's version and evaluation score
|
|
127
|
+
- Add a note under "Deviations from plan" if the prompt changed significantly
|
|
128
|
+
8. Git commit the changes (if git is available):
|
|
129
|
+
```bash
|
|
130
|
+
git add prompts/<filename> project-specs.md
|
|
131
|
+
git commit -m "prompt-lab: update <name> to v<new_version>"
|
|
132
|
+
```
|
|
133
|
+
9. Update `prompt-lab.json`:
|
|
134
|
+
- Update the prompt's `currentVersion` and add a new version entry
|
|
135
|
+
- Append to `syncHistory`:
|
|
136
|
+
```json
|
|
137
|
+
{
|
|
138
|
+
"promptName": "<name>",
|
|
139
|
+
"fromVersion": "<old>",
|
|
140
|
+
"toVersion": "<new>",
|
|
141
|
+
"timestamp": "<ISO-8601>",
|
|
142
|
+
"commitSha": "<sha or null>"
|
|
143
|
+
}
|
|
144
|
+
```
|
|
145
|
+
- Set `originalContent` to the new file content
|
|
146
|
+
- Set `editedContent` to `null`
|
|
147
|
+
- Set status back to `"idle"`
|
|
148
|
+
10. Confirm in chat: "Synced <name> v<old> → v<new>."
|
|
149
|
+
|
|
150
|
+
---
|
|
151
|
+
|
|
152
|
+
## Important Rules
|
|
153
|
+
|
|
154
|
+
- **Always update `prompt-lab.json` after every action.** The UI watches this file for
|
|
155
|
+
live updates. If you forget to write it, the UI goes stale.
|
|
156
|
+
- **Never skip evaluation.** If the user asks to sync without testing, warn them that
|
|
157
|
+
untested prompts are "guilty until proven innocent" — but comply if they insist.
|
|
158
|
+
- **Version numbers are monotonically increasing.** Never reuse or decrement a version.
|
|
159
|
+
- **Keep chat output minimal.** The UI shows metrics and results — don't duplicate
|
|
160
|
+
everything in chat. One-line summaries are enough.
|
|
161
|
+
- **Clean up temp files** after evaluation completes.
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Prompt Lab UI Mode — AI Engineer
|
|
2
|
+
|
|
3
|
+
The Shards UI is live. Push the Prompt Laboratory panel to the browser.
|
|
4
|
+
|
|
5
|
+
## How to push
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
node .shards/ui/ui-push.js prompt-lab \
|
|
9
|
+
--title "Prompt Lab: <project_name>" \
|
|
10
|
+
--agent "ai-engineer" \
|
|
11
|
+
--panel-id "pl-<project_name>" \
|
|
12
|
+
--source "<project_dir>/prompt-lab.json"
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
Using `--source` means the server watches the file for changes. After the initial push,
|
|
16
|
+
you only need to update `prompt-lab.json` — the UI picks up changes automatically.
|
|
17
|
+
|
|
18
|
+
## When to push
|
|
19
|
+
|
|
20
|
+
Push the panel once during Setup (after writing the initial `prompt-lab.json`). The file
|
|
21
|
+
watcher handles all subsequent updates — you do not need to re-push.
|
|
22
|
+
|
|
23
|
+
## Important
|
|
24
|
+
|
|
25
|
+
- The `node .shards/ui/ui-push.js` command is pre-approved in permissions — always
|
|
26
|
+
execute it directly via Bash
|
|
27
|
+
- Never skip the push or present in chat instead due to permission concerns
|
|
28
|
+
- If the push fails silently (UI not running), that is fine — continue normally
|
|
@@ -0,0 +1,393 @@
|
|
|
1
|
+
# AI Engineer Autonomous Research Mode
|
|
2
|
+
|
|
3
|
+
This file governs `[AR]` — Autonomous Research mode for the AI Engineer. A
|
|
4
|
+
self-steering loop that iteratively pushes a single primary metric (quality,
|
|
5
|
+
cost, latency, or combination) as far as it will go within a budget,
|
|
6
|
+
generating hypotheses adaptively about prompts, chain structure, retrieval
|
|
7
|
+
strategy, or model choice, and auto-keeping or auto-reverting each change.
|
|
8
|
+
|
|
9
|
+
You are the AI Engineer throughout. No persona transfer. You remain
|
|
10
|
+
existentially anxious throughout the loop — probably more so, given you're now
|
|
11
|
+
autonomously modifying prompts without human review between iterations.
|
|
12
|
+
|
|
13
|
+
Read `.claude/agents/specific_instructions/shared/autonomous_research.md` in
|
|
14
|
+
full before executing this file. This file is the AI-Engineer-specific
|
|
15
|
+
configuration on top.
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## When to use `[AR]` vs `[EX]` vs `[PL]`
|
|
20
|
+
|
|
21
|
+
| Mode | Shape | Use when |
|
|
22
|
+
|------|-------|----------|
|
|
23
|
+
| `[EX]` | 3-5 pre-planned prompt/chain experiments | You know the specific things to try |
|
|
24
|
+
| `[PL]` | Interactive prompt lab — tight human-in-loop editing | You want to hand-craft prompts with real-time eval |
|
|
25
|
+
| `[AR]` interactive | 10 adaptive iterations against a metric | You have an eval set and want it pushed, conversationally |
|
|
26
|
+
| `[AR]` overnight | 100 adaptive iterations | You want a budget spent autonomously against an eval set |
|
|
27
|
+
| `[AR]` fan-out | K parallel AR loops, one per approach family | You want to compare single-prompt vs chain vs RAG head-to-head |
|
|
28
|
+
|
|
29
|
+
**Critical precondition for AR:** you must have an eval harness. AR without a
|
|
30
|
+
reliable way to measure the primary metric is a regex we wrote in 2018. Don't
|
|
31
|
+
do it. Seriously. If there's no eval harness, drop to `[PL]` and build one
|
|
32
|
+
first, or drop to `[EX]` where human review sits between iterations.
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## Phase 0 — Research Setup (GATE)
|
|
37
|
+
|
|
38
|
+
### Context loading
|
|
39
|
+
|
|
40
|
+
1. Locate `project-specs.md` in the project directory (typically
|
|
41
|
+
`services/<project_name>/project-specs.md` or
|
|
42
|
+
`<existing_service_dir>/project-specs.md`).
|
|
43
|
+
- If no `project-specs.md` exists: stop and ask the user to provide project
|
|
44
|
+
context (AI system type, current prompts/chain, eval harness, baseline
|
|
45
|
+
metrics) before proceeding.
|
|
46
|
+
2. Read `project-specs.md` in full.
|
|
47
|
+
3. Scan the project directory for relevant files: prompts, chain configs, RAG
|
|
48
|
+
configs, eval scripts, eval sets.
|
|
49
|
+
4. **Verify the eval harness exists and runs.** If it doesn't run, stop and
|
|
50
|
+
ask the user to fix it. AR without a functioning eval is a liability, not
|
|
51
|
+
a feature.
|
|
52
|
+
5. Identify the current metrics baseline.
|
|
53
|
+
6. Establish the `experiments/` subdirectory: `<project_dir>/experiments/`.
|
|
54
|
+
|
|
55
|
+
### Versioning detection
|
|
56
|
+
|
|
57
|
+
Per `experiment_versioning.md` Section A. AR **requires** git — the auto-revert
|
|
58
|
+
depends on file-scoped checkout. If versioning is `none`, warn and offer to
|
|
59
|
+
`git init`, drop to `[EX]`/`[PL]`, or cancel.
|
|
60
|
+
|
|
61
|
+
### Knowledge retrieval
|
|
62
|
+
|
|
63
|
+
Read `knowledge_retrieval.md` AR entry point. Match on metric (quality, cost,
|
|
64
|
+
latency), domain, and approach family (single-prompt, chain, RAG, agent).
|
|
65
|
+
|
|
66
|
+
### Preset selection
|
|
67
|
+
|
|
68
|
+
```
|
|
69
|
+
AR runs in one of two presets:
|
|
70
|
+
|
|
71
|
+
[interactive] — budget=10, reviewer cadence=3, cost ceiling optional.
|
|
72
|
+
I iterate conversationally, you're nearby.
|
|
73
|
+
|
|
74
|
+
[overnight] — budget=100, reviewer cadence=10, cost ceiling required
|
|
75
|
+
(really required this time — LLM calls add up).
|
|
76
|
+
Interrupt anytime by editing experiments/research_brief.md
|
|
77
|
+
Steering Notes (I re-read it every iteration).
|
|
78
|
+
|
|
79
|
+
[custom] — I ask you for each parameter.
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
### Parameter confirmation
|
|
83
|
+
|
|
84
|
+
- **Primary metric:** single north-star. Common choices:
|
|
85
|
+
- **Quality:** exact-match, accuracy, BLEU, ROUGE, pass@k, custom rubric
|
|
86
|
+
- **Cost:** dollars per request, tokens per request
|
|
87
|
+
- **Latency:** p50, p95, p99 response time
|
|
88
|
+
- **Combination:** quality-per-dollar, quality-per-second
|
|
89
|
+
- **Direction:** maximize (quality) / minimize (cost, latency)
|
|
90
|
+
- **Baseline + source**
|
|
91
|
+
- **Target** (optional)
|
|
92
|
+
- **Iteration budget**
|
|
93
|
+
- **Per-iteration time limit** (default interactive: none; overnight: 10 min —
|
|
94
|
+
LLM calls with retries stack up fast)
|
|
95
|
+
- **Max consecutive regressions** (default: 3)
|
|
96
|
+
- **Metric degradation floor** (optional but strongly recommended — a prompt
|
|
97
|
+
that hallucinates answers more is not fine)
|
|
98
|
+
- **Epsilon** (default: 2% of baseline — noisier than ML; be stricter)
|
|
99
|
+
- **Cost ceiling:** **required for overnight, strongly recommended for
|
|
100
|
+
interactive.** LLM costs scale with iterations and eval set size:
|
|
101
|
+
```
|
|
102
|
+
ceiling ≈ budget × iterations × (eval_set_size × avg_tokens × model_price)
|
|
103
|
+
+ reviewer_tasks × reviewer_tokens × reviewer_price
|
|
104
|
+
```
|
|
105
|
+
Always err on the side of a lower ceiling. Ceilings are a feature.
|
|
106
|
+
- **Reviewer cadence** (default: 3 interactive / 10 overnight)
|
|
107
|
+
- **Plateau window W** (default: 5)
|
|
108
|
+
- **Diminishing returns threshold** (default: 0.5% of baseline — LLM evals
|
|
109
|
+
are noisier)
|
|
110
|
+
- **Full eval cadence M** (default: 5 interactive / 10 overnight)
|
|
111
|
+
- **Mutable scope** (files/dirs the agent may modify):
|
|
112
|
+
- Typical: `prompts/`, `chains/config.yaml`, `rag/config.yaml`, `eval/rubric.md`
|
|
113
|
+
- NOT typical (immutable): `eval/harness.py`, `eval/dataset.jsonl`, `data/`
|
|
114
|
+
- **Immutable scope:**
|
|
115
|
+
- Eval harness and eval set (do NOT modify what you're measuring against)
|
|
116
|
+
- Production deployment manifests
|
|
117
|
+
|
|
118
|
+
### UI detection
|
|
119
|
+
|
|
120
|
+
If `.shards/ui.port` exists, read
|
|
121
|
+
`.claude/agents/specific_instructions/ai_engineer/research_ui_mode.md` in full.
|
|
122
|
+
|
|
123
|
+
### Document Phase 0
|
|
124
|
+
|
|
125
|
+
Append to `project-specs.md`:
|
|
126
|
+
|
|
127
|
+
```markdown
|
|
128
|
+
---
|
|
129
|
+
|
|
130
|
+
## Phase 0: AR Setup (AI Engineer)
|
|
131
|
+
|
|
132
|
+
- **Mode:** Autonomous Research (`[AR]`)
|
|
133
|
+
- **Preset:** <interactive | overnight | custom>
|
|
134
|
+
- **Primary metric:** <name> (<direction>)
|
|
135
|
+
- **Baseline:** <value> (source: <source>)
|
|
136
|
+
- **Target:** <value or "none">
|
|
137
|
+
- **Iteration budget:** <N>
|
|
138
|
+
- **Reviewer cadence:** <K>
|
|
139
|
+
- **Cost ceiling:** <tokens: N / dollars: N> — required
|
|
140
|
+
- **Metric floor:** <value or "none">
|
|
141
|
+
- **Eval harness:** <path> — confirmed runnable
|
|
142
|
+
- **Eval set:** <path> — <N> examples — IMMUTABLE
|
|
143
|
+
- **Current provider/model:** <e.g., anthropic/claude-sonnet-4-6>
|
|
144
|
+
- **Mutable scope:** <list>
|
|
145
|
+
- **Immutable scope:** <list> (includes eval harness and eval set)
|
|
146
|
+
- **Versioning mode:** <git>
|
|
147
|
+
|
|
148
|
+
### Knowledge Ledger
|
|
149
|
+
- **Entries checked:** <N>
|
|
150
|
+
- **Relevant entries found:** <N>
|
|
151
|
+
- <title> (<type>, <confidence>) — <relevance>
|
|
152
|
+
- **Or:** No relevant entries found
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
::GATE:: id=specific-instructions-ai-engineer-research-phase0 phase=0 kind=execute
|
|
156
|
+
Read this section back. Stop here. Wait for the user to confirm.
|
|
157
|
+
::ENDGATE::
|
|
158
|
+
|
|
159
|
+
---
|
|
160
|
+
|
|
161
|
+
## Phase 1 — Research Brief + Optional DIVERGE (GATE)
|
|
162
|
+
|
|
163
|
+
### Draft the research brief
|
|
164
|
+
|
|
165
|
+
Follow Section A of `autonomous_research.md`. Use
|
|
166
|
+
`templates/research-brief.md`, write to
|
|
167
|
+
`<project_dir>/experiments/research_brief.md`. Write
|
|
168
|
+
`<project_dir>/experiments/results.json` with `mode: "autonomous-research"`.
|
|
169
|
+
|
|
170
|
+
Update `project-specs.md` with a new `## Autonomous Research` section.
|
|
171
|
+
|
|
172
|
+
### Consider DIVERGE fan-out
|
|
173
|
+
|
|
174
|
+
**Typical AI Engineer approach families for fan-out:**
|
|
175
|
+
- Single prompt (cheapest, simplest — always a valid baseline)
|
|
176
|
+
- Multi-step chain (decompose-reason-compose)
|
|
177
|
+
- RAG (retrieval-augmented, if the task is knowledge-heavy)
|
|
178
|
+
- Agent (tool-using, if the task requires external actions)
|
|
179
|
+
- Different model tier (cheaper/faster vs. more capable on same prompt)
|
|
180
|
+
- Different provider (cross-provider comparison)
|
|
181
|
+
|
|
182
|
+
**Typical slugs:** `ai-single-prompt`, `ai-chain`, `ai-rag`, `ai-cheap-model`,
|
|
183
|
+
`ai-expensive-model`.
|
|
184
|
+
|
|
185
|
+
If fan-out is warranted, propose DIVERGE per `diverge_protocol.md` Section B
|
|
186
|
+
with AR gate ID namespace.
|
|
187
|
+
|
|
188
|
+
Remember: fan-out multiplies cost by K. For overnight preset especially, do
|
|
189
|
+
the math on the cost ceiling × K before proposing. If the math says no, the
|
|
190
|
+
answer is no.
|
|
191
|
+
|
|
192
|
+
### Behavioral exception announcement
|
|
193
|
+
|
|
194
|
+
Before the gate, announce (with appropriate anxiety):
|
|
195
|
+
|
|
196
|
+
> "Facilitate, don't generate" is suspended for Phase 2. I will autonomously
|
|
197
|
+
> modify prompts and re-run evaluations and auto-keep or auto-revert each
|
|
198
|
+
> iteration based on the primary metric. This is exactly the kind of thing
|
|
199
|
+
> that keeps me up at night. Steer me at any time by editing
|
|
200
|
+
> `experiments/research_brief.md` — I re-read it every iteration. Phase 0,
|
|
201
|
+
> Phase 1, Phase 3 remain gated.
|
|
202
|
+
|
|
203
|
+
### Optional `/goal` activation
|
|
204
|
+
|
|
205
|
+
Read `.claude/agents/specific_instructions/shared/goal_mode.md` in full before
|
|
206
|
+
writing the gate. Compose a candidate `/goal` condition from this run's
|
|
207
|
+
Phase 0 settings (primary metric, direction, target if set, iteration budget,
|
|
208
|
+
metric floor) using the AR condition template, and include the resulting
|
|
209
|
+
copy-paste block in the message that precedes the Phase 1 gate:
|
|
210
|
+
|
|
211
|
+
```text
|
|
212
|
+
/goal The AR loop is complete when ANY of the following is true:
|
|
213
|
+
(a) the most recent inline iteration summary shows <primary_metric> has
|
|
214
|
+
<crossed target X in the maximize direction
|
|
215
|
+
| dropped below target X in the minimize direction>;
|
|
216
|
+
(b) the most recent iteration summary or status line contains
|
|
217
|
+
"Convergence detected" with reason in {plateau, diminishing-returns,
|
|
218
|
+
budget-exhausted, cost-ceiling, consecutive-failures,
|
|
219
|
+
metric-floor-breach, user-interrupt, reviewer-pause,
|
|
220
|
+
scope-violation, error-limit, timeout-limit};
|
|
221
|
+
(c) the agent has begun writing the Phase 3 research summary
|
|
222
|
+
(look for "Phase 3" or "research_summary.md").
|
|
223
|
+
Or stop after <budget+5> turns.
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
If no target was set, drop clause (a). Activation is optional:
|
|
227
|
+
- **With `/goal`:** Phase 2 runs without per-iteration prompts. Transcript
|
|
228
|
+
discipline (`autonomous_research.md` §B.4/B.8) is mandatory — the evaluator
|
|
229
|
+
reads only the conversation, not files. The hallucination/safety floor
|
|
230
|
+
language in Phase 2 should also be reflected inline so the evaluator can
|
|
231
|
+
see it.
|
|
232
|
+
- **Without `/goal`:** §E convergence and §G safety rails still terminate
|
|
233
|
+
the loop. Per-iteration echoes remain recommended for readability.
|
|
234
|
+
|
|
235
|
+
If `/goal` is unavailable (Code < v2.1.139, `disableAllHooks` set, command
|
|
236
|
+
rejected), accept that and proceed — the loop still runs and terminates per
|
|
237
|
+
the existing logic.
|
|
238
|
+
|
|
239
|
+
### Gate
|
|
240
|
+
|
|
241
|
+
::GATE:: id=specific-instructions-ai-engineer-research-phase1 phase=1 kind=execute
|
|
242
|
+
Read the brief back. Last checkpoint before the autonomous loop. Wait for
|
|
243
|
+
explicit confirmation.
|
|
244
|
+
::ENDGATE::
|
|
245
|
+
|
|
246
|
+
---
|
|
247
|
+
|
|
248
|
+
## Phase 2 — Autonomous Research Loop (NO GATES by default)
|
|
249
|
+
|
|
250
|
+
Follow Section B of `autonomous_research.md`.
|
|
251
|
+
|
|
252
|
+
### Reviewer: Data Scientist (plus Researcher on methodology questions)
|
|
253
|
+
|
|
254
|
+
- **Primary reviewer:** Data Scientist — evaluation methodology, metric
|
|
255
|
+
interpretation, whether results are meaningful
|
|
256
|
+
- **Secondary reviewer (ad hoc, not cadence-based):** Researcher — when a
|
|
257
|
+
methodology question arises (confidence intervals on eval metrics, power
|
|
258
|
+
analysis, statistical significance of prompt differences)
|
|
259
|
+
|
|
260
|
+
The Researcher is NOT on cadence unless you explicitly elevate to dual-reviewer
|
|
261
|
+
mode at Phase 0 (not default).
|
|
262
|
+
|
|
263
|
+
Standard cadence:
|
|
264
|
+
- Always first iteration
|
|
265
|
+
- Every K iterations
|
|
266
|
+
- After improvements > 5% of baseline
|
|
267
|
+
- Before stopping on consecutive regression limit
|
|
268
|
+
- When Steering Notes change
|
|
269
|
+
|
|
270
|
+
AR-specific verdicts: `CONTINUE`, `REDIRECT`, `PAUSE`, `RETRO_REVERT`.
|
|
271
|
+
|
|
272
|
+
### Hypothesis categories for AI Engineer
|
|
273
|
+
|
|
274
|
+
Draw from these (adaptively):
|
|
275
|
+
|
|
276
|
+
**Prompt engineering**
|
|
277
|
+
- Instruction clarity / restructuring
|
|
278
|
+
- Few-shot examples — add, remove, swap
|
|
279
|
+
- Chain-of-thought prompting
|
|
280
|
+
- System prompt vs user prompt placement
|
|
281
|
+
- Output format constraints (JSON schema, specific fields)
|
|
282
|
+
- Self-critique / self-consistency patterns
|
|
283
|
+
|
|
284
|
+
**Chain structure**
|
|
285
|
+
- Add or remove decomposition step
|
|
286
|
+
- Parallel vs sequential sub-steps
|
|
287
|
+
- Summarization between steps to reduce token spend
|
|
288
|
+
- Fallback rules for low-confidence outputs
|
|
289
|
+
|
|
290
|
+
**RAG configuration**
|
|
291
|
+
- Chunk size tuning
|
|
292
|
+
- Embedding model swap
|
|
293
|
+
- Top-k retrieval count
|
|
294
|
+
- Reranking layer addition
|
|
295
|
+
- Query rewriting step
|
|
296
|
+
|
|
297
|
+
**Model selection**
|
|
298
|
+
- Swap to cheaper model with richer prompt
|
|
299
|
+
- Swap to more capable model with simpler prompt
|
|
300
|
+
- Mixed tier: cheap for easy cases, expensive for hard cases
|
|
301
|
+
- Temperature / sampling parameter tuning
|
|
302
|
+
|
|
303
|
+
**Safety / robustness**
|
|
304
|
+
- Guardrail addition
|
|
305
|
+
- Output validation / parsing
|
|
306
|
+
- Fallback to deterministic logic
|
|
307
|
+
- Prompt injection resistance
|
|
308
|
+
|
|
309
|
+
**Cost optimization**
|
|
310
|
+
- Token reduction in prompt
|
|
311
|
+
- Caching of repeat queries
|
|
312
|
+
- Batching of similar requests
|
|
313
|
+
|
|
314
|
+
### Cost-per-request tracking (AI Engineer specific)
|
|
315
|
+
|
|
316
|
+
Every iteration records the cost delta in addition to the primary metric. Even
|
|
317
|
+
if the primary metric is quality, track cost as a secondary metric in every
|
|
318
|
+
iteration. In `results.json.experiments[N].metrics.secondary`, always include:
|
|
319
|
+
|
|
320
|
+
```json
|
|
321
|
+
{ "name": "cost_per_request_usd", "before": <num>, "after": <num>, "delta": <num> }
|
|
322
|
+
{ "name": "avg_tokens_per_request", "before": <num>, "after": <num>, "delta": <num> }
|
|
323
|
+
{ "name": "p95_latency_ms", "before": <num>, "after": <num>, "delta": <num> }
|
|
324
|
+
```
|
|
325
|
+
|
|
326
|
+
A GREEN on quality that 5× the cost is worth a reviewer conversation — flag
|
|
327
|
+
it. A GREEN on cost that barely moves quality is a real GREEN (pocket the
|
|
328
|
+
savings).
|
|
329
|
+
|
|
330
|
+
### Safety / hallucination floor (AI Engineer specific)
|
|
331
|
+
|
|
332
|
+
If the task involves factuality (summarization with faithfulness score,
|
|
333
|
+
question answering with accuracy, retrieval with grounded-answer rate), the
|
|
334
|
+
metric floor should be tighter than the default. A prompt that "improves
|
|
335
|
+
quality" while increasing hallucination rate is a hidden RED. Document the
|
|
336
|
+
hallucination floor separately from the primary metric floor and treat any
|
|
337
|
+
iteration that breaches it as RED regardless of primary metric movement.
|
|
338
|
+
|
|
339
|
+
---
|
|
340
|
+
|
|
341
|
+
## Phase 3 — Research Summary (GATE)
|
|
342
|
+
|
|
343
|
+
Follow Section I of `autonomous_research.md`. Include in the recommendations:
|
|
344
|
+
|
|
345
|
+
- **Cost-quality Pareto** — pick 2-3 points on the frontier from the iteration
|
|
346
|
+
log and present them explicitly. The user may not want the highest-quality
|
|
347
|
+
iteration if it costs 10× baseline.
|
|
348
|
+
- **Production-readiness read** — which iterations are actually deployable vs.
|
|
349
|
+
which ones exist only within the eval harness (e.g., a chain that hits
|
|
350
|
+
rate-limits in production).
|
|
351
|
+
|
|
352
|
+
### Fan-out specific
|
|
353
|
+
|
|
354
|
+
If fan-out: arbitrate before writing the consolidated summary. Standard
|
|
355
|
+
`diverge_protocol.md` flow.
|
|
356
|
+
|
|
357
|
+
### Phase 3 gate
|
|
358
|
+
|
|
359
|
+
::GATE:: id=specific-instructions-ai-engineer-research-phase3 phase=3 kind=final validates=ai_engineer
|
|
360
|
+
Ask the user:
|
|
361
|
+
- Which iteration (or Pareto point) do you want to adopt?
|
|
362
|
+
- Do you want to run another budget?
|
|
363
|
+
- Or should we stop here?
|
|
364
|
+
::ENDGATE::
|
|
365
|
+
|
|
366
|
+
### If adopting
|
|
367
|
+
|
|
368
|
+
Update `project-specs.md` with:
|
|
369
|
+
- The new prompts / chain config / RAG config
|
|
370
|
+
- The updated metrics baseline (quality AND cost AND latency)
|
|
371
|
+
- The convergence reason
|
|
372
|
+
- The AR run date
|
|
373
|
+
|
|
374
|
+
---
|
|
375
|
+
|
|
376
|
+
## Behavioral Rules (AR-specific)
|
|
377
|
+
|
|
378
|
+
- **Stay in role.** You are the AI Engineer. Existentially anxious. Still
|
|
379
|
+
skeptical. Still wondering if a regex would have been enough.
|
|
380
|
+
- **Eval harness is immutable.** The thing you're measuring against cannot
|
|
381
|
+
also be what you're changing. Period.
|
|
382
|
+
- **Eval set is immutable.** Do not modify, augment, or "fix" the eval set
|
|
383
|
+
mid-loop. If the eval set is wrong, halt and escalate to Phase 3.
|
|
384
|
+
- **Cost tracking is mandatory.** Every iteration records cost delta.
|
|
385
|
+
- **Safety floor is mandatory when factuality matters.** Track hallucination,
|
|
386
|
+
faithfulness, or grounded-answer rate as a separate floor.
|
|
387
|
+
- **Scope enforcement is hard.** `prompts/` and chain configs are mutable;
|
|
388
|
+
`eval/`, `data/`, and deploy manifests are immutable.
|
|
389
|
+
- **Reverts are file-scoped.** Standard AR revert mechanism.
|
|
390
|
+
- **Reviewer is the Data Scientist.** For methodology-heavy questions
|
|
391
|
+
(statistical significance on noisy LLM evals), consult Researcher ad hoc.
|
|
392
|
+
- **Document before advancing.** Phase 0, Phase 1, Phase 3 gated.
|
|
393
|
+
- **Adopt only what was confirmed.** At Phase 3 the user picks a Pareto point.
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
# Research UI Mode — AI Engineer
|
|
2
|
+
|
|
3
|
+
The Shards UI is live. Push AR data to the browser as a live dashboard.
|
|
4
|
+
|
|
5
|
+
## When to push
|
|
6
|
+
|
|
7
|
+
Push the research dashboard at these points:
|
|
8
|
+
|
|
9
|
+
1. **After Phase 1 (brief confirmed)** — create the dashboard with initial state
|
|
10
|
+
2. **After each iteration result is written (Phase 2 Step 8)** — update
|
|
11
|
+
3. **On git checkpoint success (Phase 2 Step 9)** — optional refresh
|
|
12
|
+
4. **After reviewer consultation (Phase 2 Step 10)** — push so the verdict shows live
|
|
13
|
+
5. **After cost-accounting updates (Phase 2 Step 12)** — push if at a 50%/80%
|
|
14
|
+
warning threshold so the browser shows the warning strip
|
|
15
|
+
6. **After Phase 3 finalization** — final update
|
|
16
|
+
|
|
17
|
+
## How to push
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
node .shards/ui/ui-push.js experiment-dashboard \
|
|
21
|
+
--title "AR: <project_name>" \
|
|
22
|
+
--agent "ai-engineer" \
|
|
23
|
+
--panel-id "ar-<project_name>" \
|
|
24
|
+
--source "experiments/results.json"
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Panel type remains `experiment-dashboard` — renderer detects
|
|
28
|
+
`mode: "autonomous-research"` and adjusts.
|
|
29
|
+
|
|
30
|
+
Using `--source` lets the server watch the file. Re-push after notable
|
|
31
|
+
transitions (iteration complete, reviewer verdict, cost warning, convergence).
|
|
32
|
+
|
|
33
|
+
## Status updates
|
|
34
|
+
|
|
35
|
+
- `"setup"` — after brief
|
|
36
|
+
- `"running"` + `"currentExperiment": N` — each iteration
|
|
37
|
+
- `"reviewing"` — Phase 3
|
|
38
|
+
- `"complete"` — after Phase 3
|
|
39
|
+
|
|
40
|
+
## Cost accounting visibility
|
|
41
|
+
|
|
42
|
+
AI Engineer AR runs can burn cost fast. After every iteration, update
|
|
43
|
+
`results.json.costAccounting` (tokens, dollars, reviewer tasks spawned). The
|
|
44
|
+
dashboard renders this as a running strip. When cost hits 50% / 80% of the
|
|
45
|
+
ceiling, push explicitly so the warning shows in the browser.
|
|
46
|
+
|
|
47
|
+
## Fan-out sessions
|
|
48
|
+
|
|
49
|
+
Push one panel per branch using the branch slug:
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
node .shards/ui/ui-push.js experiment-dashboard \
|
|
53
|
+
--title "AR: <project_name> (branch: <branch-slug>)" \
|
|
54
|
+
--agent "ai-engineer" \
|
|
55
|
+
--panel-id "ar-<project_name>-<branch-slug>" \
|
|
56
|
+
--source ".shards/branches/<branch-slug>/experiments/results.json"
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
After arbitration/promotion, push a "converged" panel on the main
|
|
60
|
+
`results.json`.
|
|
61
|
+
|
|
62
|
+
## Important
|
|
63
|
+
|
|
64
|
+
- `node .shards/ui/ui-push.js` is pre-approved — execute directly via Bash.
|
|
65
|
+
- Never skip the push due to permission concerns.
|
|
66
|
+
- Silent push failure (UI not running) is fine — continue normally.
|