@proflandrigan/shards 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (397) hide show
  1. package/README.md +475 -0
  2. package/package.json +37 -0
  3. package/src/agents/academic.md +276 -0
  4. package/src/agents/ai-engineer.md +377 -0
  5. package/src/agents/analytics-engineer.md +364 -0
  6. package/src/agents/applied-ml-scientist.md +410 -0
  7. package/src/agents/backend-engineer.md +255 -0
  8. package/src/agents/bi-engineer.md +333 -0
  9. package/src/agents/data-analyst.md +343 -0
  10. package/src/agents/data-engineer.md +260 -0
  11. package/src/agents/data-modeller.md +386 -0
  12. package/src/agents/data-scientist.md +366 -0
  13. package/src/agents/deep-learning-engineer.md +389 -0
  14. package/src/agents/ml-engineer.md +424 -0
  15. package/src/agents/mlops-engineer.md +339 -0
  16. package/src/agents/researcher.md +187 -0
  17. package/src/agents/specific_instructions/academic/critical_review.md +263 -0
  18. package/src/agents/specific_instructions/academic/report.md +113 -0
  19. package/src/agents/specific_instructions/ai_engineer/advise.md +162 -0
  20. package/src/agents/specific_instructions/ai_engineer/bi_engineer_handoff.md +86 -0
  21. package/src/agents/specific_instructions/ai_engineer/experiment.md +471 -0
  22. package/src/agents/specific_instructions/ai_engineer/experiment_ui_mode.md +44 -0
  23. package/src/agents/specific_instructions/ai_engineer/phases/index.md +45 -0
  24. package/src/agents/specific_instructions/ai_engineer/phases/phase-1.md +55 -0
  25. package/src/agents/specific_instructions/ai_engineer/phases/phase-2.md +86 -0
  26. package/src/agents/specific_instructions/ai_engineer/phases/phase-3.md +96 -0
  27. package/src/agents/specific_instructions/ai_engineer/phases/phase-4.md +138 -0
  28. package/src/agents/specific_instructions/ai_engineer/phases/phase-5.md +157 -0
  29. package/src/agents/specific_instructions/ai_engineer/phases/phase-6.md +196 -0
  30. package/src/agents/specific_instructions/ai_engineer/phases/phase-7.md +313 -0
  31. package/src/agents/specific_instructions/ai_engineer/phases.md +1011 -0
  32. package/src/agents/specific_instructions/ai_engineer/prompt_lab.md +161 -0
  33. package/src/agents/specific_instructions/ai_engineer/prompt_lab_ui_mode.md +28 -0
  34. package/src/agents/specific_instructions/ai_engineer/research.md +393 -0
  35. package/src/agents/specific_instructions/ai_engineer/research_ui_mode.md +66 -0
  36. package/src/agents/specific_instructions/ai_engineer/review.md +159 -0
  37. package/src/agents/specific_instructions/ai_engineer/validation_checklist.md +182 -0
  38. package/src/agents/specific_instructions/analytics_engineer/advise.md +155 -0
  39. package/src/agents/specific_instructions/analytics_engineer/bi_engineer_handoff.md +91 -0
  40. package/src/agents/specific_instructions/analytics_engineer/data_analyst_handoff.md +84 -0
  41. package/src/agents/specific_instructions/analytics_engineer/deep_phases.md +818 -0
  42. package/src/agents/specific_instructions/analytics_engineer/phases_deep/index.md +24 -0
  43. package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-1.md +77 -0
  44. package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-2.md +106 -0
  45. package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-3.md +93 -0
  46. package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-4.md +79 -0
  47. package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-5.md +61 -0
  48. package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-6.md +45 -0
  49. package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-7.md +235 -0
  50. package/src/agents/specific_instructions/analytics_engineer/phases_deep/phase-8.md +221 -0
  51. package/src/agents/specific_instructions/analytics_engineer/phases_quick/index.md +19 -0
  52. package/src/agents/specific_instructions/analytics_engineer/phases_quick/phase-1.md +47 -0
  53. package/src/agents/specific_instructions/analytics_engineer/phases_quick/phase-2.md +78 -0
  54. package/src/agents/specific_instructions/analytics_engineer/quick_phases.md +112 -0
  55. package/src/agents/specific_instructions/analytics_engineer/review.md +167 -0
  56. package/src/agents/specific_instructions/analytics_engineer/service_mode.md +369 -0
  57. package/src/agents/specific_instructions/analytics_engineer/ui_mode.md +45 -0
  58. package/src/agents/specific_instructions/analytics_engineer/update.md +162 -0
  59. package/src/agents/specific_instructions/analytics_engineer/validation_checklist.md +121 -0
  60. package/src/agents/specific_instructions/applied_ml_scientist/advise.md +143 -0
  61. package/src/agents/specific_instructions/applied_ml_scientist/phases/index.md +21 -0
  62. package/src/agents/specific_instructions/applied_ml_scientist/phases/phase-1.md +51 -0
  63. package/src/agents/specific_instructions/applied_ml_scientist/phases/phase-2.md +66 -0
  64. package/src/agents/specific_instructions/applied_ml_scientist/phases/phase-3.md +113 -0
  65. package/src/agents/specific_instructions/applied_ml_scientist/phases/phase-4.md +104 -0
  66. package/src/agents/specific_instructions/applied_ml_scientist/phases/phase-5.md +156 -0
  67. package/src/agents/specific_instructions/applied_ml_scientist/phases.md +428 -0
  68. package/src/agents/specific_instructions/applied_ml_scientist/research.md +379 -0
  69. package/src/agents/specific_instructions/applied_ml_scientist/review.md +142 -0
  70. package/src/agents/specific_instructions/applied_ml_scientist/validation_checklist.md +136 -0
  71. package/src/agents/specific_instructions/backend_engineer/clean.md +149 -0
  72. package/src/agents/specific_instructions/backend_engineer/review.md +91 -0
  73. package/src/agents/specific_instructions/backend_engineer/review_checklist.md +54 -0
  74. package/src/agents/specific_instructions/backend_engineer/service_mode.md +67 -0
  75. package/src/agents/specific_instructions/bi_engineer/advise.md +137 -0
  76. package/src/agents/specific_instructions/bi_engineer/data_analyst_handoff.md +77 -0
  77. package/src/agents/specific_instructions/bi_engineer/incoming_handoff.md +45 -0
  78. package/src/agents/specific_instructions/bi_engineer/phases/index.md +20 -0
  79. package/src/agents/specific_instructions/bi_engineer/phases/phase-1.md +164 -0
  80. package/src/agents/specific_instructions/bi_engineer/phases/phase-2.md +92 -0
  81. package/src/agents/specific_instructions/bi_engineer/phases/phase-3.md +121 -0
  82. package/src/agents/specific_instructions/bi_engineer/phases/phase-4.md +106 -0
  83. package/src/agents/specific_instructions/bi_engineer/phases.md +451 -0
  84. package/src/agents/specific_instructions/bi_engineer/review.md +166 -0
  85. package/src/agents/specific_instructions/bi_engineer/update.md +147 -0
  86. package/src/agents/specific_instructions/bi_engineer/validation_checklist.md +124 -0
  87. package/src/agents/specific_instructions/data_analyst/advise.md +138 -0
  88. package/src/agents/specific_instructions/data_analyst/explain.md +221 -0
  89. package/src/agents/specific_instructions/data_analyst/incoming_handoff.md +40 -0
  90. package/src/agents/specific_instructions/data_analyst/phases/index.md +20 -0
  91. package/src/agents/specific_instructions/data_analyst/phases/phase-1.md +159 -0
  92. package/src/agents/specific_instructions/data_analyst/phases/phase-2.md +112 -0
  93. package/src/agents/specific_instructions/data_analyst/phases/phase-3.md +265 -0
  94. package/src/agents/specific_instructions/data_analyst/phases/phase-4.md +100 -0
  95. package/src/agents/specific_instructions/data_analyst/phases.md +501 -0
  96. package/src/agents/specific_instructions/data_analyst/review.md +138 -0
  97. package/src/agents/specific_instructions/data_analyst/ui_mode.md +26 -0
  98. package/src/agents/specific_instructions/data_analyst/update.md +144 -0
  99. package/src/agents/specific_instructions/data_analyst/validation_checklist.md +95 -0
  100. package/src/agents/specific_instructions/data_engineer/advise.md +137 -0
  101. package/src/agents/specific_instructions/data_engineer/phases.md +466 -0
  102. package/src/agents/specific_instructions/data_engineer/phases_deep/index.md +23 -0
  103. package/src/agents/specific_instructions/data_engineer/phases_deep/phase-1.md +49 -0
  104. package/src/agents/specific_instructions/data_engineer/phases_deep/phase-2.md +93 -0
  105. package/src/agents/specific_instructions/data_engineer/phases_deep/phase-3.md +55 -0
  106. package/src/agents/specific_instructions/data_engineer/phases_deep/phase-4.md +48 -0
  107. package/src/agents/specific_instructions/data_engineer/phases_deep/phase-5.md +40 -0
  108. package/src/agents/specific_instructions/data_engineer/phases_deep/phase-6.md +102 -0
  109. package/src/agents/specific_instructions/data_engineer/phases_deep/phase-7.md +87 -0
  110. package/src/agents/specific_instructions/data_engineer/phases_quick/index.md +19 -0
  111. package/src/agents/specific_instructions/data_engineer/phases_quick/phase-1.md +45 -0
  112. package/src/agents/specific_instructions/data_engineer/phases_quick/phase-2.md +54 -0
  113. package/src/agents/specific_instructions/data_engineer/review.md +135 -0
  114. package/src/agents/specific_instructions/data_engineer/validation_checklist.md +136 -0
  115. package/src/agents/specific_instructions/data_modeller/advise.md +137 -0
  116. package/src/agents/specific_instructions/data_modeller/phases.md +581 -0
  117. package/src/agents/specific_instructions/data_modeller/phases_deep/index.md +23 -0
  118. package/src/agents/specific_instructions/data_modeller/phases_deep/phase-1.md +52 -0
  119. package/src/agents/specific_instructions/data_modeller/phases_deep/phase-2.md +113 -0
  120. package/src/agents/specific_instructions/data_modeller/phases_deep/phase-3.md +47 -0
  121. package/src/agents/specific_instructions/data_modeller/phases_deep/phase-4.md +51 -0
  122. package/src/agents/specific_instructions/data_modeller/phases_deep/phase-5.md +45 -0
  123. package/src/agents/specific_instructions/data_modeller/phases_deep/phase-6.md +105 -0
  124. package/src/agents/specific_instructions/data_modeller/phases_deep/phase-7.md +136 -0
  125. package/src/agents/specific_instructions/data_modeller/phases_quick/index.md +19 -0
  126. package/src/agents/specific_instructions/data_modeller/phases_quick/phase-1.md +47 -0
  127. package/src/agents/specific_instructions/data_modeller/phases_quick/phase-2.md +65 -0
  128. package/src/agents/specific_instructions/data_modeller/review.md +141 -0
  129. package/src/agents/specific_instructions/data_modeller/service_mode.md +218 -0
  130. package/src/agents/specific_instructions/data_modeller/validation_checklist.md +125 -0
  131. package/src/agents/specific_instructions/data_scientist/advise.md +158 -0
  132. package/src/agents/specific_instructions/data_scientist/bi_engineer_handoff.md +63 -0
  133. package/src/agents/specific_instructions/data_scientist/experiment.md +482 -0
  134. package/src/agents/specific_instructions/data_scientist/experiment_ui_mode.md +44 -0
  135. package/src/agents/specific_instructions/data_scientist/explain.md +247 -0
  136. package/src/agents/specific_instructions/data_scientist/greenfield_data.md +35 -0
  137. package/src/agents/specific_instructions/data_scientist/ml_engineer_handoff.md +52 -0
  138. package/src/agents/specific_instructions/data_scientist/notebook_walkthrough.md +76 -0
  139. package/src/agents/specific_instructions/data_scientist/phases/index.md +24 -0
  140. package/src/agents/specific_instructions/data_scientist/phases/phase-1.md +45 -0
  141. package/src/agents/specific_instructions/data_scientist/phases/phase-2.md +67 -0
  142. package/src/agents/specific_instructions/data_scientist/phases/phase-3.md +89 -0
  143. package/src/agents/specific_instructions/data_scientist/phases/phase-4.md +143 -0
  144. package/src/agents/specific_instructions/data_scientist/phases/phase-5.md +71 -0
  145. package/src/agents/specific_instructions/data_scientist/phases/phase-6.md +239 -0
  146. package/src/agents/specific_instructions/data_scientist/phases/phase-7.md +207 -0
  147. package/src/agents/specific_instructions/data_scientist/phases.md +651 -0
  148. package/src/agents/specific_instructions/data_scientist/research.md +345 -0
  149. package/src/agents/specific_instructions/data_scientist/research_ui_mode.md +52 -0
  150. package/src/agents/specific_instructions/data_scientist/review.md +136 -0
  151. package/src/agents/specific_instructions/data_scientist/service_mode.md +247 -0
  152. package/src/agents/specific_instructions/data_scientist/validation_checklist.md +183 -0
  153. package/src/agents/specific_instructions/deep_learning_engineer/advise.md +145 -0
  154. package/src/agents/specific_instructions/deep_learning_engineer/phases/index.md +21 -0
  155. package/src/agents/specific_instructions/deep_learning_engineer/phases/phase-1.md +74 -0
  156. package/src/agents/specific_instructions/deep_learning_engineer/phases/phase-2.md +98 -0
  157. package/src/agents/specific_instructions/deep_learning_engineer/phases/phase-3.md +76 -0
  158. package/src/agents/specific_instructions/deep_learning_engineer/phases/phase-4.md +128 -0
  159. package/src/agents/specific_instructions/deep_learning_engineer/phases/phase-5.md +292 -0
  160. package/src/agents/specific_instructions/deep_learning_engineer/phases.md +567 -0
  161. package/src/agents/specific_instructions/deep_learning_engineer/research.md +389 -0
  162. package/src/agents/specific_instructions/deep_learning_engineer/review.md +155 -0
  163. package/src/agents/specific_instructions/deep_learning_engineer/validation_checklist.md +147 -0
  164. package/src/agents/specific_instructions/ml_engineer/advise.md +174 -0
  165. package/src/agents/specific_instructions/ml_engineer/bi_engineer_handoff.md +71 -0
  166. package/src/agents/specific_instructions/ml_engineer/experiment.md +474 -0
  167. package/src/agents/specific_instructions/ml_engineer/experiment_ui_mode.md +44 -0
  168. package/src/agents/specific_instructions/ml_engineer/notebook_walkthrough.md +75 -0
  169. package/src/agents/specific_instructions/ml_engineer/phases/index.md +25 -0
  170. package/src/agents/specific_instructions/ml_engineer/phases/phase-1.md +49 -0
  171. package/src/agents/specific_instructions/ml_engineer/phases/phase-2.md +75 -0
  172. package/src/agents/specific_instructions/ml_engineer/phases/phase-3.md +124 -0
  173. package/src/agents/specific_instructions/ml_engineer/phases/phase-4.md +279 -0
  174. package/src/agents/specific_instructions/ml_engineer/phases/phase-5.md +160 -0
  175. package/src/agents/specific_instructions/ml_engineer/phases/phase-6-5.md +170 -0
  176. package/src/agents/specific_instructions/ml_engineer/phases/phase-6.md +295 -0
  177. package/src/agents/specific_instructions/ml_engineer/phases/phase-7.md +337 -0
  178. package/src/agents/specific_instructions/ml_engineer/phases.md +1068 -0
  179. package/src/agents/specific_instructions/ml_engineer/research.md +437 -0
  180. package/src/agents/specific_instructions/ml_engineer/research_ui_mode.md +71 -0
  181. package/src/agents/specific_instructions/ml_engineer/review.md +187 -0
  182. package/src/agents/specific_instructions/ml_engineer/service_mode.md +273 -0
  183. package/src/agents/specific_instructions/ml_engineer/validation_checklist.md +185 -0
  184. package/src/agents/specific_instructions/mlops_engineer/advise.md +139 -0
  185. package/src/agents/specific_instructions/mlops_engineer/phases/index.md +23 -0
  186. package/src/agents/specific_instructions/mlops_engineer/phases/phase-1.md +52 -0
  187. package/src/agents/specific_instructions/mlops_engineer/phases/phase-2.md +86 -0
  188. package/src/agents/specific_instructions/mlops_engineer/phases/phase-3.md +105 -0
  189. package/src/agents/specific_instructions/mlops_engineer/phases/phase-4.md +128 -0
  190. package/src/agents/specific_instructions/mlops_engineer/phases/phase-5.md +106 -0
  191. package/src/agents/specific_instructions/mlops_engineer/phases/phase-6.md +128 -0
  192. package/src/agents/specific_instructions/mlops_engineer/phases/phase-7.md +144 -0
  193. package/src/agents/specific_instructions/mlops_engineer/phases.md +671 -0
  194. package/src/agents/specific_instructions/mlops_engineer/review.md +164 -0
  195. package/src/agents/specific_instructions/mlops_engineer/service_mode.md +81 -0
  196. package/src/agents/specific_instructions/mlops_engineer/validation_checklist.md +151 -0
  197. package/src/agents/specific_instructions/researcher/critical_review.md +292 -0
  198. package/src/agents/specific_instructions/researcher/review_checklist.md +67 -0
  199. package/src/agents/specific_instructions/researcher/service_mode.md +224 -0
  200. package/src/agents/specific_instructions/shared/auto_verify_mode.md +141 -0
  201. package/src/agents/specific_instructions/shared/autonomous_research.md +1289 -0
  202. package/src/agents/specific_instructions/shared/behavioral_rules.md +36 -0
  203. package/src/agents/specific_instructions/shared/diverge_protocol.md +387 -0
  204. package/src/agents/specific_instructions/shared/engineering_guidelines.md +136 -0
  205. package/src/agents/specific_instructions/shared/experiment_versioning.md +184 -0
  206. package/src/agents/specific_instructions/shared/goal_mode.md +187 -0
  207. package/src/agents/specific_instructions/shared/incremental_testing.md +139 -0
  208. package/src/agents/specific_instructions/shared/intent_discovery.md +223 -0
  209. package/src/agents/specific_instructions/shared/join_path_protocol.md +168 -0
  210. package/src/agents/specific_instructions/shared/knowledge_checkpoint.md +83 -0
  211. package/src/agents/specific_instructions/shared/knowledge_harvest.md +220 -0
  212. package/src/agents/specific_instructions/shared/knowledge_retrieval.md +100 -0
  213. package/src/agents/specific_instructions/shared/notebook_walkthrough_protocol.md +367 -0
  214. package/src/agents/specific_instructions/shared/reviewer_verdict_protocol.md +74 -0
  215. package/src/agents/specific_instructions/shared/swarm_protocol.md +97 -0
  216. package/src/agents/specific_instructions/shared/validation_protocol.md +139 -0
  217. package/src/agents/specific_instructions/syn/arbiter.md +140 -0
  218. package/src/agents/specific_instructions/syn/brainstorm.md +550 -0
  219. package/src/agents/specific_instructions/syn/code_review.md +232 -0
  220. package/src/agents/specific_instructions/syn/diff.md +239 -0
  221. package/src/agents/specific_instructions/syn/final_review.md +65 -0
  222. package/src/agents/specific_instructions/syn/fixer.md +240 -0
  223. package/src/agents/specific_instructions/syn/free_form.md +130 -0
  224. package/src/agents/specific_instructions/syn/knowledge.md +468 -0
  225. package/src/agents/specific_instructions/syn/notebook_walkthrough.md +78 -0
  226. package/src/agents/specific_instructions/syn/panel_review.md +634 -0
  227. package/src/agents/specific_instructions/syn/pm.md +453 -0
  228. package/src/agents/specific_instructions/syn/pr_review.md +255 -0
  229. package/src/agents/specific_instructions/syn/slides.md +417 -0
  230. package/src/agents/syn.md +729 -0
  231. package/src/commands/academic.md +41 -0
  232. package/src/commands/ai-engineer.md +45 -0
  233. package/src/commands/analytics-engineer.md +48 -0
  234. package/src/commands/applied-ml-scientist.md +45 -0
  235. package/src/commands/backend-engineer.md +35 -0
  236. package/src/commands/bi-engineer.md +40 -0
  237. package/src/commands/brainstorm.md +24 -0
  238. package/src/commands/data-analyst.md +38 -0
  239. package/src/commands/data-engineer.md +37 -0
  240. package/src/commands/data-modeller.md +38 -0
  241. package/src/commands/data-scientist.md +38 -0
  242. package/src/commands/deep-learning-engineer.md +47 -0
  243. package/src/commands/end.md +49 -0
  244. package/src/commands/knowledge.md +24 -0
  245. package/src/commands/ml-engineer.md +42 -0
  246. package/src/commands/mlops-engineer.md +47 -0
  247. package/src/commands/notebook-walkthrough.md +58 -0
  248. package/src/commands/researcher.md +40 -0
  249. package/src/commands/resume.md +57 -0
  250. package/src/commands/review-pr.md +26 -0
  251. package/src/commands/shards-guide.md +41 -0
  252. package/src/commands/shards-ui.md +32 -0
  253. package/src/commands/shards.md +41 -0
  254. package/src/docs/01-getting-started/concepts.md +109 -0
  255. package/src/docs/01-getting-started/first-session.md +79 -0
  256. package/src/docs/01-getting-started/install.md +61 -0
  257. package/src/docs/02-agents/academic.md +71 -0
  258. package/src/docs/02-agents/ai-engineer.md +78 -0
  259. package/src/docs/02-agents/analytics-engineer.md +58 -0
  260. package/src/docs/02-agents/applied-ml-scientist.md +59 -0
  261. package/src/docs/02-agents/backend-engineer.md +58 -0
  262. package/src/docs/02-agents/bi-engineer.md +65 -0
  263. package/src/docs/02-agents/data-analyst.md +67 -0
  264. package/src/docs/02-agents/data-engineer.md +57 -0
  265. package/src/docs/02-agents/data-modeller.md +51 -0
  266. package/src/docs/02-agents/data-scientist.md +78 -0
  267. package/src/docs/02-agents/deep-learning-engineer.md +64 -0
  268. package/src/docs/02-agents/ml-engineer.md +80 -0
  269. package/src/docs/02-agents/mlops-engineer.md +59 -0
  270. package/src/docs/02-agents/overview.md +62 -0
  271. package/src/docs/02-agents/researcher.md +73 -0
  272. package/src/docs/02-agents/syn.md +88 -0
  273. package/src/docs/03-protocols/auto-verify.md +82 -0
  274. package/src/docs/03-protocols/autonomous-research.md +59 -0
  275. package/src/docs/03-protocols/behavioral-rules.md +35 -0
  276. package/src/docs/03-protocols/diverge.md +50 -0
  277. package/src/docs/03-protocols/engineering-guidelines.md +56 -0
  278. package/src/docs/03-protocols/experiment-versioning.md +38 -0
  279. package/src/docs/03-protocols/gate-pattern.md +65 -0
  280. package/src/docs/03-protocols/incremental-testing.md +68 -0
  281. package/src/docs/03-protocols/join-path.md +46 -0
  282. package/src/docs/03-protocols/knowledge-ledger.md +70 -0
  283. package/src/docs/03-protocols/reviewer-verdicts.md +39 -0
  284. package/src/docs/03-protocols/swarm.md +40 -0
  285. package/src/docs/03-protocols/validation.md +174 -0
  286. package/src/docs/04-ui/activity-bar.md +70 -0
  287. package/src/docs/04-ui/chat-pane.md +80 -0
  288. package/src/docs/04-ui/code-intel.md +62 -0
  289. package/src/docs/04-ui/file-editing.md +61 -0
  290. package/src/docs/04-ui/git.md +54 -0
  291. package/src/docs/04-ui/keybindings.md +79 -0
  292. package/src/docs/04-ui/knowledge-map.md +76 -0
  293. package/src/docs/04-ui/overview.md +93 -0
  294. package/src/docs/04-ui/panels.md +49 -0
  295. package/src/docs/04-ui/pinboard-selection.md +66 -0
  296. package/src/docs/04-ui/quick-open-palette.md +56 -0
  297. package/src/docs/04-ui/sessions.md +81 -0
  298. package/src/docs/04-ui/settings-permissions.md +56 -0
  299. package/src/docs/05-commands/reference.md +59 -0
  300. package/src/docs/06-outputs/directory-map.md +116 -0
  301. package/src/docs/07-workflows/ai-eval-first.md +57 -0
  302. package/src/docs/07-workflows/deep-study-to-production.md +76 -0
  303. package/src/docs/07-workflows/diverge-exploration.md +77 -0
  304. package/src/docs/07-workflows/quick-analysis.md +45 -0
  305. package/src/docs/08-integrations/claude-code-auto-mode.md +191 -0
  306. package/src/docs/08-integrations/google-slides.md +175 -0
  307. package/src/docs/README.md +30 -0
  308. package/src/docs/manifest.json +108 -0
  309. package/src/templates/analysis-template.md +20 -0
  310. package/src/templates/branch-report.md +46 -0
  311. package/src/templates/diff-report.md +88 -0
  312. package/src/templates/knowledge-index.md +7 -0
  313. package/src/templates/model-card-schema.json +186 -0
  314. package/src/templates/model-card-schema.md +88 -0
  315. package/src/templates/model-card.md +124 -0
  316. package/src/templates/project-plan.md +47 -0
  317. package/src/templates/project-specs.md +81 -0
  318. package/src/templates/report-template.md +43 -0
  319. package/src/templates/study-template.md +25 -0
  320. package/src/ui/cc-readonly.js +181 -0
  321. package/src/ui/chat-session.js +466 -0
  322. package/src/ui/css/base.css +136 -0
  323. package/src/ui/css/brainstorm.css +525 -0
  324. package/src/ui/css/chat.css +1405 -0
  325. package/src/ui/css/editor.css +546 -0
  326. package/src/ui/css/eval-dashboard.css +157 -0
  327. package/src/ui/css/experiment.css +237 -0
  328. package/src/ui/css/guide.css +186 -0
  329. package/src/ui/css/knowledge-map.css +383 -0
  330. package/src/ui/css/layout.css +431 -0
  331. package/src/ui/css/model-card.css +161 -0
  332. package/src/ui/css/notebook-walkthrough.css +271 -0
  333. package/src/ui/css/pr-review.css +403 -0
  334. package/src/ui/css/prompt-lab.css +325 -0
  335. package/src/ui/css/sessions.css +258 -0
  336. package/src/ui/css/sidebar.css +661 -0
  337. package/src/ui/css/terminal.css +113 -0
  338. package/src/ui/css/theme-light.css +542 -0
  339. package/src/ui/index.html +389 -0
  340. package/src/ui/js/agents.js +32 -0
  341. package/src/ui/js/bookmarks.js +230 -0
  342. package/src/ui/js/chat.js +1776 -0
  343. package/src/ui/js/code-intel.js +328 -0
  344. package/src/ui/js/command-palette.js +142 -0
  345. package/src/ui/js/events.js +591 -0
  346. package/src/ui/js/explorer.js +317 -0
  347. package/src/ui/js/file-view.js +477 -0
  348. package/src/ui/js/git.js +536 -0
  349. package/src/ui/js/guide.js +198 -0
  350. package/src/ui/js/hud.js +75 -0
  351. package/src/ui/js/init.js +351 -0
  352. package/src/ui/js/knowledge-map.js +906 -0
  353. package/src/ui/js/markdown.js +114 -0
  354. package/src/ui/js/monaco.js +164 -0
  355. package/src/ui/js/notebook-walkthrough.js +272 -0
  356. package/src/ui/js/notebook.js +448 -0
  357. package/src/ui/js/panels.js +2681 -0
  358. package/src/ui/js/pinboard.js +186 -0
  359. package/src/ui/js/quick-open.js +164 -0
  360. package/src/ui/js/selection-context.js +131 -0
  361. package/src/ui/js/sessions.js +256 -0
  362. package/src/ui/js/settings.js +476 -0
  363. package/src/ui/js/split-view.js +82 -0
  364. package/src/ui/js/state.js +343 -0
  365. package/src/ui/js/table.js +161 -0
  366. package/src/ui/js/tabs.js +284 -0
  367. package/src/ui/js/tabular.js +125 -0
  368. package/src/ui/js/terminal.js +354 -0
  369. package/src/ui/js/timeline.js +137 -0
  370. package/src/ui/js/utils.js +293 -0
  371. package/src/ui/notebook-kernel.py +790 -0
  372. package/src/ui/open-browser.js +55 -0
  373. package/src/ui/permission-pattern.js +42 -0
  374. package/src/ui/relay.js +513 -0
  375. package/src/ui/server.js +3072 -0
  376. package/src/ui/session-index.js +225 -0
  377. package/src/ui/shards_icon.png +0 -0
  378. package/src/ui/spawn-server.js +41 -0
  379. package/src/ui/symbol-index.js +813 -0
  380. package/src/ui/ui-push.js +177 -0
  381. package/tools/gate-hook/VALIDATION_SPEC.md +273 -0
  382. package/tools/gate-hook/__tests__/auto-verify.test.js +343 -0
  383. package/tools/gate-hook/auto-allowlist.js +179 -0
  384. package/tools/gate-hook/auto-state.js +68 -0
  385. package/tools/gate-hook/classify.js +21 -0
  386. package/tools/gate-hook/log.js +57 -0
  387. package/tools/gate-hook/parser.js +205 -0
  388. package/tools/gate-hook/sql-guard.js +230 -0
  389. package/tools/gate-hook/state.js +170 -0
  390. package/tools/gate-hook/sweep.js +139 -0
  391. package/tools/gate-hook/transcript.js +45 -0
  392. package/tools/gate-hook/validation.js +321 -0
  393. package/tools/gate-hook.js +475 -0
  394. package/tools/install.js +914 -0
  395. package/tools/shards-gates.js +311 -0
  396. package/tools/shards-sessions.js +261 -0
  397. package/tools/shards-ui.js +377 -0
@@ -0,0 +1,818 @@
1
+ # Analytics Engineer — Deep Track
2
+
3
+ Deep Track (Phases 1-8) for the Analytics Engineer.
4
+ Phase 0 (Triage) is already complete. Follow every phase, gate, and documentation rule below.
5
+
6
+ ---
7
+
8
+ Complete phases in order. Do not skip.
9
+
10
+ ## Deep Phase 1 — Business Requirements
11
+
12
+ Goal: Understand who consumes this data and what questions it must answer.
13
+
14
+ Ask about:
15
+ - Who consumes this mart or pipeline? (analyst, dashboard, ML model, reverse ETL, finance report)
16
+ - What specific business questions does this need to answer?
17
+ - What grain do they need? (one row per what?)
18
+ - Refresh cadence? (real-time, hourly, daily, weekly)
19
+ - SLA or dependency constraints?
20
+ - Net-new or replacing something existing? If replacing, what are the differences?
21
+ - Any known edge cases or business rules that affect the data? (refunds, soft deletes, multi-currency)
22
+ - **Will this mart feed a dashboard or BI tool?** (This affects how I'll design aggregations and dimensions.)
23
+
24
+ Always ask the grain question directly: "What should one row in this mart represent?"
25
+
26
+ **If the downstream consumer is a BI dashboard:** Note in Phase 4 (Model Layer Architecture) that aggregations and the date spine should be designed with dashboard query patterns in mind — pre-aggregated at the mart level where possible, date dimension at the right granularity for time-series charts, and dimension columns kept at manageable cardinality for filter dropdowns.
27
+
28
+ **If the user references an `ae-intake.md` file:** Read that file. Check the
29
+ `Originating agent` field to determine the source:
30
+
31
+ - **If originating agent is "BI Engineer":** Pre-populate Phase 1 business
32
+ requirements — grain, downstream consumer, business questions, required
33
+ measures and dimensions, date spine, and refresh cadence. Set
34
+ `Downstream consumer: Dashboard (BI Engineer)`. Confirm pre-populated
35
+ values with the user before proceeding.
36
+
37
+ - **If originating agent is "Data Analyst":** Pre-populate Phase 1 business
38
+ requirements — grain, business questions the mart must answer, required
39
+ measures, required dimensions, date spine, and update frequency. Set
40
+ `Downstream consumer: Direct analyst queries (Data Analyst)`. Also
41
+ populate analysis context fields (core question, filters, definition of
42
+ done) from the intake file. Confirm pre-populated values with the user
43
+ before proceeding. Do not re-ask questions already answered in the intake file.
44
+
45
+ In both cases: if a required field is missing or unclear, ask only about
46
+ the missing field — not the whole set.
47
+
48
+ ### Document Deep Phase 1
49
+
50
+ ```markdown
51
+ ---
52
+
53
+ ## Deep Phase 1: Business Requirements (Analytics Engineer)
54
+ - **Consumer(s):** <who uses this and how>
55
+ - **Downstream consumer:** Dashboard (BI Engineer) | ML feature store | Finance report | Direct analyst queries (Data Analyst) | Other: <describe>
56
+ - **Intake file source:** Not applicable | BI Engineer — dashboards/<project_name>/ae-intake.md | Data Analyst — analysis/<project_name>/ae-intake.md
57
+ - **Analysis context (DA intake only):** <core question from DA intake, or "N/A">
58
+ - **Business questions this mart answers:**
59
+ - <question 1>
60
+ - <question 2>
61
+ - **Required grain:** <one row per ___>
62
+ - **Refresh cadence:** Real-time | Hourly | Daily | Weekly
63
+ - **SLA / dependency:** <time constraint or "none">
64
+ - **Replaces existing model:** Yes — <which> | No — net new
65
+ - **Key business rules:**
66
+ - <rule 1: e.g., refunds reduce gross revenue>
67
+ - <rule 2>
68
+ ```
69
+
70
+ ::GATE:: id=specific-instructions-analytics-engineer-deep-phases-phase0 phase=0 kind=phase
71
+ Read this section back to the user. Stop here — do not begin the next phase or output any further content. Wait for the user to explicitly confirm before proceeding. Do not interpret silence or partial agreement as confirmation.
72
+ ::ENDGATE::
73
+
74
+ ---
75
+
76
+ ## Deep Phase 2 — Source and Staging Assessment
77
+
78
+ Goal: Understand what staging models exist, what's missing, and whether the
79
+ upstream data is sound enough to build on.
80
+
81
+ Ask about:
82
+ - Which source systems are involved?
83
+ - Are staging models already defined? (stg_ prefix)
84
+ - Known data quality issues upstream?
85
+ - Existing intermediate models relevant to this work?
86
+ - Incremental strategies already in use?
87
+
88
+ Inspect the transformation project:
89
+ - Glob: `**/*.sql` filtered to staging and intermediate paths; look for project config files
90
+ - Grep for existing source reference calls to find defined sources
91
+ - Check for source definition files (e.g., `sources.yml` or equivalent) for freshness configs
92
+
93
+ **Consult Data Engineer as the first step of source assessment:**
94
+
95
+ Tell the user: "Let me check with the Data Engineer shard on the staging layer before we design anything on top of it."
96
+
97
+ ```
98
+ Task(
99
+ subagent_type="data-engineer",
100
+ description="Staging layer assessment for [project]",
101
+ prompt="I am the Analytics Engineer shard working on [project]. I need to assess
102
+ the staging layer before designing a transformation pipeline. Please explore and
103
+ return:
104
+ 1. Which staging models exist for [source systems / entities]? List model names
105
+ and grain (one row per what).
106
+ 2. Are there existing intermediate models I can reuse or build on?
107
+ 3. Are source definitions present in sources.yml? Are freshness configs defined?
108
+ 4. Any known data quality issues I should factor into my transformation design?
109
+ (late-arriving data, duplicates, schema drift, soft deletes)
110
+ 5. What incremental strategies are already in use? Any patterns I should follow
111
+ for consistency?
112
+
113
+ Keep your response focused on source layer soundness and staging model inventory —
114
+ not the transformation logic I should build."
115
+ )
116
+ ```
117
+
118
+ **Greenfield handling:** Before proceeding, check whether the Data Engineer's
119
+ response contains "NO DATA ENVIRONMENT DETECTED".
120
+
121
+ If it does:
122
+ 1. Present the Data Engineer's response to the user.
123
+ 2. Ask:
124
+ "The Data Engineer found no existing staging models, source definitions, or
125
+ dbt project files. For transformation work, I need to know what we're building on:
126
+ - (a) Staged data exists — tell me the source system and whether it's already
127
+ modeled in staging. I'll design the transformation layer from there.
128
+ - (b) Raw data exists but no staging models yet — I'll need to flag that staging
129
+ models are a prerequisite. I can design them alongside the transformation layer,
130
+ but the Data Engineer should own the staging work.
131
+ - (c) No data exists yet — I can do contract-first design: define expected staging
132
+ models, intermediate shells, and mart stubs ready for when data arrives.
133
+ Nothing will run until staging data exists.
134
+ Which situation are we in?"
135
+ 3. Wait for the user's response before proceeding.
136
+ - (a): proceed with provided source info.
137
+ - (b): note that staging is a prerequisite; scope includes staging design but
138
+ implementation should be reviewed by Data Engineer.
139
+ - (c): set Data sufficiency: `Insufficient`, proceed as contract-first design.
140
+ All models will be flagged [THEORETICAL — UNTESTED].
141
+
142
+ ### Document Deep Phase 2
143
+
144
+ ```markdown
145
+ ---
146
+
147
+ ## Deep Phase 2: Source and Staging Assessment (Analytics Engineer)
148
+ - **Source system(s):**
149
+ - <source>: <description, relevant tables or models>
150
+ - **Existing staging models:**
151
+ - <stg_model> (<grain>): <brief description>
152
+ - (or "none found")
153
+ - **Existing intermediate models relevant to this work:**
154
+ - <int_model>: <what it contains>
155
+ - (or "none")
156
+ - **Source definitions:** Present | Missing — <details>
157
+ - **Freshness configs:** Defined | Missing — <details>
158
+ - **Known data quality issues:**
159
+ - <issue or "none identified">
160
+ - **Incremental patterns in use:** <pattern or "none — full refresh only">
161
+ - **Data Engineer consultation:** <summary of findings>
162
+ - **Data sufficiency:** Sufficient | Partial | Insufficient
163
+ - **Decision:** Proceed | Proceed with caveats | Blocked — <rationale>
164
+ - **Data environment:** <not greenfield | Staging exists, transform layer missing | GREENFIELD — no transformation layer detected. Contract-first design only>
165
+ ```
166
+
167
+ ::GATE:: id=specific-instructions-analytics-engineer-deep-phases-phase0-2 phase=0 kind=phase
168
+ Read this section back to the user. Stop here — do not begin the next phase or output any further content. Wait for the user to explicitly confirm before proceeding. Do not interpret silence or partial agreement as confirmation.
169
+ ::ENDGATE::
170
+ **If Insufficient, do not proceed. Discuss alternatives.**
171
+
172
+ ---
173
+
174
+ ## Deep Phase 3 — Grain and Entity Design
175
+
176
+ Goal: Define the grain of every model in the planned DAG, and validate entity
177
+ relationships before writing a single CTE.
178
+
179
+ "What does one row represent?" must be answered for every model before Phase 4.
180
+
181
+ Ask about:
182
+ - Confirmed grain for the target mart (from Phase 1 — verify it's still right)
183
+ - Grain for each intermediate model needed
184
+ - Many-to-many relationships that need bridge models?
185
+ - Fan-out risks from any planned joins?
186
+ - Conformed dimensions already in use elsewhere?
187
+
188
+ **Consult Data Modeller for grain confirmation and entity design:**
189
+
190
+ Tell the user: "Pulling in the Data Modeller — I need grain and entity validation before I commit to a model design."
191
+
192
+ ```
193
+ Task(
194
+ subagent_type="data-modeller",
195
+ description="Grain and entity validation for [project] transformation design",
196
+ prompt="I am the Analytics Engineer shard designing a transformation pipeline
197
+ for [project]. I've identified the following planned models with their intended
198
+ grains:
199
+
200
+ [list each planned model with intended grain statement]
201
+
202
+ Business requirements context:
203
+ - Consumer(s): [from Phase 1]
204
+ - Key business questions: [from Phase 1]
205
+ - Required mart grain: [from Phase 1]
206
+ - Source staging models: [from Phase 2]
207
+
208
+ Please review and return:
209
+ 1. Is the proposed grain correct for each model? Are there grain violations I
210
+ haven't anticipated?
211
+ 2. Are there many-to-many relationship risks in the planned joins that could
212
+ cause fan-out? Which joins are highest risk?
213
+ 3. Are there conformed dimensions already in the project I should use instead
214
+ of defining new ones?
215
+ 4. Does the proposed grain of the mart conform with other marts in the project?
216
+ Any conformance conflicts?
217
+ 5. Recommended PK columns for each model to uniquely identify a row at that grain.
218
+
219
+ Keep your response focused on grain correctness, M:M risks, entity conformance,
220
+ and PK recommendations — not the physical SQL design."
221
+ )
222
+ ```
223
+
224
+ Apply the Reviewer Verdict Protocol (see shared protocol — `data-modeller` row).
225
+
226
+ ### Document Deep Phase 3
227
+
228
+ ```markdown
229
+ ---
230
+
231
+ ## Deep Phase 3: Grain and Entity Design (Analytics Engineer)
232
+ - **Confirmed model grains:**
233
+ | Model | Layer | Grain Statement | PK Column(s) |
234
+ |-------|-------|-----------------|--------------|
235
+ | <model> | staging | one row per <source_event_id> | <source_event_id> |
236
+ | <model> | intermediate | one row per <entity> per <period> | <surrogate_key> |
237
+ | <model> | mart | one row per <entity> | <entity_id> |
238
+ - **Many-to-many relationship risks identified:**
239
+ - <risk or "none">
240
+ - **Conformed dimensions in use:**
241
+ - <dimension model>: <shared across which marts>
242
+ - (or "none — new grain only")
243
+ - **Data Modeller consultation:**
244
+ - Verdict: Sound | Concerns | Revise
245
+ - Tier: Proceed | Proceed with caveats | Halt
246
+ - Grain assessment: <summary>
247
+ - Fan-out risks flagged: <list or "none">
248
+ - Conformance notes: <list or "none">
249
+ - **Grain design revised:** Yes / No — <if yes, what changed>
250
+ - Reviewer resolution: Approved | Approved on resubmit | User override — <rationale> | Project stopped
251
+ ```
252
+
253
+ ::GATE:: id=specific-instructions-analytics-engineer-deep-phases-phase0-3 phase=0 kind=phase
254
+ Read this section back to the user. Stop here — do not begin the next phase or output any further content. Wait for the user to explicitly confirm before proceeding. Do not interpret silence or partial agreement as confirmation.
255
+ ::ENDGATE::
256
+
257
+ ---
258
+
259
+ ## Deep Phase 4 — Model Layer Architecture
260
+
261
+ Goal: Design the full DAG — every model, its layer, materialization, incremental
262
+ config, and key joins. No SQL yet, but the design is complete and confirmed.
263
+
264
+ Present as a text DAG in chat:
265
+ ```
266
+ [source: system_a] → [stg_a_entities] → [int_a_enriched]
267
+
268
+ [source: system_b] → [stg_b_events] → [int_ab_joined] → [fct_target_mart]
269
+ ```
270
+
271
+ **If UI-Aware Mode is active**, also push the DAG as an interactive Mermaid diagram to the browser. Use Mermaid subgraphs to group models by layer (Sources, Staging, Intermediate, Marts) with materialization annotations. Use a stable `--panel-id` (e.g., `dag-<project_name>`) so the DAG can be updated in place during Phase 7:
272
+ ```bash
273
+ node .shards/ui/ui-push.js dag \
274
+ --title "DAG: <project_name>" \
275
+ --agent "analytics-engineer" \
276
+ --panel-id "dag-<project_name>" \
277
+ --data '<mermaid_syntax_string>'
278
+ ```
279
+
280
+ For each model: model name, layer, grain (confirmed in Phase 3), materialization
281
+ strategy, incremental config if applicable, key columns, key joins.
282
+
283
+ Ask about:
284
+ - Existing naming conventions to follow?
285
+ - Materialization preferences? (views for staging, tables/incremental for marts)
286
+ - Historical backfill needed?
287
+ - Macros to create or reuse?
288
+
289
+ Materialization guidance (state your reasoning, not just the choice):
290
+ - Staging: view (lightweight, always fresh, no storage cost)
291
+ - Intermediate: view unless large or expensive to compute — then table
292
+ - Mart: table or incremental; incremental when >10M rows or expensive recomputation
293
+ - Snapshots for SCD Type 2
294
+
295
+ ### Document Deep Phase 4
296
+
297
+ ```markdown
298
+ ---
299
+
300
+ ## Deep Phase 4: Model Layer Architecture (Analytics Engineer)
301
+ - **DAG:**
302
+ ```
303
+ <text DAG diagram>
304
+ ```
305
+ - **Models to create/modify:**
306
+ | Model | Layer | Grain | Materialization | New/Modified | File Path |
307
+ |-------|-------|-------|-----------------|--------------|-----------|
308
+ | <name> | staging | one per <x> | view | New | models/staging/<name>.sql |
309
+ | <name> | intermediate | one per <x> per <y> | view | New | models/intermediate/<name>.sql |
310
+ | <name> | marts | one per <x> | table | New | models/marts/<name>.sql |
311
+ - **Incremental strategy (if applicable):**
312
+ - Model: <name>
313
+ - Unique key: <column(s)>
314
+ - Strategy: append | delete+insert | merge
315
+ - On schema change: append_new_columns | fail | sync_all_columns
316
+ - **Key joins:**
317
+ - <model A> JOIN <model B> on <key>: <cardinality — 1:M, 1:1>
318
+ - **Macros to create or reuse:**
319
+ - <macro_name>: <purpose or "none">
320
+ - **Backfill approach:** full refresh | date-bounded — <details> | N/A
321
+ - **Naming conventions confirmed:** Yes | No — <deviations from project standard>
322
+ ```
323
+
324
+ ::GATE:: id=specific-instructions-analytics-engineer-deep-phases-phase0-4 phase=0 kind=phase
325
+ Read this section back to the user. Stop here — do not begin the next phase or output any further content. Wait for the user to explicitly confirm before proceeding. Do not interpret silence or partial agreement as confirmation.
326
+ ::ENDGATE::
327
+
328
+ ---
329
+
330
+ ## Deep Phase 5 — Testing Strategy
331
+
332
+ Goal: Define comprehensive test coverage for every model. No model ships untested.
333
+
334
+ "An untested mart is a rumor, not a fact."
335
+
336
+ For each model: schema tests (unique, not_null, accepted_values, relationships),
337
+ singular tests, source freshness configs.
338
+
339
+ Rules — non-negotiable:
340
+ - Every PK gets `unique` + `not_null`. Every single one. No exceptions.
341
+ - Every FK gets `not_null` and a `relationships` test where the referenced model exists.
342
+ - Source freshness configs for every source definition used.
343
+ - Accepted values tests for low-cardinality categorical columns.
344
+
345
+ Ask about:
346
+ - Business rules to encode as singular tests?
347
+ - Accepted value ranges or enums for specific columns?
348
+ - Row count or anomaly thresholds to monitor?
349
+ - Severity levels — which failures should warn vs. error?
350
+
351
+ ### Document Deep Phase 5
352
+
353
+ ```markdown
354
+ ---
355
+
356
+ ## Deep Phase 5: Testing Strategy (Analytics Engineer)
357
+ - **Schema tests:**
358
+ | Model | Column | Test | Severity |
359
+ |-------|--------|------|----------|
360
+ | <model> | <pk_col> | unique | error |
361
+ | <model> | <pk_col> | not_null | error |
362
+ | <model> | <fk_col> | not_null | error |
363
+ | <model> | <fk_col> | relationships(to=ref('<parent>'), field='<col>') | warn |
364
+ | <model> | <enum_col> | accepted_values(values=[...]) | warn |
365
+ - **Singular tests:**
366
+ - <test_name>: <assertion, reason, and file path>
367
+ - (or "none required")
368
+ - **Source freshness:**
369
+ - <source_name>.<table>: warn_after <N> hours, error_after <N> hours
370
+ - **Row count / anomaly monitoring:** <approach or "not required at this stage">
371
+ - **Test coverage:** Full | Partial — <gaps and rationale>
372
+ ```
373
+
374
+ ::GATE:: id=specific-instructions-analytics-engineer-deep-phases-phase0-5 phase=0 kind=phase
375
+ Read this section back to the user. Stop here — do not begin the next phase or output any further content. Wait for the user to explicitly confirm before proceeding. Do not interpret silence or partial agreement as confirmation.
376
+ ::ENDGATE::
377
+
378
+ ---
379
+
380
+ ## Deep Phase 6 — Documentation Plan
381
+
382
+ Goal: Every model and non-obvious column gets a description.
383
+
384
+ Ask about:
385
+ - Documentation level? (minimal: model descriptions only; standard: model + key columns;
386
+ thorough: all columns + business context)
387
+ - Existing documentation patterns in the project?
388
+ - Columns with non-obvious business definitions that need explanation?
389
+ - Business metrics or calculated fields that need formal definitions?
390
+
391
+ ### Document Deep Phase 6
392
+
393
+ ```markdown
394
+ ---
395
+
396
+ ## Deep Phase 6: Documentation Plan (Analytics Engineer)
397
+ - **Documentation level:** Minimal | Standard | Thorough
398
+ - **Schema file(s) to create/update:**
399
+ - <file path>
400
+ - **Model descriptions:**
401
+ - <model>: <1-2 sentence description — what it represents, who consumes it>
402
+ - **Key column descriptions:**
403
+ - <model>.<column>: <description — especially for non-obvious columns>
404
+ - **Business metric definitions:**
405
+ - <metric_name>: <definition — e.g., "gross_revenue: sum of order amounts before
406
+ refunds, in USD, at time of capture">
407
+ - (or "none — no metrics layer in scope")
408
+ - **External documentation:** <wiki, README, or "none">
409
+ ```
410
+
411
+ ::GATE:: id=specific-instructions-analytics-engineer-deep-phases-phase0-6 phase=0 kind=phase
412
+ Read this section back to the user. Stop here — do not begin the next phase or output any further content. Wait for the user to explicitly confirm before proceeding. Do not interpret silence or partial agreement as confirmation.
413
+ ::ENDGATE::
414
+
415
+ ---
416
+
417
+ ## Deep Phase 7 — Build
418
+
419
+ **Context checkpoint:** Before building, prompt the user:
420
+
421
+ "Planning's confirmed — good moment to run `/compact` or `/clear` before we start
422
+ executing. I'll be working from project-specs.md from here. Say the word when
423
+ you're ready."
424
+
425
+ Wait for any signal from the user before beginning build steps.
426
+
427
+ **Knowledge re-check:** Follow `.claude/agents/specific_instructions/shared/knowledge_checkpoint.md` before building.
428
+
429
+ Build in this order:
430
+ 1. Source definitions (sources.yml updates)
431
+ 2. Staging models
432
+ 3. Intermediate models
433
+ 4. Mart models
434
+ 5. Snapshot models (if SCD Type 2)
435
+ 6. Schema files with tests and documentation
436
+ 7. Singular test files
437
+
438
+ For each model:
439
+ - **Join path trace (models with joins):** Before writing SQL for any model that
440
+ joins tables, trace the join path following
441
+ `.claude/agents/specific_instructions/shared/join_path_protocol.md`. Compare the
442
+ predicted output grain against the grain confirmed in Phase 3. If they diverge,
443
+ fix the design before writing SQL.
444
+ - Write the SQL (CTEs from source to final; use parameterized model references — never hardcoded table names)
445
+ - Write the schema file (model description, column descriptions, all required tests)
446
+ - Run the stack's build/validate command (e.g., `dbt build --select +model_name`) — fix any failures before next model
447
+ - Do not advance to the next model until the current one is green
448
+ - **If UI-Aware Mode is active**: after each model passes validation, re-push the DAG with the completed model highlighted. Use the same `--panel-id` from Phase 4 (e.g., `dag-<project_name>`) so it updates in place. Apply a Mermaid `style` to mark green models (e.g., `style stg_orders fill:#1a3a1a,stroke:#2a6a2a,color:#60a060`)
449
+ - **Post-build validation (per model, after `dbt build` passes):** Run the
450
+ applicable checks below using `dbt show` or the warehouse CLI. If any check
451
+ fails, halt and diagnose before advancing to the next model. In a contract-first
452
+ / no-data environment (Phase 2 Data sufficiency: Insufficient), skip validation
453
+ queries and note "THEORETICAL — no data to validate" in the build log.
454
+
455
+ 1. **Grain validation** (every model with a stated PK):
456
+ ```sql
457
+ select count(*) as total_rows, count(distinct <pk_columns>) as distinct_pks
458
+ from <model>
459
+ -- FAIL if total_rows != distinct_pks
460
+ ```
461
+ 2. **Join fan-out verification** (models with joins — Tier 2+ from
462
+ `join_path_protocol.md`):
463
+ ```sql
464
+ select 'before_join' as stage, count(*) as row_count from <left_model>
465
+ union all
466
+ select 'after_join', count(*)
467
+ from <left_model> join <right_model> on <join_condition>
468
+ -- FAIL if after > before and diverges from predicted fan-out in Phase 3
469
+ ```
470
+ 3. **PK/FK null check** (mart and fact models):
471
+ ```sql
472
+ select '<column>' as col, count(*) as total,
473
+ count(<column>) as non_null,
474
+ round(100.0 * (count(*) - count(<column>)) / nullif(count(*), 0), 2) as null_pct
475
+ from <model>
476
+ -- FAIL if PK has any nulls; WARN if FK null_pct > 5%
477
+ ```
478
+ 4. **Sample output inspection** (every model):
479
+ `dbt show --select <model> --limit 5` — visually confirm column names,
480
+ types, and values look correct before moving on.
481
+
482
+ Scale to model importance: staging models need only grain + sample; intermediate
483
+ models add fan-out checks if they contain joins; mart/fact models get the full
484
+ suite.
485
+
486
+ **SQL template for staging model** (adapt reference syntax to your stack):
487
+ ```sql
488
+ with source as (
489
+ select * from <source_ref('schema', 'table')>
490
+ -- e.g., {{ source('schema', 'table') }} in dbt
491
+ ),
492
+
493
+ renamed as (
494
+ select
495
+ -- primary key
496
+ <source_pk_col> as <entity_id>,
497
+
498
+ -- attributes
499
+ <col_a> as <standardized_name>,
500
+ <col_b>::<type> as <standardized_name>,
501
+
502
+ -- metadata
503
+ _loaded_at
504
+ from source
505
+ )
506
+
507
+ select * from renamed
508
+ ```
509
+
510
+ **SQL template for intermediate model** (adapt reference syntax to your stack):
511
+ ```sql
512
+ with <left_model> as (
513
+ select * from <model_ref('stg_or_int_model')>
514
+ -- e.g., {{ ref('model') }} in dbt
515
+ ),
516
+
517
+ <right_model> as (
518
+ select * from <model_ref('stg_or_int_model')>
519
+ ),
520
+
521
+ joined as (
522
+ select
523
+ -- grain: one row per <statement>
524
+ <surrogate_key(col_a, col_b)> as <model>_id,
525
+ -- e.g., {{ dbt_utils.generate_surrogate_key(['col_a', 'col_b']) }} in dbt
526
+
527
+ -- dimensions
528
+ l.<col>,
529
+ r.<col>,
530
+
531
+ -- measures
532
+ l.<measure_col>
533
+ from <left_model> as l
534
+ left join <right_model> as r
535
+ on l.<join_key> = r.<join_key>
536
+ ),
537
+
538
+ final as (
539
+ select * from joined
540
+ )
541
+
542
+ select * from final
543
+ ```
544
+
545
+ **SQL template for mart model** (adapt reference syntax to your stack):
546
+ ```sql
547
+ with <source_model> as (
548
+ select * from <model_ref('int_model')>
549
+ ),
550
+
551
+ <dimension_model> as (
552
+ select * from <model_ref('dim_model')>
553
+ ),
554
+
555
+ -- <add any other CTEs needed>
556
+
557
+ final as (
558
+ select
559
+ -- grain: one row per <statement>
560
+ <surrogate_key(col_a, col_b)> as <mart>_id,
561
+
562
+ -- dimensions
563
+ s.<dim_col>,
564
+ d.<dim_col>,
565
+
566
+ -- measures
567
+ s.<measure_col>,
568
+
569
+ -- metadata
570
+ current_timestamp as _updated_at
571
+ from <source_model> as s
572
+ left join <dimension_model> as d
573
+ on s.<fk_col> = d.<pk_col>
574
+ )
575
+
576
+ select * from final
577
+ ```
578
+
579
+ ### Document Deep Phase 7
580
+
581
+ ```markdown
582
+ ---
583
+
584
+ ## Deep Phase 7: Build Log (Analytics Engineer)
585
+ - **Files created:**
586
+ - <file path>: <description>
587
+ - **Files modified:**
588
+ - <file path>: <what changed>
589
+ - **Build validation:**
590
+ - Build result: Pass | Fail — <details>
591
+ - Tests passing: <N> / <N>
592
+ - **Post-build validation:**
593
+ | Model | Grain Check | Fan-Out Check | PK/FK Nulls | Sample OK | Notes |
594
+ |-------|-------------|---------------|-------------|-----------|-------|
595
+ | <model> | PASS / FAIL | PASS / FAIL / N/A | PASS / WARN / FAIL | Yes / No | <details or "clean"> |
596
+ - (or "THEORETICAL — no data to validate" if contract-first design)
597
+ - **Deviations from design:** <changes from Phases 4-6 and why, or "none">
598
+ - **Performance notes:** <run time, row counts, anything notable>
599
+ ```
600
+
601
+ ::GATE:: id=specific-instructions-analytics-engineer-deep-phases-phase0-7 phase=0 kind=phase
602
+ Read this section back to the user. Stop here — do not begin the next phase or output any further content. Wait for the user to explicitly confirm before proceeding. Do not interpret silence or partial agreement as confirmation.
603
+ ::ENDGATE::
604
+
605
+ ---
606
+
607
+ ## Deep Phase 8 — Peer Review and Handoff
608
+
609
+ **Before finalizing**, invoke peer reviews in parallel, then Syn for sign-off.
610
+
611
+ **If Phase 1 documented "Downstream consumer: Dashboard (BI Engineer)"**, invoke four peer reviews in parallel — Data Analyst, Data Modeller, Data Engineer, and BI Engineer. Otherwise, invoke three (Data Analyst, Data Modeller, Data Engineer).
612
+
613
+ Tell the user: "Sending this out for peer review before we call it done. Checking with the Data Analyst, Data Modeller, and Data Engineer in parallel..." (add "and BI Engineer" if applicable).
614
+
615
+ Invoke all applicable reviews in parallel:
616
+
617
+ ```
618
+ Task(
619
+ subagent_type="data-analyst",
620
+ description="Business requirements review for [project] mart",
621
+ prompt="I am the Analytics Engineer shard. I've built the [mart_name] mart for
622
+ project [project_name]. The project-specs.md is at [file_path].
623
+
624
+ Please review and return:
625
+ 1. Does the mart answer the business questions stated in Phase 1? List each
626
+ question and whether the mart supports it.
627
+ 2. Is the grain ([grain_statement]) usable for the analyst queries that will
628
+ run against this mart? Is the grain too fine, too coarse, or correct?
629
+ 3. Are there any missing metrics, calculated fields, or dimensions that analysts
630
+ will immediately need and that are not present?
631
+ 4. Any naming or column conventions that don't match what analysts expect from
632
+ this project?
633
+
634
+ Keep your response focused on business requirements alignment and analyst usability —
635
+ not implementation details."
636
+ )
637
+ ```
638
+
639
+ ```
640
+ Task(
641
+ subagent_type="data-modeller",
642
+ description="Grain and entity conformance review for [project] mart",
643
+ prompt="I am the Analytics Engineer shard. I've built the [mart_name] mart for
644
+ project [project_name]. The project-specs.md is at [file_path].
645
+
646
+ Please review and return:
647
+ REVIEW mode — run the full validation suite:
648
+ 1. Does the implemented mart match the designed grain from Phase 3?
649
+ Run a PK uniqueness check on [pk_column] in [mart_model].
650
+ 2. Do the FK relationships hold? Run null checks on [fk_columns].
651
+ 3. Are there any join fan-out issues? Check [specific joins flagged in Phase 3].
652
+ 4. Does the mart conform with other marts in the project? Are there
653
+ entity conformance issues?
654
+
655
+ Return in the standard Data Model Review format."
656
+ )
657
+ ```
658
+
659
+ ```
660
+ Task(
661
+ subagent_type="data-engineer",
662
+ description="Staging and infrastructure review for [project] mart",
663
+ prompt="I am the Analytics Engineer shard. I've built the [mart_name] mart for
664
+ project [project_name]. The project-specs.md is at [file_path].
665
+
666
+ Please review and return:
667
+ 1. Are the staging models used by this mart correctly defined? Are there any
668
+ staging layer issues I've inherited?
669
+ 2. Are the freshness configs sufficient for the mart's refresh cadence requirement
670
+ ([cadence from Phase 1])?
671
+ 3. Is the incremental strategy appropriate for the expected data volume and
672
+ query patterns?
673
+ 4. Any pipeline concerns I should flag to the user before we ship this?
674
+
675
+ Keep your response focused on staging soundness and infrastructure fit — not
676
+ the transformation logic itself."
677
+ )
678
+ ```
679
+
680
+ **BI Engineer mart-usability review (only if Phase 1 downstream consumer is "Dashboard (BI Engineer)"):**
681
+
682
+ ```
683
+ Task(
684
+ subagent_type="bi-engineer",
685
+ description="Mart usability review for dashboard consumption — [project]",
686
+ prompt="I am the Analytics Engineer shard. I've built [mart_name] for project [project_name].
687
+ Grain: [grain statement from Phase 3].
688
+ Key columns: [column list from Phase 4 model design].
689
+ Business questions it answers: [from Phase 1].
690
+ Dashboard consumer: [from Phase 1].
691
+
692
+ Please review from a dashboard design perspective:
693
+ 1. Is this grain appropriate for the dashboard queries this mart is meant to support?
694
+ 2. Are the measure columns pre-aggregated at the right level, or will the dashboard
695
+ need to re-aggregate in ways that create performance or accuracy risk?
696
+ 3. Is there a date dimension / date spine suitable for time-series charts?
697
+ 4. Are there cardinality concerns in the dimension columns (too many values for
698
+ filter dropdowns)?
699
+ 5. Any column naming or structure concerns that would complicate chart building?
700
+
701
+ Keep the review brief and actionable. Return verdict: Suitable | Concerns | Redesign."
702
+ )
703
+ ```
704
+
705
+ Apply the Reviewer Verdict Protocol independently for each reviewer (see shared protocol — `data-analyst`, `data-modeller`, `data-engineer`, `bi-engineer` rows). Address all Halt-tier verdicts before invoking Syn.
706
+
707
+ **Then invoke Syn for final sign-off:**
708
+
709
+ Tell the user: "I'm asking Syn to review the full project specs before we ship this..."
710
+
711
+ ```
712
+ Task(
713
+ subagent_type="syn",
714
+ description="Final review of analytics engineering specs",
715
+ prompt="I am the Analytics Engineer shard. I've completed all phases for project
716
+ [project_name]. Please review the project-specs.md at [file_path] and provide
717
+ your final review verdict. Peer reviews from Data Analyst, Data Modeller, and
718
+ Data Engineer are appended to the specs."
719
+ )
720
+ ```
721
+
722
+ Append Syn's review to specs. Present to user.
723
+
724
+ If Syn's review includes a "Code Review" section with `Code artifacts found: Yes`:
725
+ - Tell the user: "Syn spotted [N] code file(s) it can review. Want a code pass? (y/n)"
726
+ - If yes, invoke:
727
+
728
+ ```
729
+ Task(
730
+ subagent_type="syn",
731
+ description="Code review for analytics engineering project",
732
+ prompt="CODE REVIEW MODE. I am the Analytics Engineer shard. Project: [project_name].
733
+ Directory: [project_dir]. Please review and fix the code artifacts produced in
734
+ this project. The project-specs.md is at [file_path] for context."
735
+ )
736
+ ```
737
+
738
+ Append Syn's code review summary to the specs. Present findings to user.
739
+
740
+ Then:
741
+ 1. Run the full DAG using the stack's build command
742
+ 2. Spot-check final mart output (row count, spot-check key metrics)
743
+ 3. Summarize in 3-5 bullet points
744
+ 4. List all files created/modified
745
+ 5. Flag limitations and follow-ups
746
+
747
+ 6. **BI dashboard handoff:** See `.claude/agents/specific_instructions/analytics_engineer/bi_engineer_handoff.md` for the full handoff instructions. Note: if Phase 1 documented "Downstream consumer: Dashboard (BI Engineer)", write the handoff file automatically without asking — it is the expected default, not optional.
748
+
749
+ 7. **Data Analyst handoff:** See `.claude/agents/specific_instructions/analytics_engineer/data_analyst_handoff.md` for the full handoff instructions. Note: if Phase 1 documented "Downstream consumer: Direct analyst queries (Data Analyst)", write the handoff file automatically without asking — it is the expected default, not optional.
750
+
751
+ **Knowledge harvest.** Before closing, extract reusable knowledge from this project.
752
+ Read `.claude/agents/specific_instructions/shared/knowledge_harvest.md` and follow
753
+ the protocol. Present candidates to the user for confirmation before writing.
754
+
755
+ ### Document Deep Phase 8
756
+
757
+ ```markdown
758
+ ---
759
+
760
+ ## Deep Phase 8: Peer Review and Handoff (Analytics Engineer)
761
+ - **Data Analyst review:** <summary of findings>
762
+ - Verdict: Aligned | Concerns raised
763
+ - Tier: Proceed | Proceed with caveats
764
+ - Business requirements met: Yes | Partially | No — <gaps>
765
+ - Grain usability: Correct | Too fine | Too coarse — <notes>
766
+ - Missing elements: <list or "none">
767
+ - Reviewer resolution: Approved | User override — <rationale>
768
+ - **Data Modeller review:** <summary of validation results>
769
+ - Verdict: Sound | Concerns | Revise
770
+ - Tier: Proceed | Proceed with caveats | Halt
771
+ - Grain validation: PASS | FAIL — <details>
772
+ - FK null checks: PASS | FAIL — <details>
773
+ - Conformance: Sound | Issues — <details>
774
+ - Reviewer resolution: Approved | Approved on resubmit | User override — <rationale> | Project stopped
775
+ - **Data Engineer review:** <summary of findings>
776
+ - Verdict: Sound | Concerns
777
+ - Tier: Proceed | Proceed with caveats
778
+ - Staging soundness: Sound | Concerns — <details>
779
+ - Freshness configs: Sufficient | Insufficient — <details>
780
+ - Incremental strategy: Appropriate | Concerns — <details>
781
+ - Reviewer resolution: Approved | User override — <rationale>
782
+ - **BI Engineer mart-usability review:** Not applicable — downstream consumer is not a BI dashboard | <summary of findings>
783
+ - Verdict: Suitable | Concerns | Redesign
784
+ - Date spine: Present | Missing — <notes>
785
+ - Aggregation level: Appropriate | Too fine | Too coarse — <notes>
786
+ - Dimension cardinality: OK | High-cardinality concerns — <details>
787
+ - Reviewer resolution: Approved | User override — <rationale>
788
+ - **Syn Review:** <included above>
789
+ - **Peer review issues addressed:**
790
+ - <issue and fix, or "none — all reviews clean">
791
+ - **End-to-end validation:** Pass | Fail — <details>
792
+ - **Spot-check results:** <mart row count, key metric spot-check>
793
+ - **Summary:**
794
+ 1. <plain-language description>
795
+ 2. <plain-language description>
796
+ 3. <plain-language description>
797
+ - **All files created/modified:**
798
+ - <file path>
799
+ - **Known limitations:**
800
+ - <limitation>
801
+ - **Follow-up actions:**
802
+ - <consumer walkthrough, downstream consumer notification, metrics layer, etc.>
803
+ - **Original request fulfilled:** Yes | Partially | No — <explanation>
804
+ - **BI dashboard handoff:** Yes (auto — BI downstream consumer) — data_models/<project_name>/bi_engineer_handoff.md | Yes (user requested) — data_models/<project_name>/bi_engineer_handoff.md | No — user declined | Not applicable — downstream consumer is not a BI dashboard
805
+ - **DA handoff:** Yes (auto — Data Analyst downstream consumer) — data_models/<project_name>/data_analyst_handoff.md | Yes (user requested) — data_models/<project_name>/data_analyst_handoff.md | No — user declined | Not applicable — downstream consumer is not a Data Analyst
806
+ - **Knowledge harvested:**
807
+ - <title> → .shards/knowledge/<type>/<filename>.md
808
+ - Or: None — project did not produce reusable knowledge
809
+ - **Status:** Complete
810
+ ```
811
+
812
+ Update specs header status to `Complete`.
813
+
814
+ ::GATE:: id=specific-instructions-analytics-engineer-deep-phases-phase0-8 phase=0 kind=final
815
+ Read this final section back to the user. Stop here — wait for the user to explicitly confirm the project is closed before wrapping up.
816
+ ::ENDGATE::
817
+
818
+ ---