self-evolve-framework 1.6.0 → 1.6.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (301) hide show
  1. package/README.md +33 -23
  2. package/bin/cli.js +24 -67
  3. package/package.json +1 -1
  4. package/template/rules/knowledge/README.md +56 -0
  5. package/template/rules/knowledge/ant-design-vue.mdc +54 -0
  6. package/template/rules/knowledge/general.mdc +56 -0
  7. package/template/rules/knowledge/rust.mdc +55 -0
  8. package/template/rules/knowledge/shadcn-svelte.mdc +64 -0
  9. package/template/rules/knowledge/svelte.mdc +135 -0
  10. package/template/rules/knowledge/tailwind.mdc +66 -0
  11. package/template/rules/knowledge/tauri.mdc +65 -0
  12. package/template/rules/knowledge/typescript.mdc +41 -0
  13. package/template/rules/knowledge/vue.mdc +48 -0
  14. package/template/rules/ponytail.mdc +23 -98
  15. package/template/rules/self-evolve.mdc +24 -0
  16. package/template/skills/impeccable/SKILL.md +47 -174
  17. package/template/skills/skillopt-sleep/SKILL.md +32 -88
  18. package/template/skills/impeccable/agents/impeccable_asset_producer.toml +0 -92
  19. package/template/skills/impeccable/agents/impeccable_manual_edit_applier.toml +0 -95
  20. package/template/skills/impeccable/agents/openai.yaml +0 -4
  21. package/template/skills/impeccable/reference/adapt.md +0 -311
  22. package/template/skills/impeccable/reference/animate.md +0 -201
  23. package/template/skills/impeccable/reference/audit.md +0 -133
  24. package/template/skills/impeccable/reference/bolder.md +0 -113
  25. package/template/skills/impeccable/reference/brand.md +0 -108
  26. package/template/skills/impeccable/reference/clarify.md +0 -288
  27. package/template/skills/impeccable/reference/codex.md +0 -105
  28. package/template/skills/impeccable/reference/colorize.md +0 -257
  29. package/template/skills/impeccable/reference/craft.md +0 -123
  30. package/template/skills/impeccable/reference/critique.md +0 -790
  31. package/template/skills/impeccable/reference/delight.md +0 -302
  32. package/template/skills/impeccable/reference/distill.md +0 -111
  33. package/template/skills/impeccable/reference/document.md +0 -429
  34. package/template/skills/impeccable/reference/extract.md +0 -69
  35. package/template/skills/impeccable/reference/harden.md +0 -347
  36. package/template/skills/impeccable/reference/hooks.md +0 -90
  37. package/template/skills/impeccable/reference/init.md +0 -172
  38. package/template/skills/impeccable/reference/interaction-design.md +0 -189
  39. package/template/skills/impeccable/reference/layout.md +0 -161
  40. package/template/skills/impeccable/reference/live.md +0 -720
  41. package/template/skills/impeccable/reference/onboard.md +0 -234
  42. package/template/skills/impeccable/reference/optimize.md +0 -258
  43. package/template/skills/impeccable/reference/overdrive.md +0 -130
  44. package/template/skills/impeccable/reference/polish.md +0 -241
  45. package/template/skills/impeccable/reference/product.md +0 -60
  46. package/template/skills/impeccable/reference/quieter.md +0 -99
  47. package/template/skills/impeccable/reference/shape.md +0 -165
  48. package/template/skills/impeccable/reference/typeset.md +0 -279
  49. package/template/skills/impeccable/scripts/command-metadata.json +0 -94
  50. package/template/skills/impeccable/scripts/context-signals.mjs +0 -225
  51. package/template/skills/impeccable/scripts/context.mjs +0 -961
  52. package/template/skills/impeccable/scripts/critique-storage.mjs +0 -242
  53. package/template/skills/impeccable/scripts/detect-csp.mjs +0 -198
  54. package/template/skills/impeccable/scripts/detect.mjs +0 -21
  55. package/template/skills/impeccable/scripts/detector/browser/injected/index.mjs +0 -1937
  56. package/template/skills/impeccable/scripts/detector/cli/main.mjs +0 -290
  57. package/template/skills/impeccable/scripts/detector/design-system.mjs +0 -750
  58. package/template/skills/impeccable/scripts/detector/detect-antipatterns-browser.js +0 -5138
  59. package/template/skills/impeccable/scripts/detector/detect-antipatterns.mjs +0 -50
  60. package/template/skills/impeccable/scripts/detector/engines/browser/detect-url.mjs +0 -277
  61. package/template/skills/impeccable/scripts/detector/engines/regex/detect-text.mjs +0 -568
  62. package/template/skills/impeccable/scripts/detector/engines/static-html/css-cascade.mjs +0 -1015
  63. package/template/skills/impeccable/scripts/detector/engines/static-html/detect-html.mjs +0 -234
  64. package/template/skills/impeccable/scripts/detector/engines/visual/screenshot-contrast.mjs +0 -189
  65. package/template/skills/impeccable/scripts/detector/findings.mjs +0 -12
  66. package/template/skills/impeccable/scripts/detector/node/file-system.mjs +0 -198
  67. package/template/skills/impeccable/scripts/detector/profile/profiler.mjs +0 -166
  68. package/template/skills/impeccable/scripts/detector/registry/antipatterns.mjs +0 -448
  69. package/template/skills/impeccable/scripts/detector/rules/checks.mjs +0 -2671
  70. package/template/skills/impeccable/scripts/detector/shared/color.mjs +0 -124
  71. package/template/skills/impeccable/scripts/detector/shared/constants.mjs +0 -101
  72. package/template/skills/impeccable/scripts/detector/shared/inline-ignores.mjs +0 -148
  73. package/template/skills/impeccable/scripts/detector/shared/page.mjs +0 -7
  74. package/template/skills/impeccable/scripts/hook-admin.mjs +0 -661
  75. package/template/skills/impeccable/scripts/hook-before-edit.mjs +0 -476
  76. package/template/skills/impeccable/scripts/hook-lib.mjs +0 -1632
  77. package/template/skills/impeccable/scripts/hook.mjs +0 -61
  78. package/template/skills/impeccable/scripts/lib/design-parser.mjs +0 -842
  79. package/template/skills/impeccable/scripts/lib/impeccable-config.mjs +0 -638
  80. package/template/skills/impeccable/scripts/lib/impeccable-paths.mjs +0 -128
  81. package/template/skills/impeccable/scripts/lib/is-generated.mjs +0 -69
  82. package/template/skills/impeccable/scripts/lib/target-args.mjs +0 -42
  83. package/template/skills/impeccable/scripts/live/browser-script-parts.mjs +0 -49
  84. package/template/skills/impeccable/scripts/live/completion.mjs +0 -19
  85. package/template/skills/impeccable/scripts/live/event-validation.mjs +0 -137
  86. package/template/skills/impeccable/scripts/live/insert-ui.mjs +0 -458
  87. package/template/skills/impeccable/scripts/live/manual-apply.mjs +0 -939
  88. package/template/skills/impeccable/scripts/live/manual-edit-routes.mjs +0 -357
  89. package/template/skills/impeccable/scripts/live/manual-edits-buffer.mjs +0 -152
  90. package/template/skills/impeccable/scripts/live/session-store.mjs +0 -289
  91. package/template/skills/impeccable/scripts/live/svelte-component.mjs +0 -826
  92. package/template/skills/impeccable/scripts/live/sveltekit-adapter.mjs +0 -274
  93. package/template/skills/impeccable/scripts/live/ui-core.mjs +0 -180
  94. package/template/skills/impeccable/scripts/live/vocabulary.mjs +0 -36
  95. package/template/skills/impeccable/scripts/live-accept.mjs +0 -812
  96. package/template/skills/impeccable/scripts/live-browser-dom.js +0 -146
  97. package/template/skills/impeccable/scripts/live-browser-session.js +0 -123
  98. package/template/skills/impeccable/scripts/live-browser.js +0 -11173
  99. package/template/skills/impeccable/scripts/live-commit-manual-edits.mjs +0 -1241
  100. package/template/skills/impeccable/scripts/live-complete.mjs +0 -75
  101. package/template/skills/impeccable/scripts/live-copy-edit-agent.mjs +0 -683
  102. package/template/skills/impeccable/scripts/live-discard-manual-edits.mjs +0 -51
  103. package/template/skills/impeccable/scripts/live-inject.mjs +0 -583
  104. package/template/skills/impeccable/scripts/live-insert.mjs +0 -272
  105. package/template/skills/impeccable/scripts/live-manual-edit-evidence.mjs +0 -363
  106. package/template/skills/impeccable/scripts/live-poll.mjs +0 -384
  107. package/template/skills/impeccable/scripts/live-resume.mjs +0 -94
  108. package/template/skills/impeccable/scripts/live-server.mjs +0 -1135
  109. package/template/skills/impeccable/scripts/live-status.mjs +0 -61
  110. package/template/skills/impeccable/scripts/live-target.mjs +0 -30
  111. package/template/skills/impeccable/scripts/live-wrap.mjs +0 -894
  112. package/template/skills/impeccable/scripts/live.mjs +0 -297
  113. package/template/skills/impeccable/scripts/modern-screenshot.umd.js +0 -14
  114. package/template/skills/impeccable/scripts/palette.mjs +0 -633
  115. package/template/skills/impeccable/scripts/pin.mjs +0 -214
  116. package/template/skills/skillopt-sleep/configs/_base_/default.yaml +0 -103
  117. package/template/skills/skillopt-sleep/configs/alfworld/default.yaml +0 -29
  118. package/template/skills/skillopt-sleep/configs/docvqa/default.yaml +0 -28
  119. package/template/skills/skillopt-sleep/configs/features/soft_gate.yaml +0 -47
  120. package/template/skills/skillopt-sleep/configs/livemathematicianbench/default.yaml +0 -22
  121. package/template/skills/skillopt-sleep/configs/officeqa/default.yaml +0 -34
  122. package/template/skills/skillopt-sleep/configs/searchqa/default.yaml +0 -32
  123. package/template/skills/skillopt-sleep/configs/spreadsheetbench/default.yaml +0 -34
  124. package/template/skills/skillopt-sleep/scripts/framework/skillopt/__init__.py +0 -28
  125. package/template/skills/skillopt-sleep/scripts/framework/skillopt/config.py +0 -282
  126. package/template/skills/skillopt-sleep/scripts/framework/skillopt/datasets/__init__.py +0 -7
  127. package/template/skills/skillopt-sleep/scripts/framework/skillopt/datasets/base.py +0 -512
  128. package/template/skills/skillopt-sleep/scripts/framework/skillopt/engine/__init__.py +0 -9
  129. package/template/skills/skillopt-sleep/scripts/framework/skillopt/engine/trainer.py +0 -2379
  130. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/__init__.py +0 -1
  131. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/_template/README.md +0 -43
  132. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/_template/config_template.yaml +0 -55
  133. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/_template/env_template.py +0 -151
  134. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/_template/loader_template.py +0 -87
  135. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/__init__.py +0 -5
  136. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/adapter.py +0 -428
  137. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/dataloader.py +0 -123
  138. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/prompts/analyst_error.md +0 -55
  139. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/prompts/analyst_success.md +0 -33
  140. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/prompts/rollout_no_history.md +0 -8
  141. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/prompts/rollout_with_history.md +0 -9
  142. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/prompts/rollout_with_memory.md +0 -16
  143. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/reflect.py +0 -4
  144. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/rollout.py +0 -366
  145. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/skills/initial.md +0 -45
  146. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/vendor/__init__.py +0 -9
  147. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/vendor/alfworld_envs.py +0 -221
  148. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/vendor/alfworld_projection.py +0 -60
  149. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/vendor/alfworld_prompts.py +0 -8
  150. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/vendor/config_tw.yaml +0 -145
  151. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/vendor/env_base.py +0 -84
  152. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/vendor/env_manager.py +0 -139
  153. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/alfworld/vendor/memory.py +0 -87
  154. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/base.py +0 -329
  155. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/docvqa/__init__.py +0 -1
  156. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/docvqa/adapter.py +0 -90
  157. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/docvqa/dataloader.py +0 -61
  158. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/docvqa/evaluator.py +0 -113
  159. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/docvqa/prompts/analyst_error.md +0 -35
  160. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/docvqa/prompts/analyst_success.md +0 -24
  161. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/docvqa/prompts/rollout_system.md +0 -12
  162. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/docvqa/rollout.py +0 -391
  163. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/docvqa/skills/initial.md +0 -11
  164. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/livemathematicianbench/__init__.py +0 -1
  165. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/livemathematicianbench/adapter.py +0 -129
  166. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/livemathematicianbench/dataloader.py +0 -308
  167. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/livemathematicianbench/evaluator.py +0 -62
  168. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/livemathematicianbench/prompts/analyst_error.md +0 -37
  169. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/livemathematicianbench/prompts/analyst_success.md +0 -25
  170. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/livemathematicianbench/prompts/rollout_system.md +0 -12
  171. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/livemathematicianbench/reflect.py +0 -4
  172. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/livemathematicianbench/rollout.py +0 -434
  173. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/livemathematicianbench/skills/initial.md +0 -16
  174. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/officeqa/__init__.py +0 -1
  175. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/officeqa/adapter.py +0 -112
  176. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/officeqa/dataloader.py +0 -71
  177. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/officeqa/evaluator.py +0 -46
  178. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/officeqa/prompts/analyst_error.md +0 -37
  179. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/officeqa/prompts/analyst_success.md +0 -25
  180. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/officeqa/prompts/rollout_system.md +0 -15
  181. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/officeqa/rollout.py +0 -799
  182. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/officeqa/skills/initial.md +0 -15
  183. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/officeqa/tool_runtime.py +0 -552
  184. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/searchqa/__init__.py +0 -1
  185. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/searchqa/adapter.py +0 -96
  186. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/searchqa/dataloader.py +0 -42
  187. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/searchqa/evaluator.py +0 -100
  188. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/searchqa/prompts/analyst_error.md +0 -46
  189. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/searchqa/prompts/analyst_success.md +0 -32
  190. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/searchqa/prompts/rollout_system.md +0 -13
  191. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/searchqa/reflect.py +0 -4
  192. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/searchqa/rollout.py +0 -494
  193. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/searchqa/skills/initial.md +0 -3
  194. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/spreadsheetbench/__init__.py +0 -5
  195. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/spreadsheetbench/adapter.py +0 -159
  196. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/spreadsheetbench/codegen_agent.py +0 -731
  197. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/spreadsheetbench/dataloader.py +0 -37
  198. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/spreadsheetbench/evaluator.py +0 -158
  199. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/spreadsheetbench/executor.py +0 -67
  200. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/spreadsheetbench/prompts/analyst_error.md +0 -46
  201. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/spreadsheetbench/prompts/analyst_success.md +0 -32
  202. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/spreadsheetbench/prompts/codegen_system.md +0 -1
  203. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/spreadsheetbench/prompts/critical_rules.md +0 -9
  204. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/spreadsheetbench/prompts/react_system.md +0 -21
  205. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/spreadsheetbench/react_agent.py +0 -395
  206. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/spreadsheetbench/reflect.py +0 -4
  207. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/spreadsheetbench/rollout.py +0 -979
  208. package/template/skills/skillopt-sleep/scripts/framework/skillopt/envs/spreadsheetbench/skills/initial.md +0 -56
  209. package/template/skills/skillopt-sleep/scripts/framework/skillopt/evaluation/__init__.py +0 -13
  210. package/template/skills/skillopt-sleep/scripts/framework/skillopt/evaluation/gate.py +0 -148
  211. package/template/skills/skillopt-sleep/scripts/framework/skillopt/gradient/__init__.py +0 -15
  212. package/template/skills/skillopt-sleep/scripts/framework/skillopt/gradient/aggregate.py +0 -253
  213. package/template/skills/skillopt-sleep/scripts/framework/skillopt/gradient/reflect.py +0 -635
  214. package/template/skills/skillopt-sleep/scripts/framework/skillopt/model/__init__.py +0 -514
  215. package/template/skills/skillopt-sleep/scripts/framework/skillopt/model/azure_openai.py +0 -915
  216. package/template/skills/skillopt-sleep/scripts/framework/skillopt/model/backend_config.py +0 -185
  217. package/template/skills/skillopt-sleep/scripts/framework/skillopt/model/claude_backend.py +0 -371
  218. package/template/skills/skillopt-sleep/scripts/framework/skillopt/model/codex_backend.py +0 -666
  219. package/template/skills/skillopt-sleep/scripts/framework/skillopt/model/codex_harness.py +0 -1057
  220. package/template/skills/skillopt-sleep/scripts/framework/skillopt/model/common.py +0 -229
  221. package/template/skills/skillopt-sleep/scripts/framework/skillopt/model/minimax_backend.py +0 -277
  222. package/template/skills/skillopt-sleep/scripts/framework/skillopt/model/qwen_backend.py +0 -456
  223. package/template/skills/skillopt-sleep/scripts/framework/skillopt/model/router.py +0 -236
  224. package/template/skills/skillopt-sleep/scripts/framework/skillopt/optimizer/__init__.py +0 -15
  225. package/template/skills/skillopt-sleep/scripts/framework/skillopt/optimizer/appendix.py +0 -156
  226. package/template/skills/skillopt-sleep/scripts/framework/skillopt/optimizer/clip.py +0 -109
  227. package/template/skills/skillopt-sleep/scripts/framework/skillopt/optimizer/lr_autonomous.py +0 -108
  228. package/template/skills/skillopt-sleep/scripts/framework/skillopt/optimizer/meta_skill.py +0 -79
  229. package/template/skills/skillopt-sleep/scripts/framework/skillopt/optimizer/rewrite.py +0 -59
  230. package/template/skills/skillopt-sleep/scripts/framework/skillopt/optimizer/scheduler.py +0 -127
  231. package/template/skills/skillopt-sleep/scripts/framework/skillopt/optimizer/select.py +0 -4
  232. package/template/skills/skillopt-sleep/scripts/framework/skillopt/optimizer/skill.py +0 -201
  233. package/template/skills/skillopt-sleep/scripts/framework/skillopt/optimizer/skill_aware.py +0 -206
  234. package/template/skills/skillopt-sleep/scripts/framework/skillopt/optimizer/slow_update.py +0 -396
  235. package/template/skills/skillopt-sleep/scripts/framework/skillopt/optimizer/update_modes.py +0 -135
  236. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/__init__.py +0 -63
  237. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/analyst_error.md +0 -41
  238. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/analyst_error_full_rewrite.md +0 -32
  239. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/analyst_error_rewrite.md +0 -44
  240. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/analyst_success.md +0 -36
  241. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/analyst_success_full_rewrite.md +0 -30
  242. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/analyst_success_rewrite.md +0 -33
  243. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/lr_autonomous.md +0 -20
  244. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/merge_failure.md +0 -30
  245. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/merge_failure_full_rewrite.md +0 -28
  246. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/merge_failure_rewrite.md +0 -26
  247. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/merge_final.md +0 -33
  248. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/merge_final_full_rewrite.md +0 -28
  249. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/merge_final_rewrite.md +0 -25
  250. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/merge_success.md +0 -28
  251. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/merge_success_full_rewrite.md +0 -28
  252. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/merge_success_rewrite.md +0 -25
  253. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/meta_skill.md +0 -40
  254. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/ranking.md +0 -20
  255. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/ranking_rewrite.md +0 -15
  256. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/rewrite_skill.md +0 -25
  257. package/template/skills/skillopt-sleep/scripts/framework/skillopt/prompts/slow_update.md +0 -60
  258. package/template/skills/skillopt-sleep/scripts/framework/skillopt/scheduler/__init__.py +0 -8
  259. package/template/skills/skillopt-sleep/scripts/framework/skillopt/types.py +0 -306
  260. package/template/skills/skillopt-sleep/scripts/framework/skillopt/utils/__init__.py +0 -4
  261. package/template/skills/skillopt-sleep/scripts/framework/skillopt/utils/json_utils.py +0 -172
  262. package/template/skills/skillopt-sleep/scripts/framework/skillopt/utils/scoring.py +0 -28
  263. package/template/skills/skillopt-sleep/scripts/python/__init__.py +0 -20
  264. package/template/skills/skillopt-sleep/scripts/python/__main__.py +0 -343
  265. package/template/skills/skillopt-sleep/scripts/python/backend.py +0 -1371
  266. package/template/skills/skillopt-sleep/scripts/python/budget.py +0 -75
  267. package/template/skills/skillopt-sleep/scripts/python/config.py +0 -162
  268. package/template/skills/skillopt-sleep/scripts/python/consolidate.py +0 -238
  269. package/template/skills/skillopt-sleep/scripts/python/cycle.py +0 -291
  270. package/template/skills/skillopt-sleep/scripts/python/dream.py +0 -138
  271. package/template/skills/skillopt-sleep/scripts/python/experiments/__init__.py +0 -1
  272. package/template/skills/skillopt-sleep/scripts/python/experiments/gbrain_bench.py +0 -119
  273. package/template/skills/skillopt-sleep/scripts/python/experiments/personas.py +0 -86
  274. package/template/skills/skillopt-sleep/scripts/python/experiments/report.py +0 -132
  275. package/template/skills/skillopt-sleep/scripts/python/experiments/run_experiment.py +0 -178
  276. package/template/skills/skillopt-sleep/scripts/python/experiments/run_gbrain.py +0 -209
  277. package/template/skills/skillopt-sleep/scripts/python/experiments/run_transfer.py +0 -155
  278. package/template/skills/skillopt-sleep/scripts/python/experiments/sweep.py +0 -164
  279. package/template/skills/skillopt-sleep/scripts/python/gate.py +0 -50
  280. package/template/skills/skillopt-sleep/scripts/python/harvest.py +0 -304
  281. package/template/skills/skillopt-sleep/scripts/python/harvest_codex.py +0 -253
  282. package/template/skills/skillopt-sleep/scripts/python/harvest_sources.py +0 -41
  283. package/template/skills/skillopt-sleep/scripts/python/judges.py +0 -84
  284. package/template/skills/skillopt-sleep/scripts/python/llm_miner.py +0 -134
  285. package/template/skills/skillopt-sleep/scripts/python/memory.py +0 -129
  286. package/template/skills/skillopt-sleep/scripts/python/mine.py +0 -312
  287. package/template/skills/skillopt-sleep/scripts/python/replay.py +0 -146
  288. package/template/skills/skillopt-sleep/scripts/python/rollout.py +0 -153
  289. package/template/skills/skillopt-sleep/scripts/python/scheduler.py +0 -138
  290. package/template/skills/skillopt-sleep/scripts/python/slow_update.py +0 -142
  291. package/template/skills/skillopt-sleep/scripts/python/staging.py +0 -103
  292. package/template/skills/skillopt-sleep/scripts/python/state.py +0 -96
  293. package/template/skills/skillopt-sleep/scripts/python/tasks_file.py +0 -81
  294. package/template/skills/skillopt-sleep/scripts/python/types.py +0 -146
  295. package/template/skills/skillopt-sleep/scripts/shell/__init__.py +0 -0
  296. package/template/skills/skillopt-sleep/scripts/shell/eval_only.py +0 -466
  297. package/template/skills/skillopt-sleep/scripts/shell/materialize_searchqa.py +0 -148
  298. package/template/skills/skillopt-sleep/scripts/shell/run_alfworld.sh +0 -60
  299. package/template/skills/skillopt-sleep/scripts/shell/run_searchqa.sh +0 -40
  300. package/template/skills/skillopt-sleep/scripts/shell/run_spreadsheetbench.sh +0 -39
  301. package/template/skills/skillopt-sleep/scripts/shell/train.py +0 -556
@@ -1,428 +0,0 @@
1
- """ALFWorld environment adapter for ReflACT.
2
-
3
- Connects the ReflACT training loop to ALFWorld by implementing
4
- :class:`~skillopt.envs.base.EnvAdapter`.
5
- """
6
- from __future__ import annotations
7
-
8
- from dataclasses import dataclass
9
- import json
10
- import os
11
-
12
- from skillopt.datasets.base import BatchSpec
13
- from skillopt.envs.base import EnvAdapter
14
- from skillopt.envs.alfworld.dataloader import ALFWorldDataLoader
15
- from skillopt.envs.alfworld.rollout import (
16
- build_alfworld_env,
17
- run_alfworld_batch,
18
- TASKS,
19
- )
20
- from skillopt.utils import compute_score
21
-
22
-
23
- @dataclass(frozen=True)
24
- class ALFWorldBatchRun:
25
- """Lazy ALFWorld batch description.
26
-
27
- The adapter materializes this in rollout chunks so a large evaluation set
28
- does not keep every ALFWorld simulator open at once.
29
- """
30
-
31
- env_num: int
32
- eval_dataset: str
33
- seed: int
34
- is_train: bool
35
- workers: int
36
- specific_gamefiles: list[str] | None = None
37
- result_ids: list[str] | None = None
38
- items: list[dict] | None = None
39
-
40
- def __iter__(self):
41
- return iter(self.items or [])
42
-
43
- def __len__(self) -> int:
44
- return int(self.env_num or 0)
45
-
46
-
47
- class ALFWorldAdapter(EnvAdapter):
48
- """ALFWorld environment adapter.
49
-
50
- Parameters
51
- ----------
52
- max_steps : int
53
- Maximum steps per ALFWorld episode (default 50).
54
- max_api_workers : int
55
- Maximum concurrent API calls during rollout (default 8).
56
- analyst_workers : int
57
- Parallel workers for analyst stage (default 16).
58
- failure_only : bool
59
- If True, only run error analyst (skip success analyst).
60
- minibatch_size : int
61
- Trajectories per analyst group, M (default 8).
62
- edit_budget : int
63
- Maximum edits per minibatch, L (default 4).
64
- """
65
-
66
- def __init__(
67
- self,
68
- split_dir: str = "",
69
- data_path: str = "",
70
- split_mode: str = "split_dir",
71
- split_ratio: str = "2:1:7",
72
- split_seed: int = 42,
73
- split_output_dir: str = "",
74
- seed: int = 42,
75
- limit: int = 0,
76
- train_size: int = 0,
77
- max_steps: int = 50,
78
- workers: int = 8,
79
- max_api_workers: int = 8,
80
- analyst_workers: int = 16,
81
- failure_only: bool = False,
82
- minibatch_size: int = 8,
83
- edit_budget: int = 4,
84
- max_completion_tokens: int = 16384,
85
- ) -> None:
86
- self.max_steps = max_steps
87
- self.workers = max(int(workers or 1), 1)
88
- self.max_api_workers = max_api_workers
89
- self.max_completion_tokens = int(max_completion_tokens)
90
- self.analyst_workers = analyst_workers
91
- self.failure_only = failure_only
92
- self.minibatch_size = minibatch_size
93
- self.edit_budget = edit_budget
94
- self.dataloader = ALFWorldDataLoader(
95
- split_dir=split_dir,
96
- data_path=data_path,
97
- split_mode=split_mode,
98
- split_ratio=split_ratio,
99
- split_seed=split_seed,
100
- split_output_dir=split_output_dir,
101
- seed=seed,
102
- limit=limit,
103
- train_size=train_size,
104
- )
105
- self._traj_cache: dict[str, dict | None] = {}
106
-
107
- def setup(self, cfg: dict) -> None:
108
- super().setup(cfg)
109
- self.dataloader.setup(cfg)
110
-
111
- def _load_traj_data(self, item: dict) -> dict | None:
112
- gamefile = str(item.get("gamefile") or "").strip()
113
- if not gamefile:
114
- return None
115
- if gamefile in self._traj_cache:
116
- return self._traj_cache[gamefile]
117
-
118
- traj_path = os.path.join(os.path.dirname(gamefile), "traj_data.json")
119
- try:
120
- with open(traj_path, encoding="utf-8") as f:
121
- data = json.load(f)
122
- except Exception:
123
- data = None
124
- self._traj_cache[gamefile] = data
125
- return data
126
-
127
- @staticmethod
128
- def _unique_lines(values: list[str], *, limit: int = 0) -> list[str]:
129
- lines: list[str] = []
130
- seen: set[str] = set()
131
- for raw in values:
132
- line = str(raw or "").strip()
133
- if not line or line in seen:
134
- continue
135
- seen.add(line)
136
- lines.append(line)
137
- if limit > 0 and len(lines) >= limit:
138
- break
139
- return lines
140
-
141
- @staticmethod
142
- def _format_high_pddl(high_pddl: list[dict]) -> list[str]:
143
- steps: list[str] = []
144
- for idx, step in enumerate(high_pddl or [], start=1):
145
- discrete = step.get("discrete_action") or {}
146
- action = str(discrete.get("action") or "").strip()
147
- args = [str(arg).strip() for arg in (discrete.get("args") or []) if str(arg).strip()]
148
- if action and args:
149
- text = f"{action}({', '.join(args)})"
150
- elif action:
151
- text = action
152
- else:
153
- planner_action = step.get("planner_action") or {}
154
- text = str(planner_action.get("action") or "").strip()
155
- if text:
156
- steps.append(f"{idx}. {text}")
157
- return steps
158
-
159
- def _build_reference_bundle(self, item: dict) -> dict:
160
- data = self._load_traj_data(item)
161
- if not data:
162
- return {}
163
-
164
- anns = ((data.get("turk_annotations") or {}).get("anns") or [])
165
- task_descs = self._unique_lines(
166
- [ann.get("task_desc", "") for ann in anns],
167
- limit=3,
168
- )
169
- high_descs = self._unique_lines(
170
- [step for ann in anns for step in (ann.get("high_descs") or [])],
171
- limit=12,
172
- )
173
- pddl_params = {
174
- key: value
175
- for key, value in (data.get("pddl_params") or {}).items()
176
- if value not in ("", None, [], {})
177
- }
178
- scene = data.get("scene") or {}
179
- scene_summary = {
180
- key: scene.get(key)
181
- for key in ("floor_plan", "scene_num", "dirty_and_empty")
182
- if scene.get(key) not in ("", None, [], {})
183
- }
184
- high_pddl = self._format_high_pddl((data.get("plan") or {}).get("high_pddl") or [])
185
- task_type = str(data.get("task_type") or item.get("task_type") or "").strip()
186
- return {
187
- "task_type": task_type,
188
- "task_descs": task_descs,
189
- "high_descs": high_descs,
190
- "pddl_params": pddl_params,
191
- "high_pddl": high_pddl,
192
- "scene_summary": scene_summary,
193
- }
194
-
195
- def build_reference_text(self, item: dict) -> str:
196
- bundle = self._build_reference_bundle(item)
197
- if not bundle:
198
- return ""
199
-
200
- parts: list[str] = []
201
- if bundle["task_type"]:
202
- parts.append(f"## Reference Task Type\n{bundle['task_type']}")
203
- if bundle["task_descs"]:
204
- parts.append(
205
- "## Reference Human Task Descriptions\n"
206
- + "\n".join(f"- {line}" for line in bundle["task_descs"])
207
- )
208
- if bundle["high_descs"]:
209
- parts.append(
210
- "## Reference Human High-Level Steps\n"
211
- + "\n".join(f"{idx}. {line}" for idx, line in enumerate(bundle["high_descs"], start=1))
212
- )
213
- if bundle["pddl_params"]:
214
- parts.append(
215
- "## Reference PDDL Params\n"
216
- + "\n".join(f"- {key}: {value}" for key, value in bundle["pddl_params"].items())
217
- )
218
- if bundle["high_pddl"]:
219
- parts.append(
220
- "## Reference Planner High-Level Plan\n" + "\n".join(bundle["high_pddl"])
221
- )
222
- if bundle["scene_summary"]:
223
- parts.append(
224
- "## Reference Scene Summary\n"
225
- + "\n".join(f"- {key}: {value}" for key, value in bundle["scene_summary"].items())
226
- )
227
- return "\n\n".join(parts)
228
-
229
- def get_reference_metadata(self, item: dict) -> dict:
230
- bundle = self._build_reference_bundle(item)
231
- if not bundle:
232
- return {"fields": [], "preview": ""}
233
-
234
- fields: list[str] = []
235
- previews: list[str] = []
236
- if bundle["task_type"]:
237
- fields.append("task_type")
238
- previews.append(f"[task_type] {bundle['task_type']}")
239
- if bundle["task_descs"]:
240
- fields.append("task_desc")
241
- previews.append("[task_desc]\n" + "\n".join(bundle["task_descs"][:2]))
242
- if bundle["high_descs"]:
243
- fields.append("high_descs")
244
- previews.append("[high_descs]\n" + "\n".join(bundle["high_descs"][:3]))
245
- if bundle["pddl_params"]:
246
- fields.append("pddl_params")
247
- previews.append(
248
- "[pddl_params]\n"
249
- + "\n".join(
250
- f"{key}: {value}" for key, value in list(bundle["pddl_params"].items())[:4]
251
- )
252
- )
253
- if bundle["high_pddl"]:
254
- fields.append("plan.high_pddl")
255
- previews.append("[plan.high_pddl]\n" + "\n".join(bundle["high_pddl"][:3]))
256
- if bundle["scene_summary"]:
257
- fields.append("scene")
258
- previews.append(
259
- "[scene]\n"
260
- + "\n".join(
261
- f"{key}: {value}" for key, value in bundle["scene_summary"].items()
262
- )
263
- )
264
- return {
265
- "fields": fields,
266
- "preview": "\n\n".join(previews)[:600],
267
- }
268
-
269
- @staticmethod
270
- def _infer_dataset_from_gamefile(gamefile: str) -> tuple[str, bool]:
271
- path = str(gamefile or "")
272
- if "/valid_seen/" in path:
273
- return "eval_in_distribution", False
274
- if "/valid_unseen/" in path:
275
- return "eval_out_of_distribution", False
276
- return "train", True
277
-
278
- def get_dataloader(self):
279
- return self.dataloader
280
-
281
- def _comparison_items(self, items: list[dict]) -> list[dict]:
282
- enriched: list[dict] = []
283
- for item in items:
284
- row = dict(item)
285
- bundle = self._build_reference_bundle(row)
286
- if bundle.get("task_descs"):
287
- row["task_description"] = bundle["task_descs"][0]
288
- elif bundle.get("task_type"):
289
- row["task_description"] = bundle["task_type"]
290
- enriched.append(row)
291
- return enriched
292
-
293
- def requires_ray(self) -> bool:
294
- return False
295
-
296
- def build_env_from_batch(self, batch: BatchSpec, **kwargs):
297
- gamefiles = list(batch.metadata.get("gamefiles") or [])
298
- result_ids = list(batch.metadata.get("result_ids") or [])
299
- items = self._comparison_items(list(batch.payload or []))
300
- return ALFWorldBatchRun(
301
- env_num=batch.batch_size,
302
- eval_dataset=batch.metadata.get("eval_dataset", batch.split),
303
- seed=batch.seed,
304
- is_train=batch.metadata.get("is_train", batch.phase == "train"),
305
- specific_gamefiles=gamefiles or None,
306
- result_ids=result_ids or None,
307
- items=items,
308
- workers=self.workers,
309
- )
310
-
311
- def build_train_env(self, batch_size: int, seed: int, **kwargs):
312
- batch = self.dataloader.build_train_batch(batch_size=batch_size, seed=seed, **kwargs)
313
- return self.build_env_from_batch(batch, **kwargs)
314
-
315
- def build_eval_env(self, env_num: int, split: str, seed: int, **kwargs):
316
- batch = self.dataloader.build_eval_batch(env_num=env_num, split=split, seed=seed, **kwargs)
317
- return self.build_env_from_batch(batch, **kwargs)
318
-
319
- def rollout(
320
- self,
321
- env_manager,
322
- skill_content: str,
323
- out_dir: str,
324
- **kwargs,
325
- ) -> list[dict]:
326
- results_path = os.path.join(out_dir, "results.jsonl")
327
- os.makedirs(out_dir, exist_ok=True)
328
-
329
- # Resume support
330
- if os.path.exists(results_path):
331
- existing: list[dict] = []
332
- with open(results_path) as f:
333
- for line in f:
334
- try:
335
- existing.append(json.loads(line))
336
- except Exception:
337
- pass
338
- if existing:
339
- return existing
340
-
341
- if isinstance(env_manager, ALFWorldBatchRun):
342
- results = self._run_batch(
343
- env_manager,
344
- skill_content=skill_content,
345
- out_dir=out_dir,
346
- )
347
- else:
348
- results = run_alfworld_batch(
349
- env_manager=env_manager,
350
- skill_content=skill_content,
351
- max_steps=self.max_steps,
352
- out_root=out_dir,
353
- max_api_workers=self.max_api_workers,
354
- max_completion_tokens=self.max_completion_tokens,
355
- result_ids=getattr(env_manager, "_skillopt_result_ids", None),
356
- )
357
-
358
- with open(results_path, "w") as f:
359
- for r in results:
360
- f.write(json.dumps(r, ensure_ascii=False) + "\n")
361
-
362
- return results
363
-
364
- @staticmethod
365
- def _close_env(env_manager) -> None:
366
- close = getattr(env_manager, "close", None)
367
- if callable(close):
368
- close()
369
-
370
- def _run_batch(
371
- self,
372
- batch: ALFWorldBatchRun,
373
- skill_content: str,
374
- out_dir: str,
375
- *,
376
- diagnostic_mode: bool = False,
377
- diagnostic_instruction: str = "",
378
- ) -> list[dict]:
379
- total = int(batch.env_num or 0)
380
- if total <= 0:
381
- return []
382
-
383
- workers = max(1, min(int(batch.workers or self.workers), total))
384
- if total > workers:
385
- print(
386
- f" [alfworld rollout] episodes={total} "
387
- f"env_workers={workers} chunks={(total + workers - 1) // workers}"
388
- )
389
-
390
- all_results: list[dict] = []
391
- for start in range(0, total, workers):
392
- chunk_size = min(workers, total - start)
393
- chunk_gamefiles = (
394
- batch.specific_gamefiles[start:start + chunk_size]
395
- if batch.specific_gamefiles
396
- else None
397
- )
398
- chunk_ids = (
399
- batch.result_ids[start:start + chunk_size]
400
- if batch.result_ids
401
- else [f"env_{idx:03d}" for idx in range(start, start + chunk_size)]
402
- )
403
- chunk_env = build_alfworld_env(
404
- env_num=chunk_size,
405
- eval_dataset=batch.eval_dataset,
406
- seed=batch.seed + start,
407
- is_train=batch.is_train,
408
- specific_gamefiles=chunk_gamefiles,
409
- )
410
- try:
411
- chunk_results = run_alfworld_batch(
412
- env_manager=chunk_env,
413
- skill_content=skill_content,
414
- max_steps=self.max_steps,
415
- out_root=out_dir,
416
- max_api_workers=min(self.max_api_workers, chunk_size),
417
- max_completion_tokens=self.max_completion_tokens,
418
- diagnostic_mode=diagnostic_mode,
419
- diagnostic_instruction=diagnostic_instruction,
420
- result_ids=chunk_ids,
421
- )
422
- finally:
423
- self._close_env(chunk_env)
424
- all_results.extend(chunk_results)
425
- return all_results
426
-
427
- def get_task_types(self) -> list[str]:
428
- return list(TASKS)
@@ -1,123 +0,0 @@
1
- """ALFWorld task dataloader."""
2
- from __future__ import annotations
3
-
4
- from skillopt.datasets.base import BatchSpec, SplitDataLoader
5
-
6
-
7
- class ALFWorldDataLoader(SplitDataLoader):
8
- """ALFWorld batch planner.
9
-
10
- In split_dir mode, batches are fixed gamefile items so ablations differ
11
- only in how the same training set is batched.
12
- """
13
-
14
- def __init__(
15
- self,
16
- split_dir: str = "",
17
- data_path: str = "",
18
- split_mode: str = "split_dir",
19
- split_ratio: str = "2:1:7",
20
- split_seed: int = 42,
21
- split_output_dir: str = "",
22
- seed: int = 42,
23
- limit: int = 0,
24
- train_size: int = 0,
25
- **kwargs,
26
- ) -> None:
27
- super().__init__(
28
- split_dir=split_dir,
29
- data_path=data_path,
30
- split_mode=split_mode,
31
- split_ratio=split_ratio,
32
- split_seed=split_seed,
33
- split_output_dir=split_output_dir,
34
- seed=seed,
35
- limit=limit,
36
- )
37
- self.train_size_override = int(train_size or 0)
38
-
39
- @staticmethod
40
- def _metadata_for_items(items: list[dict], split: str, phase: str) -> dict:
41
- gamefiles = [str(item.get("gamefile") or "") for item in items]
42
- if any(not gamefile for gamefile in gamefiles):
43
- raise ValueError("ALFWorld split items must contain non-empty gamefile paths.")
44
- eval_dataset = "train"
45
- is_train = phase == "train"
46
- first = gamefiles[0] if gamefiles else ""
47
- if "/valid_seen/" in first:
48
- eval_dataset = "eval_in_distribution"
49
- is_train = False
50
- elif "/valid_unseen/" in first:
51
- eval_dataset = "eval_out_of_distribution"
52
- is_train = False
53
- return {
54
- "eval_dataset": eval_dataset,
55
- "is_train": is_train,
56
- "gamefiles": gamefiles,
57
- "result_ids": [str(item.get("id") or idx) for idx, item in enumerate(items)],
58
- }
59
-
60
- def get_train_size(self) -> int:
61
- if self.train_size_override > 0:
62
- return self.train_size_override
63
- return super().get_train_size()
64
-
65
- def build_train_batch(self, batch_size: int, seed: int, **kwargs) -> BatchSpec:
66
- batch = super().build_train_batch(batch_size=batch_size, seed=seed, **kwargs)
67
- items = list(batch.payload or [])
68
- batch.metadata.update(self._metadata_for_items(items, "train", "train"))
69
- return BatchSpec(
70
- phase="train",
71
- split="train",
72
- seed=seed,
73
- batch_size=len(items),
74
- payload=items,
75
- metadata=batch.metadata,
76
- )
77
-
78
- def plan_train_epoch(
79
- self,
80
- *,
81
- epoch: int,
82
- steps_per_epoch: int,
83
- accumulation: int,
84
- batch_size: int,
85
- seed: int,
86
- **kwargs,
87
- ) -> list[BatchSpec]:
88
- batches = super().plan_train_epoch(
89
- epoch=epoch,
90
- steps_per_epoch=steps_per_epoch,
91
- accumulation=accumulation,
92
- batch_size=batch_size,
93
- seed=seed,
94
- **kwargs,
95
- )
96
- for batch in batches:
97
- items = list(batch.payload or [])
98
- batch.metadata.update(self._metadata_for_items(items, "train", "train"))
99
- return batches
100
-
101
- def build_eval_batch(
102
- self,
103
- env_num: int,
104
- split: str,
105
- seed: int,
106
- **kwargs,
107
- ) -> BatchSpec:
108
- batch = super().build_eval_batch(
109
- env_num=env_num,
110
- split=split,
111
- seed=seed,
112
- **kwargs,
113
- )
114
- items = list(batch.payload or [])
115
- batch.metadata.update(self._metadata_for_items(items, split, "eval"))
116
- return BatchSpec(
117
- phase="eval",
118
- split=split,
119
- seed=seed,
120
- batch_size=len(items),
121
- payload=items,
122
- metadata=batch.metadata,
123
- )
@@ -1,55 +0,0 @@
1
- You are an expert failure-analysis agent for ALFWorld embodied household tasks.
2
-
3
- You will be given MULTIPLE failed agent trajectories from a single minibatch
4
- and the current skill document.
5
- Your job is to identify the most important COMMON failure patterns across
6
- the batch and propose a concise set of skill edits.
7
-
8
- ## ALFWorld Task Types
9
- - pick_and_place: Put object in/on a receptacle
10
- - pick_two_obj_and_place: Put two instances of an object in/on a receptacle
11
- - look_at_obj_in_light: Examine an object under a desklamp
12
- - pick_heat_then_place_in_recep: Heat an object and put it in/on a receptacle
13
- - pick_cool_then_place_in_recep: Cool an object and put it in/on a receptacle
14
- - pick_clean_then_place_in_recep: Clean an object and put it in/on a receptacle
15
-
16
- ## Failure Type Categories
17
- - **navigation_loop**: the agent revisits the same locations repeatedly without progress
18
- - **missed_object**: the agent fails to pick up a visible/reachable goal object
19
- - **wrong_sequence**: the agent performs actions in the wrong order (e.g., placing before transforming)
20
- - **premature_stop**: the agent stops or gets stuck before completing all goal conditions
21
- - **action_loop**: the agent repeats the same action without advancing
22
- - **appliance_error**: the agent misuses or skips an appliance (microwave, fridge, sink)
23
- - **rule_missing**: the skill lacks a relevant rule for this situation
24
- - **rule_wrong**: an existing skill rule is misleading or incorrect
25
- - **rule_ignored**: the skill has the right rule but the agent did not follow it
26
- - **other**: none of the above
27
-
28
- ## Analysis Process
29
- 1. Read ALL trajectories in the minibatch.
30
- 2. Identify the most prevalent, systematic failure patterns across them.
31
- 3. For each pattern, classify its failure type.
32
- 4. Propose skill edits that address the COMMON patterns — not individual edge cases.
33
- 5. Edits must be generalizable; do not hardcode task-specific values.
34
- 6. Only patch gaps in the skill — do not duplicate existing content.
35
-
36
- You will be told the maximum number of edits (the budget L). Produce AT MOST L edits,
37
- focusing on the highest-impact patterns. You may produce fewer if warranted.
38
-
39
- Respond ONLY with a valid JSON object (no markdown fences, no extra text):
40
- {
41
- "batch_size": <number of trajectories analysed>,
42
- "failure_summary": [
43
- {"failure_type": "<type>", "count": <int>, "description": "<one-line>"}
44
- ],
45
- "patch": {
46
- "reasoning": "<why these edits address the batch's common failures>",
47
- "edits": [
48
- {"op": "append", "content": "<markdown to add at end of skill>"},
49
- {"op": "insert_after", "target": "<exact heading/text to insert after>", "content": "<markdown>"},
50
- {"op": "replace", "target": "<exact text to replace>", "content": "<replacement>"},
51
- {"op": "delete", "target": "<exact text to remove>"}
52
- ]
53
- }
54
- }
55
- Only include edits that are needed. "edits" can be an empty list if no patch is warranted.
@@ -1,33 +0,0 @@
1
- You are an expert success-pattern analyst for AI agents operating in ALFWorld,
2
- a text-based embodied household environment.
3
-
4
- You will be given MULTIPLE successful agent trajectories from a single minibatch
5
- and the current skill document. Your job is to identify generalizable behavior
6
- patterns that are COMMON across the batch and worth encoding in the skill.
7
-
8
- ## Rules
9
- - Only propose patches for patterns NOT already covered in the skill.
10
- - Focus on patterns that appear across MULTIPLE trajectories in the batch.
11
- - Be concise. Patterns must generalize beyond specific tasks.
12
- - Prefer reinforcing existing sections over adding new top-level sections.
13
- - If the agents' success involved efficient exploration or smart appliance usage,
14
- consider reinforcing that in the patch.
15
-
16
- You will be told the maximum number of edits (the budget L). Produce AT MOST L edits,
17
- focusing on the most broadly applicable patterns. You may produce fewer if warranted.
18
-
19
- Respond ONLY with a valid JSON object:
20
- {
21
- "batch_size": <number of trajectories analysed>,
22
- "success_patterns": ["<pattern 1>", "<pattern 2>"],
23
- "patch": {
24
- "reasoning": "<why these patterns are worth encoding>",
25
- "edits": [
26
- {"op": "append", "content": "<markdown>"},
27
- {"op": "insert_after", "target": "<heading/text>", "content": "<markdown>"},
28
- {"op": "replace", "target": "<old text>", "content": "<new text>"},
29
- {"op": "delete", "target": "<exact text to remove>"}
30
- ]
31
- }
32
- }
33
- "edits" may be empty if the skill already covers all observed patterns.
@@ -1,8 +0,0 @@
1
-
2
- You are an expert agent operating in the ALFRED Embodied Environment.
3
- Your current observation is: {current_observation}
4
- Your admissible actions of the current situation are: [{admissible_actions}].
5
-
6
- Now it's your turn to take an action.
7
- You should first reason step-by-step about the current situation. This reasoning process MUST be enclosed within <think> </think> tags.
8
- Once you've finished your reasoning, you should choose an admissible action for current step and present it within <action> </action> tags.
@@ -1,9 +0,0 @@
1
-
2
- You are an expert agent operating in the ALFRED Embodied Environment. Your task is to: {task_description}
3
- Prior to this step, you have already taken {step_count} step(s). Below are the most recent {history_length} observations and the corresponding actions you took: {action_history}
4
- You are now at step {current_step} and your current observation is: {current_observation}
5
- Your admissible actions of the current situation are: [{admissible_actions}].
6
-
7
- Now it's your turn to take an action.
8
- You should first reason step-by-step about the current situation. This reasoning process MUST be enclosed within <think> </think> tags.
9
- Once you've finished your reasoning, you should choose an admissible action for current step and present it within <action> </action> tags.
@@ -1,16 +0,0 @@
1
-
2
- You are an expert agent operating in the ALFRED Embodied Environment. Your task is to: {task_description}
3
-
4
- ## Retrieved Relevant Experience
5
-
6
- {retrieved_memories}
7
-
8
- ## Current Progress
9
-
10
- Prior to this step, you have already taken {step_count} step(s). Below are the most recent {history_length} observations and the corresponding actions you took: {action_history}
11
- You are now at step {current_step} and your current observation is: {current_observation}
12
- Your admissible actions of the current situation are: [{admissible_actions}].
13
-
14
- Now it's your turn to take an action.
15
- You should first reason step-by-step about the current situation. This reasoning process MUST be enclosed within <think> </think> tags.
16
- Once you've finished your reasoning, you should choose an admissible action for current step and present it within <action> </action> tags.
@@ -1,4 +0,0 @@
1
- """ALFWorld Reflect stage.
2
-
3
- Prompts are now loaded from .md files by the base adapter.
4
- """