oh-my-opencode 4.19.3 → 5.0.0-beta.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (347) hide show
  1. package/.agents/command/get-unpublished-changes.md +2 -0
  2. package/.agents/command/omomomo.md +1 -1
  3. package/.agents/command/publish.md +7 -0
  4. package/.agents/skills/get-unpublished-changes/SKILL.md +2 -0
  5. package/.agents/skills/hyperplan/SKILL.md +3 -3
  6. package/.agents/skills/omomomo/SKILL.md +1 -1
  7. package/.agents/skills/publish/SKILL.md +7 -0
  8. package/.opencode/command/get-unpublished-changes.md +2 -0
  9. package/.opencode/command/omomomo.md +1 -1
  10. package/.opencode/command/publish.md +7 -0
  11. package/.opencode/skills/hyperplan/SKILL.md +3 -3
  12. package/README.ja.md +1 -1
  13. package/README.ko.md +1 -1
  14. package/README.md +6 -5
  15. package/README.ru.md +1 -1
  16. package/README.zh-cn.md +1 -1
  17. package/bin/oh-my-opencode.js +14 -1
  18. package/bin/oh-my-opencode.test.ts +21 -0
  19. package/dist/agents/sisyphus-junior/agent.d.ts +1 -1
  20. package/dist/cli/doctor/checks/deprecated-reasoning-keys.d.ts +2 -0
  21. package/dist/cli/index.js +2925 -1919
  22. package/dist/cli-node/index.js +2925 -1919
  23. package/dist/config/schema/agent-overrides.d.ts +1343 -15
  24. package/dist/config/schema/categories.d.ts +132 -0
  25. package/dist/config/schema/fallback-models.d.ts +50 -0
  26. package/dist/config/schema/oh-my-opencode-config.d.ts +1322 -11
  27. package/dist/config-migration/index.d.ts +1 -0
  28. package/dist/config-migration/migration-plans.d.ts +1 -0
  29. package/dist/config-migration/reasoning-unification.d.ts +3 -0
  30. package/dist/features/monitor/batcher.d.ts +3 -1
  31. package/dist/features/monitor/manager-internals.d.ts +1 -0
  32. package/dist/features/monitor/output-injector-types.d.ts +2 -0
  33. package/dist/features/monitor/output-injector.d.ts +6 -0
  34. package/dist/features/team-mode/tools/lifecycle-test-fixture.d.ts +2 -0
  35. package/dist/hooks/codegraph-bootstrap/command-runner.d.ts +1 -0
  36. package/dist/hooks/model-fallback/next-fallback.d.ts +1 -0
  37. package/dist/hooks/runtime-fallback/constants.d.ts +1 -1
  38. package/dist/hooks/todo-continuation-enforcer/types.d.ts +1 -0
  39. package/dist/hooks/todo-continuation-enforcer/unrecoverable-request-error.d.ts +9 -0
  40. package/dist/hooks/tool-pair-validator/hook.test-support.d.ts +29 -0
  41. package/dist/hooks/tool-pair-validator/tool-part-ids.d.ts +14 -5
  42. package/dist/hooks/tool-pair-validator/tool-result-repair.d.ts +4 -3
  43. package/dist/hooks/tool-pair-validator/types.d.ts +5 -22
  44. package/dist/index.js +4864 -4011
  45. package/dist/mcp/lsp.d.ts +1 -0
  46. package/dist/oh-my-opencode.schema.json +3221 -222
  47. package/dist/plugin-handlers/prometheus-agent-config-builder.d.ts +1 -0
  48. package/dist/shared/agent-variant.d.ts +11 -0
  49. package/dist/shared/session-prompt-params-helpers.d.ts +6 -1
  50. package/dist/shared/tmux/constants.d.ts +1 -1
  51. package/dist/skills/ast-grep/SOURCE +1 -1
  52. package/dist/skills/ast-grep/install.ps1 +2 -2
  53. package/dist/skills/ast-grep/install.sh +1 -1
  54. package/dist/skills/ast-grep/references/install.md +2 -2
  55. package/dist/skills/ast-grep/tests/smoke.sh +1 -1
  56. package/dist/skills/coding-agent-sessions/SKILL.md +4 -3
  57. package/dist/skills/coding-agent-sessions/references/all-platforms.md +3 -1
  58. package/dist/skills/coding-agent-sessions/scripts/agent_sessions/aside_scanner.py +140 -0
  59. package/dist/skills/coding-agent-sessions/scripts/agent_sessions/scanners.py +3 -0
  60. package/dist/skills/data-scientist/SKILL.md +243 -0
  61. package/dist/skills/data-scientist/references/common-scenarios.md +176 -0
  62. package/dist/skills/data-scientist/references/execution-templates.md +197 -0
  63. package/dist/skills/data-scientist/references/integration-patterns.md +153 -0
  64. package/dist/skills/data-scientist/references/performance-benchmarks.md +37 -0
  65. package/dist/skills/data-scientist/references/uv-setup.md +78 -0
  66. package/dist/skills/data-scientist/scripts/quick-query.py +111 -0
  67. package/dist/skills/data-scientist/scripts/setup-uv.ps1 +53 -0
  68. package/dist/skills/data-scientist/scripts/setup-uv.sh +60 -0
  69. package/dist/skills/debugging/SKILL.md +1 -1
  70. package/dist/skills/programming/SKILL.md +1 -2
  71. package/dist/skills/start-work/SKILL.md +54 -9
  72. package/dist/skills/ultimate-browsing/SKILL.md +2 -2
  73. package/dist/skills/ultimate-browsing/engine/__main__.py +8 -1
  74. package/dist/skills/ultimate-browsing/engine/bias_check.py +11 -0
  75. package/dist/skills/ultimate-browsing/engine/fetch_chain.py +90 -52
  76. package/dist/skills/ultimate-browsing/engine/result_schema.py +10 -1
  77. package/dist/skills/ultimate-browsing/engine/surrogate.py +214 -0
  78. package/dist/skills/ultimate-browsing/engine/surrogates.yaml +60 -0
  79. package/dist/skills/ultimate-browsing/engine/tests/fixtures/amp_redirect_stub.html +7 -0
  80. package/dist/skills/ultimate-browsing/engine/tests/fixtures/search_interstitial.html +19 -0
  81. package/dist/skills/ultimate-browsing/engine/tests/fixtures/wayback_available.json +1 -0
  82. package/dist/skills/ultimate-browsing/engine/tests/fixtures/wayback_snapshot.html +1128 -0
  83. package/dist/skills/ultimate-browsing/engine/tests/test_surrogate.py +252 -0
  84. package/dist/skills/ultimate-browsing/engine/tests/test_surrogate_validators.py +78 -0
  85. package/dist/skills/ultimate-browsing/engine/validators.py +46 -0
  86. package/dist/skills/ultimate-browsing/engine/waf_detector.py +1 -1
  87. package/dist/skills/ultimate-browsing/engine/waf_profiles.yaml +10 -5
  88. package/dist/skills/ultimate-browsing/references/agent-reach/social.md +1 -1
  89. package/dist/skills/ultimate-browsing/references/chrome-stealth.md +13 -11
  90. package/dist/skills/ultimate-browsing/references/insane-search/README.md +4 -4
  91. package/dist/skills/ultimate-browsing/references/insane-search/cache-archive.md +51 -50
  92. package/dist/skills/ultimate-browsing/references/insane-search/fallback.md +1 -1
  93. package/dist/skills/ultimate-browsing/references/insane-search/jina.md +8 -2
  94. package/dist/skills/ultimate-browsing/references/insane-search/naver.md +1 -1
  95. package/dist/skills/ultimate-browsing/references/insane-search/twitter.md +3 -3
  96. package/dist/skills/ulw-plan/SKILL.md +2 -2
  97. package/dist/skills/ulw-plan/references/full-workflow.md +3 -3
  98. package/dist/skills/ulw-plan/scripts/scaffold-plan.mjs +1 -1
  99. package/dist/skills/ulw-research/SKILL.md +128 -12
  100. package/dist/tools/delegate-task/builtin-categories.d.ts +1 -0
  101. package/dist/tools/delegate-task/builtin-category-definition.d.ts +1 -0
  102. package/dist/tools/delegate-task/constants.d.ts +1 -1
  103. package/dist/tui.js +1271 -1108
  104. package/docs/reference/web-terminal-visual-qa.md +1 -1
  105. package/package.json +27 -19
  106. package/packages/lsp-core/src/lsp/client-diagnostics-freshness.integration.test.ts +2 -2
  107. package/packages/lsp-core/src/lsp/connection.ts +1 -1
  108. package/packages/lsp-daemon/dist/cli.js +27 -13
  109. package/packages/lsp-daemon/dist/client.js +49 -35
  110. package/packages/lsp-daemon/dist/ensure-daemon.d.ts +1 -0
  111. package/packages/lsp-daemon/dist/ensure-daemon.js +18 -5
  112. package/packages/lsp-daemon/dist/index.js +34 -20
  113. package/packages/lsp-tools-mcp/dist/cli.js +1 -1
  114. package/packages/lsp-tools-mcp/dist/lsp/manager.js +1 -1
  115. package/packages/lsp-tools-mcp/dist/mcp.js +1 -1
  116. package/packages/lsp-tools-mcp/dist/tools.js +1 -1
  117. package/packages/omo-codex/plugin/.codex-plugin/plugin.json +1 -1
  118. package/packages/omo-codex/plugin/components/bootstrap/dist/cli.js +289 -95
  119. package/packages/omo-codex/plugin/components/bootstrap/hooks/hooks.json +1 -1
  120. package/packages/omo-codex/plugin/components/bootstrap/package.json +1 -1
  121. package/packages/omo-codex/plugin/components/bootstrap/src/setup.ts +7 -7
  122. package/packages/omo-codex/plugin/components/bootstrap/src/worker.ts +3 -0
  123. package/packages/omo-codex/plugin/components/codegraph/dist/cli.js +385 -64
  124. package/packages/omo-codex/plugin/components/codegraph/dist/serve.js +336 -47
  125. package/packages/omo-codex/plugin/components/codegraph/package.json +1 -1
  126. package/packages/omo-codex/plugin/components/codegraph/test/hook.test.ts +6 -0
  127. package/packages/omo-codex/plugin/components/comment-checker/hooks/hooks.json +1 -1
  128. package/packages/omo-codex/plugin/components/comment-checker/package.json +1 -1
  129. package/packages/omo-codex/plugin/components/git-bash/hooks/hooks.json +2 -2
  130. package/packages/omo-codex/plugin/components/git-bash/package.json +1 -1
  131. package/packages/omo-codex/plugin/components/lazycodex-executor-verify/hooks/hooks.json +1 -1
  132. package/packages/omo-codex/plugin/components/lazycodex-executor-verify/package.json +1 -1
  133. package/packages/omo-codex/plugin/components/lsp/dist/.omo-runtime-manifest.json +3 -3
  134. package/packages/omo-codex/plugin/components/lsp/dist/cli.js +56 -42
  135. package/packages/omo-codex/plugin/components/lsp/hooks/hooks.json +2 -2
  136. package/packages/omo-codex/plugin/components/lsp/package.json +1 -1
  137. package/packages/omo-codex/plugin/components/rules/dist/cli.js +1 -1
  138. package/packages/omo-codex/plugin/components/rules/hooks/hooks.json +4 -4
  139. package/packages/omo-codex/plugin/components/rules/package.json +1 -1
  140. package/packages/omo-codex/plugin/components/rules/src/post-compact-budget.ts +1 -1
  141. package/packages/omo-codex/plugin/components/start-work-continuation/AGENTS.md +1 -0
  142. package/packages/omo-codex/plugin/components/start-work-continuation/README.md +5 -1
  143. package/packages/omo-codex/plugin/components/start-work-continuation/directive.md +2 -1
  144. package/packages/omo-codex/plugin/components/start-work-continuation/dist/cli.js +18 -0
  145. package/packages/omo-codex/plugin/components/start-work-continuation/hooks/hooks.json +2 -2
  146. package/packages/omo-codex/plugin/components/start-work-continuation/package.json +1 -1
  147. package/packages/omo-codex/plugin/components/start-work-continuation/src/codex-hook.ts +21 -0
  148. package/packages/omo-codex/plugin/components/start-work-continuation/test/codex-hook.test.ts +105 -0
  149. package/packages/omo-codex/plugin/components/teammode/hooks/hooks.json +1 -1
  150. package/packages/omo-codex/plugin/components/teammode/package.json +1 -1
  151. package/packages/omo-codex/plugin/components/telemetry/hooks/hooks.json +1 -1
  152. package/packages/omo-codex/plugin/components/telemetry/package.json +1 -1
  153. package/packages/omo-codex/plugin/components/ultrawork/CHANGELOG.md +2 -0
  154. package/packages/omo-codex/plugin/components/ultrawork/agents/lazycodex-code-reviewer.toml +1 -1
  155. package/packages/omo-codex/plugin/components/ultrawork/agents/lazycodex-gate-reviewer.toml +1 -1
  156. package/packages/omo-codex/plugin/components/ultrawork/agents/lazycodex-qa-executor.toml +1 -1
  157. package/packages/omo-codex/plugin/components/ultrawork/agents/lazycodex-worker-high.toml +1 -1
  158. package/packages/omo-codex/plugin/components/ultrawork/agents/lazycodex-worker-low.toml +1 -1
  159. package/packages/omo-codex/plugin/components/ultrawork/agents/lazycodex-worker-medium.toml +1 -1
  160. package/packages/omo-codex/plugin/components/ultrawork/agents/plan.toml +2 -2
  161. package/packages/omo-codex/plugin/components/ultrawork/directive.md +3 -2
  162. package/packages/omo-codex/plugin/components/ultrawork/hooks/hooks.json +1 -1
  163. package/packages/omo-codex/plugin/components/ultrawork/package.json +1 -1
  164. package/packages/omo-codex/plugin/components/ultrawork/skills/ultrawork/SKILL.md +3 -2
  165. package/packages/omo-codex/plugin/components/ultrawork/skills/ulw-plan/SKILL.md +2 -2
  166. package/packages/omo-codex/plugin/components/ultrawork/skills/ulw-plan/references/full-workflow.md +3 -3
  167. package/packages/omo-codex/plugin/components/ultrawork/skills/ulw-plan/scripts/scaffold-plan.mjs +1 -1
  168. package/packages/omo-codex/plugin/components/ulw-loop/AGENTS.md +1 -1
  169. package/packages/omo-codex/plugin/components/ulw-loop/CHANGELOG.md +2 -0
  170. package/packages/omo-codex/plugin/components/ulw-loop/README.md +11 -11
  171. package/packages/omo-codex/plugin/components/ulw-loop/directive.md +3 -2
  172. package/packages/omo-codex/plugin/components/ulw-loop/dist/checkpoint-reconciliation.js +1 -1
  173. package/packages/omo-codex/plugin/components/ulw-loop/dist/cli-output.d.ts +1 -1
  174. package/packages/omo-codex/plugin/components/ulw-loop/dist/cli-output.js +9 -9
  175. package/packages/omo-codex/plugin/components/ulw-loop/dist/cli-steering.js +1 -1
  176. package/packages/omo-codex/plugin/components/ulw-loop/dist/cli.js +66 -66
  177. package/packages/omo-codex/plugin/components/ulw-loop/dist/codex-goal-instruction.js +4 -4
  178. package/packages/omo-codex/plugin/components/ulw-loop/dist/codex-hook.js +1 -1
  179. package/packages/omo-codex/plugin/components/ulw-loop/dist/plan-crud.js +1 -1
  180. package/packages/omo-codex/plugin/components/ulw-loop/dist/plan-io.js +1 -1
  181. package/packages/omo-codex/plugin/components/ulw-loop/dist/steering.js +1 -1
  182. package/packages/omo-codex/plugin/components/ulw-loop/dist/stop-resume-hook.js +2 -2
  183. package/packages/omo-codex/plugin/components/ulw-loop/hooks/hooks.json +4 -4
  184. package/packages/omo-codex/plugin/components/ulw-loop/package.json +1 -1
  185. package/packages/omo-codex/plugin/components/ulw-loop/skills/ulw-loop/SKILL.md +3 -3
  186. package/packages/omo-codex/plugin/components/ulw-loop/skills/ulw-loop/references/full-workflow.md +22 -25
  187. package/packages/omo-codex/plugin/components/ulw-loop/src/checkpoint-reconciliation.ts +1 -1
  188. package/packages/omo-codex/plugin/components/ulw-loop/src/cli-output.ts +9 -9
  189. package/packages/omo-codex/plugin/components/ulw-loop/src/cli-steering.ts +1 -1
  190. package/packages/omo-codex/plugin/components/ulw-loop/src/cli.ts +1 -1
  191. package/packages/omo-codex/plugin/components/ulw-loop/src/codex-goal-instruction.ts +4 -4
  192. package/packages/omo-codex/plugin/components/ulw-loop/src/codex-hook.ts +1 -1
  193. package/packages/omo-codex/plugin/components/ulw-loop/src/plan-crud.ts +1 -1
  194. package/packages/omo-codex/plugin/components/ulw-loop/src/plan-io.ts +1 -1
  195. package/packages/omo-codex/plugin/components/ulw-loop/src/steering.ts +1 -1
  196. package/packages/omo-codex/plugin/components/ulw-loop/src/stop-resume-hook.ts +2 -2
  197. package/packages/omo-codex/plugin/components/ulw-loop/test/cli-commands.test.ts +2 -2
  198. package/packages/omo-codex/plugin/components/ulw-loop/test/cli-entrypoint.test.ts +1 -1
  199. package/packages/omo-codex/plugin/components/ulw-loop/test/cli-helpers.test.ts +2 -2
  200. package/packages/omo-codex/plugin/components/ulw-loop/test/cli-steering-kind-guidance.test.ts +1 -1
  201. package/packages/omo-codex/plugin/components/ulw-loop/test/codex-hook.test.ts +2 -2
  202. package/packages/omo-codex/plugin/components/ulw-loop/test/fixtures/quality-gate-builder.ts +1 -1
  203. package/packages/omo-codex/plugin/components/ulw-loop/test/package-smoke.test.ts +5 -5
  204. package/packages/omo-codex/plugin/components/ulw-loop/test/plan-io.test.ts +1 -1
  205. package/packages/omo-codex/plugin/components/ulw-loop/test/quality-gate-roles.test.ts +1 -1
  206. package/packages/omo-codex/plugin/components/ulw-loop/test/steering.test.ts +1 -1
  207. package/packages/omo-codex/plugin/components/ulw-loop/test/stop-resume-hook.test.ts +1 -1
  208. package/packages/omo-codex/plugin/hooks/post-compact-resetting-git-bash-mcp-reminder.json +1 -1
  209. package/packages/omo-codex/plugin/hooks/post-compact-resetting-lsp-diagnostics-cache.json +1 -1
  210. package/packages/omo-codex/plugin/hooks/post-compact-resetting-project-rule-cache.json +1 -1
  211. package/packages/omo-codex/plugin/hooks/post-tool-use-checking-codegraph-init-guidance.json +1 -1
  212. package/packages/omo-codex/plugin/hooks/post-tool-use-checking-comments.json +1 -1
  213. package/packages/omo-codex/plugin/hooks/post-tool-use-checking-lsp-diagnostics.json +1 -1
  214. package/packages/omo-codex/plugin/hooks/post-tool-use-checking-thread-title-hygiene.json +1 -1
  215. package/packages/omo-codex/plugin/hooks/post-tool-use-matching-project-rules.json +1 -1
  216. package/packages/omo-codex/plugin/hooks/pre-tool-use-enforcing-unlimited-goal-budget.json +1 -1
  217. package/packages/omo-codex/plugin/hooks/pre-tool-use-guarding-ulw-loop-spawns.json +1 -1
  218. package/packages/omo-codex/plugin/hooks/pre-tool-use-recommending-git-bash-mcp.json +1 -1
  219. package/packages/omo-codex/plugin/hooks/session-start-checking-auto-update.json +1 -1
  220. package/packages/omo-codex/plugin/hooks/session-start-checking-bootstrap-provisioning.json +1 -1
  221. package/packages/omo-codex/plugin/hooks/session-start-checking-codegraph-bootstrap.json +1 -1
  222. package/packages/omo-codex/plugin/hooks/session-start-loading-project-rules.json +1 -1
  223. package/packages/omo-codex/plugin/hooks/session-start-recording-session-telemetry.json +1 -1
  224. package/packages/omo-codex/plugin/hooks/stop-checking-start-work-continuation.json +1 -1
  225. package/packages/omo-codex/plugin/hooks/stop-checking-ulw-loop-resume.json +1 -1
  226. package/packages/omo-codex/plugin/hooks/subagent-stop-checking-start-work-continuation.json +1 -1
  227. package/packages/omo-codex/plugin/hooks/subagent-stop-verifying-lazycodex-executor-evidence.json +1 -1
  228. package/packages/omo-codex/plugin/hooks/user-prompt-submit-checking-ultrawork-trigger.json +1 -1
  229. package/packages/omo-codex/plugin/hooks/user-prompt-submit-checking-ulw-loop-steering.json +1 -1
  230. package/packages/omo-codex/plugin/hooks/user-prompt-submit-loading-project-rules.json +1 -1
  231. package/packages/omo-codex/plugin/package-lock.json +13 -13
  232. package/packages/omo-codex/plugin/package.json +1 -1
  233. package/packages/omo-codex/plugin/scripts/sync-skills.mjs +8 -2
  234. package/packages/omo-codex/plugin/skills/ast-grep/SOURCE +1 -1
  235. package/packages/omo-codex/plugin/skills/ast-grep/install.ps1 +2 -2
  236. package/packages/omo-codex/plugin/skills/ast-grep/install.sh +1 -1
  237. package/packages/omo-codex/plugin/skills/ast-grep/references/install.md +2 -2
  238. package/packages/omo-codex/plugin/skills/ast-grep/tests/smoke.sh +1 -1
  239. package/packages/omo-codex/plugin/skills/coding-agent-sessions/SKILL.md +4 -3
  240. package/packages/omo-codex/plugin/skills/coding-agent-sessions/references/all-platforms.md +3 -1
  241. package/packages/omo-codex/plugin/skills/coding-agent-sessions/scripts/agent_sessions/aside_scanner.py +140 -0
  242. package/packages/omo-codex/plugin/skills/coding-agent-sessions/scripts/agent_sessions/scanners.py +3 -0
  243. package/packages/omo-codex/plugin/skills/data-scientist/SKILL.md +243 -0
  244. package/packages/omo-codex/plugin/skills/data-scientist/agents/openai.yaml +2 -0
  245. package/packages/omo-codex/plugin/skills/data-scientist/references/common-scenarios.md +176 -0
  246. package/packages/omo-codex/plugin/skills/data-scientist/references/execution-templates.md +197 -0
  247. package/packages/omo-codex/plugin/skills/data-scientist/references/integration-patterns.md +153 -0
  248. package/packages/omo-codex/plugin/skills/data-scientist/references/performance-benchmarks.md +37 -0
  249. package/packages/omo-codex/plugin/skills/data-scientist/references/uv-setup.md +78 -0
  250. package/packages/omo-codex/plugin/skills/data-scientist/scripts/quick-query.py +111 -0
  251. package/packages/omo-codex/plugin/skills/data-scientist/scripts/setup-uv.ps1 +53 -0
  252. package/packages/omo-codex/plugin/skills/data-scientist/scripts/setup-uv.sh +60 -0
  253. package/packages/omo-codex/plugin/skills/debugging/SKILL.md +1 -1
  254. package/packages/omo-codex/plugin/skills/programming/SKILL.md +1 -2
  255. package/packages/omo-codex/plugin/skills/start-work/SKILL.md +54 -9
  256. package/packages/omo-codex/plugin/skills/ultimate-browsing/SKILL.md +2 -2
  257. package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/__main__.py +8 -1
  258. package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/bias_check.py +11 -0
  259. package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/fetch_chain.py +90 -52
  260. package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/result_schema.py +10 -1
  261. package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/surrogate.py +214 -0
  262. package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/surrogates.yaml +60 -0
  263. package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/tests/fixtures/amp_redirect_stub.html +7 -0
  264. package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/tests/fixtures/search_interstitial.html +19 -0
  265. package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/tests/fixtures/wayback_available.json +1 -0
  266. package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/tests/fixtures/wayback_snapshot.html +1128 -0
  267. package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/tests/test_surrogate.py +252 -0
  268. package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/tests/test_surrogate_validators.py +78 -0
  269. package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/validators.py +46 -0
  270. package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/waf_detector.py +1 -1
  271. package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/waf_profiles.yaml +10 -5
  272. package/packages/omo-codex/plugin/skills/ultimate-browsing/references/agent-reach/social.md +1 -1
  273. package/packages/omo-codex/plugin/skills/ultimate-browsing/references/chrome-stealth.md +13 -11
  274. package/packages/omo-codex/plugin/skills/ultimate-browsing/references/insane-search/README.md +4 -4
  275. package/packages/omo-codex/plugin/skills/ultimate-browsing/references/insane-search/cache-archive.md +51 -50
  276. package/packages/omo-codex/plugin/skills/ultimate-browsing/references/insane-search/fallback.md +1 -1
  277. package/packages/omo-codex/plugin/skills/ultimate-browsing/references/insane-search/jina.md +8 -2
  278. package/packages/omo-codex/plugin/skills/ultimate-browsing/references/insane-search/naver.md +1 -1
  279. package/packages/omo-codex/plugin/skills/ultimate-browsing/references/insane-search/twitter.md +3 -3
  280. package/packages/omo-codex/plugin/skills/ultrawork/SKILL.md +3 -2
  281. package/packages/omo-codex/plugin/skills/ulw-loop/SKILL.md +3 -3
  282. package/packages/omo-codex/plugin/skills/ulw-loop/references/full-workflow.md +22 -25
  283. package/packages/omo-codex/plugin/skills/ulw-plan/SKILL.md +2 -2
  284. package/packages/omo-codex/plugin/skills/ulw-plan/references/full-workflow.md +3 -3
  285. package/packages/omo-codex/plugin/skills/ulw-plan/scripts/scaffold-plan.mjs +1 -1
  286. package/packages/omo-codex/plugin/skills/ulw-research/SKILL.md +127 -12
  287. package/packages/omo-codex/plugin/test/bootstrap-binlinks.test.mjs +12 -12
  288. package/packages/omo-codex/plugin/test/bootstrap-orchestration.test.mjs +36 -4
  289. package/packages/omo-codex/plugin/test/sync-skills-orchestration.test.mjs +1 -1
  290. package/packages/omo-codex/plugin/test/sync-skills-test-support.mjs +9 -2
  291. package/packages/omo-codex/plugin/test/sync-skills.test.mjs +12 -0
  292. package/packages/omo-codex/scripts/install-bin-links.test.mjs +56 -2
  293. package/packages/omo-codex/scripts/install-delegated-command.test.mjs +6 -6
  294. package/packages/omo-codex/scripts/install-dist/install-local.mjs +138 -65
  295. package/packages/omo-codex/scripts/install-local-entrypoint.test.mjs +4 -4
  296. package/packages/omo-codex/scripts/install-local.test.mjs +5 -2
  297. package/packages/shared-skills/index.mjs +19 -1
  298. package/packages/shared-skills/skills/ast-grep/SOURCE +1 -1
  299. package/packages/shared-skills/skills/ast-grep/install.ps1 +2 -2
  300. package/packages/shared-skills/skills/ast-grep/install.sh +1 -1
  301. package/packages/shared-skills/skills/ast-grep/references/install.md +2 -2
  302. package/packages/shared-skills/skills/ast-grep/tests/smoke.sh +1 -1
  303. package/packages/shared-skills/skills/coding-agent-sessions/SKILL.md +4 -3
  304. package/packages/shared-skills/skills/coding-agent-sessions/references/all-platforms.md +3 -1
  305. package/packages/shared-skills/skills/coding-agent-sessions/scripts/agent_sessions/aside_scanner.py +140 -0
  306. package/packages/shared-skills/skills/coding-agent-sessions/scripts/agent_sessions/scanners.py +3 -0
  307. package/packages/shared-skills/skills/data-scientist/SKILL.md +243 -0
  308. package/packages/shared-skills/skills/data-scientist/references/common-scenarios.md +176 -0
  309. package/packages/shared-skills/skills/data-scientist/references/execution-templates.md +197 -0
  310. package/packages/shared-skills/skills/data-scientist/references/integration-patterns.md +153 -0
  311. package/packages/shared-skills/skills/data-scientist/references/performance-benchmarks.md +37 -0
  312. package/packages/shared-skills/skills/data-scientist/references/uv-setup.md +78 -0
  313. package/packages/shared-skills/skills/data-scientist/scripts/quick-query.py +111 -0
  314. package/packages/shared-skills/skills/data-scientist/scripts/setup-uv.ps1 +53 -0
  315. package/packages/shared-skills/skills/data-scientist/scripts/setup-uv.sh +60 -0
  316. package/packages/shared-skills/skills/debugging/SKILL.md +1 -1
  317. package/packages/shared-skills/skills/programming/SKILL.md +1 -2
  318. package/packages/shared-skills/skills/start-work/SKILL.md +54 -9
  319. package/packages/shared-skills/skills/ultimate-browsing/SKILL.md +2 -2
  320. package/packages/shared-skills/skills/ultimate-browsing/engine/__main__.py +8 -1
  321. package/packages/shared-skills/skills/ultimate-browsing/engine/bias_check.py +11 -0
  322. package/packages/shared-skills/skills/ultimate-browsing/engine/fetch_chain.py +90 -52
  323. package/packages/shared-skills/skills/ultimate-browsing/engine/result_schema.py +10 -1
  324. package/packages/shared-skills/skills/ultimate-browsing/engine/surrogate.py +214 -0
  325. package/packages/shared-skills/skills/ultimate-browsing/engine/surrogates.yaml +60 -0
  326. package/packages/shared-skills/skills/ultimate-browsing/engine/tests/fixtures/amp_redirect_stub.html +7 -0
  327. package/packages/shared-skills/skills/ultimate-browsing/engine/tests/fixtures/search_interstitial.html +19 -0
  328. package/packages/shared-skills/skills/ultimate-browsing/engine/tests/fixtures/wayback_available.json +1 -0
  329. package/packages/shared-skills/skills/ultimate-browsing/engine/tests/fixtures/wayback_snapshot.html +1128 -0
  330. package/packages/shared-skills/skills/ultimate-browsing/engine/tests/test_surrogate.py +252 -0
  331. package/packages/shared-skills/skills/ultimate-browsing/engine/tests/test_surrogate_validators.py +78 -0
  332. package/packages/shared-skills/skills/ultimate-browsing/engine/validators.py +46 -0
  333. package/packages/shared-skills/skills/ultimate-browsing/engine/waf_detector.py +1 -1
  334. package/packages/shared-skills/skills/ultimate-browsing/engine/waf_profiles.yaml +10 -5
  335. package/packages/shared-skills/skills/ultimate-browsing/references/agent-reach/social.md +1 -1
  336. package/packages/shared-skills/skills/ultimate-browsing/references/chrome-stealth.md +13 -11
  337. package/packages/shared-skills/skills/ultimate-browsing/references/insane-search/README.md +4 -4
  338. package/packages/shared-skills/skills/ultimate-browsing/references/insane-search/cache-archive.md +51 -50
  339. package/packages/shared-skills/skills/ultimate-browsing/references/insane-search/fallback.md +1 -1
  340. package/packages/shared-skills/skills/ultimate-browsing/references/insane-search/jina.md +8 -2
  341. package/packages/shared-skills/skills/ultimate-browsing/references/insane-search/naver.md +1 -1
  342. package/packages/shared-skills/skills/ultimate-browsing/references/insane-search/twitter.md +3 -3
  343. package/packages/shared-skills/skills/ulw-plan/SKILL.md +2 -2
  344. package/packages/shared-skills/skills/ulw-plan/references/full-workflow.md +3 -3
  345. package/packages/shared-skills/skills/ulw-plan/scripts/scaffold-plan.mjs +1 -1
  346. package/packages/shared-skills/skills/ulw-research/SKILL.md +128 -12
  347. package/postinstall.mjs +6 -0
@@ -0,0 +1,252 @@
1
+ """Surrogate registry/fetch/ordering tests; registry entries and responses are faked, never live network."""
2
+ from __future__ import annotations
3
+
4
+ import json
5
+ import os
6
+ import sys
7
+ import unittest
8
+ from pathlib import Path
9
+ from unittest.mock import patch
10
+
11
+ sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
12
+
13
+ from engine.result_schema import Attempt # noqa: E402
14
+ from engine.validators import Verdict # noqa: E402
15
+
16
+ FIXTURES = Path(__file__).resolve().parent / "fixtures"
17
+ TARGET = "https://en.wikipedia.org/wiki/Web_archiving" # NOTE-BIAS-OK — fixture URL from a third-party surrogate service, not a target site
18
+
19
+
20
+ def _fixture(name: str) -> str:
21
+ return (FIXTURES / name).read_text(encoding="utf-8", errors="replace")
22
+
23
+
24
+ class _Resp:
25
+ def __init__(self, text: str = "", status: int = 200, url: str = "", payload: dict | None = None):
26
+ self.text = text
27
+ self.status_code = status
28
+ self.url = url
29
+ self.cookies = {}
30
+ self.headers = {}
31
+ self._payload = payload
32
+
33
+ def json(self):
34
+ if self._payload is None:
35
+ raise ValueError("no json payload")
36
+ return self._payload
37
+
38
+
39
+ def _fail_attempt(phase: str) -> Attempt:
40
+ return Attempt(
41
+ phase=phase,
42
+ executor="curl_cffi",
43
+ url=TARGET,
44
+ url_transform="original",
45
+ impersonate="safari",
46
+ referer="self_root",
47
+ verdict=Verdict.CHALLENGE.value,
48
+ )
49
+
50
+
51
+ class SurrogateRegistry(unittest.TestCase):
52
+ def test_default_registry_contains_wayback(self) -> None:
53
+ from engine import surrogate
54
+
55
+ reg = surrogate.load_surrogates()
56
+ self.assertIn("wayback", reg)
57
+ self.assertEqual(reg["wayback"]["kind"], "archive")
58
+ self.assertEqual(reg["wayback"]["trust"], "archive")
59
+
60
+ def test_stale_entries_detected(self) -> None:
61
+ from engine import surrogate
62
+
63
+ self.assertTrue(surrogate.is_stale({"last_verified": "2025-01-01"}, max_age_days=90))
64
+ self.assertFalse(surrogate.is_stale({"last_verified": "2999-01-01"}, max_age_days=90))
65
+
66
+
67
+ class SurrogateFetch(unittest.TestCase):
68
+ def test_wayback_success_sets_snapshot_provenance(self) -> None:
69
+ from engine import surrogate
70
+
71
+ discovery = json.loads(_fixture("wayback_available.json"))
72
+
73
+ def fake_http(u: str, **kw):
74
+ if "wayback/available" in u:
75
+ return _Resp(json.dumps(discovery), 200, u, payload=discovery)
76
+ return _Resp(_fixture("wayback_snapshot.html"), 200, u)
77
+
78
+ with patch.object(surrogate, "_http_get", side_effect=fake_http):
79
+ atts, meta, content = surrogate.run_surrogate(TARGET, registry=surrogate.load_surrogates(), timeout=5)
80
+
81
+ att = atts[-1]
82
+ self.assertEqual(att.phase, "surrogate")
83
+ self.assertEqual(att.executor, "surrogate_wayback")
84
+ self.assertEqual(att.verdict, Verdict.WEAK_OK.value)
85
+ self.assertEqual(meta["provenance"], "snapshot")
86
+ self.assertTrue(meta["snapshot_timestamp"])
87
+ self.assertEqual(meta["trust"], "archive")
88
+ self.assertGreater(len(content), 3000)
89
+
90
+ def test_stub_snapshot_is_not_accepted(self) -> None:
91
+ from engine import surrogate
92
+
93
+ disc = {"archived_snapshots": {"closest": {"available": True, "url": "https://web.archive.org/web/2024/x", "timestamp": "20240101000000"}}} # NOTE-BIAS-OK — fixture URL from a third-party surrogate service, not a target site
94
+
95
+ def fake_http(u: str, **kw):
96
+ if "wayback/available" in u:
97
+ return _Resp(json.dumps(disc), 200, u, payload=disc)
98
+ return _Resp(_fixture("amp_redirect_stub.html"), 200, u)
99
+
100
+ with patch.object(surrogate, "_http_get", side_effect=fake_http):
101
+ atts, _, _ = surrogate.run_surrogate(TARGET, registry={"wayback": surrogate.DEFAULT_SURROGATES["wayback"]}, timeout=5)
102
+ att = atts[-1]
103
+ self.assertNotIn(att.verdict, (Verdict.STRONG_OK.value, Verdict.WEAK_OK.value))
104
+
105
+ def test_proxy_skips_without_allow_flag(self) -> None:
106
+ from engine import surrogate
107
+
108
+ reg = {"anon_relay": {"kind": "proxy", "trust": "untrusted", "enabled": False,
109
+ "last_verified": "2999-01-01", "fetch": "https://relay.invalid/{target}"}} # NOTE-BIAS-OK — fixture URL from a third-party surrogate service, not a target site
110
+ called: list[str] = []
111
+
112
+ with patch.object(surrogate, "_http_get", side_effect=lambda u, **kw: (called.append(u), _Resp("<html>" + "y" * 4000 + "</html>", 200, u))[1]):
113
+ atts, _, _ = surrogate.run_surrogate(TARGET, registry=reg, allow_proxy=False, timeout=5)
114
+ self.assertEqual(len(called), 0)
115
+
116
+ with patch.object(surrogate, "_http_get", side_effect=lambda u, **kw: (called.append(u), _Resp("<html>" + "y" * 4000 + "</html>", 200, u))[1]):
117
+ atts, meta, _ = surrogate.run_surrogate(TARGET, registry=reg, allow_proxy=True, timeout=5)
118
+ self.assertEqual(len(called), 1)
119
+ att = atts[-1]
120
+ self.assertEqual(att.executor, "surrogate_anon_relay")
121
+ self.assertEqual(meta["trust"], "untrusted")
122
+
123
+ def test_proxy_never_carries_credentials(self) -> None:
124
+ from engine import surrogate
125
+
126
+ seen: list[dict] = []
127
+
128
+ def fake_http(u: str, **kw):
129
+ seen.append(kw.get("headers", {}))
130
+ return _Resp("<html>" + "z" * 4000 + "</html>", 200, u)
131
+
132
+ reg = {"anon_relay": {"kind": "proxy", "trust": "untrusted", "enabled": False,
133
+ "last_verified": "2999-01-01", "fetch": "https://relay.invalid/{target}"}} # NOTE-BIAS-OK — fixture URL from a third-party surrogate service, not a target site
134
+ os.environ["OMOB_TEST_TOKEN"] = "secret"
135
+ try:
136
+ with patch.object(surrogate, "_http_get", side_effect=fake_http):
137
+ surrogate.run_surrogate(TARGET, registry=reg, allow_proxy=True, timeout=5)
138
+ finally:
139
+ os.environ.pop("OMOB_TEST_TOKEN", None)
140
+
141
+ self.assertEqual(len(seen), 1)
142
+ joined = json.dumps(seen[0]).lower()
143
+ self.assertNotIn("secret", joined)
144
+ self.assertNotIn("authorization", joined)
145
+ self.assertNotIn("cookie", joined)
146
+
147
+
148
+ class ChainOrdering(unittest.TestCase):
149
+ def _profile(self) -> dict:
150
+ return {
151
+ "tls_impersonate_candidates": [[]],
152
+ "referer_strategies": [],
153
+ "url_transform_order": ["original"],
154
+ "fallback_when_challenge": ["surrogate_wayback", "playwright_real_chrome"],
155
+ }
156
+
157
+ def test_surrogate_success_short_circuits_playwright(self) -> None:
158
+ from engine.fetch_chain import fetch
159
+ from engine import surrogate
160
+ from engine import executor as ex
161
+
162
+ def fake_surrogate(*a, **kw):
163
+ att = Attempt(
164
+ phase="surrogate",
165
+ executor="surrogate_wayback",
166
+ url=TARGET,
167
+ url_transform="original",
168
+ impersonate=None,
169
+ referer="",
170
+ verdict=Verdict.WEAK_OK.value,
171
+ body_size=5000,
172
+ )
173
+ return [att], {"provenance": "snapshot", "snapshot_timestamp": "20240101000000", "trust": "archive"}, "<html>content</html>"
174
+
175
+ def fail_playwright(*a, **kw):
176
+ raise AssertionError("playwright must not run after surrogate success")
177
+
178
+ with patch("engine.fetch_chain._load_profiles", return_value={}), \
179
+ patch("engine.fetch_chain.last_load_error", return_value=None), \
180
+ patch("engine.fetch_chain.run_attempt", side_effect=lambda *a, **kw: (_fail_attempt(str(kw.get("phase", "grid"))), None)), \
181
+ patch("engine.fetch_chain.detect", return_value=[]), \
182
+ patch("engine.fetch_chain.load_profile", side_effect=lambda *a, **kw: self._profile()), \
183
+ patch.object(surrogate, "run_surrogate", side_effect=fake_surrogate), \
184
+ patch.object(ex, "run_playwright_fallback", side_effect=fail_playwright):
185
+ result = fetch(TARGET, enable_playwright=True, max_attempts=1)
186
+
187
+ self.assertTrue(result.ok)
188
+ self.assertEqual(result.provenance, "snapshot")
189
+ self.assertEqual(result.snapshot_timestamp, "20240101000000")
190
+
191
+ def test_surrogate_still_runs_when_playwright_disabled(self) -> None:
192
+ from engine.fetch_chain import fetch
193
+ from engine import surrogate
194
+
195
+ calls: list[str] = []
196
+
197
+ def fake_surrogate(*a, **kw):
198
+ calls.append("surrogate")
199
+ att = Attempt(phase="surrogate", executor="surrogate_wayback", url=TARGET,
200
+ url_transform="original", impersonate=None, referer="",
201
+ verdict=Verdict.WEAK_OK.value, body_size=5000)
202
+ return [att], {"provenance": "snapshot", "snapshot_timestamp": "20240101000000", "trust": "archive"}, "<html>snap</html>"
203
+
204
+ with patch("engine.fetch_chain._load_profiles", return_value={}), \
205
+ patch("engine.fetch_chain.last_load_error", return_value=None), \
206
+ patch("engine.fetch_chain.run_attempt", side_effect=lambda *a, **kw: (_fail_attempt(str(kw.get("phase", "grid"))), None)), \
207
+ patch("engine.fetch_chain.detect", return_value=[]), \
208
+ patch("engine.fetch_chain.load_profile", side_effect=lambda *a, **kw: self._profile()), \
209
+ patch.object(surrogate, "run_surrogate", side_effect=fake_surrogate):
210
+ result = fetch(TARGET, enable_playwright=False, max_attempts=1)
211
+
212
+ self.assertEqual(calls, ["surrogate"])
213
+ self.assertTrue(result.ok)
214
+ self.assertEqual(result.provenance, "snapshot")
215
+
216
+ def test_playwright_runs_when_surrogate_fails(self) -> None:
217
+ from engine.fetch_chain import fetch
218
+ from engine import surrogate
219
+ from engine import executor as ex
220
+
221
+ calls: list[str] = []
222
+
223
+ def fake_surrogate(*a, **kw):
224
+ calls.append("surrogate")
225
+ att = Attempt(phase="surrogate", executor="surrogate_wayback", url=TARGET,
226
+ url_transform="original", impersonate=None, referer="",
227
+ verdict=Verdict.UNKNOWN.value)
228
+ return [att], {"provenance": "live", "snapshot_timestamp": None, "trust": "origin"}, ""
229
+
230
+ def fake_playwright(*a, **kw):
231
+ calls.append("playwright")
232
+ att = Attempt(phase="fallback", executor="playwright_real_chrome", url=TARGET,
233
+ url_transform="original", impersonate=None, referer="",
234
+ verdict=Verdict.WEAK_OK.value, body_size=5000)
235
+ return att, "<html>chrome content</html>"
236
+
237
+ with patch("engine.fetch_chain._load_profiles", return_value={}), \
238
+ patch("engine.fetch_chain.last_load_error", return_value=None), \
239
+ patch("engine.fetch_chain.run_attempt", side_effect=lambda *a, **kw: (_fail_attempt(str(kw.get("phase", "grid"))), None)), \
240
+ patch("engine.fetch_chain.detect", return_value=[]), \
241
+ patch("engine.fetch_chain.load_profile", side_effect=lambda *a, **kw: self._profile()), \
242
+ patch.object(surrogate, "run_surrogate", side_effect=fake_surrogate), \
243
+ patch.object(ex, "run_playwright_fallback", side_effect=fake_playwright):
244
+ result = fetch(TARGET, enable_playwright=True, max_attempts=1)
245
+
246
+ self.assertTrue(result.ok)
247
+ self.assertEqual(result.provenance, "live")
248
+ self.assertEqual(calls, ["surrogate", "playwright"])
249
+
250
+
251
+ if __name__ == "__main__":
252
+ unittest.main()
@@ -0,0 +1,78 @@
1
+ """Surrogate-tier validation tests; fixtures only, never live network, so the suite cannot flake."""
2
+ from __future__ import annotations
3
+
4
+ import sys
5
+ import unittest
6
+ from pathlib import Path
7
+
8
+ sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
9
+
10
+ from engine.result_schema import FetchResult # noqa: E402
11
+ from engine.validators import Verdict, validate # noqa: E402
12
+
13
+ FIXTURES = Path(__file__).resolve().parent / "fixtures"
14
+
15
+
16
+ def _fixture(name: str) -> str:
17
+ return (FIXTURES / name).read_text(encoding="utf-8", errors="replace")
18
+
19
+
20
+ class _Resp:
21
+ def __init__(self, text: str = "", status: int = 200, url: str = ""):
22
+ self.text = text
23
+ self.status_code = status
24
+ self.url = url
25
+ self.cookies = {}
26
+ self.headers = {}
27
+
28
+
29
+ class RedirectStubRejection(unittest.TestCase):
30
+ def test_amp_redirect_stub_is_not_ok(self) -> None:
31
+ body = _fixture("amp_redirect_stub.html")
32
+ self.assertLess(len(body), 1000)
33
+ resp = _Resp(body, 200, "https://en-wikipedia-org.cdn.ampproject.org/c/s/x") # NOTE-BIAS-OK — fixture URL from a third-party surrogate service, not a target site
34
+ vr = validate(resp, target_url="https://en.wikipedia.org/wiki/Web_archiving") # NOTE-BIAS-OK — fixture URL from a third-party surrogate service, not a target site
35
+ self.assertIn(vr.verdict, (Verdict.CHALLENGE, Verdict.UNKNOWN))
36
+ self.assertFalse(vr.ok)
37
+
38
+ def test_real_snapshot_still_passes(self) -> None:
39
+ resp = _Resp(_fixture("wayback_snapshot.html"), 200, "https://web.archive.org/web/2024/x") # NOTE-BIAS-OK — fixture URL from a third-party surrogate service, not a target site
40
+ vr = validate(resp, target_url="https://en.wikipedia.org/wiki/Web_archiving") # NOTE-BIAS-OK — fixture URL from a third-party surrogate service, not a target site
41
+ self.assertTrue(vr.ok)
42
+
43
+ def test_plain_page_still_weak_ok(self) -> None:
44
+ resp = _Resp("<html><body>" + ("x" * 4000) + "</body></html>", 200, "https://example.com/")
45
+ vr = validate(resp, target_url="https://example.com/")
46
+ self.assertTrue(vr.ok)
47
+
48
+
49
+ class InterstitialRejection(unittest.TestCase):
50
+ def test_search_engine_interstitial_is_not_ok(self) -> None:
51
+ body = _fixture("search_interstitial.html")
52
+ resp = _Resp(body, 200, "https://webcache.googleusercontent.com/search?q=cache:x") # NOTE-BIAS-OK — fixture URL from a third-party surrogate service, not a target site
53
+ vr = validate(resp, target_url="https://en.wikipedia.org/wiki/Web_archiving") # NOTE-BIAS-OK — fixture URL from a third-party surrogate service, not a target site
54
+ self.assertIn(vr.verdict, (Verdict.CHALLENGE, Verdict.UNKNOWN))
55
+ self.assertFalse(vr.ok)
56
+
57
+
58
+ class ResultSchemaProvenance(unittest.TestCase):
59
+ def test_defaults_are_live_origin(self) -> None:
60
+ r = FetchResult(ok=True)
61
+ self.assertEqual(r.provenance, "live")
62
+ self.assertEqual(r.trust, "origin")
63
+ self.assertIsNone(r.snapshot_timestamp)
64
+ d = r.to_dict()
65
+ self.assertEqual(d["provenance"], "live")
66
+ self.assertEqual(d["trust"], "origin")
67
+ self.assertIn("snapshot_timestamp", d)
68
+
69
+ def test_snapshot_fields_roundtrip(self) -> None:
70
+ r = FetchResult(ok=True, provenance="snapshot", snapshot_timestamp="20241227132135", trust="archive")
71
+ d = r.to_dict()
72
+ self.assertEqual(d["provenance"], "snapshot")
73
+ self.assertEqual(d["snapshot_timestamp"], "20241227132135")
74
+ self.assertEqual(d["trust"], "archive")
75
+
76
+
77
+ if __name__ == "__main__":
78
+ unittest.main()
@@ -43,6 +43,41 @@ CHALLENGE_MARKERS: list[str] = [
43
43
  # that know their response type should pass success_selectors instead.
44
44
  SMALL_BODY_THRESHOLD = 3000
45
45
 
46
+ # Surrogate-route interstitial: a search-engine front page returned in place
47
+ # of the cached page (dead cache services fall back to their home page).
48
+ INTERSTITIAL_TITLE_MARKERS: list[str] = [
49
+ "<title>google search</title>",
50
+ ]
51
+
52
+
53
+ def is_interstitial_title(body_lower: str) -> bool:
54
+ return any(m in body_lower for m in INTERSTITIAL_TITLE_MARKERS)
55
+
56
+
57
+ def is_redirect_stub(text: str, *, target_url: Optional[str] = None) -> bool:
58
+ """True when the body only exists to send the reader back to `target_url`.
59
+
60
+ AMP-cache style stubs answer HTTP 200 with a few hundred bytes whose sole
61
+ content is a meta-refresh / JS redirect pointing at the origin we failed
62
+ to fetch — accepting one loops the caller straight back into the block.
63
+ """
64
+ if not text or len(text) >= SMALL_BODY_THRESHOLD:
65
+ return False
66
+ lowered = text.lower()
67
+ has_redirect = (
68
+ "http-equiv=\"refresh\"" in lowered
69
+ or "location.replace" in lowered
70
+ or "window.location" in lowered
71
+ )
72
+ if not has_redirect:
73
+ return False
74
+ if target_url is None:
75
+ return True
76
+ host_match = "".join(c for c in target_url.split("//")[-1].split("/")[0])
77
+ if not host_match:
78
+ return True
79
+ return host_match.lower() in lowered
80
+
46
81
 
47
82
  class Verdict(Enum):
48
83
  """Three-level classification (Codex suggestion — avoid binary)."""
@@ -115,6 +150,7 @@ def validate(
115
150
  success_selectors: Optional[list[str]] = None,
116
151
  known_bad_sizes: Optional[list[int]] = None,
117
152
  size_tolerance: int = 20,
153
+ target_url: Optional[str] = None,
118
154
  ) -> ValidationResult:
119
155
  """Validate a `curl_cffi` / `requests` response.
120
156
 
@@ -153,6 +189,16 @@ def validate(
153
189
  r.reasons.extend(f"marker:{m}" for m in markers[:3])
154
190
  return r
155
191
 
192
+ # --- Layer 1.5: surrogate-route dead ends (interstitial / redirect stub) --
193
+ if is_interstitial_title(lowered):
194
+ r.verdict = Verdict.CHALLENGE
195
+ r.reasons.append("interstitial_title")
196
+ return r
197
+ if is_redirect_stub(text, target_url=target_url):
198
+ r.verdict = Verdict.CHALLENGE
199
+ r.reasons.append("redirect_stub")
200
+ return r
201
+
156
202
  # --- Layer 2: size fingerprints (caller hint, tolerant match) ---
157
203
  # Fingerprint match is a strong negative signal — override even selectors.
158
204
  if known_bad_sizes:
@@ -39,7 +39,7 @@ _DEFAULT_PROFILES: dict = {
39
39
  ],
40
40
  "referer_strategies": ["self_root", "google_search", "none"],
41
41
  "url_transform_order": ["original", "mobile_subdomain"],
42
- "fallback_when_challenge": ["playwright_mcp", "playwright_real_chrome"],
42
+ "fallback_when_challenge": ["surrogate_wayback", "playwright_mcp", "playwright_real_chrome"],
43
43
  "notes": "in-code default — waf_profiles.yaml unavailable",
44
44
  },
45
45
  }
@@ -34,21 +34,21 @@ akamai_bot_manager:
34
34
  # at least a challenge page (i.e. IP still alive) rather than an outright
35
35
  # TLS reject. Grouped by family; planner tries top groups first.
36
36
  # Refresh quarterly — vendor WAFs shift which TLS fingerprints they trust.
37
- - [safari, safari15_3, safari15_5, safari17_0, safari260]
37
+ - [safari, safari15_3, safari15_5, safari17_0, safari260, safari2601]
38
38
  - [safari_ios, safari17_2_ios, safari260_ios]
39
- - [chrome99, chrome100, chrome101, chrome104, chrome110, chrome116, chrome119, chrome124, chrome131, chrome133a, chrome136, chrome145, chrome146]
39
+ - [chrome99, chrome100, chrome101, chrome104, chrome110, chrome116, chrome119, chrome124, chrome131, chrome133a, chrome136, chrome142, chrome145, chrome146]
40
40
  - [chrome_android, chrome131_android]
41
41
  - [edge99, edge101]
42
42
  tls_impersonate_avoid:
43
43
  # Empirically observed to 403 immediately (TLS fingerprint blacklisted).
44
44
  # DO NOT hard-block — planner deprioritizes only. Refresh quarterly.
45
+ # NOTE: requires curl_cffi >= 0.15.0 for chrome142/145/146 + safari260/2601 +
46
+ # firefox144/147 targets (added 0.14.0-0.15.0; default aliases resolve to
47
+ # chrome146/safari2601/firefox147 in 0.16.0+). Older installs raise ImpersonateError.
45
48
  - safari18_0
46
49
  - chrome107
47
50
  - chrome120
48
51
  - chrome123
49
- - chrome145 # curl_cffi 0.11+ needed — mark avoid for older installs
50
- - chrome146
51
- - firefox
52
52
  - firefox133
53
53
  - firefox135
54
54
  referer_strategies:
@@ -57,6 +57,7 @@ akamai_bot_manager:
57
57
  - original
58
58
  - mobile_subdomain # www.* → m.* — strong observational win in SSR sites
59
59
  fallback_when_challenge:
60
+ - surrogate_wayback # Phase 2.5: archive/reader copy before a browser spin-up
60
61
  - curl_grid_exhaust # try more impersonate × referer × url combos
61
62
  - playwright_real_chrome
62
63
  notes: |
@@ -77,6 +78,7 @@ cloudflare_turnstile:
77
78
  - google_search
78
79
  - self_root
79
80
  fallback_when_challenge:
81
+ - surrogate_wayback # Phase 2.5: archive/reader copy before a browser spin-up
80
82
  - playwright_mcp # MCP sufficient; Chromium TLS passes CF baseline
81
83
  - playwright_real_chrome
82
84
 
@@ -112,6 +114,7 @@ datadome_probable:
112
114
  tls_impersonate_candidates:
113
115
  - [safari, chrome]
114
116
  fallback_when_challenge:
117
+ - surrogate_wayback # Phase 2.5: archive/reader copy before a browser spin-up
115
118
  - playwright_real_chrome
116
119
  notes: |
117
120
  "_probable" suffix reminds us this is a growing attack surface — mark
@@ -127,6 +130,7 @@ perimeterx_human:
127
130
  tls_impersonate_candidates:
128
131
  - [safari, chrome]
129
132
  fallback_when_challenge:
133
+ - surrogate_wayback # Phase 2.5: archive/reader copy before a browser spin-up
130
134
  - playwright_real_chrome
131
135
  notes: |
132
136
  PerimeterX (now HUMAN Bot Defender). Distinct cookie family from
@@ -154,6 +158,7 @@ unknown_challenge:
154
158
  - original
155
159
  - mobile_subdomain
156
160
  fallback_when_challenge:
161
+ - surrogate_wayback # Phase 2.5: archive/reader copy before a browser spin-up
157
162
  - playwright_mcp
158
163
  - playwright_real_chrome
159
164
  notes: |
@@ -40,7 +40,7 @@ xhs favorites # 可能返回 API error
40
40
  >
41
41
  > **频率控制**: 高频请求(批量搜索、深翻评论)会触发验证码,这是平台限制无法绕过。建议每次操作间隔 2-3 秒。
42
42
  >
43
- > **POST 操作风险**: 发帖(post)、评论(comment)、点赞(like) 等写操作在 v0.6.x 可能因签名问题返回 406。如需使用,建议降级到 v0.3.5 (`pipx install xiaohongshu-cli==0.3.5`)
43
+ > **POST 操作**: 发帖(post)、评论(comment)、点赞(like) 等写操作在 v0.6.4 已修复签名问题 (PR [#19](https://github.com/jackwener/xiaohongshu-cli/pull/19)),可正常使用。
44
44
 
45
45
  ## 抖音 / Douyin
46
46
 
@@ -2,8 +2,8 @@
2
2
 
3
3
  Real interaction (clicks, forms, screenshots, video, persistent login) for pages that defeat Tier 1/1.5. Two runtime tools, both installed on demand — neither is vendored in this skill:
4
4
 
5
- - **CloakBrowser** (`pip`) — stealth Chromium with source-level C++ fingerprint patches. The Python wrapper source is MIT; the downloaded Chromium binary is covered by CloakBrowser's separate binary license and is not redistributed by this package. Passes Cloudflare Turnstile, FingerprintJS, BrowserScan, and 30+ detectors. Pin **0.4.10**.
6
- - **agent-browser** (`npm`, Apache-2.0) — native CDP automation CLI that drives CloakBrowser. AX-tree snapshots, `@eN` refs, click/fill/type/scroll, screenshots, video, cookie/state/session management. Pin **0.31.1**.
5
+ - **CloakBrowser** (`pip`) — stealth Chromium with source-level C++ fingerprint patches. The Python wrapper source is MIT; the downloaded Chromium binary is covered by CloakBrowser's separate binary license and is not redistributed by this package. Passes Cloudflare Turnstile, FingerprintJS, BrowserScan, and 30+ detectors. Pin **0.5.5**.
6
+ - **agent-browser** (`npm`, Apache-2.0) — native CDP automation CLI that drives CloakBrowser. AX-tree snapshots, `@eN` refs, click/fill/type/scroll, screenshots, video, cookie/state/session management. Pin **0.33.2**.
7
7
 
8
8
  ```
9
9
  CloakBrowser (stealth Chromium) <- CDP port 9242 -> agent-browser CLI
@@ -18,22 +18,22 @@ CloakBrowser (stealth Chromium) <- CDP port 9242 -> agent-browser CLI
18
18
  CloakBrowser runs in a dedicated Python venv. Cross-platform: macOS, Linux, and Windows all supported by both tools (use the venv path convention for your OS).
19
19
 
20
20
  ```bash
21
- # CloakBrowser (MIT wrapper source; separate binary license, pin 0.4.10):
21
+ # CloakBrowser (MIT wrapper source; separate binary license, pin 0.5.5):
22
22
  uv venv .cloak-venv --python 3.13
23
23
  # macOS/Linux: source .cloak-venv/bin/activate Windows: .cloak-venv\Scripts\activate
24
- uv pip install "cloakbrowser==0.4.10"
24
+ uv pip install "cloakbrowser==0.5.5"
25
25
  python -c "import cloakbrowser; cloakbrowser.ensure_binary()" # downloads stealth Chromium on first import
26
26
 
27
- # agent-browser (Apache-2.0, pin 0.31.1):
28
- npm i -g agent-browser@0.31.1 && agent-browser install
29
- agent-browser --version # 0.31.1
27
+ # agent-browser (Apache-2.0, pin 0.33.2):
28
+ npm i -g agent-browser@0.33.2 && agent-browser install
29
+ agent-browser --version # 0.33.2
30
30
  ```
31
31
 
32
32
  Verify CloakBrowser:
33
33
 
34
34
  ```bash
35
35
  python -c "import cloakbrowser; print(cloakbrowser.__version__, cloakbrowser.CHROMIUM_VERSION, cloakbrowser.binary_info()['installed'])"
36
- # -> 0.4.10 <chromium-version> True
36
+ # -> 0.5.5 <chromium-version> True
37
37
  ```
38
38
 
39
39
  ## Launch + drive
@@ -76,7 +76,9 @@ agent-browser skills list # everything available on the installed
76
76
  agent-browser --cdp 9242 eval 'navigator.webdriver' # must print false
77
77
  ```
78
78
 
79
- Verified 2026-07 with CloakBrowser 0.4.10 + agent-browser 0.31.1: `navigator.webdriver` reads the boolean false with no init-script, bot.sannysoft.com all-green, browserscan.net "Normal" (15/15), nowsecure.nl Turnstile bypassed.
79
+ Verified 2026-07 with CloakBrowser 0.5.5 + agent-browser 0.33.2: `navigator.webdriver` reads the boolean false with no init-script, bot.sannysoft.com all-green, browserscan.net "Normal" (15/15), nowsecure.nl Turnstile bypassed.
80
+
81
+ > **agent-browser 0.33.x behavior note:** the daemon now defaults to a 1-hour idle timeout (saves restore state, closes the browser, exits after 1 h of no commands). Set `AGENT_BROWSER_IDLE_TIMEOUT_MS=0` to restore the old always-persist behavior. External WebSocket stream consumers see latest-wins frame delivery; `record` (CDP) and the dashboard are unaffected.
80
82
 
81
83
  ## Cookie login (cross-platform)
82
84
 
@@ -115,6 +117,6 @@ lsof -ti:9242 | xargs kill -9
115
117
  # agent-browser can't connect:
116
118
  curl -s http://127.0.0.1:9242/json/version | head -5 # empty -> CloakBrowser not running
117
119
  # Update either tool:
118
- uv pip install --upgrade "cloakbrowser==0.4.10" && python -c "import cloakbrowser; cloakbrowser.ensure_binary()"
119
- npm i -g agent-browser@0.31.1
120
+ uv pip install --upgrade "cloakbrowser==0.5.5" && python -c "import cloakbrowser; cloakbrowser.ensure_binary()"
121
+ npm i -g agent-browser@0.33.2
120
122
  ```
@@ -71,9 +71,9 @@
71
71
  |------------|------|
72
72
  | URL 제공 (`https://...`) | → Phase 0 검사 후 없으면 Phase 1 (generic fetch chain) |
73
73
  | 핸들 제공 (`@username`) | → Phase 0 syndication/API |
74
- | 키워드만 ("X에서 AI 검색") | → WebSearch(`site:{domain} {keyword}`) 먼저 → URL 확보 후 재진입 |
74
+ | 키워드만 ("X에서 AI 검색") | → 웹 검색 도구(`site:{domain} {keyword}`) 먼저 → URL 확보 후 재진입 |
75
75
 
76
- > **한국어 신규 콘텐츠 한계**: 네이버/다음/한국 커뮤니티의 키워드 검색은 WebSearch 경유가 유일하며, 신규 콘텐츠 인덱싱이 지연될 수 있다.
76
+ > **한국어 신규 콘텐츠 한계**: 네이버/다음/한국 커뮤니티의 키워드 검색은 검색 도구 경유가 유일하며, 신규 콘텐츠 인덱싱이 지연될 수 있다.
77
77
 
78
78
  ## Phase 0 — 플랫폼 공식 API 인덱스
79
79
 
@@ -83,7 +83,7 @@
83
83
 
84
84
  | 플랫폼 | 방법 | 상세 |
85
85
  |--------|------|------|
86
- | X/Twitter | syndication (타임라인) + oEmbed (개별 트윗) + 키워드 검색: WebSearch → oEmbed | [twitter.md](twitter.md) |
86
+ | X/Twitter | syndication (타임라인) + oEmbed (개별 트윗) + 키워드 검색: 검색 도구 → oEmbed | [twitter.md](twitter.md) |
87
87
  | Reddit | URL + `.json` + Mobile UA | [json-api.md](json-api.md) |
88
88
  | Bluesky | AT Protocol (`public.api.bsky.app/xrpc/...`) | [public-api.md](public-api.md) |
89
89
  | Mastodon | 인스턴스별 공개 API | [public-api.md](public-api.md) |
@@ -301,7 +301,7 @@ yt-dlp --write-sub --write-auto-sub --sub-lang "en,ko" --skip-download -o "/tmp/
301
301
  |------|-------------|-----------------|
302
302
  | [`json-api.md`](json-api.md) | Reddit/Wikipedia/HN/npm/PyPI 등 **URL 변형만으로** JSON을 주는 사이트 | Reddit `/json` suffix + Mobile UA, HN Firebase, Algolia Search, Wikipedia REST, npm/PyPI Registry API |
303
303
  | [`public-api.md`](public-api.md) | Bluesky/Mastodon/arXiv/Stack Overflow/CrossRef/GitHub/OpenLibrary/Wayback 공식 API 사용 시 | 인증 없이 쓰는 공식 공개 REST/AT/Atom API 엔드포인트, 요청 형식, 공통 파라미터 |
304
- | [`twitter.md`](twitter.md) | X/Twitter 접근 — 프로필 타임라인, 특정 트윗, 키워드 검색 | `syndication.twitter.com` 타임라인, oEmbed 개별 트윗, 검색은 WebSearch로 URL 확보 후 oEmbed |
304
+ | [`twitter.md`](twitter.md) | X/Twitter 접근 — 프로필 타임라인, 특정 트윗, 키워드 검색 | `syndication.twitter.com` 타임라인, oEmbed 개별 트윗, 검색은 검색 도구로 URL 확보 후 oEmbed |
305
305
  | [`naver.md`](naver.md) | 네이버 블로그·뉴스·증권·검색 접근 | 서비스별 우회(블로그는 `m.blog.naver.com` 변환, 증권은 비공식 JSON, 검색은 `search.naver.com`), 한글 검색 쿼리 패턴 |
306
306
  | [`media.md`](media.md) | YouTube/Vimeo/Twitch/TikTok/SoundCloud 등 미디어 메타·자막·오디오 필요 시 | `yt-dlp --dump-json` 기반 1,858개 사이트 커버, 자막 다운로드(`--write-sub`), 포맷 선택, 라이브/팟캐스트 |
307
307