argo-search 1.0.1 → 2.9.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (463) hide show
  1. package/LICENSE +201 -21
  2. package/README.en.md +822 -0
  3. package/README.es.md +864 -0
  4. package/README.ja.md +864 -0
  5. package/README.ko.md +864 -0
  6. package/README.md +867 -320
  7. package/SKILL.md +135 -0
  8. package/assets/readme/hero.svg +121 -0
  9. package/assets/readme/made-with-beautify.svg +10 -0
  10. package/assets/readme/proof-routes.svg +73 -0
  11. package/assets/readme/settings-keys.png +0 -0
  12. package/assets/readme/settings-tools.png +0 -0
  13. package/assets/readme/why-better.svg +104 -0
  14. package/assets/readme/workflow.svg +83 -0
  15. package/backends/domain_profiles.json +2308 -96
  16. package/backends/engine_registry.yaml +3314 -503
  17. package/backends/query_synonyms_cn.json +35 -0
  18. package/backends/quota_profiles.json +2409 -161
  19. package/backends/source_types_cn.json +83 -0
  20. package/backends/text_lexicon.json +44 -0
  21. package/bin/argo +703 -0
  22. package/bin/argo.js +71 -39
  23. package/config.yaml +4976 -363
  24. package/cordis.patch.yml +21 -0
  25. package/data/countries.json +40 -0
  26. package/data/lang_matrix/lang_capability.json +842 -0
  27. package/data/lang_matrix/matrix_2026-09-10.json +11921 -0
  28. package/data/regions_cn.json +1 -0
  29. package/docs/ADDING_NEW_ENGINE.md +184 -0
  30. package/docs/ARGO_INTRO.md +124 -0
  31. package/docs/DATA_SOURCE_GAPS.md +81 -0
  32. package/docs/DESIGN_LOCAL_DATA_FUSION.md +90 -0
  33. package/docs/ENGINE_CATALOG.md +564 -0
  34. package/docs/MCP_SETUP.md +106 -0
  35. package/docs/PLATFORM_PATHS.md +111 -0
  36. package/docs/RELEASE_NOTES_v2.8.0.md +71 -0
  37. package/docs/RELEASE_NOTES_v2.8.2.md +63 -0
  38. package/docs/RELEASE_NOTES_v2.8.3.md +55 -0
  39. package/docs/RELEASE_NOTES_v2.8.4.md +79 -0
  40. package/docs/RELEASE_NOTES_v2.8.5.md +102 -0
  41. package/docs/RELEASE_NOTES_v2.8.6.md +66 -0
  42. package/docs/RELEASE_NOTES_v2.8.7.md +65 -0
  43. package/docs/RELEASE_NOTES_v2.8.8.md +67 -0
  44. package/docs/RELEASE_NOTES_v2.8.9.md +67 -0
  45. package/docs/RELEASE_NOTES_v2.9.0.md +64 -0
  46. package/docs/SEARCH_ARCHIVE.md +115 -0
  47. package/docs/assets/architecture.svg +104 -0
  48. package/docs/assets/hero.svg +98 -0
  49. package/docs/engines/bangumi.md +27 -0
  50. package/docs/engines/biorxiv.md +27 -0
  51. package/docs/engines/clawhub.md +27 -0
  52. package/docs/engines/crt_sh.md +27 -0
  53. package/docs/engines/datacite.md +27 -0
  54. package/docs/engines/deps_dev.md +27 -0
  55. package/docs/engines/douban_movie.md +27 -0
  56. package/docs/engines/endoflife.md +27 -0
  57. package/docs/engines/flk_law.md +27 -0
  58. package/docs/engines/fxtwitter.md +27 -0
  59. package/docs/engines/google_news.md +27 -0
  60. package/docs/engines/gov_regulations.md +27 -0
  61. package/docs/engines/guqian.md +27 -0
  62. package/docs/engines/hackernews.md +27 -0
  63. package/docs/engines/iplant.md +27 -0
  64. package/docs/engines/keenable.md +27 -0
  65. package/docs/engines/met_museum.md +27 -0
  66. package/docs/engines/nasa_images.md +27 -0
  67. package/docs/engines/netease_music.md +27 -0
  68. package/docs/engines/npm.md +27 -0
  69. package/docs/engines/nvd.md +27 -0
  70. package/docs/engines/openreview.md +27 -0
  71. package/docs/engines/openstd.md +27 -0
  72. package/docs/engines/parallel_free.md +27 -0
  73. package/docs/engines/people_daily.md +27 -0
  74. package/docs/engines/realtime_index.md +27 -0
  75. package/docs/engines/sec_edgar.md +27 -0
  76. package/docs/engines/seltz.md +27 -0
  77. package/docs/engines/skillsmp.md +27 -0
  78. package/docs/engines/sspai.md +27 -0
  79. package/docs/engines/std_samr.md +27 -0
  80. package/docs/engines/train.md +27 -0
  81. package/docs/engines/un_comtrade.md +27 -0
  82. package/docs/engines/wikisource.md +27 -0
  83. package/docs/engines/zdic.md +27 -0
  84. package/docs/engines/zenodo.md +27 -0
  85. package/docs/engines/zhihu_hot_app.md +27 -0
  86. package/docs//344/270/272/344/273/200/344/271/210/351/200/211/346/213/251argo.md +176 -0
  87. package/engines/README.md +10 -0
  88. package/engines/_template_cli.yaml +39 -0
  89. package/engines/_template_http.yaml +37 -0
  90. package/engines/plugins/_template_plugin.py +50 -0
  91. package/engines/specs/_cisa_kev.yaml +54 -0
  92. package/engines/specs/_osv.yaml +63 -0
  93. package/engines/specs/aljazeera.yaml +43 -0
  94. package/engines/specs/antara.yaml +43 -0
  95. package/engines/specs/artic.yaml +20 -0
  96. package/engines/specs/aviation_weather.yaml +39 -0
  97. package/engines/specs/bangumi.yaml +21 -0
  98. package/engines/specs/biorxiv.yaml +24 -0
  99. package/engines/specs/carbon_intensity.yaml +32 -0
  100. package/engines/specs/cinii.yaml +44 -0
  101. package/engines/specs/clawhub.yaml +49 -0
  102. package/engines/specs/cleveland.yaml +19 -0
  103. package/engines/specs/cn_ai_news.yaml +43 -0
  104. package/engines/specs/core.yaml +35 -0
  105. package/engines/specs/crt_sh.yaml +38 -0
  106. package/engines/specs/ct24.yaml +43 -0
  107. package/engines/specs/datacite.yaml +27 -0
  108. package/engines/specs/datagov_uk.yaml +41 -0
  109. package/engines/specs/ddgs_images.yaml +45 -0
  110. package/engines/specs/deepseek_search.yaml +74 -0
  111. package/engines/specs/deezer.yaml +20 -0
  112. package/engines/specs/deps_dev.yaml +21 -0
  113. package/engines/specs/digup.yaml +36 -0
  114. package/engines/specs/doi.yaml +39 -0
  115. package/engines/specs/douban_movie.yaml +22 -0
  116. package/engines/specs/dw_news.yaml +47 -0
  117. package/engines/specs/egov_law.yaml +21 -0
  118. package/engines/specs/elpais.yaml +43 -0
  119. package/engines/specs/endoflife.yaml +21 -0
  120. package/engines/specs/energy_charts.yaml +21 -0
  121. package/engines/specs/factcheck_org.yaml +52 -0
  122. package/engines/specs/faz.yaml +43 -0
  123. package/engines/specs/federal_register.yaml +57 -0
  124. package/engines/specs/firecrawl.yaml +53 -0
  125. package/engines/specs/flk_law.yaml +24 -0
  126. package/engines/specs/folha.yaml +43 -0
  127. package/engines/specs/france24.yaml +44 -0
  128. package/engines/specs/full_fact.yaml +47 -0
  129. package/engines/specs/fxtwitter.yaml +31 -0
  130. package/engines/specs/gbfs_nyc.yaml +32 -0
  131. package/engines/specs/gdacs.yaml +21 -0
  132. package/engines/specs/gdelt.yaml +64 -0
  133. package/engines/specs/google_news.yaml +22 -0
  134. package/engines/specs/gov_regulations.yaml +38 -0
  135. package/engines/specs/guardian_rss.yaml +59 -0
  136. package/engines/specs/hurriyet.yaml +43 -0
  137. package/engines/specs/iplant.yaml +25 -0
  138. package/engines/specs/jikan.yaml +24 -0
  139. package/engines/specs/jolpica.yaml +20 -0
  140. package/engines/specs/k10plus.yaml +20 -0
  141. package/engines/specs/keenable.yaml +44 -0
  142. package/engines/specs/lefigaro.yaml +43 -0
  143. package/engines/specs/listenbrainz.yaml +19 -0
  144. package/engines/specs/matichon.yaml +43 -0
  145. package/engines/specs/met_museum.yaml +21 -0
  146. package/engines/specs/nasa_images.yaml +45 -0
  147. package/engines/specs/netease_music.yaml +38 -0
  148. package/engines/specs/nhk.yaml +43 -0
  149. package/engines/specs/nhtsa_vpic.yaml +20 -0
  150. package/engines/specs/noaa_swpc.yaml +20 -0
  151. package/engines/specs/nvd.yaml +38 -0
  152. package/engines/specs/obis.yaml +20 -0
  153. package/engines/specs/opencitations.yaml +52 -0
  154. package/engines/specs/openf1.yaml +21 -0
  155. package/engines/specs/openfoodfacts.yaml +45 -0
  156. package/engines/specs/openligadb.yaml +20 -0
  157. package/engines/specs/openreview.yaml +37 -0
  158. package/engines/specs/openstd.yaml +21 -0
  159. package/engines/specs/people_daily.yaml +22 -0
  160. package/engines/specs/realtime_index.yaml +34 -0
  161. package/engines/specs/ror.yaml +20 -0
  162. package/engines/specs/satnogs.yaml +20 -0
  163. package/engines/specs/sec_edgar.yaml +46 -0
  164. package/engines/specs/skillsmp.yaml +50 -0
  165. package/engines/specs/sogou_weixin_browser.yaml +30 -0
  166. package/engines/specs/soilgrids.yaml +25 -0
  167. package/engines/specs/sspai.yaml +36 -0
  168. package/engines/specs/stackexchange.yaml +46 -0
  169. package/engines/specs/std_samr.yaml +21 -0
  170. package/engines/specs/tass.yaml +43 -0
  171. package/engines/specs/tle_mirror.yaml +21 -0
  172. package/engines/specs/train.yaml +37 -0
  173. package/engines/specs/tvmaze.yaml +20 -0
  174. package/engines/specs/twitter_syndication.yaml +31 -0
  175. package/engines/specs/un_comtrade.yaml +23 -0
  176. package/engines/specs/unpaywall.yaml +54 -0
  177. package/engines/specs/vnpress.yaml +43 -0
  178. package/engines/specs/weather.yaml +32 -0
  179. package/engines/specs/who_don.yaml +21 -0
  180. package/engines/specs/who_gho.yaml +20 -0
  181. package/engines/specs/wikimedia_commons.yaml +34 -0
  182. package/engines/specs/wikisource.yaml +22 -0
  183. package/engines/specs/worms.yaml +19 -0
  184. package/engines/specs/x_trending.yaml +34 -0
  185. package/engines/specs/yna.yaml +43 -0
  186. package/engines/specs/ynet.yaml +43 -0
  187. package/engines/specs/zbmath.yaml +40 -0
  188. package/engines/specs/zdic.yaml +22 -0
  189. package/engines/specs/zenodo.yaml +27 -0
  190. package/engines/specs/zhihu_hot_app.yaml +35 -0
  191. package/mcp/clients.yaml +74 -0
  192. package/package.json +51 -12
  193. package/packages/dsh-plugin/README.md +134 -0
  194. package/packages/dsh-plugin/cordis.patch.yml +48 -0
  195. package/packages/dsh-plugin/dsh/client.js +1244 -0
  196. package/packages/dsh-plugin/dsh/index.js +2290 -0
  197. package/packages/dsh-plugin/dsh/native-tools.mjs +743 -0
  198. package/packages/dsh-plugin/package.json +38 -0
  199. package/packages/dsh-plugin/test/client.test.js +232 -0
  200. package/packages/dsh-plugin/test/config.test.js +30 -0
  201. package/packages/dsh-plugin/test/index.test.js +1338 -0
  202. package/references/query-templates.md +96 -0
  203. package/references/query-templates.yaml +118 -0
  204. package/scripts/ab_eval.py +219 -0
  205. package/scripts/ab_eval_p0p1.py +266 -0
  206. package/scripts/adaptive.py +271 -70
  207. package/scripts/answer.py +154 -0
  208. package/scripts/archive_run.py +591 -0
  209. package/scripts/argo_engine_registry.py +172 -0
  210. package/scripts/argo_paths.py +728 -0
  211. package/scripts/article.py +136 -0
  212. package/scripts/arxiv_source.py +201 -0
  213. package/scripts/badcase_ledger.py +252 -0
  214. package/scripts/batch_probe.py +241 -0
  215. package/scripts/benchmark.py +19 -0
  216. package/scripts/bounded_run.py +144 -0
  217. package/scripts/browser_auth.py +323 -0
  218. package/scripts/burstiness_detector.py +147 -0
  219. package/scripts/cache.py +1078 -67
  220. package/scripts/cache_guard.py +271 -0
  221. package/scripts/cache_key_vdom.py +56 -0
  222. package/scripts/cache_similarity.py +170 -0
  223. package/scripts/caliber_isolation.py +195 -0
  224. package/scripts/calibrate_lowquality.py +210 -0
  225. package/scripts/candidate_envelope.py +366 -0
  226. package/scripts/chrome_cdp.py +871 -0
  227. package/scripts/circuit_breaker.py +389 -0
  228. package/scripts/citations.py +334 -0
  229. package/scripts/cite_cli.py +59 -0
  230. package/scripts/cjk_tokens.py +95 -0
  231. package/scripts/clarify.py +47 -24
  232. package/scripts/cli_io.py +240 -0
  233. package/scripts/config.py +683 -45
  234. package/scripts/config_cache.py +402 -0
  235. package/scripts/conn_pool.py +220 -0
  236. package/scripts/content_security.py +685 -0
  237. package/scripts/content_signals.py +730 -0
  238. package/scripts/crawl.py +74 -35
  239. package/scripts/daemon.py +866 -0
  240. package/scripts/digup_bridge.py +214 -0
  241. package/scripts/engine_admission.py +366 -0
  242. package/scripts/engine_dispatch.py +1019 -0
  243. package/scripts/engine_env.py +534 -0
  244. package/scripts/engine_failure.py +411 -0
  245. package/scripts/engine_families.py +732 -0
  246. package/scripts/engine_policy.py +218 -0
  247. package/scripts/engine_requires.py +190 -0
  248. package/scripts/engine_status.py +322 -0
  249. package/scripts/engine_validate.py +638 -0
  250. package/scripts/engines.py +778 -294
  251. package/scripts/engines_base.py +1521 -0
  252. package/scripts/engines_builders.py +298 -0
  253. package/scripts/engines_builders_batch10.py +159 -0
  254. package/scripts/engines_builders_batch11.py +174 -0
  255. package/scripts/engines_builders_batch12.py +103 -0
  256. package/scripts/engines_builders_batch13.py +340 -0
  257. package/scripts/engines_builders_batch9.py +1468 -0
  258. package/scripts/engines_builders_cn.py +2283 -0
  259. package/scripts/engines_builders_data.py +2923 -0
  260. package/scripts/engines_builders_data_macro.py +775 -0
  261. package/scripts/engines_builders_feeds.py +255 -0
  262. package/scripts/engines_builders_intl.py +1239 -0
  263. package/scripts/engines_builders_search.py +440 -0
  264. package/scripts/engines_builders_tech.py +1360 -0
  265. package/scripts/evidence.py +588 -162
  266. package/scripts/evidence_loop.py +592 -0
  267. package/scripts/evidence_tier.py +298 -0
  268. package/scripts/except_sets.py +25 -0
  269. package/scripts/extract.py +55 -16
  270. package/scripts/fact_align.py +207 -0
  271. package/scripts/fact_crosscheck.py +230 -0
  272. package/scripts/fetch.py +25 -12
  273. package/scripts/fetch_browser.py +87 -0
  274. package/scripts/fetch_jsrun.py +122 -0
  275. package/scripts/fetch_quality.py +225 -0
  276. package/scripts/fetch_render_tinyfish.py +117 -0
  277. package/scripts/fetch_v3.py +1779 -0
  278. package/scripts/focus_extract.py +350 -0
  279. package/scripts/fulltext_store.py +169 -0
  280. package/scripts/fxtwitter_api.py +325 -0
  281. package/scripts/gen_engine_catalog.py +364 -0
  282. package/scripts/gen_native_tools.py +91 -0
  283. package/scripts/gen_routing_snapshot.py +184 -0
  284. package/scripts/health_check.py +140 -402
  285. package/scripts/health_probe.py +23 -8
  286. package/scripts/hot_state.py +134 -0
  287. package/scripts/http_client.py +1053 -0
  288. package/scripts/image_ops.py +430 -0
  289. package/scripts/image_vision/vision_probe +0 -0
  290. package/scripts/image_vision/vision_probe.swift +126 -0
  291. package/scripts/install.ps1 +148 -0
  292. package/scripts/install.sh +206 -0
  293. package/scripts/job.py +1186 -0
  294. package/scripts/jump_resolver.py +235 -0
  295. package/scripts/lang_capability.py +150 -0
  296. package/scripts/lang_detect.py +289 -0
  297. package/scripts/lang_matrix_probe.py +329 -0
  298. package/scripts/lang_pref.py +311 -0
  299. package/scripts/link_source.py +340 -0
  300. package/scripts/local_engines.py +321 -0
  301. package/scripts/local_image.py +970 -0
  302. package/scripts/local_seek.py +366 -0
  303. package/scripts/macro_countries.py +82 -0
  304. package/scripts/matrix_search_eval.py +1105 -0
  305. package/scripts/mcp_diag.py +8 -3
  306. package/scripts/mcp_handlers.py +1074 -0
  307. package/scripts/mcp_handlers_surface.py +157 -0
  308. package/scripts/mcp_launch.sh +122 -0
  309. package/scripts/mcp_server.py +118 -417
  310. package/scripts/mcp_setup.py +407 -0
  311. package/scripts/mcp_tools.py +330 -0
  312. package/scripts/mcp_transport.py +191 -0
  313. package/scripts/min_version_scan.py +335 -0
  314. package/scripts/net_proxy.py +258 -0
  315. package/scripts/network_aware.py +152 -0
  316. package/scripts/paper.py +274 -0
  317. package/scripts/pdf_extract.py +518 -0
  318. package/scripts/phrase_filter.py +66 -0
  319. package/scripts/plan.py +422 -0
  320. package/scripts/postprocess_signals.py +150 -0
  321. package/scripts/pre-commit +48 -0
  322. package/scripts/query_enhance.py +157 -0
  323. package/scripts/query_operators.py +196 -0
  324. package/scripts/query_rewriter.py +116 -7
  325. package/scripts/query_signals.py +300 -0
  326. package/scripts/query_templates.py +167 -0
  327. package/scripts/query_understanding.py +411 -0
  328. package/scripts/query_variants.py +298 -0
  329. package/scripts/quota.py +488 -76
  330. package/scripts/rank_signals.py +775 -0
  331. package/scripts/ranking_eval.py +319 -0
  332. package/scripts/readability_extract.py +433 -0
  333. package/scripts/recompute.py +415 -0
  334. package/scripts/recovery.py +616 -0
  335. package/scripts/recovery_l0.py +160 -0
  336. package/scripts/redskill/redskill_engine.py +332 -0
  337. package/scripts/regression_p0p1.py +318 -0
  338. package/scripts/replay_eval.py +640 -0
  339. package/scripts/rerank_tfidf.py +225 -0
  340. package/scripts/research.py +591 -403
  341. package/scripts/research_cli.py +361 -0
  342. package/scripts/research_dossier.py +492 -0
  343. package/scripts/research_expand.py +146 -0
  344. package/scripts/research_gates.py +272 -0
  345. package/scripts/research_report.py +209 -0
  346. package/scripts/research_strategy.py +37 -0
  347. package/scripts/research_work_packages.py +195 -0
  348. package/scripts/result_lang.py +311 -0
  349. package/scripts/robots_guard.py +259 -0
  350. package/scripts/route.py +921 -254
  351. package/scripts/route_bangs.py +40 -0
  352. package/scripts/route_cache.py +225 -0
  353. package/scripts/route_combo.py +910 -0
  354. package/scripts/route_domains.py +328 -0
  355. package/scripts/route_lang.py +605 -0
  356. package/scripts/route_log.py +63 -0
  357. package/scripts/route_policy.py +250 -0
  358. package/scripts/rrf_classic.py +66 -0
  359. package/scripts/screenshot.py +73 -0
  360. package/scripts/search.py +897 -405
  361. package/scripts/search_attribution.py +88 -0
  362. package/scripts/search_benchmark.py +407 -0
  363. package/scripts/search_cli.py +627 -0
  364. package/scripts/search_entry.py +389 -0
  365. package/scripts/search_output.py +606 -0
  366. package/scripts/search_pipeline.py +1077 -0
  367. package/scripts/search_rank.py +991 -0
  368. package/scripts/search_types.py +27 -12
  369. package/scripts/seek_locator.py +68 -0
  370. package/scripts/semantic_evidence.py +215 -0
  371. package/scripts/serp_guard.py +228 -0
  372. package/scripts/serp_kill_log.py +51 -0
  373. package/scripts/single_flight.py +120 -0
  374. package/scripts/social_engines/__init__.py +10 -0
  375. package/scripts/social_engines/bilibili_engine.py +131 -0
  376. package/scripts/social_engines/hackernews_engine.py +113 -0
  377. package/scripts/social_engines/reddit_engine.py +172 -0
  378. package/scripts/social_engines/twitter_engine.py +397 -0
  379. package/scripts/social_engines/v2ex_engine.py +101 -0
  380. package/scripts/social_engines/weibo_engine.py +128 -0
  381. package/scripts/social_engines/xiaohongshu_engine.py +137 -0
  382. package/scripts/social_engines/zhihu_engine.py +139 -0
  383. package/scripts/social_research.py +201 -0
  384. package/scripts/sogou_weixin_browser.py +318 -0
  385. package/scripts/source_ledger.py +249 -0
  386. package/scripts/stage_timing.py +58 -0
  387. package/scripts/stats_cli.py +95 -0
  388. package/scripts/straggler_policy.py +60 -0
  389. package/scripts/stylometry_detector.py +231 -0
  390. package/scripts/sync_backends.py +353 -0
  391. package/scripts/sync_installs.py +29 -0
  392. package/scripts/template_cli.py +127 -0
  393. package/scripts/text_lexicon.py +72 -0
  394. package/scripts/tfidf_router.py +109 -37
  395. package/scripts/time_utils.py +233 -0
  396. package/scripts/topic_research_profiles.py +643 -0
  397. package/scripts/train.py +737 -0
  398. package/scripts/tweet.py +596 -0
  399. package/scripts/url_canon.py +217 -0
  400. package/scripts/url_optimize.py +56 -0
  401. package/scripts/url_safety.py +236 -0
  402. package/scripts/usage_log.py +116 -0
  403. package/scripts/usage_log_clean.py +113 -0
  404. package/scripts/v2ex_nodes.py +311 -0
  405. package/scripts/variant_recall.py +188 -0
  406. package/scripts/watch.py +205 -0
  407. package/scripts/wechat_article.py +258 -0
  408. package/scripts/wide_research.py +402 -0
  409. package/scripts/wx.py +349 -0
  410. package/scripts/x_article_fetch.py +141 -0
  411. package/scripts/x_search.py +234 -0
  412. package/scripts/x_trending.py +189 -0
  413. package/scripts/yaml_load.py +67 -0
  414. package/sub-skills/ego-search/SKILL.md +253 -0
  415. package/sub-skills/ego-search/references/browser-runtime.md +43 -0
  416. package/sub-skills/ego-search/references/install.md +67 -0
  417. package/sub-skills/ego-search/references/original-ego-upgrade.md +51 -0
  418. package/sub-skills/ego-search/scripts/ego_search.py +800 -0
  419. package/sub-skills/ego-search/scripts/merge.py +226 -0
  420. package/sub-skills/ego-search/scripts/quality.py +64 -0
  421. package/sub-skills/ego-search/scripts/runtime.py +172 -0
  422. package/sub-skills/ego-search/scripts/safety.py +109 -0
  423. package/sub-skills/ego-search/scripts/serp_spec.py +109 -0
  424. package/sub-skills/ego-search/scripts/webbridge_adapter.py +374 -0
  425. package/sub-skills/ego-search/tests/test_query_normalize.py +68 -0
  426. package/sub-skills/js-run/SKILL.md +67 -0
  427. package/sub-skills/js-run/scripts/jsrun.py +112 -0
  428. package/sub-skills/js-run/scripts/shim.js +188 -0
  429. package/sub-skills/js-run/tests/fixtures/challenge_like.js +27 -0
  430. package/sub-skills/js-run/tests/test_jsrun.py +165 -0
  431. package/sub-skills/local-search/SKILL.md +50 -53
  432. package/sub-skills/local-search/bm25.py +162 -0
  433. package/sub-skills/local-search/config.yaml +152 -55
  434. package/sub-skills/local-search/engine_registry.py +91 -26
  435. package/sub-skills/local-search/{health_check.py → local_health_check.py} +140 -10
  436. package/sub-skills/local-search/local_search_adapter.py +13 -2
  437. package/sub-skills/local-search/parse_maps.yaml +40 -42
  438. package/sub-skills/local-search/search_v3.py +1182 -65
  439. package/sub-skills/local-search/smart_router.py +47 -7
  440. package/sub-skills/local-seek/SKILL.md +110 -0
  441. package/sub-skills/local-seek/config/domains.yaml +20 -0
  442. package/sub-skills/local-seek/references/auto-behaviors.md +24 -0
  443. package/sub-skills/local-seek/references/fd-mdfind-recipes.md +62 -0
  444. package/sub-skills/local-seek/references/git-integration.md +54 -0
  445. package/sub-skills/local-seek/references/remote-handoff.md +46 -0
  446. package/sub-skills/local-seek/references/rg-recipes.md +96 -0
  447. package/sub-skills/local-seek/references/strategies.md +52 -0
  448. package/sub-skills/local-seek/references/structural-search.md +59 -0
  449. package/sub-skills/local-seek/scripts/eval_rank.py +216 -0
  450. package/sub-skills/local-seek/scripts/eval_seek.py +224 -0
  451. package/sub-skills/local-seek/scripts/seek.py +1465 -0
  452. package/sub-skills/local-seek/scripts/seek_frecency.py +126 -0
  453. package/sub-skills/local-seek/scripts/seek_fts.py +430 -0
  454. package/sub-skills/local-seek/scripts/seek_rank.py +193 -0
  455. package/sub-skills/local-seek/tests/test_content_rank.py +222 -0
  456. package/sub-skills/local-seek/tests/test_determinism_20261006.py +145 -0
  457. package/sub-skills/local-seek/tests/test_noise_tier.py +101 -0
  458. package/sub-skills/local-seek/tests/test_pcre2_probe.py +136 -0
  459. package/sub-skills/local-seek/tests/test_rank_pinyin.py +77 -0
  460. package/sub-skills/local-seek/tests/test_regression_20261003.py +169 -0
  461. package/sub-skills/local-seek/tests/test_regression_20261006.py +146 -0
  462. package/sub-skills/local-seek/tests/test_seek_degradation.py +111 -0
  463. package/sub-skills/local-seek/tests/test_seek_fts.py +313 -0
@@ -0,0 +1,184 @@
1
+ # 接入新搜索引擎(标准化流程)
2
+
3
+ 目标:在现有 Argo 上**自由增减、组合**搜索 API,走统一配置与准入,而不是每次改一堆散落文件。
4
+
5
+ ## 引擎三档
6
+
7
+ | 档 | 何时用 | 你要交付什么 | 改 Python? |
8
+ |----|--------|--------------|-------------|
9
+ | **L1 声明式 HTTP** | 标准 REST JSON 搜索 | `engines/specs/<id>.yaml` + 环境变量 | 否 |
10
+ | **L2 增强声明式** | 特殊 header/body/output_map | 同上,补全 `output_map` / `required_env` | 通常否 |
11
+ | **CLI 桥接** | 本机已装 CLI 输出 YAML/JSON(结构化) | `engines/specs/<id>.yaml`,`type: cli` + `cmd`/`search_args`/`output_format`/`filter_args` | 否 |
12
+ | **L3 插件** | 多端点、HTML、会话、非标协议 | `engines/plugins/<id>.py` + spec `type` | 是 |
13
+
14
+ 模板:
15
+
16
+ - L1/L2:`engines/_template_http.yaml`
17
+ - CLI 桥接:`engines/_template_cli.yaml`(参考实现 `engines/specs/realtime_index.yaml`)
18
+ - L3:`engines/plugins/_template_plugin.py`
19
+
20
+ CLI 桥接关键字段:
21
+
22
+ - `cmd` / `search_args`:命令与参数模板,支持 `{query}` / `{n}` 占位符;裸命令(PATH 中)与绝对路径均可。
23
+ - `output_format: yaml`:走通用 YAML 解析,支持 `results`/`items`/`data` 或顶层 list,字段别名 `title|name`、`url|link`、`snippet|description|content`,并保留 `published_at` 发布时间维度。
24
+ - `filter_args`:条件参数表。查询带 `--since`/`--until`(如 `7d` / `2026-08-01`)时自动追加对应 CLI 参数;未携带则不追加。缓存按时间窗隔离,不会串结果。
25
+
26
+ ## 可选结果字段(图源 / 全文源必看)
27
+
28
+ 结果 dict 只要求 `title` / `url`,其余字段原样透传到融合、精排与 JSON 输出
29
+ (无字段白名单,`rrf_merge` 与 `local_five_dim_rerank` 都不会裁字段)。三个
30
+ **可选**字段有下游消费者,写对了才有价值:
31
+
32
+ | 字段 | 语义 | 谁在用 |
33
+ |------|------|--------|
34
+ | `image_url` | 可直接打开的图片地址 | 结果展示;判断「这是图源」的唯一依据 |
35
+ | `image_license` | 该图的授权/权利状态 | 合规展示(如 `CC0`、`公有领域(NASA)`) |
36
+ | `full_text_url` | **可确定性取到正文**的端点 | `evidence_loop.gate_results` 的取数建议优先用它,见下 |
37
+
38
+ > **字段落点**:这些字段进的是 `results`(以及归档用的 `candidates` 视图),
39
+ > **不进** `sources`。`build_sources` 是「底部相关链接」形态的**稳定 5 字段投影**
40
+ > (`ref/title/url/engine/score/snippet`),有意裁剪,别往里加字段。消费方要拿
41
+ > 图片/正文端点请读 `results`。
42
+
43
+ ### `image_url` 的三条硬约束
44
+
45
+ 1. **不对就不写**:上游没给图(或 Met 这类对非公版件不给链接)时**不要**填空串或详情页 URL——宁缺勿假,下游据此判断「有无图」。
46
+ 2. **路径必须指向字符串**:声明式 `output_map` 里写 `item_image: links.0.href`。NASA 的 `links.0` 是 `{href, rel, render, …}`,写成 `links.0` 会被 `_coerce_field` 当 dict 丢成空串。
47
+ 3. **图床可能有额外要求**:AIC 的 IIIF 要求 `AIC-User-Agent` 头才放行(否则 403),复用该 URL 抓图时要带上。
48
+
49
+ 声明式源可在 spec 顶层写 `image_license: "…"` 作为常量(授权不随条目变化时用,
50
+ 如 NASA 公版)。
51
+
52
+ ### `full_text_url` 什么时候该写
53
+
54
+ 当**给人看的页面拿不到正文**、而该源另有确定性正文端点时写它。两个实测例子:
55
+
56
+ - `gutenberg`:`/ebooks/{id}` 是下载门户页(去标签首段全是 noprint 脚本),gutendex 在 `formats` 里已给纯文本 URL;
57
+ - `egov_law`:`/law/{id}` 是 JS 空壳页(去标签后只剩「e-Gov 法令検索」),`/api/1/lawdata/{id}` 才是官方全文 XML。
58
+
59
+ 写它的收益是让取数建议走直连,省掉「门户页找下载链」或浏览器渲染那几级;
60
+ **如果人类页面本身就能取到正文,不要写**(那是无消费方的装饰字段)。
61
+
62
+ 回归门禁:`tests/test_image_sources.py`、`tests/test_full_text_sources.py`。
63
+
64
+ ## 环境变量规范
65
+
66
+ ```bash
67
+ # 推荐(新)
68
+ export ARGO_TAVILY_API_KEY=...
69
+ export ARGO_EXA_API_KEY=...
70
+ export ARGO_MYENGINE_API_KEY=...
71
+
72
+ # 兼容(旧名仍可用)
73
+ export TAVILY_API_KEY=...
74
+ export EXA_API_KEY=...
75
+
76
+ # 可选:白名单 / 黑名单(逗号分隔)
77
+ export ARGO_ENABLE_ENGINES="hackernews,duckduckgo,eastmoney,tavily"
78
+ export ARGO_DISABLE_ENGINES="brave,felo"
79
+ ```
80
+
81
+ 策略类配置(路由权重、TTL、RRF)仍在 `config.yaml`,与密钥分离。
82
+
83
+ ## 生命周期
84
+
85
+ ```
86
+ 注册声明 → 配置注入 → 标准化验证 → 生产准入
87
+ YAML env validate admission
88
+ ```
89
+
90
+ ### 1. 注册声明(L1 示例)
91
+
92
+ ```bash
93
+ cp engines/_template_http.yaml engines/specs/myengine.yaml
94
+ # 编辑 engine_id / url / headers / body / output_map / required_env
95
+ ```
96
+
97
+ 启动时 `config.py` 自动 merge `engines/*.yaml` 与 `engines/specs/*.yaml`(`_` 前缀跳过)。
98
+
99
+ 也可继续写在主 `config.yaml` 的 `engines:` 段(存量方式)。
100
+
101
+ ### 2. 配置注入
102
+
103
+ ```bash
104
+ export ARGO_MYENGINE_API_KEY="..."
105
+ # 可选:仅启用子集做试验
106
+ export ARGO_ENABLE_ENGINES="myengine,hackernews,duckduckgo"
107
+ ```
108
+
109
+ 缺 Key 的引擎:
110
+
111
+ - **自动路由**:跳过
112
+ - **强制** `--engine myengine`:仍可调用(通常返回 `[]`)
113
+
114
+ ### 3. 标准化验证
115
+
116
+ ```bash
117
+ cd ~/.workbuddy/skills/argo # 或你的安装路径
118
+
119
+ # 连通性 + schema
120
+ python3 scripts/engine_validate.py --engine myengine --stage health
121
+
122
+ # 质量基准(固定 query 集)
123
+ python3 scripts/engine_validate.py --engine myengine --stage quality
124
+
125
+ # 全量 + 写入准入 + 生成文档
126
+ python3 scripts/engine_validate.py --engine myengine --stage all --admit --write-doc
127
+
128
+ # 批量:所有 free 且 env 就绪
129
+ python3 scripts/engine_validate.py --all-free --stage health --admit
130
+ ```
131
+
132
+ 通过 health 且 `--admit` → `~/.cache/unified-search/admission/<id>.json` 中 `blocked: false`。
133
+
134
+ ### 4. 生产准入与观测
135
+
136
+ ```bash
137
+ # 表格式状态
138
+ python3 scripts/search.py --list-engines --detail
139
+
140
+ # 仅可自动路由
141
+ python3 scripts/search.py --list-engines --detail --routable-only
142
+
143
+ # JSON
144
+ python3 scripts/search.py --list-engines --detail --json
145
+
146
+ # 单引擎状态
147
+ python3 scripts/engine_status.py --engine myengine --json
148
+ ```
149
+
150
+ 手动熔断:
151
+
152
+ ```python
153
+ from engine_admission import set_blocked
154
+ set_blocked("myengine", True, reason="quota_exhausted")
155
+ ```
156
+
157
+ ## 组合与优化
158
+
159
+ - **减少**:`enabled: false` 或 `ARGO_DISABLE_ENGINES`
160
+ - **增加**:丢 YAML / 插件 → validate → admit
161
+ - **组合**:改 `config.yaml` 的 `domains[].engines_combo`,或 TF-IDF `domain_profiles`
162
+ - **预算**:`--mode fast|auto|deep|budget` 继续按 cost_tier 过滤
163
+
164
+ ## 目录真源
165
+
166
+ | 路径 | 职责 |
167
+ |------|------|
168
+ | `config.yaml` | 主配置、域规则、cost_tiers、存量引擎 |
169
+ | `engines/specs/*.yaml` | 外置引擎声明(推荐新引擎落点) |
170
+ | `engines/plugins/*.py` | L3 自定义 builder |
171
+ | `scripts/engine_env.py` | Key 别名与 ENABLE/DISABLE |
172
+ | `scripts/engine_admission.py` | 准入状态 |
173
+ | `scripts/engine_validate.py` | 验证 CLI |
174
+ | `backends/engine_registry.yaml` | 元数据/观测目录(非运行时唯一真源) |
175
+
176
+ 运行时引擎列表以 **merge 后的 config engines** 为准。
177
+
178
+ ## 验收清单
179
+
180
+ - [ ] `engine_validate --stage health` 为 `pass` 或合理 `skipped`(缺 Key)
181
+ - [ ] `--list-engines --detail` 中 `routable=True`(需要进自动路由时)
182
+ - [ ] `search.py "查询" --engine <id>` 有结构化结果
183
+ - [ ] 无 Key 时自动路由不含该引擎
184
+ - [ ] `blocked=true` 后自动路由剔除
@@ -0,0 +1,124 @@
1
+ # Argo 阿尔戈 — 统一搜索与证据核验
2
+
3
+ > 一个开源的「搜索 + 证据核验」工具,给 AI Agent 和终端使用。把 218 个搜索源、多语言路由、深度研究与证据判定整合在一个命令行 / MCP 里,让「搜」和「核」变成一件事。
4
+
5
+ ---
6
+
7
+ ## 安装方式
8
+
9
+ > **唯一真实源是 GitHub。** 请用下面任一方式,勿用 `npm install argo-search`(npm registry 上那份是**非官方陈旧版 v1.0.1**,不随本项目更新)。
10
+
11
+ ### 方式一:一键脚本(推荐本机长期用)
12
+
13
+ ```bash
14
+ curl -fsSL https://raw.githubusercontent.com/taxueseek/argo/main/scripts/install.sh | bash
15
+ ```
16
+
17
+ ### 方式二:npx(不加装,即用即走)
18
+
19
+ ```bash
20
+ npx -y github:taxueseek/argo
21
+ ```
22
+
23
+ ### 方式三:给 Agent 挂 MCP
24
+
25
+ Claude / Codex / Kimi 等支持 MCP 的客户端,直接挂:
26
+
27
+ ```json
28
+ {
29
+ "mcpServers": {
30
+ "argo": { "command": "npx", "args": ["-y", "github:taxueseek/argo"] }
31
+ }
32
+ }
33
+ ```
34
+
35
+ 挂上后获得 19 个工具(与 CLI 能力面对齐):`argo_search`、`argo_local_search`、`argo_local_read`、`argo_recompute`、`argo_research`、`argo_evidence`、`argo_clarify`、`argo_crawl`、`argo_fetch`、`argo_screenshot`、`argo_pdf`、`argo_social_search`、`argo_article`(公众号文章全文)、`argo_job`(招聘多平台聚合)、`argo_extract`(结构化提取)、`argo_preflight`(URL 批量预检)、`argo_answer`(直答)、`argo_watch`(网页变化监控)、`argo_cite`(DOI 引用条目)。
36
+
37
+ ### 依赖
38
+
39
+ - **必需**:Python 3.9+、`pip install pyyaml`
40
+ - 下限由语法决定,与 `bin/argo` 的 `MIN_PYTHON` / `scripts/install.sh` 保持一致(改一处就要改另外两处)。唯一越界过的 `job.py` 用了 3.12+ 的 PEP 701 f-string 写法,已按 3.9 改写,此后由跨解释器解析检查兜住。
41
+ - **可选**:`curl_cffi`(TLS 指纹,反爬站抓取更稳)、Chrome(截图)
42
+ - 只有 npx 入口才需 Node.js 18+
43
+
44
+ ---
45
+
46
+ ## 使用示例
47
+
48
+ ### 1. 搜索
49
+
50
+ ```bash
51
+ argo search "Rust async runtime tokio 性能"
52
+ argo search "python asyncio error handling" # 编程问答域 → StackOverflow 官方 API
53
+ argo search "미국 연준 금리 인하 전망" # 多语言,返回韩语结果
54
+ ```
55
+
56
+ ### 2. 证据核验(高后果问题)
57
+
58
+ 金融 / 医疗 / 法律 / 事实核查类查询,搜索结果自带证据门控:高后果问题标 `fetch_required`,每条结果标 `fetch_suggested`、`has_fetched_evidence`。
59
+
60
+ ```bash
61
+ # 对 top-3 核验正文,回填「核实后证据分」
62
+ argo search "某公司财报" --verify 3
63
+ ```
64
+
65
+ ### 3. 深度研究(多子查询取证)
66
+
67
+ 复杂问题用 `argo research`,产出 **dossier**(来源 / 覆盖 / 缺口 / 可判定检查),判断稿交给 Agent 写(事实 / 推断 / 建议 / 未知 分开)。
68
+
69
+ ```bash
70
+ argo research "半导体行业 2026 供需格局"
71
+ argo research "iPhone 16 用户口碑" --mode social-sentiment
72
+ ```
73
+
74
+ ### 4. 抓取与结构化
75
+
76
+ ```bash
77
+ argo fetch "https://example.com" --focus "关键词" # 自动降级抓取
78
+ argo pdf "https://example.com/paper.pdf" --pages "1-5"
79
+ ```
80
+
81
+ ### 5. 单代理里直接用
82
+
83
+ ```python
84
+ from scripts.search import search # 或走 MCP 工具
85
+ results = search("查一下")
86
+ ```
87
+
88
+ ---
89
+
90
+ ## 为什么是现在(2026 检索趋势)
91
+
92
+ - **Agent 检索进入证据管线时代**:交付物从链接清单变成可排序、可复核、不撑爆上下文的结构化证据
93
+ - **上下文经济**:token 是第一成本,Agent 档(约 3.7KB/次)与字节预算检查让检索不烧窗口
94
+ - **AI 友好内容标准铺开**:llms.txt 与 `.md` 直出在头部文档站普及,为 Agent 备料的网站越来越多
95
+ - **免费开放生态成熟**:政府/学术/标准/安全机构的开放 API + 免 key 引擎覆盖大多数领域
96
+ - **质量可度量**:排序金标(MRR/nDCG)、融合增益消融、路由负向控制——检索质量有数字可查
97
+
98
+ Argo v2.8.8 把这五条全部落在代码里:220 源 / 89 域 / 185 免配置。更详细的场景对照与重点源清单见 [为什么选择 Argo](为什么选择argo.md)。
99
+
100
+ ---
101
+
102
+ ## 能力边界(诚实说明)
103
+
104
+ **能力强项**
105
+
106
+ - **多语言**:中 / 英 / 日 / 韩 / 欧语(德法西意)/ 西里尔等路由,多数查询返回对应目标语言(ja/ko 已大幅修复)
107
+ - **218 搜索源、89 业务域**(184 个免配置开箱)+ TF-IDF 语义路由 + RRF 融合(weighted,弱源自动降权)
108
+ - **三条拿正文直出通道**:站点根 llms.txt 与页面 `.md` 探测(命中跳过反爬链)+ r.jina.ai 阅读器级保底
109
+ - **垂直官方 API 新成员**:StackExchange(180+ 问答站)、DOI 元数据(CSL JSON)、endoflife、NVD、法律法规库
110
+ - **垂直域**:金融行情、宏观数据、化学、学术、本地、社交、模态卡(火车票/油价/万年历等)
111
+ - **证据完整链路**:`fetch_required` / `--verify` / 证据分回填,适合高后果问题
112
+ - **兼容开源生态**:可直接挂 MCP / DSH 插件(`wide_research` 并行研究调度)
113
+
114
+ **已知限制**
115
+
116
+ - 部分高质量源(`exa`、`bocha` 等)需 API Key;无 Key 自动走免费引擎 + 本地引擎,质量略有折损
117
+ - 日 / 韩等技术类专业查询的分源质量仍有提升空间(宏观/日常已明显改善,但非完美)
118
+ - 网络不稳定时(SSL 波动)个别源会暂时失败,会熔断降级自动切备选(证据分能提示是否可下结论)
119
+ - **安装请用 GitHub 来源**;npm registry 上的 `argo-search` 是陈旧的 v1.0.1,功能残缺
120
+ - 判断稿、结论始终由 Agent 负责;`argo` 负责「取证 + 给可判定检查」,不替你下结论
121
+
122
+ ---
123
+
124
+ **License**:MIT · 仓库:https://github.com/taxueseek/argo
@@ -0,0 +1,81 @@
1
+ # 数据源缺口评估与归档(2026-08-02)
2
+
3
+ 本文件归档数据源缺口分析结论,供后续按投入产出比分批吸纳。已接入的源不重复列出,
4
+ 只记录「缺口 → 候选 → 结论」以及中文垂直源的特殊情况。
5
+
6
+ ## 一、缺口领域与候选数据源总表
7
+
8
+ | 缺口领域 | 推荐数据源 | 接入成本 | 备注 |
9
+ |---------|-----------|---------|------|
10
+ | 化学/药学结构检索 | PubChem PUG REST、ChEMBL | 零 cost,JSON 原生 | 化学结构检索是通用搜索死角 |
11
+ | 蛋白质/基因组 | UniProt、RCSB PDB | 零 cost | 23 万实验结构 + 100 万 AlphaFold 模型 |
12
+ | 医学术语统一 | UMLS/UTS | 需账号 | 200+ 术语表统一到 CUI |
13
+ | 法律判例 | CourtListener | 免费但 2026-05 后限额骤降 | 5 次/分、125 次/天;美国判例全文唯一免费路径 |
14
+ | 欧盟法律 | EUR-Lex CELLAR | 零 cost | 270 万文档 RDF |
15
+ | 专利全文/权利要求 | Espacenet OPS、Lens、USPTO PatentsView | OPS 需免费 key | Lens 1.4 亿专利 |
16
+ | 经济时间序列 | FRED、World Bank | 已接入(fred/worldbank) | FRED 正吞并 Fed 官方 DDP,是美国数据唯一 API 出口 |
17
+ | 地球科学/遥感 | NASA CMR、USGS、NOAA/NWS | 零 cost | 地震/水文/高程/天气 |
18
+ | 社科微观数据 | ICPSR | 部分受限 | 25 万+ 文件、1.6 万研究 |
19
+ | 政府开放数据 | data.gov、data.europa.eu | 零 cost | 37 万+/200 万+ 数据集 |
20
+ | 公版书籍语料 | Project Gutenberg、Internet Archive Wayback CDX | 零 cost | 7.5 万+ 公版书;CDX 可检索已删除内容 |
21
+ | 全网正文语料 | Common Crawl | 零 cost 但量大 | 单月 27 亿页/377 TiB |
22
+ | 中文社科全文 | NSSD | 无正式公开 API | 2000+ 期刊、930 万+ 篇 |
23
+ | 中文统计数据 | 国家统计局 data.stats.gov.cn | 无正式公开 API | easyquery POST 为社区方案,稳定性待验证 |
24
+ | 地理实体解析 | GeoNames | 零 cost | 1100 万+ 地名 |
25
+ | 音乐元数据 | MusicBrainz | 已在 Argo | 仅作搜索源,未做实体消歧层 |
26
+ | 通用知识图谱 | Wikidata SPARQL | 已在 Argo | 未做图查询增强 |
27
+
28
+ ## 二、优先补齐 Top 5(按投入产出比)
29
+
30
+ 1. **PubChem PUG REST + ChEMBL** —— 零 cost、JSON 原生、化学结构检索是通用搜索死角,5 分钟可集成。
31
+ 2. **ClinicalTrials.gov v2** —— 已在引擎列表(clinicaltrials),未在 SKILL.md 垂直分类中显式标注为医疗入口,建议在路由规则中强化。
32
+ 3. **FRED + World Bank** —— 经济事实核查黄金组合,已接入;FRED 正在吞并 Fed 官方 DDP。
33
+ 4. **Common Crawl + Wayback CDX** —— 全网语料与已删除内容检索,回报率最高的「搜索增强」投资。
34
+ 5. **CourtListener** —— 注意 2026-05 限额骤降(5 次/分、125 次/天),仍是美国判例全文检索唯一免费路径。
35
+
36
+ ## 三、中文垂直源现实情况
37
+
38
+ 中文区没有「小而美、免费、有正式 API」的垂直源。两个最可行入口:
39
+
40
+ - **NSSD**(社科全文):无正式公开 API 文档,接入需逆向或爬取,风险与维护成本高。
41
+ - **国家统计局 easyquery**(宏观数据):社区广泛使用但未文档化的 POST 接口,稳定性需自行验证。
42
+
43
+ 结论:两源暂不接入,归档待定。优先把已有源的查询质量做扎实(如 worldbank 国家词分流),
44
+ 而非堆砌不稳定的中文源。
45
+
46
+ ## 四、已实施改动
47
+
48
+ - **worldbank 国家词分流**(ddf60f1 后续提交):`is_foreign_macro_query` 识别非美国国家词,
49
+ fred 引擎对「中国GDP」「日本通胀」类查询返回空,route 层将 worldbank 前置,
50
+ 杜绝「搜中国给出来美国数据」的错误(此前 fred 按关键词匹配 series_id 直接返回美国序列,
51
+ 且 fast 模式 early-stop 会短路 worldbank 出场机会)。
52
+ - 修复文件:`scripts/engines_builders_data.py`、`scripts/route.py`。
53
+
54
+ ## 五、未接入源待办(按优先级)
55
+
56
+ - [x] PubChem PUG REST(chem 域,30d21da 后接入)
57
+ - [x] ChEMBL 活性数据(chem 域,并入 pubchem 引擎兜底)
58
+ - [x] Eurostat SDMX(macro_data 域,EU 国家 GDP/人均GDP/失业率/人口)
59
+ - [x] GBIF 物种检索(species 域,学名/英文俗名)
60
+ - [x] RFC Editor / IETF Datatracker(rfc_search 域)
61
+ - [ ] GDELT 2.0(news 域)—— 持续 429 限流(本机 IP 两次实测均被拒),暂缓
62
+ - [ ] Common Crawl 索引 + Wayback CDX(通用增强)
63
+ - [ ] CourtListener(legal 域,注意配额)
64
+ - [ ] GeoNames 地理消歧(geo 层增强)
65
+ - [ ] MusicBrainz 实体消歧层(music 域)
66
+ - [ ] Wikidata SPARQL 图查询增强
67
+ - [ ] ClinicalTrials.gov 路由强化(medical 域)
68
+
69
+ ## 六、新增引擎实测结论(2026-08-02)
70
+
71
+ - **PubChem**:名称查询走 `compound/name/{q}/cids` + `property`(分子式/分子量/IUPAC/SMILES);
72
+ `compound/formula` 端点是异步等待模式(返回 Waiting + ListKey),不做轮询,分子式查询由
73
+ ChEMBL 兜底(C9H8O4 → ASPIRIN/CAFFEIC ACID)。ChEMBL 需 `search.json` 后缀或
74
+ `Accept: application/json`,否则返回 XML。
75
+ - **Eurostat**:SDMX 2.1 免认证;已验证维度 GDP(nama_10_gdp/B1GQ/CP_MEUR)、
76
+ 人均GDP(nama_10_pc/CP_EUR_HAB)、失业率(une_rt_a/Y15-74/PC_ACT)、人口(demo_pjan/age=TOTAL)。
77
+ HICP 数据集(prc_hicp_manr)400 未用,通胀由 worldbank 覆盖。unit 维度 label 的 key 是编码非索引。
78
+ - **GBIF**:`/v1/species/search?q=` 对拉丁学名准确,对中文俗名返回无关属种
79
+ (「大熊猫」→ 植物属 Panda 等),引擎加 ASCII 守卫:纯中文查询直接放弃,交给百科类引擎。
80
+ - **RFC**:datatracker `name__icontains` 只匹配文档名(rfc9000 名里无 quic),
81
+ 必须用 `title__icontains` + `type__slug=rfc`;`type=rfc` 直接过滤无效(type 是 URI)。
@@ -0,0 +1,90 @@
1
+ # 设计:本地数据与网络数据的分层融合(深度研究 L1)
2
+
3
+ > 状态:P0-1 已实施(2026-08-25)。决策点:常规层 `--include-local` 默认关闭;recompute 执行器用本机受限子进程 + 审批。
4
+ > 目标:常规查询保持轻量双通道;深度研究才启用「本地数据入账 + 可复算 + 攻击验证 + 任务板」完整栈。
5
+
6
+ ## 1. 问题重定义
7
+
8
+ 上一轮审查发现 argo 深度研究只能吃 URL/文本,用户本地一手数据(CSV/XLSX/文献 PDF)无法入账,也无代码执行语义。经讨论确认:**这个缺口只属于深度研究层,常规层不需要**。
9
+
10
+ - 常规查询验收标准:快 + 来源对得上。本地检索(local-seek)与联网检索(argo_search)独立并列即可,交叉验证成本只在结论承重时值得付。
11
+ - 深度研究验收标准:结论可核查。本地一手数据与网络公开证据需要血缘合并、相互印证、数值重算。
12
+
13
+ ## 2. 分层架构
14
+
15
+ | 层 | 通道 | 本地+网络关系 | 新增组件 |
16
+ |---|---|---|---|
17
+ | L0 常规 | argo_search + argo_local_search | 独立并列,Agent 自选;可选 `--include-local`(默认关) | `search.py` 一个参数 + seek 结果并尾 |
18
+ | L1 深度 | argo_research / wide_research | 血缘合并 + 可复算 + 攻击验证 | 账本扩展 / recompute 执行器 / claim 验证 / 任务板 |
19
+
20
+ 分流规则(写进 SKILL.md):
21
+ - 单跳、来源明确 → 两个常规工具任选或都用
22
+ - 多跳、结论承重、涉本地数据文件 → argo_research + 工作包(file_inputs / recompute)
23
+ - 中间地带(如「我的笔记怎么说 + 网上怎么说」单跳对照)→ `--include-local`(默认关)
24
+
25
+ ## 3. P0-1 本地数据入账(file:// 通道)
26
+
27
+ 现状:来源账本只认 http(s) URL(wide_research 防本地注入)。
28
+
29
+ - 账本条目:`{type: file, path, sha256, mtime, role}`;工作包增 `file_inputs: [{path, kind, role}]`(kind: csv|xlsx|pdf|json|md;role: 定义|基线|数据)
30
+ - 白名单制:仅工作包显式声明的文件可入账(`--forensics-dir` 或 `file_inputs`),其余 fail-closed 拒绝
31
+ - 隐私:内容不入账,只存哈希、摘要字段与引用行号;dossier 输出 `local_sources` 章节
32
+ - 门禁挂钩:`no_primary_sources` 把本地一手文件计入一手命中(防「有原始数据却判 no sources」假阴性)
33
+
34
+ ## 4. P0-2 recompute 可复算契约
35
+
36
+ 工作包可选字段:
37
+
38
+ ```json
39
+ {
40
+ "id": "wp-rev",
41
+ "question": "公司 2025 年度营收同比增速是多少",
42
+ "query": "公司名 2025 年报 营收 同比",
43
+ "file_inputs": [{"path": "~/data/company_2025.xlsx", "role": "原始年报数据"}],
44
+ "recompute": {
45
+ "script": "读取 xlsx 营收列,计算 (2025-2024)/2024",
46
+ "expect": "0.23 左右",
47
+ "budget": {"timeout_s": 30, "max_mem_mb": 512}
48
+ }
49
+ }
50
+ ```
51
+
52
+ 执行器 `scripts/recompute.py`(新组件):
53
+ - 只读挂载 file_inputs → 本机受限子进程(决策确认:非 Docker)
54
+ - Fail-closed:默认禁运行,需 `--allow-exec` 或 MCP 参数显式放行;输入只读、断网(网络命名空间/权限不可用则跳过联网工具)、超时硬杀、无写目录
55
+ - 输出 `{exit_code, stdout, stderr, elapsed_ms}`;改产物进 dossier `recomputed_values`
56
+ - 闭环:检索数字与重算结果对照——对齐标 `recomputed=true`;冲突命中 `fact_conflicts` 门禁(结论上限 medium)
57
+ - 安全:审批制(变异操作须显式确认);执行记录进 trace
58
+
59
+ ## 5. P0-3 claim 级攻击验证(非对称验证)
60
+
61
+ - 从 dossier key_findings 提取 load-bearing claims(含数字/判断)
62
+ - 每个 claim 生成攻击子任务:验证器只见「主张 + 证据 + 交付约束」,不见生成全轨迹(防同源锚定);任务 = 独立源类三角 + 数字/日期/公式原子核验 + 反例搜索
63
+ - 输出 `claim_audit: [{claim, verdict: confirmed|contested|rejected, basis, repair_hint}]`
64
+ - contested/rejected → conclusion_cap 降级;repair_hint 直接生成下一轮工作包
65
+ - 复用现有 verify/evidence 抓取能力,新增验证提示词模板 + 调度
66
+
67
+ ## 6. P2 任务板 + 介入(协调层)
68
+
69
+ - wide_research 轨道状态机显式化:`board: [{id, objective, state: pending|active|completed|blocked|cancelled, depends_on, returned}]`,输出进 dossier
70
+ - 失败只失效后代:轨道失败 → 依赖轨道 `blocked`,已完成的独立进展保留
71
+ - 介入:研究会话执行中侧信道输入(MCP `intervene` / `--intervene-file` 指令文件,下一轮次边界注入);支持暂停/取消单轨道、补充 file_inputs、改优先级
72
+ - 预算可见:有界并发预算按轨道分配到板
73
+
74
+ ## 7. L0 常规层明确不做
75
+
76
+ 不融合重排、不加 checkpoint/resume/回放、不加 recompute、不加任务板。
77
+
78
+ 可选开关 `--include-local`(默认关):`argo_search` 把 seek 结果并入输出尾部(source=local_files),不做交叉评分,零成本扩展口。
79
+
80
+ ## 8. 实施顺序与验收
81
+
82
+ | 步 | 内容 | 验收 |
83
+ |---|---|---|
84
+ | 1 ✅ | P0-1 账本扩展 + work_packages schema | dossier 含 local_sources;白名单外文件拒绝;`no_primary_sources` 计入(已实施) |
85
+ | 2 ✅ | P0-2 recompute 执行器 + 门禁闭环 | 只读/断网/超时/审批四组测试;recompute_skipped / recompute_conflict 门禁触发测试(已实施) |
86
+ | 3 | P0-3 claim 验证模板 + 调度 | claim_audit 三态判定测试 |
87
+ | 4 | P2 任务板 + intervene | 轨道状态机测试;失败仅后代 blocked |
88
+ | 5 | --include-local(默认关) | 开关默认不改变现有输出 |
89
+
90
+ 与「深度研究不写判断稿」一致:recompute 只产数值事实,判断仍归 Agent。