@researai/deepscientist 1.5.16 → 1.6.0

package/AISB/catalog/aisb.t3.080_latentscorereweight.zh.yaml

@@ -0,0 +1,123 @@
+schema_version: 1
+id: aisb.t3.080_latentscorereweight
+name: Latent Score-Based Reweighting for Robust Classification on Imbalanced Tabular Data
+version: 0.1.0
+one_line: 在分布偏移下用于稳健表格分类的基于分数的密度重加权方法，无需群组标签。
+task_description: 该基准面向稳健表格分类：利用 score-based（扩散）模型的潜在分数信号识别并上调联合分布 P(X, Y) 中代表性不足区域。不同于 JTT 等只在决策边界 P(Y|X) 附近上调误分类样本的方法，本方法通过与邻近样本的分数向量相似性估计局部密度，并重加权低密度区域，从而得到更全局平衡的训练分布，且不需要群组标签先验。评测目标为分布偏移下的 worst-group accuracy 与 mean accuracy。当前本地快照不含可执行训练/评估代码，两项主指标均为 blocked-no-code，执行前必须先恢复源码。
+task_mode: analysis_driven
+requires_execution: true
+requires_paper: true
+integrity_level: cas_plus_canary
+snapshot_status: restore_needed
+support_level: recovery
+time_band: 6-24h
+cost_band: medium
+difficulty: medium
+data_access: public
+capability_tags:
+- robust_classification
+- tabular_ml
+- score_based_models
+- distribution_shift
+- data_reweighting
+- imbalanced_data
+aisb_direction: T3
+track_fit:
+- paper_track
+- benchmark_track
+primary_outputs:
+- worst_group_accuracy
+- mean_accuracy
+launch_profiles:
+- id: analysis_only
+  label: Analysis Only
+  description: 'Review preserved benchmark metadata, metric contracts, and paper artefacts. No executable code is present in the current snapshot; this profile supports planning and paper-backed analysis only.
+
+    '
+- id: restore_first
+  label: Restore First
+  description: 'Restore the missing latent-score reweighting implementation from the upstream repository (https://github.com/YunzeTong/latent-score-based-reweighting) before treating this entry as runnable. Both metrics remain blocked until restoration.
+
+    '
+dataset_download:
+  primary_method: mixed
+  sources: []
+  notes:
+  - 'The current local snapshot does not preserve an executable benchmark route. Dataset setup should be treated as part of upstream restoration rather than a bundled run path. Refer to the paper and upstream GitHub repository for dataset identities and loading instructions.
+
+    '
+credential_requirements:
+  mode: none
+  items: []
+  notes: []
+resources:
+  minimum:
+    cpu_cores: 8
+    ram_gb: 32
+    disk_gb: 80
+    gpu_count: 1
+    gpu_vram_gb: 16
+  recommended:
+    cpu_cores: 16
+    ram_gb: 64
+    disk_gb: 150
+    gpu_count: 1
+    gpu_vram_gb: 24
+environment:
+  python: null
+  cuda: null
+  pytorch: null
+  flash_attn: null
+  key_packages: []
+  notes:
+  - 'The current local snapshot does not preserve executable setup files or runnable source code. Environment requirements should be treated as provisional until upstream restoration from https://github.com/YunzeTong/latent-score-based-reweighting is complete.
+
+    '
+  - 'Expected dependencies include a score-based or diffusion model library, standard tabular ML packages (e.g. scikit-learn, XGBoost), and a PyTorch backend for the generative density estimation stage. Confirm exact versions from the upstream repository''s requirements file after restoration.
+
+    '
+risk_flags:
+- source_snapshot_incomplete
+- route_caveat
+- metrics_blocked_no_code
+risk_notes:
+- 'Both primary metrics (worst_group_accuracy, mean_accuracy) are marked blocked-no-code in the metric contract. No executable code anchor was detected in the staged snapshot. Do not report metric values until trusted execution outputs exist from a restored environment.
+
+  '
+- 'The top-level README is absent from the current snapshot. Do not rely on README-only interpretation; prefer the paper (latex.md) and metric contract (json/metric_contract.json) as authoritative references.
+
+  '
+recommended_when: 当你需要基于论文进行方案规划、方法分析，或为 latent score-based reweighting 做上游恢复范围界定时可使用本条目。也适合在设计/综述阶段比较密度重加权与边界重加权（如 JTT）。
+not_recommended_when: 不要将当前本地包作为可直接运行的稳健性基准。当前分阶段快照缺少可执行生成式重加权流程，且两项指标均被阻塞。任何结果都应来自完整恢复并验证后的执行环境。
+paper:
+  title: Latent Score-Based Reweighting for Robust Classification on Imbalanced Tabular Data
+  authors:
+  - Yunze Tong
+  - Fengda Zhang
+  - Zihao Tang
+  - Kaifeng Gao
+  - Kai Huang
+  - Pengfei Lyu
+  - Jun Xiao
+  - Kun Kuang
+  venue: ICML 2025
+  year: 2025
+  url: https://proceedings.mlr.press/v267/tong25c.html
+  code_url: https://github.com/YunzeTong/latent-score-based-reweighting
+download:
+  url: https://github.com/ResearAI/DeepScientist/releases/download/aisb-v0.0.1/aisb.t3.080_latentscorereweight.zip
+  archive_type: zip
+  local_dir_name: paper-80-LatentScoreReweight
+  provider: github_release
+  repo: ResearAI/DeepScientist
+  tag: aisb-v0.0.1
+  asset_name: aisb.t3.080_latentscorereweight.zip
+  sha256: 9c70bcc4a57a17ffc712a81763731ba5c5f46c987ca83e9b5cd0d730a18e0d03
+  size_bytes: 36629
+commercial:
+  annual_fee: null
+display:
+  palette_seed: copper-teal-reweight
+  art_style: robust-tabular
+  accent_priority: high
+image_path: ../image/080_aisb.t3.080_latentscorereweight.jpg

package/AISB/catalog/aisb.t3.081_qmamba.yaml

@@ -0,0 +1,150 @@
+schema_version: 1
+id: aisb.t3.081_qmamba
+name: Meta-Black-Box-Optimization through Offline Q-function Learning
+version: 0.1.0
+one_line: Offline meta-learning framework for black-box optimization that uses Q-function
+  decomposition and Mamba architecture over long configuration trajectories.
+task_description: 'This benchmark entry covers Q-Mamba, an offline meta-black-box
+  optimization framework that addresses the effectiveness-efficiency dilemma in MetaBBO.
+  The approach transforms dynamic algorithm configuration (DAC) into a long-sequence
+  decision process with decomposed Q-functions for each hyper-parameter dimension.
+  Key components include: (1) offline data collection from strong baselines and random
+  policies for balanced exploration/exploitation, (2) conservative Q-learning with
+  compositional Q-loss to handle distribution shift, and (3) a Mamba-based RL agent
+  leveraging selective state modeling and hardware-aware parallel scan for efficient
+  long-sequence training. The current local snapshot preserves metadata and paper
+  references but lacks runnable implementation code, requiring upstream restoration
+  before execution.
+
+  '
+task_mode: analysis_driven
+requires_execution: true
+requires_paper: true
+integrity_level: cas_plus_canary
+snapshot_status: restore_needed
+support_level: recovery
+time_band: 6-24h
+cost_band: medium
+difficulty: hard
+data_access: public
+primary_outputs:
+- mean_reward
+launch_profiles:
+- id: analysis_only
+  label: Analysis Only
+  description: 'Review the preserved benchmark metadata, paper assets, and offline
+    Q-learning route documentation without attempting execution.
+
+    '
+- id: restore_first
+  label: Restore First
+  description: 'Restore the missing Q-Mamba implementation from upstream sources before
+    treating this benchmark as runnable. Requires fetching executable code from the
+    published repository.
+
+    '
+dataset_download:
+  primary_method: pending_restoration
+  sources: []
+  notes:
+  - The current local package does not bundle executable benchmark code or runnable
+    evaluation data.
+  - Dataset and implementation must be restored from upstream sources referenced in
+    the paper.
+  - The benchmark route is not available in the current snapshot state.
+credential_requirements:
+  mode: none
+  items: []
+  notes: []
+resources:
+  minimum:
+    cpu_cores: 8
+    ram_gb: 32
+    disk_gb: 80
+    gpu_count: 1
+    gpu_vram_gb: 16
+  recommended:
+    cpu_cores: 16
+    ram_gb: 64
+    disk_gb: 150
+    gpu_count: 1
+    gpu_vram_gb: 24
+environment:
+  python: '3.10'
+  cuda: '12.1'
+  pytorch: 2.1.1
+  flash_attn: null
+  key_packages:
+  - mamba-ssm>=1.2.0
+  notes:
+  - Environment specifications are provisional based on the paper and upstream restoration
+    requirements.
+  - The current staged snapshot does not preserve executable setup files or runnable
+    source code.
+  - Full dependency set should be obtained from the published Q-Mamba source repository.
+  - mamba-ssm is the core architectural dependency for the Mamba-based RL agent.
+risk_flags:
+- source_snapshot_incomplete
+- route_caveat
+- no_executable_anchor
+risk_notes:
+- No executable code files detected in the current snapshot that can anchor staged
+  metrics.
+- No benchmark execution was performed during this packaging pass.
+- The primary metric mean_reward is blocked pending source restoration.
+- Treat this entry as blocked until upstream restoration provides runnable implementation.
+recommended_when: 'Use this entry when you need paper-backed analysis of the Q-Mamba
+  offline meta-optimization approach, or when planning upstream restoration of the
+  Mamba-based meta-learning route for black-box optimization. Suitable for understanding
+  Q-function decomposition in long-horizon configuration tasks.
+
+  '
+not_recommended_when: 'Do not treat this local package as a ready-to-run benchmark
+  for offline meta-black-box optimization. The executable code, evaluation protocols,
+  and benchmark data are absent from the staged snapshot. Do not attempt sequential
+  optimization runs without first restoring the missing implementation.
+
+  '
+paper:
+  title: Meta-Black-Box-Optimization through Offline Q-function Learning
+  authors: Zeyuan Ma, Zhiguang Cao, Zhou Jiang, Hongshu Guo, Yue-Jiao Gong
+  venue: ICML 2025
+  year: 2025
+  url: https://proceedings.mlr.press/v267/ma25b.html
+  notes:
+  - Proposes Q-Mamba framework for offline MetaBBO with Q-function decomposition.
+  - Addresses effectiveness-efficiency trade-off in meta-learning for dynamic algorithm
+    configuration.
+  - Uses Mamba architecture for long-sequence learning with parallel scan efficiency.
+  - Source codes available online per the paper.
+download:
+  url: https://github.com/ResearAI/DeepScientist/releases/download/aisb-v0.0.1/aisb.t3.081_qmamba.zip
+  archive_type: zip
+  local_dir_name: paper-81-QMamba
+  notes:
+  - Archive contains metadata and documentation; executable code requires restoration
+    from upstream.
+  provider: github_release
+  repo: ResearAI/DeepScientist
+  tag: aisb-v0.0.1
+  asset_name: aisb.t3.081_qmamba.zip
+  sha256: f565acdff164658ee030066f9a6c81cb7685ad3616de081df7c56a837b33cfd4
+  size_bytes: 31451
+display:
+  palette_seed: slate-gold-offline
+  art_style: trajectory-systems
+  accent_priority: high
+image_path: ../image/081_aisb.t3.081_qmamba.jpg
+capability_tags:
+- research_code_optimization
+- black_box_optimization
+- offline_rl
+- mamba_models
+- meta_learning
+- q_function_decomposition
+- dynamic_algorithm_configuration
+- long_sequence_decision
+aisb_direction: T3
+track_fit:
+- paper_track
+- benchmark_track

package/AISB/catalog/aisb.t3.081_qmamba.zh.yaml

@@ -0,0 +1,117 @@
+schema_version: 1
+id: aisb.t3.081_qmamba
+name: Meta-Black-Box-Optimization through Offline Q-function Learning
+version: 0.1.0
+one_line: 面向黑盒优化的离线元学习框架，使用 Q 函数分解与 Mamba 架构处理长配置轨迹。
+task_description: 该条目覆盖 Q-Mamba：一种离线元黑盒优化框架，用于缓解 MetaBBO 的效果-效率矛盾。方法把动态算法配置（DAC）转化为长序列决策过程，并为每个超参数维度分解 Q 函数。核心包括：(1) 结合强基线与随机策略的离线数据采集以平衡探索与利用；(2) 通过组合式 Q-loss 的保守 Q-learning 处理分布偏移；(3) 使用具备选择性状态建模与硬件友好并行扫描的 Mamba RL 代理进行高效长序列训练。当前本地快照保留元数据与论文引用，但缺少可运行代码，需上游恢复后执行。
+task_mode: analysis_driven
+requires_execution: true
+requires_paper: true
+integrity_level: cas_plus_canary
+snapshot_status: restore_needed
+support_level: recovery
+time_band: 6-24h
+cost_band: medium
+difficulty: hard
+data_access: public
+primary_outputs:
+- mean_reward
+launch_profiles:
+- id: analysis_only
+  label: Analysis Only
+  description: 'Review the preserved benchmark metadata, paper assets, and offline Q-learning route documentation without attempting execution.
+
+    '
+- id: restore_first
+  label: Restore First
+  description: 'Restore the missing Q-Mamba implementation from upstream sources before treating this benchmark as runnable. Requires fetching executable code from the published repository.
+
+    '
+dataset_download:
+  primary_method: pending_restoration
+  sources: []
+  notes:
+  - The current local package does not bundle executable benchmark code or runnable evaluation data.
+  - Dataset and implementation must be restored from upstream sources referenced in the paper.
+  - The benchmark route is not available in the current snapshot state.
+credential_requirements:
+  mode: none
+  items: []
+  notes: []
+resources:
+  minimum:
+    cpu_cores: 8
+    ram_gb: 32
+    disk_gb: 80
+    gpu_count: 1
+    gpu_vram_gb: 16
+  recommended:
+    cpu_cores: 16
+    ram_gb: 64
+    disk_gb: 150
+    gpu_count: 1
+    gpu_vram_gb: 24
+environment:
+  python: '3.10'
+  cuda: '12.1'
+  pytorch: 2.1.1
+  flash_attn: null
+  key_packages:
+  - mamba-ssm>=1.2.0
+  notes:
+  - Environment specifications are provisional based on the paper and upstream restoration requirements.
+  - The current staged snapshot does not preserve executable setup files or runnable source code.
+  - Full dependency set should be obtained from the published Q-Mamba source repository.
+  - mamba-ssm is the core architectural dependency for the Mamba-based RL agent.
+risk_flags:
+- source_snapshot_incomplete
+- route_caveat
+- no_executable_anchor
+risk_notes:
+- No executable code files detected in the current snapshot that can anchor staged metrics.
+- No benchmark execution was performed during this packaging pass.
+- The primary metric mean_reward is blocked pending source restoration.
+- Treat this entry as blocked until upstream restoration provides runnable implementation.
+recommended_when: 当你需要对 Q-Mamba 离线元优化方法进行基于论文的分析，或规划恢复 Mamba 元学习黑盒优化路线时可用本条目。也适合理解长时程配置任务中的 Q 函数分解。
+not_recommended_when: 不要把当前本地包视为可直接运行的离线元黑盒优化基准。分阶段快照缺少可执行代码、评估协议和基准数据，恢复实现前不应尝试顺序优化运行。
+paper:
+  title: Meta-Black-Box-Optimization through Offline Q-function Learning
+  authors: Zeyuan Ma, Zhiguang Cao, Zhou Jiang, Hongshu Guo, Yue-Jiao Gong
+  venue: ICML 2025
+  year: 2025
+  url: https://proceedings.mlr.press/v267/ma25b.html
+  notes:
+  - Proposes Q-Mamba framework for offline MetaBBO with Q-function decomposition.
+  - Addresses effectiveness-efficiency trade-off in meta-learning for dynamic algorithm configuration.
+  - Uses Mamba architecture for long-sequence learning with parallel scan efficiency.
+  - Source codes available online per the paper.
+download:
+  url: https://github.com/ResearAI/DeepScientist/releases/download/aisb-v0.0.1/aisb.t3.081_qmamba.zip
+  archive_type: zip
+  local_dir_name: paper-81-QMamba
+  notes:
+  - Archive contains metadata and documentation; executable code requires restoration from upstream.
+  provider: github_release
+  repo: ResearAI/DeepScientist
+  tag: aisb-v0.0.1
+  asset_name: aisb.t3.081_qmamba.zip
+  sha256: f565acdff164658ee030066f9a6c81cb7685ad3616de081df7c56a837b33cfd4
+  size_bytes: 31451
+display:
+  palette_seed: slate-gold-offline
+  art_style: trajectory-systems
+  accent_priority: high
+image_path: ../image/081_aisb.t3.081_qmamba.jpg
+capability_tags:
+- research_code_optimization
+- black_box_optimization
+- offline_rl
+- mamba_models
+- meta_learning
+- q_function_decomposition
+- dynamic_algorithm_configuration
+- long_sequence_decision
+aisb_direction: T3
+track_fit:
+- paper_track
+- benchmark_track

package/AISB/catalog/aisb.t3.082_onlinellmrouting.yaml

@@ -0,0 +1,160 @@
+id: aisb.t3.082_onlinellmrouting
+name: Online LLM Routing
+version: 0.1.0
+one_line: Training-free online router allocating prompts across multiple LLMs under
+  token budget constraints using ANNS and one-time optimization.
+task_description: 'This benchmark implements a training-free online routing algorithm
+  for high-volume multi-LLM serving under constrained token budgets. The system uses
+  Approximate Nearest Neighbor Search (ANNS) to efficiently estimate query features
+  for each deployed LLM from a historical auxiliary dataset, then performs a one-time
+  Mixed-Integer Linear Programming (MILP) optimization over a small sample set of
+  observed queries (typically ~250) to learn routing weights. These weights guide
+  routing decisions for subsequent queries arriving sequentially. The approach requires
+  no model training and introduces negligible deployment overhead, making it adaptable
+  to dynamic LLM deployment configurations. Theoretical analysis guarantees a competitive
+  ratio of 1 - o(1) under mild assumptions.
+
+  '
+task_mode: evaluation_driven
+requires_execution: true
+requires_paper: true
+integrity_level: cas_plus_canary
+snapshot_status: restore_needed
+support_level: recovery
+time_band: 1-2h
+cost_band: low
+difficulty: medium
+data_access: public
+primary_outputs:
+- quality_score
+- routing_decisions
+- cost_efficiency_metrics
+- throughput_metrics
+launch_profiles:
+- id: analysis_only
+  label: Analysis Only
+  description: Review preserved routing benchmark metadata, target objective function,
+    and MILP formulation. No execution performed.
+- id: restore_first
+  label: Restore First
+  description: Restore missing online routing implementation and ANNS components before
+    treating this benchmark as runnable. Current snapshot lacks executable code anchors.
+dataset_download:
+  primary_method: mixed
+  sources:
+  - type: external_archive
+    url: https://deepscientist.cc/AISB/082_onlinellmrouting
+    archive_type: zip
+    local_dir_name: paper-82-OnlineLLMRouting
+  - type: github_repository
+    url: https://github.com/fzwark/PORT
+    note: Original implementation source referenced in paper
+  notes:
+  - Archive contains bundled README and requirements for full dependency set
+  - Benchmark datasets referenced in paper not included in snapshot
+credential_requirements:
+  mode: none
+  items: []
+  notes:
+  - No API credentials required for CPU-only routing evaluation
+  - LLM inference APIs not required for the core routing algorithm evaluation
+resources:
+  minimum:
+    cpu_cores: 8
+    ram_gb: 16
+    disk_gb: 30
+    gpu_count: 0
+    gpu_vram_gb: 0
+  recommended:
+    cpu_cores: 16
+    ram_gb: 32
+    disk_gb: 80
+    gpu_count: 0
+    gpu_vram_gb: 0
+  notes:
+  - CPU-only execution is fully viable for the minimum routing implementation
+  - ANNS operations (FAISS) are CPU-optimized with faiss_cpu package
+  - No GPU required for MILP solver or routing weight optimization
+  - Memory requirements scale with auxiliary dataset size for ANNS indexing
+environment:
+  python: '3.10'
+  cuda: null
+  pytorch: 2.6.0
+  flash_attn: null
+  key_packages:
+  - faiss_cpu==1.10.0
+  - transformers==4.51.0
+  notes:
+  - faiss_cpu provides ANNS index for query feature estimation
+  - transformers used for embedding generation if auxiliary dataset requires
+  - Full dependency list provided in bundled requirements.txt
+risk_flags:
+- snapshot_incomplete
+- blocked_no_code
+risk_notes:
+- Current snapshot does not expose executable code files anchoring staged metrics
+- No benchmark execution was performed in this packaging pass
+- Treat as blocked until source restoration or richer local snapshot is recovered
+- Top-level README absent in current snapshot
+recommended_when: 'Use this benchmark for systems-routing research focused on retrieval,
+  optimization, and budget allocation under token constraints. Ideal for evaluating
+  training-free approaches to multi-LLM serving that require rapid routing decisions
+  without model retraining.
+
+  '
+not_recommended_when: 'Do not use this benchmark if you require local GPU finetuning
+  of language models, offline ensemble methods with multiple model calls per query,
+  or tasks requiring model-based predictors that need training overhead.
+
+  '
+paper:
+  title: Efficient Training-Free Online Routing for High-Volume Multi-LLM Serving
+  venue: NeurIPS 2025
+  year: 2025
+  url: https://openreview.net/forum?id=d4mZyZB5I9
+  authors:
+  - Fangzhou Wu
+  - Sandeep Silwal
+  institution: University of Wisconsin-Madison
+display:
+  palette_seed: silver-blue-router
+  art_style: serving-console
+  accent_priority: high
+image_path: ../image/082_aisb.t3.082_onlinellmrouting.jpg
+capability_tags:
+- research_code_optimization
+- llm_routing
+- approximate_nearest_neighbors
+- serving_systems
+- online_optimization
+- milp
+- budget_allocation
+track_fit:
+- paper_track
+- benchmark_track
+aisb_direction: T3
+metric_contract:
+  primary_metric: quality_score
+  status: blocked_no_code
+  notes:
+  - No executable code anchor in current snapshot
+  - Metric surface blocked until routing implementation is restored
+  - Original paper reports 3.55x overall performance improvement and 1.85x cost efficiency
+    against 8 baselines
+commercial:
+  annual_fee: null
+recovery_rule: 'Prefer existing code files over README-only claims when choosing origin_path,
+  source_ref, and evaluation_protocol.code_paths. Keep metrics_summary empty until
+  trusted execution outputs exist. If the current snapshot has no executable anchor
+  for a metric, mark that metric as provisional or blocked instead of inventing support.
+
+  '
+download:
+  provider: github_release
+  repo: ResearAI/DeepScientist
+  tag: aisb-v0.0.1
+  asset_name: aisb.t3.082_onlinellmrouting.zip
+  url: https://github.com/ResearAI/DeepScientist/releases/download/aisb-v0.0.1/aisb.t3.082_onlinellmrouting.zip
+  archive_type: zip
+  sha256: 60d8ccca2c47273b6fa1cd022d1625bd395bd8358d3ef7f79234036d9b1d8a04
+  size_bytes: 54262

package/AISB/catalog/aisb.t3.082_onlinellmrouting.zh.yaml

@@ -0,0 +1,133 @@
+id: aisb.t3.082_onlinellmrouting
+name: Online LLM Routing
+version: 0.1.0
+one_line: 无需训练的在线路由器：在 token 预算约束下，结合 ANNS 与一次性优化在多个 LLM 间分配请求。
+task_description: 该基准实现了面向高吞吐多 LLM 服务的无训练在线路由算法。系统先用近似最近邻搜索（ANNS）从历史辅助数据中高效估计每个已部署 LLM 的查询特征，再在少量已观测查询样本（通常约 250）上进行一次混合整数线性规划（MILP）以学习路由权重，随后用于后续顺序到达查询的路由决策。该方法无需模型训练，部署额外开销极低，适配动态 LLM 部署配置。理论上在温和假设下可保证 1 - o(1) 的竞争比。
+task_mode: evaluation_driven
+requires_execution: true
+requires_paper: true
+integrity_level: cas_plus_canary
+snapshot_status: restore_needed
+support_level: recovery
+time_band: 1-2h
+cost_band: low
+difficulty: medium
+data_access: public
+primary_outputs:
+- quality_score
+- routing_decisions
+- cost_efficiency_metrics
+- throughput_metrics
+launch_profiles:
+- id: analysis_only
+  label: Analysis Only
+  description: Review preserved routing benchmark metadata, target objective function, and MILP formulation. No execution performed.
+- id: restore_first
+  label: Restore First
+  description: Restore missing online routing implementation and ANNS components before treating this benchmark as runnable. Current snapshot lacks executable code anchors.
+dataset_download:
+  primary_method: mixed
+  sources:
+  - type: external_archive
+    url: https://deepscientist.cc/AISB/082_onlinellmrouting
+    archive_type: zip
+    local_dir_name: paper-82-OnlineLLMRouting
+  - type: github_repository
+    url: https://github.com/fzwark/PORT
+    note: Original implementation source referenced in paper
+  notes:
+  - Archive contains bundled README and requirements for full dependency set
+  - Benchmark datasets referenced in paper not included in snapshot
+credential_requirements:
+  mode: none
+  items: []
+  notes:
+  - No API credentials required for CPU-only routing evaluation
+  - LLM inference APIs not required for the core routing algorithm evaluation
+resources:
+  minimum:
+    cpu_cores: 8
+    ram_gb: 16
+    disk_gb: 30
+    gpu_count: 0
+    gpu_vram_gb: 0
+  recommended:
+    cpu_cores: 16
+    ram_gb: 32
+    disk_gb: 80
+    gpu_count: 0
+    gpu_vram_gb: 0
+  notes:
+  - CPU-only execution is fully viable for the minimum routing implementation
+  - ANNS operations (FAISS) are CPU-optimized with faiss_cpu package
+  - No GPU required for MILP solver or routing weight optimization
+  - Memory requirements scale with auxiliary dataset size for ANNS indexing
+environment:
+  python: '3.10'
+  cuda: null
+  pytorch: 2.6.0
+  flash_attn: null
+  key_packages:
+  - faiss_cpu==1.10.0
+  - transformers==4.51.0
+  notes:
+  - faiss_cpu provides ANNS index for query feature estimation
+  - transformers used for embedding generation if auxiliary dataset requires
+  - Full dependency list provided in bundled requirements.txt
+risk_flags:
+- snapshot_incomplete
+- blocked_no_code
+risk_notes:
+- Current snapshot does not expose executable code files anchoring staged metrics
+- No benchmark execution was performed in this packaging pass
+- Treat as blocked until source restoration or richer local snapshot is recovered
+- Top-level README absent in current snapshot
+recommended_when: 当你研究在 token 预算约束下的检索、优化与预算分配型路由系统时使用本基准。适合评估无需重训练即可快速决策的多 LLM 服务路由方案。
+not_recommended_when: 如果你需要本地 GPU 微调语言模型、每查询多模型离线集成，或依赖需训练预测器的任务，不建议使用。
+paper:
+  title: Efficient Training-Free Online Routing for High-Volume Multi-LLM Serving
+  venue: NeurIPS 2025
+  year: 2025
+  url: https://openreview.net/forum?id=d4mZyZB5I9
+  authors:
+  - Fangzhou Wu
+  - Sandeep Silwal
+  institution: University of Wisconsin-Madison
+display:
+  palette_seed: silver-blue-router
+  art_style: serving-console
+  accent_priority: high
+image_path: ../image/082_aisb.t3.082_onlinellmrouting.jpg
+capability_tags:
+- research_code_optimization
+- llm_routing
+- approximate_nearest_neighbors
+- serving_systems
+- online_optimization
+- milp
+- budget_allocation
+track_fit:
+- paper_track
+- benchmark_track
+aisb_direction: T3
+metric_contract:
+  primary_metric: quality_score
+  status: blocked_no_code
+  notes:
+  - No executable code anchor in current snapshot
+  - Metric surface blocked until routing implementation is restored
+  - Original paper reports 3.55x overall performance improvement and 1.85x cost efficiency against 8 baselines
+commercial:
+  annual_fee: null
+recovery_rule: 'Prefer existing code files over README-only claims when choosing origin_path, source_ref, and evaluation_protocol.code_paths. Keep metrics_summary empty until trusted execution outputs exist. If the current snapshot has no executable anchor for a metric, mark that metric as provisional or blocked instead of inventing support.
+
+  '
+download:
+  provider: github_release
+  repo: ResearAI/DeepScientist
+  tag: aisb-v0.0.1
+  asset_name: aisb.t3.082_onlinellmrouting.zip
+  url: https://github.com/ResearAI/DeepScientist/releases/download/aisb-v0.0.1/aisb.t3.082_onlinellmrouting.zip
+  archive_type: zip
+  sha256: 60d8ccca2c47273b6fa1cd022d1625bd395bd8358d3ef7f79234036d9b1d8a04
+  size_bytes: 54262