@researai/deepscientist 1.5.16 → 1.6.0

package/AISB/catalog/aisb.t3.032_ptsolver.zh.yaml

@@ -0,0 +1,135 @@
+schema_version: 1
+id: aisb.t3.032_ptsolver
+name: '个人旅行求解器：RealTravel 数据集'
+version: 0.1.0
+one_line: 'RealTravel 数据集（77 个美国城市，1000 个测试样本 + 155 个验证样本）的纯数据快照，包含 LLM 驱动的用户偏好提取脚本；完整的 PTS 规划器和 SCIP 求解器未包含在内。
+
+'
+task_description: '该基准测试打包了 RealTravel 数据集——这是 TravelPlanner 基准测试的扩展，包含了来自 Google Local 的真实用户评论和 POI 元数据，覆盖 77 个美国城市。快照包含结构化旅行查询、用户评论历史、POI 数据库（餐厅、景点、住宿）以及四个使用 LLM API（GPT-4o / DeepSeek）的 Python 数据处理脚本，用于提取用户偏好、汇总用户画像、标准化偏好标签，以及生成景点利弊描述。完整的个人旅行求解器（PTS）系统——包括转换器、搜索、重排序（SASRec）和规划（SCIP 求解器）模块——未包含在此快照中。主要评估指标（pass_rate）被阻塞，因为没有可执行的评估或规划代码。若要将此基准测试作为可运行的评估工具，需要从论文的外部代码发布中恢复下游 PTS 规划器。
+
+'
+capability_tags:
+- research_code_optimization
+- travel_planning
+- llm_tooling
+- constraint_solving
+- personalization
+aisb_direction: T3
+track_fit:
+- paper_track
+- benchmark_track
+task_mode: evaluation_driven
+requires_execution: true
+requires_paper: true
+integrity_level: cas_plus_canary
+snapshot_status: data_only
+support_level: recovery
+cost_band: low
+time_band: 1-2h
+difficulty: medium
+data_access: public
+primary_outputs:
+- user_profiles
+- poi_summaries
+- structured_travel_data
+- preference_tags
+launch_profiles:
+- id: data_prep
+  label: 数据准备
+  description: '运行四个捆绑的数据处理脚本，提取用户偏好、汇总用户画像、标准化标签，并生成景点利弊描述。需要 LLM API 凭证（OpenAI 或 DeepSeek）。无需 GPU。
+
+'
+- id: planner_restore
+  label: 规划器恢复
+  description: '从论文的外部代码发布中恢复完整的 PTS 流程（转换器、搜索、SASRec 重排序、SCIP 求解器规划），然后才能将此基准测试作为可运行的端到端评估工具。
+
+'
+dataset_download:
+  primary_method: bundled
+  sources:
+  - kind: archive
+    url: https://deepscientist.cc/AISB/032_ptsolver
+    access: public
+    note: 'ZIP 压缩包，包含 RealTravel 数据集（1000 个测试样本 / 155 个验证样本）、77 个城市的 POI 数据库、用户评论数据及数据处理脚本。
+
+'
+  notes:
+  - 数据集来源于 Google Local（Yan et al., 2023）和 TravelPlanner（Xie et al., 2024）。
+  - 数据库子目录包含住宿、景点、餐饮和背景数据。
+  - 总磁盘占用量较小（解压后低于 20 GB）。
+credential_requirements:
+  mode: api_keys
+  items:
+  - OpenAI API 密钥（GPT-4o）或 DeepSeek API 密钥，用于运行数据处理脚本
+  - 可选：attraction-pro-con.py 中引用的 ChatGLM / SiliconFlow / Yi API 密钥
+  notes:
+  - 脚本从环境变量（Gpt_API_KEY、DEEPSEEK_API_KEY、CHATGLM_API_KEY）读取密钥。
+  - 部分源代码中存在硬编码的 API 密钥，可能已过期或无效。
+  - 仅检查预存在的数据集文件时无需任何凭据。
+resources:
+  minimum:
+    cpu_cores: 8
+    ram_gb: 16
+    disk_gb: 20
+    gpu_count: 0
+    gpu_vram_gb: 0
+  recommended:
+    cpu_cores: 16
+    ram_gb: 32
+    disk_gb: 50
+    gpu_count: 0
+    gpu_vram_gb: 0
+environment:
+  python: '3.10'
+  cuda: null
+  pytorch: null
+  flash_attn: null
+  key_packages:
+  - langchain
+  - openai
+  - tqdm
+  notes:
+  - 所有捆绑脚本均可仅用 CPU 运行。
+  - 数据处理脚本使用 langchain ChatOpenAI 和 openai 客户端库。
+  - 完整 PTS 系统（未捆绑）还需 PySCIPOpt、SASRec、BGE 嵌入和 scikit-learn（用于 PCA）。
+  - 完整的依赖项清单请参阅捆绑的 requirements 文件。
+risk_flags:
+- blocked_metric
+- incomplete_pipeline
+- api_key_exposure
+- external_code_dependency
+risk_notes:
+- 主要评估指标（pass_rate）被阻塞——快照中没有评估或规划代码。
+- 完整的 PTS 系统（5 个模块）未捆绑；仅包含数据处理脚本。
+- attraction-pro-con.py 中存在硬编码的 API 密钥；这些可能是泄露的凭据，不应重复使用。
+  - 运行数据处理脚本会产生 LLM API 费用，成本与处理的用户/POI 数量成正比。
+  - 重排序模块（SASRec + BGE 嵌入）和规划模块（SCIP 求解器）必须从外部恢复。
+recommended_when: '当您需要一个基于真实用户评论和 POI 数据的逼真旅行规划数据集，用于混合符号约束满足与用户偏好建模的任务，且无需进行繁重的 GPU 训练时，可以使用此基准测试。适用于评估基于 LLM 的偏好提取、用户画像分析，或作为构建约束型旅行规划器的数据基础。
+
+'
+not_recommended_when: '如果需要完全自包含、端到端可运行的基准测试，请勿使用此基准测试。规划和评估流程未捆绑。此外，不适用于专注于大规模模型微调、多模态数据或非美国旅行目的地的基准测试。
+
+'
+paper:
+  title: 'Personal Travel Solver: A Preference-Driven LLM-Solver System for Travel
+    Planning'
+  venue: ACL 2025
+  year: 2025
+  url: https://aclanthology.org/2025.acl-long.1339/
+download:
+  url: https://github.com/ResearAI/DeepScientist/releases/download/aisb-v0.0.1/aisb.t3.032_ptsolver.zip
+  archive_type: zip
+  local_dir_name: paper-32-PTSolver
+  provider: github_release
+  repo: ResearAI/DeepScientist
+  tag: aisb-v0.0.1
+  asset_name: aisb.t3.032_ptsolver.zip
+  sha256: 26f7f39e12eb28552ada092809b289f77090a229d0b745e9dfbcbb7b7b4f9d5c
+  size_bytes: 38141816
+commercial:
+  annual_fee: null
+display:
+  palette_seed: sand-teal-itinerary
+  art_style: trip-planner
+  accent_priority: medium
+image_path: ../image/032_aisb.t3.032_ptsolver.jpg

package/AISB/catalog/aisb.t3.033_gcse.yaml

@@ -0,0 +1,144 @@
+id: aisb.t3.033_gcse
+name: GCSE
+version: 0.1.0
+one_line: Unsupervised sentence embedding optimization using knowledge graph-guided
+  augmentation and Gaussian-decayed contrastive learning for noise-aware training.
+task_description: 'This packaged benchmark covers unsupervised sentence embedding
+  training via the Gaussian-decayed gradient-assisted Contrastive Sentence Embedding
+  (GCSE) model. The method addresses two primary challenges in unsupervised sentence
+  representation: low data diversity and high data noise. The pipeline first extracts
+  fine-grained knowledge (entities, quantities) from source data to construct a knowledge
+  graph, then uses an LLM to synthesize diverse positive samples. An evaluation model
+  annotates and filters false positives, while a Gaussian-decayed function reduces
+  the gradient impact of false negatives during training, gradually restoring weights
+  as training progresses. This preserves sample diversity while preventing noisy samples
+  from distorting the semantic space. The benchmark evaluates on multiple STS tasks
+  via SentEval.
+
+  '
+task_mode: experiment_driven
+requires_execution: true
+requires_paper: true
+integrity_level: cas_plus_canary
+snapshot_status: runnable
+support_level: advanced
+time_band: 6-24h
+cost_band: medium
+difficulty: medium
+data_access: public
+primary_outputs:
+- avg_sts
+- embedding_checkpoint
+- contrastive_eval_report
+launch_profiles:
+- id: quick_eval
+  label: Quick Eval
+  description: Run the packaged sentence-embedding evaluation route using SentEval
+    on STS benchmarks (STS12-STS16, STS-Benchmark, SICK-Relatedness).
+- id: full_train_eval
+  label: Full Train + Eval
+  description: Execute the full GCSE training pipeline with knowledge graph construction,
+    LLM-based data synthesis, denoising, and downstream evaluation.
+dataset_download:
+  primary_method: url
+  sources:
+  - url: https://deepscientist.cc/AISB/033_gcse
+    archive_type: zip
+  notes:
+  - Dataset includes source domain data and pre-processed knowledge graphs for augmentation.
+  - LLM-based data synthesis may require additional model weights depending on configuration.
+credential_requirements:
+  mode: none
+  items: []
+  notes: []
+resources:
+  minimum:
+    cpu_cores: 8
+    ram_gb: 32
+    disk_gb: 80
+    gpu_count: 1
+    gpu_vram_gb: 16
+  recommended:
+    cpu_cores: 16
+    ram_gb: 64
+    disk_gb: 150
+    gpu_count: 1
+    gpu_vram_gb: 24
+environment:
+  python: '3.12'
+  cuda: '12.0'
+  pytorch: '2.0'
+  flash_attn: null
+  key_packages:
+  - transformers==4.44.1
+  - senteval
+  - prettytable
+  notes:
+  - The README explicitly recommends a separate vLLM environment for LLM-based data
+    synthesis tasks.
+  - Default training uses bert-base-uncased as base model; other encoder variants
+    (BERT-large, RoBERTa-base, RoBERTa-large) are supported per paper experiments.
+  - See bundled requirements.txt and pipeline dependencies for full environment setup.
+risk_flags:
+- requires_llm_synthesis
+- single_gpu_training
+risk_notes:
+- LLM-based data synthesis (pipelines/5-8) requires inference access to a language
+  model; vLLM is recommended for efficiency.
+- Training time scales with dataset size and synthesis volume; the 6-24h estimate
+  assumes moderate-scale domain data.
+- No benchmark execution was performed in this packaging pass; runtime validation
+  is recommended before trusting reported metric values.
+recommended_when: 'Use this benchmark for moderate-scale NLP training tasks focused
+  on semantic textual similarity and sentence embedding robustness. Suitable when
+  exploring knowledge-driven data augmentation or noise-aware contrastive learning
+  for improving unsupervised sentence representations.
+
+  '
+not_recommended_when: 'Do not use this if you require a fully CPU-only workflow, a
+  benchmark centered on full autoregressive generation, or if no GPU resources are
+  available. Not suitable for tasks requiring multi-GPU distributed training at scale.
+
+  '
+paper:
+  title: Enhancing Unsupervised Sentence Embeddings via Knowledge-Driven Data Augmentation
+    and Gaussian-Decayed Contrastive Learning
+  venue: ACL 2025
+  year: 2025
+  url: https://aclanthology.org/2025.acl-long.244/
+  authors:
+  - Peichao Lai (Peking University)
+  - Zhengfeng Zhang (Fuzhou University)
+  - Wentao Zhang (Peking University)
+  - Fangcheng Fu (Peking University)
+  - Bin Cui (Peking University)
+download:
+  url: https://github.com/ResearAI/DeepScientist/releases/download/aisb-v0.0.1/aisb.t3.033_gcse.zip
+  archive_type: zip
+  local_dir_name: paper-33-GCSE
+  provider: github_release
+  repo: ResearAI/DeepScientist
+  tag: aisb-v0.0.1
+  asset_name: aisb.t3.033_gcse.zip
+  sha256: 1d0220f894ed883ae572eb0da0ce1a358966a280ddfafadeb19a9db1f01f7866
+  size_bytes: 912820
+display:
+  palette_seed: mint-slate-semantic
+  art_style: embedding-atlas
+  accent_priority: medium
+image_path: ../image/033_aisb.t3.033_gcse.jpg
+capability_tags:
+- research_code_optimization
+- sentence_embeddings
+- representation_learning
+- data_augmentation
+- nlp
+- contrastive_learning
+- knowledge_graphs
+- noise_aware_training
+aisb_direction: T3
+track_fit:
+- paper_track
+- benchmark_track
+commercial:
+  annual_fee: null

package/AISB/catalog/aisb.t3.033_gcse.zh.yaml

@@ -0,0 +1,126 @@
+id: aisb.t3.033_gcse
+name: GCSE
+version: 0.1.0
+one_line: 基于知识图谱引导增强和高斯衰减对比学习的无监督句子嵌入优化，实现噪声感知训练。
+task_description: '本基准测试包涵盖基于高斯衰减梯度辅助对比句子嵌入（GCSE）模型的无监督句子嵌入训练。该方法针对无监督句子表示学习中的两个主要挑战：数据多样性不足和数据噪声过高。流程首先从源数据中提取细粒度知识（实体、数量）以构建知识图谱，然后使用大语言模型合成多样化正样本。评估模型对假正例进行标注和过滤，而高斯衰减函数在训练过程中降低假负例的梯度影响，逐步恢复权重。这在保持样本多样性的同时，防止噪声样本扭曲语义空间。基准测试通过SentEval在多个STS任务上进行评估。
+
+  '
+task_mode: experiment_driven
+requires_execution: true
+requires_paper: true
+integrity_level: cas_plus_canary
+snapshot_status: runnable
+support_level: advanced
+time_band: 6-24h
+cost_band: medium
+difficulty: medium
+data_access: public
+primary_outputs:
+- avg_sts
+- embedding_checkpoint
+- contrastive_eval_report
+launch_profiles:
+- id: quick_eval
+  label: 快速评估
+  description: 使用SentEval在STS基准测试（STS12-STS16、STS-Benchmark、SICK-Relatedness）上运行打包的句子嵌入评估流程。
+- id: full_train_eval
+  label: 完整训练 + 评估
+  description: 执行完整的GCSE训练流程，包括知识图谱构建、基于大语言模型的数据合成、去噪和下游评估。
+dataset_download:
+  primary_method: url
+  sources:
+  - url: https://deepscientist.cc/AISB/033_gcse
+    archive_type: zip
+  notes:
+  - 数据集包含源领域数据及用于增强的预处理知识图谱。
+  - 基于大语言模型的数据合成可能需要额外的模型权重，具体取决于配置。
+credential_requirements:
+  mode: none
+  items: []
+  notes: []
+resources:
+  minimum:
+    cpu_cores: 8
+    ram_gb: 32
+    disk_gb: 80
+    gpu_count: 1
+    gpu_vram_gb: 16
+  recommended:
+    cpu_cores: 16
+    ram_gb: 64
+    disk_gb: 150
+    gpu_count: 1
+    gpu_vram_gb: 24
+environment:
+  python: '3.12'
+  cuda: '12.0'
+  pytorch: '2.0'
+  flash_attn: null
+  key_packages:
+  - transformers==4.44.1
+  - senteval
+  - prettytable
+  notes:
+  - README明确建议为基于大语言模型的数据合成任务配置独立的vLLM环境。
+  - 默认训练使用bert-base-uncased作为基础模型；根据论文实验，还支持其他编码器变体（BERT-large、RoBERTa-base、RoBERTa-large）。
+  - 有关完整环境设置，请参阅附带的requirements.txt和流程依赖项。
+risk_flags:
+- requires_llm_synthesis
+- single_gpu_training
+risk_notes:
+  - 基于大语言模型的数据合成（流程5-8）需要访问语言模型进行推理；推荐使用vLLM以提高效率。
+  - 训练时间随数据集大小和合成量增加而延长；6-24小时的估计假设使用中等规模的领域数据。
+  - 本次打包过程中未执行基准测试运行；在使用报告的指标值之前，建议进行运行时验证。
+recommended_when: '当您需要进行中等规模的NLP训练任务，且重点关注语义文本相似性和句子嵌入鲁棒性时，可使用此基准测试。适用于探索知识驱动数据增强或噪声感知对比学习以改进无监督句子表示的场景。
+
+  '
+not_recommended_when: '如果您需要完全基于CPU的工作流程、以完全自回归生成为中心的基准测试，或没有可用的GPU资源，请勿使用此基准测试。不适用于需要大规模多GPU分布式训练的任务。
+
+  '
+paper:
+  title: Enhancing Unsupervised Sentence Embeddings via Knowledge-Driven Data Augmentation
+    and Gaussian-Decayed Contrastive Learning
+  venue: ACL 2025
+  year: 2025
+  url: https://aclanthology.org/2025.acl-long.244/
+  authors:
+  - Peichao Lai (Peking University)
+  - Zhengfeng Zhang (Fuzhou University)
+  - Wentao Zhang (Peking University)
+  - Fangcheng Fu (Peking University)
+  - Bin Cui (Peking University)
+  abstract_summary: 本文提出了一种名为GCSE的新方法，通过知识图谱引导的数据增强和高斯衰减对比学习来提升无监督句子嵌入的质量。该方法首先利用知识图谱获取细粒度知识来增强数据多样性，然后通过高斯衰减函数降低噪声样本的影响。在多个STS基准测试上的实验表明，该方法显著优于现有无监督方法。
+  abstract: '无监督句子嵌入学习面临着数据多样性不足和噪声干扰的双重挑战。本研究提出了一种名为高斯衰减梯度辅助对比句子嵌入（GCSE）的新方法，通过知识驱动的数据增强和噪声感知对比学习来解决这些问题。首先，我们从源数据中提取细粒度知识（实体、数量）构建知识图谱，并利用大语言模型合成多样化的正样本。然后，引入高斯衰减函数来降低训练过程中假负例的梯度影响，同时通过评估模型过滤假正例。在多个标准STS基准测试上的实验表明，我们的方法显著提升了无监督句子嵌入的性能，超越了现有的最先进方法。
+
+    '
+  notes: []
+download:
+  url: https://github.com/ResearAI/DeepScientist/releases/download/aisb-v0.0.1/aisb.t3.033_gcse.zip
+  archive_type: zip
+  local_dir_name: paper-33-GCSE
+  provider: github_release
+  repo: ResearAI/DeepScientist
+  tag: aisb-v0.0.1
+  asset_name: aisb.t3.033_gcse.zip
+  sha256: 1d0220f894ed883ae572eb0da0ce1a358966a280ddfafadeb19a9db1f01f7866
+  size_bytes: 912820
+display:
+  palette_seed: mint-slate-semantic
+  art_style: embedding-atlas
+  accent_priority: medium
+image_path: ../image/033_aisb.t3.033_gcse.jpg
+capability_tags:
+- research_code_optimization
+- sentence_embeddings
+- representation_learning
+- data_augmentation
+- nlp
+- contrastive_learning
+- knowledge_graphs
+- noise_aware_training
+aisb_direction: T3
+track_fit:
+- paper_track
+- benchmark_track
+commercial:
+  annual_fee: null

package/AISB/catalog/aisb.t3.034_ensemblewm.yaml

@@ -0,0 +1,183 @@
+id: aisb.t3.034_ensemblewm
+name: Ensemble Watermarks for Large Language Models
+version: 0.1.0
+one_line: Multi-feature watermark detection combining red-green, acrostic, and sensorimotor-norm
+  signals for robust LLM output verification.
+task_description: 'This benchmark evaluates ensemble watermark detection methods for
+  large language model outputs. The ensemble combines three distinct watermark features:
+  the red-green watermark (token-level logit manipulation), acrostic embeddings (sentence-initial
+  letter encoding), and sensorimotor norms (perceptual/action category selection).
+  The task involves generating watermarked text via logit modification and detecting
+  watermarks in both clean and paraphrased outputs. Evaluation covers detection rate,
+  watermark score distributions, and robustness against paraphrasing attacks using
+  a T5-based attack pipeline. The unified detection function applies across all ensemble
+  configurations without modification.
+
+  '
+task_mode: evaluation_driven
+requires_execution: true
+requires_paper: true
+integrity_level: cas_plus_canary
+snapshot_status: runnable
+support_level: advanced
+time_band: 6-24h
+cost_band: medium
+difficulty: hard
+data_access: public
+primary_outputs:
+- detection_rate
+- watermark_scores
+- benchmark_report
+launch_profiles:
+- id: quick_eval
+  label: Quick Eval
+  description: Run one packaged watermark-detection evaluation route on generated
+    outputs. Uses a single model configuration and predefined test prompts to measure
+    baseline detection rates without attack simulation.
+- id: full_eval
+  label: Full Eval
+  description: Execute the complete multi-feature watermark detection workflow including
+    generation, soft paraphrasing attack (T5-based word replacement at configurable
+    percentage), and evaluation across all feature combinations. Produces detection
+    rate matrices and watermark score distributions.
+dataset_download:
+  primary_method: mixed
+  sources:
+  - url: https://deepscientist.cc/AISB/034_ensemblewm
+    type: archive
+    format: zip
+  - url: https://huggingface.co/datasets/know-center/Lancaster_sensorimotor_norms
+    type: external
+    format: csv
+    description: Lancaster Sensorimotor Norms dataset with 39,707 words across 11
+      perceptual and 5 action dimensions
+  notes:
+  - The Lancaster sensorimotor norms CSV is bundled in the archive
+  - Test prompts sourced from standard LLM evaluation datasets
+credential_requirements:
+  mode: none
+  items: []
+  notes:
+  - No external API keys required
+  - Local LLM weights must be obtained separately (e.g., Llama-3.1-8B via HuggingFace)
+resources:
+  minimum:
+    cpu_cores: 8
+    ram_gb: 32
+    disk_gb: 80
+    gpu_count: 1
+    gpu_vram_gb: 24
+    notes: Supports quantized models (GPTQ 4-bit) to reduce VRAM requirements
+  recommended:
+    cpu_cores: 16
+    ram_gb: 64
+    disk_gb: 150
+    gpu_count: 1
+    gpu_vram_gb: 48
+    notes: Full precision recommended for generation experiments; 4-bit quantization
+      available for memory-constrained setups
+environment:
+  python: '3.10'
+  cuda: '11.8'
+  pytorch: 2.1.0
+  flash_attn: 2.x
+  key_packages:
+  - transformers
+  - torch
+  - numpy
+  - pandas
+  - spacy
+  - datasets
+  - tqdm
+  - bitsandbytes
+  - auto-gptq
+  notes:
+  - Requires spacy en_core_web_sm model: python -m spacy download en_core_web_sm
+  - T5 model for paraphrasing attack loaded via transformers (T5-small or T5-base)
+  - Llama model loaded via AutoModelForCausalLM with optional BitsAndBytesConfig for
+    4-bit quantization
+  - See bundled requirements.txt for full dependency specification
+risk_flags:
+- high_vram
+- extended_runtime
+risk_notes:
+- Generation experiments require significant GPU memory for LLM inference
+- Full evaluation with attack simulation may take several hours depending on dataset
+  size
+- Soft attack (paraphrasing) batch_size parameter affects memory usage
+recommended_when: 'Use this benchmark when you need an LLM-safeguard evaluation pipeline
+  focused on watermark detection quality, want to evaluate ensemble detection robustness
+  against paraphrasing, or need to compare single-feature versus multi-feature watermark
+  schemes. Suitable for evaluating detection-only approaches without requiring model
+  fine-tuning.
+
+  '
+not_recommended_when: 'Do not use this benchmark if you cannot host open LLM checkpoints
+  locally, lack GPU resources with adequate VRAM, or need a benchmark without generation
+  and detection loops. Not suitable for evaluating text classification or perplexity-based
+  detection methods.
+
+  '
+paper:
+  title: Ensemble Watermarks for Large Language Models
+  authors:
+  - Georg Niess
+  - Roman Kern
+  venue: arXiv preprint
+  year: 2024
+  url: https://arxiv.org/abs/2411.19563
+  code_url: https://github.com/
+  abstract: 'As large language models reach human-like fluency, reliably distinguishing
+    AI-generated text from human authorship becomes increasingly difficult. We propose
+    a multi-feature method for generating watermarks that combines acrostica and sensorimotor
+    norms with the established red-green watermark to achieve a 98% detection rate.
+    After paraphrasing attacks, performance remains at 95% detection rate compared
+    to 49% for red-green alone.
+
+    '
+download:
+  url: https://github.com/ResearAI/DeepScientist/releases/download/aisb-v0.0.1/aisb.t3.034_ensemblewm.zip
+  archive_type: zip
+  local_dir_name: aisb-t3-034-ensemblewm
+  provider: github_release
+  repo: ResearAI/DeepScientist
+  tag: aisb-v0.0.1
+  asset_name: aisb.t3.034_ensemblewm.zip
+  sha256: 8eae0196937e9b32feade1727d417158061f0741b7b0b80a9f154bdd6aaba079
+  size_bytes: 25404700
+display:
+  palette_seed: aqua-ink-watermark
+  art_style: verification-grid
+  accent_priority: high
+  tags:
+  - watermarking
+  - detection
+  - llm-safety
+  - robustness
+  - ensemble-methods
+image_path: ../image/034_aisb.t3.034_ensemblewm.jpg
+metric_contract:
+  primary_metric: detection_rate
+  origin_path: detection_notebook.ipynb
+  source_ref: detection_rate
+  evaluation_protocol:
+    code_paths:
+    - batch_run/run_experiments_soft.py
+    - batch_run/run_attack_soft.py
+    - detection_notebook.ipynb
+    - modules/text_generation.py
+  metrics_summary: []
+  execution_status: pending
+  execution_notes: 'Static code audit confirmed executable anchors for all staged
+    metrics. No benchmark execution was performed in this packaging pass. Metric values
+    should be treated as provisional pending trusted runtime outputs.
+
+    '
+executive_summary: 'This benchmark provides a comprehensive evaluation framework for
+  ensemble watermark detection in LLM outputs. The approach combines token-level (red-green),
+  sentence-level (acrostic), and semantic-level (sensorimotor) watermark features
+  to achieve robust detection even after paraphrasing attacks. The unified detection
+  function operates across all feature combinations, enabling flexible configuration
+  for different security-robustness tradeoffs.
+
+  '