dsh-math-modeling-agent 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +127 -0
- package/cordis.patch.yml +8 -0
- package/package.json +34 -0
- package/skills/math-modeling-agent/SKILL.md +50 -0
- package/skills/math-modeling-agent/examples/minimal-run/README.md +14 -0
- package/skills/math-modeling-agent/examples/resumed-run/README.md +13 -0
- package/skills/math-modeling-agent/references/claims-evidence.md +29 -0
- package/skills/math-modeling-agent/references/data-subproblems.md +17 -0
- package/skills/math-modeling-agent/references/math-grill.md +33 -0
- package/skills/math-modeling-agent/references/modeling-methodology.md +38 -0
- package/skills/math-modeling-agent/references/problem-types.md +20 -0
- package/skills/math-modeling-agent/references/report-contract.md +29 -0
- package/skills/math-modeling-agent/references/research-breakthrough.md +27 -0
- package/skills/math-modeling-agent/references/state-recovery.md +18 -0
- package/skills/math-modeling-agent/references/tool-policy.md +21 -0
- package/skills/math-modeling-agent/references/workflow.md +39 -0
- package/skills/math-modeling-agent/schemas/attempt.schema.json +70 -0
- package/skills/math-modeling-agent/schemas/ledger.schema.json +47 -0
- package/skills/math-modeling-agent/schemas/run.schema.json +102 -0
- package/skills/math-modeling-agent/scripts/capability-probe.mjs +105 -0
- package/skills/math-modeling-agent/scripts/python-environment.mjs +408 -0
- package/skills/math-modeling-agent/scripts/run-state.mjs +547 -0
- package/skills/math-modeling-audit/SKILL.md +41 -0
- package/skills/math-modeling-audit/examples/audit-report.md +13 -0
- package/skills/math-modeling-audit/examples/mcm-final-review.md +86 -0
- package/skills/math-modeling-audit/references/data-citation-audit.md +15 -0
- package/skills/math-modeling-audit/references/evidence-levels.md +10 -0
- package/skills/math-modeling-audit/references/mcm-icm-final-judge.md +331 -0
- package/skills/math-modeling-audit/references/verification-protocol.md +27 -0
- package/skills/math-modeling-audit/scripts/mcm-score.mjs +218 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Yohan Chen
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,127 @@
|
|
|
1
|
+
# MathModelingAgent for DSH
|
|
2
|
+
|
|
3
|
+
证据驱动的数学建模智能体插件:状态机编排 + 主张/义务/证据账本 + 独立验证协议 + MCM/ICM 终审。继承 MathModelingAgent v3.1 的"分解 + 迭代"方法论,把验证裁决权从 LLM 主观打分换成可复现证据。
|
|
4
|
+
|
|
5
|
+
## 继承与改良
|
|
6
|
+
|
|
7
|
+
- **继承**:数据探查、子问题分解、顺序求解、带接受/拒绝标准的建模-分析-修正循环、停滞检测(源自 [IMO25](https://github.com/lyang36/IMO25) 的分解+迭代思想)。
|
|
8
|
+
- **改良**:原项目的验证者是"分析者 LLM 打 1-5 分";本插件改为**义务账本 + 工具执行证据 + 独立审计**——LLM 只负责提出主张和攻击,不再负责裁决正确性。旧仓库保持原样未动。
|
|
9
|
+
|
|
10
|
+
## 特性
|
|
11
|
+
|
|
12
|
+
- **状态机**:9 个可恢复终态(SOLVED / PARTIAL / CONDITIONAL / INCONCLUSIVE / REFUTED / INFEASIBLE / UNIDENTIFIABLE / BLOCKED / CANCELLED),每次转移必须引用证据或 issue。
|
|
13
|
+
- **证据账本**:每个主张强制登记验证义务(数值→独立重算+误差界;最优性→KKT/对偶/精确搜索,否则只能声称"已找到最优";预测→防泄漏划分+基线+校准……),证据强度不得弱于主张强度。
|
|
14
|
+
- **验证协议**:固定攻击顺序 + PASS / FAIL / INCONCLUSIVE 三态裁决,INCONCLUSIVE 禁止升格。
|
|
15
|
+
- **崩溃恢复**:原子快照 + 追加日志 + 跨进程锁,stale 锁与残留 guard 自动回收。
|
|
16
|
+
- **MCM/ICM 终审**:一票否决与奖项封顶 + 七类 100 分 + 固定 14 节报告。
|
|
17
|
+
- **零运行时依赖**:Python、Lean、Wolfram 全部可选,缺失时验证等级降级,绝不伪造执行。
|
|
18
|
+
|
|
19
|
+
## 工作流
|
|
20
|
+
|
|
21
|
+
```mermaid
|
|
22
|
+
stateDiagram-v2
|
|
23
|
+
[*] --> TRIAGE
|
|
24
|
+
TRIAGE --> SCOPE_FROZEN
|
|
25
|
+
SCOPE_FROZEN --> INPUT_PROFILED
|
|
26
|
+
INPUT_PROFILED --> CLAIMS_REGISTERED
|
|
27
|
+
CLAIMS_REGISTERED --> CANDIDATES_READY
|
|
28
|
+
CANDIDATES_READY --> ATTEMPT
|
|
29
|
+
ATTEMPT --> EXECUTE
|
|
30
|
+
EXECUTE --> VERIFY
|
|
31
|
+
VERIFY --> REVISE
|
|
32
|
+
VERIFY --> RESEARCH
|
|
33
|
+
VERIFY --> FORK
|
|
34
|
+
REVISE --> ATTEMPT
|
|
35
|
+
RESEARCH --> CANDIDATES_READY
|
|
36
|
+
FORK --> ATTEMPT
|
|
37
|
+
VERIFY --> SOLVED
|
|
38
|
+
VERIFY --> PARTIAL
|
|
39
|
+
VERIFY --> CONDITIONAL
|
|
40
|
+
VERIFY --> INCONCLUSIVE
|
|
41
|
+
VERIFY --> REFUTED
|
|
42
|
+
VERIFY --> INFEASIBLE
|
|
43
|
+
VERIFY --> UNIDENTIFIABLE
|
|
44
|
+
VERIFY --> BLOCKED
|
|
45
|
+
VERIFY --> CANCELLED
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
任意非终态可直达 BLOCKED / CANCELLED。ATTEMPT 永远不能直接跳到 SOLVED。
|
|
49
|
+
|
|
50
|
+
## 安装
|
|
51
|
+
|
|
52
|
+
固定 GitHub 版本(发布流程需先创建 v0.1.0 tag):
|
|
53
|
+
|
|
54
|
+
~~~bash
|
|
55
|
+
dsh plugin --profile web add github:yohanchen1/MathModelingAgent#v0.1.0
|
|
56
|
+
~~~
|
|
57
|
+
|
|
58
|
+
npm 发布后,或本地 `npm pack` 生成的 tarball 路径:
|
|
59
|
+
|
|
60
|
+
~~~bash
|
|
61
|
+
dsh plugin --profile web add dsh-math-modeling-agent
|
|
62
|
+
~~~
|
|
63
|
+
|
|
64
|
+
不要用 `dsh plugin add .` 安装本地目录:它会按目录名安装且不激活 bundle。安装后验证组合配置并重启 host:
|
|
65
|
+
|
|
66
|
+
~~~bash
|
|
67
|
+
dsh --profile web --dump-config
|
|
68
|
+
dsh web
|
|
69
|
+
~~~
|
|
70
|
+
|
|
71
|
+
## 核心机制
|
|
72
|
+
|
|
73
|
+
### 状态机与 SOLVED 门禁
|
|
74
|
+
|
|
75
|
+
每轮尝试有一个目标并记录:候选与假设增量、实际执行的命令或推导产物、新证据 ID、关闭的义务、打开/关闭的 issue、是否产生可审计进展、预算消耗、下一步行动。判定"进展"只认:关闭义务、新增可复现证据、反驳候选、收紧界或不确定区间、移除阻塞、正确弱化不支持的断言——重述、同参数重跑、更长的散文、工具 exit 0 都不算。连续两轮无进展进入停滞审查,第三轮无进展必须实质换方向(FORK)、交给用户决策,或以非 SOLVED 终态暂停。
|
|
76
|
+
|
|
77
|
+
SOLVED 硬门禁:范围冻结 + 全部必选义务 PASS + 关键对抗检查通过 + 可复现材料齐全 + 局限已声明;High-Assurance 模式还必须通过一次只拿产物、不注入思维链的独立审计。
|
|
78
|
+
|
|
79
|
+
### 主张 / 义务 / 证据
|
|
80
|
+
|
|
81
|
+
- **主张记录**:ID、原文、类型、量词范围、假设、风险、验证义务、证据 ID、独立反查、状态、局限。
|
|
82
|
+
- **证据记录**:方法、工具、时间戳、覆盖的主张、输入输出哈希、命令/环境、退出码、产物、容差、局限,以及六档等级:NOT_CHECKED / DERIVED / EXECUTED / VERIFIED / INDEPENDENTLY_VERIFIED / EXTERNALLY_VALIDATED。
|
|
83
|
+
- **强度匹配**:训练集分数不能支撑泛化结论;单个优化器返回点不能支撑全局最优;形式化证明不能支撑未形式化的现实假设。不支持的断言只能弱化,不能放宽验证规则。
|
|
84
|
+
|
|
85
|
+
### 验证协议
|
|
86
|
+
|
|
87
|
+
冻结输入(哈希题目、数据、代码、配置、环境)→ 重建主张映射 → 按固定顺序攻击:任务覆盖与代理指标替换 → 单位/量纲/定义域/约束 → 推导与实现一致性 → 数据泄漏/标签/划分/后验参数 → 基线/不确定性/敏感性/外部效度 → 可复现性与引用真实性 → 反例/失败案例/更简单替代。裁决只允许 PASS(可复现证据支撑)、FAIL(矛盾/反例/无效方法/复现失败)、INCONCLUSIVE(证据不足),且 INCONCLUSIVE 不得因为"看起来合理"升格为 PASS。
|
|
88
|
+
|
|
89
|
+
### 崩溃恢复与并发
|
|
90
|
+
|
|
91
|
+
默认运行根目录 `math-modeling-runs/<task-id>/`。`run.json` 原子快照、`events.jsonl` 追加式转移日志、`ledger.json` 主张账本;`run-state.mjs` 提供 init / transition / validate / recover 四个命令。跨进程互斥锁带 ownerId 与 stale 回收(进程已死且超时自动接管),空锁文件与残留 reclaim guard 按 mtime 回收,Windows 共享冲突自动重试;恢复时校验日志并跳过哈希未变的已完成工作,任何 provider/解析器失败都保留原始产物与最佳候选。
|
|
92
|
+
|
|
93
|
+
### MCM/ICM 终审
|
|
94
|
+
|
|
95
|
+
`math-modeling-audit` Skill 内置模拟 100 分终审(非 COMAP 官方评分表):Stage 1 一票否决与奖项封顶(14 项检查,逐项给出证据链封顶);Stage 2 七类评分共 100 分(问题理解与分解 10、数据/证据/参数 12、模型构建 22、求解算法与可复现 16、结果验证与可信度 24、结论与推广 8、写作与图表 8);Stage 3 模型逐个尸检;Stage 4 关键结果审计;Stage 5 按 MCM A/B/C 或 ICM D/E/F 启用专项检查;Stage 6 依据 93-100 Outstanding Candidate 等 band 判定奖项,输出固定 14 节报告。
|
|
96
|
+
|
|
97
|
+
### 工具与降级
|
|
98
|
+
|
|
99
|
+
`capability-probe.mjs` 探测 PATH 并实测可用性。Python 可选但推荐:仅在需要计算时用 `python-environment.mjs` 在运行目录内创建隔离 uv/venv 环境,只安装所需 PyPI 包(VCS 依赖与任意索引需用户批准);Lean 与 Wolfram 可选,永不自动安装,Lean 结果必须附无 sorry/admit 证明与自然语言-形式语句忠实性检查;缺失能力只降低证据等级(unverified / partially_verified),不伪造执行结果。
|
|
100
|
+
|
|
101
|
+
## 运行产物
|
|
102
|
+
|
|
103
|
+
~~~text
|
|
104
|
+
math-modeling-runs/<task-id>/
|
|
105
|
+
├── run.json # 原子状态快照
|
|
106
|
+
├── ledger.json # 范围、假设、主张、义务、子问题、候选、issue
|
|
107
|
+
├── events.jsonl # 追加式转移日志
|
|
108
|
+
├── problem-brief.md # 问题摘要
|
|
109
|
+
├── inputs.json # 输入与数据画像
|
|
110
|
+
├── attempts/<n>/ # 每轮:report.md + 代码/产物(存在时)
|
|
111
|
+
├── research/ # 文献检索与候选方法矩阵(发生研究时)
|
|
112
|
+
├── walls/ # 放弃方向的突破备忘录(放弃时)
|
|
113
|
+
├── reproducibility.json # 数据/代码/配置版本、锁、种子、命令
|
|
114
|
+
└── final-report.md # 终态报告
|
|
115
|
+
~~~
|
|
116
|
+
|
|
117
|
+
## 卸载
|
|
118
|
+
|
|
119
|
+
~~~bash
|
|
120
|
+
dsh plugin --profile web remove dsh-math-modeling-agent
|
|
121
|
+
~~~
|
|
122
|
+
|
|
123
|
+
卸载后重启当前 host。
|
|
124
|
+
|
|
125
|
+
## License
|
|
126
|
+
|
|
127
|
+
MIT
|
package/cordis.patch.yml
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
1
|
+
- insert:
|
|
2
|
+
- id: dsh-math-modeling-agent-skills
|
|
3
|
+
name: '@deepseek-ai/dsh-skill-filesystem'
|
|
4
|
+
config:
|
|
5
|
+
providerName: dsh-math-modeling-agent
|
|
6
|
+
includeDefaultRoots: false
|
|
7
|
+
bundledSkillDir: !!js "process.getBuiltinModule('node:url').fileURLToPath(new URL('skills/', baseUrl))"
|
|
8
|
+
watch: false
|
package/package.json
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "dsh-math-modeling-agent",
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "Evidence-driven mathematical modeling and verification skills for DeepSeek Harness",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"files": [
|
|
7
|
+
"cordis.patch.yml",
|
|
8
|
+
"skills",
|
|
9
|
+
"README.md",
|
|
10
|
+
"LICENSE"
|
|
11
|
+
],
|
|
12
|
+
"scripts": {
|
|
13
|
+
"test": "node --test",
|
|
14
|
+
"pack:check": "npm pack --dry-run --json"
|
|
15
|
+
},
|
|
16
|
+
"license": "MIT",
|
|
17
|
+
"repository": {
|
|
18
|
+
"type": "git",
|
|
19
|
+
"url": "git+https://github.com/yohanchen1/MathModelingAgent.git"
|
|
20
|
+
},
|
|
21
|
+
"keywords": [
|
|
22
|
+
"dsh",
|
|
23
|
+
"mathematical-modeling",
|
|
24
|
+
"verification",
|
|
25
|
+
"lean",
|
|
26
|
+
"wolfram",
|
|
27
|
+
"python"
|
|
28
|
+
],
|
|
29
|
+
"dsh": {
|
|
30
|
+
"bundle": {
|
|
31
|
+
"patch": "./cordis.patch.yml"
|
|
32
|
+
}
|
|
33
|
+
}
|
|
34
|
+
}
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: math-modeling-agent
|
|
3
|
+
description: This skill should be used when the user asks to "建立数学模型", "解决数学建模题", "继续上次建模", "寻找新的建模方向", "solve this modeling problem", or needs an evidence-backed mathematical model with computation, research, verification, per-attempt reports, and resumable state.
|
|
4
|
+
whenToUse: Use for open-ended mathematical modeling, prediction, optimization, estimation, simulation, mechanism, decision, or multi-part contest problems; do not use for auditing an already completed artifact without changing it.
|
|
5
|
+
user-invocable: true
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Math Modeling Agent
|
|
9
|
+
|
|
10
|
+
## Goal
|
|
11
|
+
|
|
12
|
+
Move one mathematical problem from scoped intake to an evidence-backed conclusion or a precise resumable scientific status.
|
|
13
|
+
|
|
14
|
+
## Interface
|
|
15
|
+
|
|
16
|
+
Accept problem text, attachments, or an existing run directory. Do not ask the user to choose internal phases, Python/Lean/Wolfram adapters, file names, or state fields.
|
|
17
|
+
|
|
18
|
+
Return a concise current status, supported claims, unresolved obligations, attempt delta, artifact path, and next action or final report.
|
|
19
|
+
|
|
20
|
+
## Default workflow
|
|
21
|
+
|
|
22
|
+
1. Route the task to Fast, Standard, or High-Assurance using `references/problem-types.md`.
|
|
23
|
+
2. Read available inputs before asking questions; apply the one-question math grill in `references/math-grill.md`.
|
|
24
|
+
3. Create or validate run state with `scripts/run-state.mjs` and follow `references/state-recovery.md`.
|
|
25
|
+
4. Profile inputs and build the subproblem DAG using `references/data-subproblems.md`.
|
|
26
|
+
5. Register assumptions, claims, and verification obligations from `references/claims-evidence.md`.
|
|
27
|
+
6. Build a baseline and materially different candidates using `references/modeling-methodology.md`.
|
|
28
|
+
7. Probe tools with `scripts/capability-probe.mjs`; create a run-local Python environment with `scripts/python-environment.mjs` only when computation is required.
|
|
29
|
+
8. Execute, verify, critique, and revise according to `references/workflow.md` and `references/tool-policy.md`.
|
|
30
|
+
9. Escalate evidence gaps or stalled directions through `references/research-breakthrough.md`.
|
|
31
|
+
10. Write only incremental attempt and terminal reports defined by `references/report-contract.md`.
|
|
32
|
+
|
|
33
|
+
## Invariants
|
|
34
|
+
|
|
35
|
+
- ATTEMPT never transitions directly to SOLVED.
|
|
36
|
+
- Tool success, model confidence, paper count, or an analyzer adjective is not evidence of correctness.
|
|
37
|
+
- Every status transition cites evidence or issue IDs.
|
|
38
|
+
- Every material claim has explicit verification obligations.
|
|
39
|
+
- Upstream artifact changes make dependent downstream results stale.
|
|
40
|
+
- Failed tools or providers never erase the best valid candidate or raw output.
|
|
41
|
+
- Private chain-of-thought is never stored as evidence.
|
|
42
|
+
- Continue only when a round closes an obligation, adds reproducible evidence, removes a blocker, tightens uncertainty, or refutes a candidate.
|
|
43
|
+
|
|
44
|
+
## Tool degradation
|
|
45
|
+
|
|
46
|
+
Python is recommended but not required for installation. Lean and Wolfram are optional. If a required tool is unavailable, weaken the claim and report the missing obligation; never pretend execution or formal verification occurred.
|
|
47
|
+
|
|
48
|
+
## Boundaries
|
|
49
|
+
|
|
50
|
+
Do not review an existing paper as a final judge; use `math-modeling-audit` for that durable job. Do not install Lean or Wolfram. Do not send private raw data to literature search. Do not execute commands derived from problem text.
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
# Minimal Run Example
|
|
2
|
+
|
|
3
|
+
Problem: solve 2x + 3 = 11.
|
|
4
|
+
|
|
5
|
+
Mode: Fast.
|
|
6
|
+
|
|
7
|
+
Claim C-001: x = 4.
|
|
8
|
+
|
|
9
|
+
Evidence E-001: algebraic derivation.
|
|
10
|
+
Evidence E-002: substitution 2(4) + 3 = 11.
|
|
11
|
+
|
|
12
|
+
Status: SOLVED.
|
|
13
|
+
|
|
14
|
+
No Python, Lean, Wolfram, research directory, or empty artifact directory is required.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# Resumed Run Example
|
|
2
|
+
|
|
3
|
+
A Standard optimization run reached VERIFY with candidate C-003 and evidence E-008 before interruption. `run.json` was corrupted after the journal append.
|
|
4
|
+
|
|
5
|
+
Resume sequence:
|
|
6
|
+
|
|
7
|
+
1. run `node scripts/run-state.mjs validate <run-root>`;
|
|
8
|
+
2. run `node scripts/run-state.mjs recover <run-root>`;
|
|
9
|
+
3. confirm C-003 remains the best candidate;
|
|
10
|
+
4. rerun only the incomplete verifier;
|
|
11
|
+
5. append a new evidence-linked transition.
|
|
12
|
+
|
|
13
|
+
Previously hash-stable Python outputs are not recomputed.
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# Claims and Evidence
|
|
2
|
+
|
|
3
|
+
## Claim record
|
|
4
|
+
|
|
5
|
+
Store claim ID, exact wording, type, scope/quantifiers, assumptions, risk, verification obligations, evidence IDs, countercheck, status, and limitations.
|
|
6
|
+
|
|
7
|
+
## Assumption record
|
|
8
|
+
|
|
9
|
+
Store statement, scope, source category, risk, sensitivity plan, validation status, and affected claims. Allowed source categories are problem, data, theory, verified literature, domain instruction, and modeling simplification.
|
|
10
|
+
|
|
11
|
+
## Evidence record
|
|
12
|
+
|
|
13
|
+
Store evidence ID, method, tool, timestamp, covered claim IDs, input/output hashes, command/workdir/environment, exit code, output artifacts, tolerance, limitations, and level:
|
|
14
|
+
DERIVED, EXECUTED, VERIFIED, INDEPENDENTLY_VERIFIED, EXTERNALLY_VALIDATED, or NOT_CHECKED.
|
|
15
|
+
|
|
16
|
+
## Obligation patterns
|
|
17
|
+
|
|
18
|
+
- Numeric value: independent recomputation, residual/error bound, domain, and boundary check.
|
|
19
|
+
- Feasibility: all constraints and justified tolerance on original scale.
|
|
20
|
+
- Global optimum: convexity/KKT/duality, exact search, or certified bound; otherwise say local or best found.
|
|
21
|
+
- Uniqueness: proof or explicit non-uniqueness disclaimer.
|
|
22
|
+
- Prediction: untouched test/external validation, leakage-safe split, baseline, uncertainty, and calibration.
|
|
23
|
+
- Causal effect: estimand, identification, temporal order, confounding, and sensitivity.
|
|
24
|
+
- Robustness: parameter, data, seed, specification, and scenario perturbation.
|
|
25
|
+
- Theorem: readable or kernel-checked proof plus formalization-fidelity audit.
|
|
26
|
+
- Literature support: resolvable source, exact location, excerpt, retrieval date, and mapped claim.
|
|
27
|
+
- Reproducibility: data/code/config versions, lock, seed, tolerance, commands, and clean rerun.
|
|
28
|
+
|
|
29
|
+
Evidence strength may not be weaker than claim strength. Weaken unsupported claims; never weaken the verification rule.
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# Inputs, Data, and Subproblems
|
|
2
|
+
|
|
3
|
+
## Input manifest
|
|
4
|
+
|
|
5
|
+
For each accessible artifact record stable ID, cryptographic hash, source, path, type, size, encoding, readability, and extraction provenance. For tables record sheets, rows, columns, inferred types, units, missingness, duplicates, ranges, anomalies, label source, time structure, group structure, and leakage risk.
|
|
6
|
+
|
|
7
|
+
Do not silently ignore unsupported files. Mark them unreadable and state the minimum representation needed.
|
|
8
|
+
|
|
9
|
+
## Leakage audit
|
|
10
|
+
|
|
11
|
+
Before model selection, identify target labels, future information, repeated entities, grouped observations, temporal ordering, and preprocessing fit scope. Fit imputation, scaling, feature selection, dimensionality reduction, and thresholds only inside training folds. Use time or group splits when ordinary random splits leak information.
|
|
12
|
+
|
|
13
|
+
## Subproblem DAG
|
|
14
|
+
|
|
15
|
+
Each subproblem records ID, exact task, required inputs, produced claims/artifacts, dependencies, acceptance criteria, and status. Downstream work references upstream IDs rather than copied prose.
|
|
16
|
+
|
|
17
|
+
When an upstream artifact changes, mark every dependent output stale. If an upstream node is blocked, downstream work must choose exactly one state: blocked, provisional with explicit condition, or alternative independent path.
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Math-Grill Protocol
|
|
2
|
+
|
|
3
|
+
Ask one question at a time. Read the problem and attachments first. Ask only questions whose answer can change the model, evidence obligation, or deliverable.
|
|
4
|
+
|
|
5
|
+
## Minimum routing questions
|
|
6
|
+
|
|
7
|
+
1. What output is required: number, proof, model, prediction, optimum, or decision?
|
|
8
|
+
2. Does data exist, and what is its source?
|
|
9
|
+
3. Is the requested claim proof, global optimality, causality, safety, or a useful approximation?
|
|
10
|
+
4. What error, confidence, or decision risk is acceptable?
|
|
11
|
+
5. Is the result for practice, competition, publication, production, or high-stakes use?
|
|
12
|
+
|
|
13
|
+
## Selective modeling fields
|
|
14
|
+
|
|
15
|
+
Collect only applicable fields:
|
|
16
|
+
|
|
17
|
+
- background and practical objective;
|
|
18
|
+
- explicit and hidden subproblems;
|
|
19
|
+
- data, label source, units, quality, time, and group structure;
|
|
20
|
+
- decision/state variables and parameters;
|
|
21
|
+
- objective or evaluation metric;
|
|
22
|
+
- equality, inequality, integer, nonnegative, logical, initial, and boundary constraints;
|
|
23
|
+
- assumptions and their source category;
|
|
24
|
+
- acceptance criteria and final deliverable.
|
|
25
|
+
|
|
26
|
+
## Rules
|
|
27
|
+
|
|
28
|
+
- Infer before asking.
|
|
29
|
+
- Ask the highest-impact unknown first.
|
|
30
|
+
- Mark irrelevant fields “not applicable” with a reason.
|
|
31
|
+
- Turn “自行判断” into an explicit modeling assumption with risk and sensitivity plan.
|
|
32
|
+
- Do not freeze scope while a missing answer can change the problem type or claim strength.
|
|
33
|
+
- Do not force a proof problem to invent data or an exploratory analysis to invent an objective function.
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
# Modeling Methodology
|
|
2
|
+
|
|
3
|
+
## Mechanism first
|
|
4
|
+
|
|
5
|
+
Map real objects to variables, mechanism to equations, reality constraints to mathematical constraints, and requested decisions to objective/evaluation criteria. Never choose a named method first and force the problem into it.
|
|
6
|
+
|
|
7
|
+
## Candidate portfolio
|
|
8
|
+
|
|
9
|
+
Always begin with a defensible baseline. Add a more complex candidate only when it targets a documented baseline failure. Candidates must differ in mechanism, information, assumptions, or verification route—not wording.
|
|
10
|
+
|
|
11
|
+
For each candidate record:
|
|
12
|
+
|
|
13
|
+
- stable ID and parent/fork;
|
|
14
|
+
- target subproblem and claims;
|
|
15
|
+
- mechanism and equations;
|
|
16
|
+
- assumptions and parameter sources;
|
|
17
|
+
- input requirements;
|
|
18
|
+
- expected information gain;
|
|
19
|
+
- verification obligations;
|
|
20
|
+
- stop/abandon conditions;
|
|
21
|
+
- complexity, interpretability, and compute cost.
|
|
22
|
+
|
|
23
|
+
## Selection
|
|
24
|
+
|
|
25
|
+
Prefer the simplest candidate that meets evidence and accuracy requirements. Compare verification coverage, critical issues, empirical performance, robustness, complexity, interpretability, cost, and reproducibility. Do not reward model count, deep learning, metaheuristics, AHP, TOPSIS, entropy weights, or Monte Carlo by name.
|
|
26
|
+
|
|
27
|
+
## Core checks
|
|
28
|
+
|
|
29
|
+
- units and dimensions;
|
|
30
|
+
- limiting and boundary cases;
|
|
31
|
+
- known special cases;
|
|
32
|
+
- constraint completeness;
|
|
33
|
+
- identifiability and conditioning;
|
|
34
|
+
- convergence and numerical stability;
|
|
35
|
+
- data leakage and post-hoc parameters;
|
|
36
|
+
- baseline and ablation;
|
|
37
|
+
- sensitivity of conclusion-changing parameters;
|
|
38
|
+
- model-result-conclusion traceability.
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
# Problem Types and Obligations
|
|
2
|
+
|
|
3
|
+
| Type | Primary output | Mandatory checks |
|
|
4
|
+
|---|---|---|
|
|
5
|
+
| Symbolic calculation | exact expression or roots | domain, equivalence, substitution, singular cases |
|
|
6
|
+
| Theorem/proposition | proof or counterexample | quantifiers, assumptions, boundary cases, proof completeness |
|
|
7
|
+
| Deterministic numerical | approximation | residual, convergence, stability, discretization error |
|
|
8
|
+
| Feasibility | satisfying assignment | every constraint, tolerance, certificate |
|
|
9
|
+
| Optimization | local/global solution | objective fidelity, constraints, bounds, optimality gap/certificate |
|
|
10
|
+
| Statistical estimation | estimate and uncertainty | sampling assumptions, interval, residual diagnostics |
|
|
11
|
+
| Prediction/classification | out-of-sample performance | leakage-safe split, baseline, calibration, uncertainty |
|
|
12
|
+
| Causal inference | identified effect | estimand, temporal order, confounding, sensitivity |
|
|
13
|
+
| Stochastic/risk | distribution or risk measure | distribution assumptions, Monte Carlo error, tail behavior |
|
|
14
|
+
| Inverse problem | recovered parameter/structure | identifiability, conditioning, regularization |
|
|
15
|
+
| Mechanism/system | dynamic behavior | structural validity, units, calibration, initial/boundary conditions |
|
|
16
|
+
| Decision analysis | recommendation | preference/weight source, constraints, sensitivity, regret |
|
|
17
|
+
| Exploratory analysis | hypotheses/patterns | no causal upgrade, multiplicity, robustness |
|
|
18
|
+
| Literature synthesis | supported method/fact | source verification, applicability, conflicting evidence |
|
|
19
|
+
|
|
20
|
+
Use multiple labels when necessary, but choose one primary output and one explicit assurance mode.
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# Report Contract
|
|
2
|
+
|
|
3
|
+
## Attempt report
|
|
4
|
+
|
|
5
|
+
Every attempt report contains exactly:
|
|
6
|
+
|
|
7
|
+
1. one round objective;
|
|
8
|
+
2. candidate/direction;
|
|
9
|
+
3. assumption delta;
|
|
10
|
+
4. claims changed;
|
|
11
|
+
5. actual execution or derivation artifacts;
|
|
12
|
+
6. new evidence IDs;
|
|
13
|
+
7. obligations closed;
|
|
14
|
+
8. issues opened and closed;
|
|
15
|
+
9. delta from prior attempt;
|
|
16
|
+
10. valid-progress decision;
|
|
17
|
+
11. budget usage;
|
|
18
|
+
12. current status;
|
|
19
|
+
13. one next action.
|
|
20
|
+
|
|
21
|
+
Do not copy full prior solutions or private reasoning.
|
|
22
|
+
|
|
23
|
+
## Terminal report
|
|
24
|
+
|
|
25
|
+
State the frozen question, final scientific status, answer/recommendation, verified claims and evidence links, conditional/unresolved claims, assumptions and sensitivity, candidate comparison, data and parameter sources, validation, failure cases, limitations, reproducibility commands, and resume conditions when non-SOLVED.
|
|
26
|
+
|
|
27
|
+
## Chat summary
|
|
28
|
+
|
|
29
|
+
After each round show only attempt number, direction, tool execution status, obligations passed/failed, most important failure, artifact path, and next action.
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# Research and Breakthrough
|
|
2
|
+
|
|
3
|
+
Research only when it closes a named evidence gap, validates a parameter/assumption, supplies an applicable method, or explains a failure.
|
|
4
|
+
|
|
5
|
+
## Source record
|
|
6
|
+
|
|
7
|
+
Record author, title, year, venue, DOI/stable URL, version, retrieval date, exact page/section/theorem/table, excerpt, mapped claim, source quality, applicability, and support strength. Distinguish full text, abstract only, metadata only, secondary source, conflicting evidence, and citation needed.
|
|
8
|
+
|
|
9
|
+
## Method matrix
|
|
10
|
+
|
|
11
|
+
For each candidate method record applicability, assumptions, data needs, implementation cost, verification route, reason to try, and reason to reject.
|
|
12
|
+
|
|
13
|
+
## Isolated campaign
|
|
14
|
+
|
|
15
|
+
For genuinely hard multi-direction problems, give isolated subagents independent briefs and prevent cross-talk. Require a counterexample family, the first unproved step, honest evidence level, and direction-exhaustion discipline. The coordinator reads final reports and adjudicates.
|
|
16
|
+
|
|
17
|
+
## Wall memo
|
|
18
|
+
|
|
19
|
+
A direction may be abandoned only when refuted, blocked by a precise missing tool/theory/information, or after natural variants are exhausted. Write:
|
|
20
|
+
|
|
21
|
+
- target and direction;
|
|
22
|
+
- paths tried and evidence;
|
|
23
|
+
- exact wall and wall type;
|
|
24
|
+
- breakthrough condition;
|
|
25
|
+
- restart checklist.
|
|
26
|
+
|
|
27
|
+
On a new tool, model, source, or resumed request, scan wall memos before restarting from zero.
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
# State and Recovery
|
|
2
|
+
|
|
3
|
+
Default run root: `math-modeling-runs/<task-id>/`.
|
|
4
|
+
|
|
5
|
+
Required durable files:
|
|
6
|
+
|
|
7
|
+
- run.json: atomic current snapshot;
|
|
8
|
+
- ledger.json: scope, assumptions, claims, obligations, subproblems, candidates, and issues;
|
|
9
|
+
- events.jsonl: append-only transition journal;
|
|
10
|
+
- problem-brief.md and inputs.json;
|
|
11
|
+
- attempts/<number>/report.md plus code/artifacts only when they exist;
|
|
12
|
+
- research/sources.jsonl and methods.md only when research occurs;
|
|
13
|
+
- walls/ only when a direction is abandoned;
|
|
14
|
+
- reproducibility.json and final-report.md for terminal output.
|
|
15
|
+
|
|
16
|
+
Use `scripts/run-state.mjs` for initialization, transition, validation, and recovery. A transition without evidence/issue references is invalid. Provider, parser, or verifier failure preserves raw output and the best candidate. Resume by validating state, recovering from the last journal snapshot when needed, then skipping hash-stable completed work.
|
|
17
|
+
|
|
18
|
+
Failure classes include data missing/schema/leakage, tool unavailable/dependency/license, runtime/timeout/numerical instability, model/assumption/parameter, verification/formalization, citation/research, parser/provider, no progress, and budget exhausted.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# Tool Policy
|
|
2
|
+
|
|
3
|
+
## Capability discovery
|
|
4
|
+
|
|
5
|
+
Run `scripts/capability-probe.mjs`; treat PATH hits as candidates, then verify version and usability through the available DSH shell. Record unavailable, candidate, available, or unknown with reason.
|
|
6
|
+
|
|
7
|
+
## Python
|
|
8
|
+
|
|
9
|
+
Use `scripts/python-environment.mjs`. Prefer uv, then an existing Python runtime plus venv. Keep the environment under the run directory. Install only required normalized PyPI package names; VCS URLs, local wheels, or arbitrary indexes require explicit user approval. Record lock, seed, command, workdir, timeout, stdout/stderr, exit code, and generated files.
|
|
10
|
+
|
|
11
|
+
## Lean
|
|
12
|
+
|
|
13
|
+
Never auto-install. Use only for formalizable obligations. Record Lean/Lake/library versions, axioms/imports, proof artifact, absence of sorry/admit, and a separate natural-language-to-formal-statement fidelity check.
|
|
14
|
+
|
|
15
|
+
## Wolfram
|
|
16
|
+
|
|
17
|
+
Never auto-install. Verify executable and license usability. Use for applicable symbolic, exact, or high-precision claims and state the exact coverage boundary.
|
|
18
|
+
|
|
19
|
+
## Search, PDF, and images
|
|
20
|
+
|
|
21
|
+
Use available web, PDF, vision/OCR, or local Python readers. Do not send private raw data in search queries. Missing capabilities lower evidence level rather than causing fabricated execution.
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# Modeling Workflow
|
|
2
|
+
|
|
3
|
+
## Modes
|
|
4
|
+
|
|
5
|
+
- Fast: explicit low-risk calculation; at most two clarifications; direct inverse, substitution, domain, or unit check.
|
|
6
|
+
- Standard: normal modeling; input profile, baseline, candidates, execution, sensitivity, and reproducibility.
|
|
7
|
+
- High-Assurance: publication, proof, global optimum, causal, safety, guarantee, or high-stakes use; require independent audit.
|
|
8
|
+
|
|
9
|
+
## States
|
|
10
|
+
|
|
11
|
+
TRIAGE → SCOPE_FROZEN → INPUT_PROFILED → CLAIMS_REGISTERED → CANDIDATES_READY → ATTEMPT → EXECUTE → VERIFY.
|
|
12
|
+
|
|
13
|
+
VERIFY may move to REVISE, RESEARCH, FORK, an independent audit, or one terminal status:
|
|
14
|
+
SOLVED, PARTIAL, CONDITIONAL, INCONCLUSIVE, REFUTED, INFEASIBLE, UNIDENTIFIABLE, BLOCKED, CANCELLED.
|
|
15
|
+
|
|
16
|
+
## Round contract
|
|
17
|
+
|
|
18
|
+
Each round has one objective and records:
|
|
19
|
+
|
|
20
|
+
- candidate and assumption delta;
|
|
21
|
+
- actual command or derivation artifact;
|
|
22
|
+
- new evidence IDs;
|
|
23
|
+
- obligations closed;
|
|
24
|
+
- issues opened and closed;
|
|
25
|
+
- whether auditable progress occurred;
|
|
26
|
+
- budget used;
|
|
27
|
+
- one next action.
|
|
28
|
+
|
|
29
|
+
## Progress test
|
|
30
|
+
|
|
31
|
+
Count progress only when a round closes an obligation, adds reproducible evidence, refutes a candidate, tightens a bound or uncertainty interval, removes a blocker, or weakens an unsupported claim correctly.
|
|
32
|
+
|
|
33
|
+
Do not count rewording, same-parameter reruns, training-only gains, longer prose, tool exit success, or unverified search titles.
|
|
34
|
+
|
|
35
|
+
After two no-progress rounds, run a stagnation review. A third no-progress round requires a material fork, user decision, or resumable non-SOLVED status.
|
|
36
|
+
|
|
37
|
+
## SOLVED gate
|
|
38
|
+
|
|
39
|
+
SOLVED requires frozen scope, all required obligations passed, critical adversarial checks passed, reproducibility material present, limitations stated, and—under High-Assurance—an independent artifact-only audit.
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
3
|
+
"$id": "https://github.com/yohanchen1/MathModelingAgent/schemas/attempt.schema.json",
|
|
4
|
+
"title": "Math Modeling Attempt",
|
|
5
|
+
"type": "object",
|
|
6
|
+
"additionalProperties": false,
|
|
7
|
+
"required": [
|
|
8
|
+
"schemaVersion",
|
|
9
|
+
"attempt",
|
|
10
|
+
"candidateId",
|
|
11
|
+
"objective",
|
|
12
|
+
"evidenceIds",
|
|
13
|
+
"closedObligationIds",
|
|
14
|
+
"issuesOpened",
|
|
15
|
+
"issuesClosed",
|
|
16
|
+
"progress",
|
|
17
|
+
"nextAction",
|
|
18
|
+
"status"
|
|
19
|
+
],
|
|
20
|
+
"properties": {
|
|
21
|
+
"schemaVersion": {
|
|
22
|
+
"const": 1
|
|
23
|
+
},
|
|
24
|
+
"attempt": {
|
|
25
|
+
"type": "integer",
|
|
26
|
+
"minimum": 1
|
|
27
|
+
},
|
|
28
|
+
"candidateId": {
|
|
29
|
+
"type": "string"
|
|
30
|
+
},
|
|
31
|
+
"objective": {
|
|
32
|
+
"type": "string",
|
|
33
|
+
"minLength": 1
|
|
34
|
+
},
|
|
35
|
+
"evidenceIds": {
|
|
36
|
+
"type": "array",
|
|
37
|
+
"items": {
|
|
38
|
+
"type": "string"
|
|
39
|
+
}
|
|
40
|
+
},
|
|
41
|
+
"closedObligationIds": {
|
|
42
|
+
"type": "array",
|
|
43
|
+
"items": {
|
|
44
|
+
"type": "string"
|
|
45
|
+
}
|
|
46
|
+
},
|
|
47
|
+
"issuesOpened": {
|
|
48
|
+
"type": "array",
|
|
49
|
+
"items": {
|
|
50
|
+
"type": "string"
|
|
51
|
+
}
|
|
52
|
+
},
|
|
53
|
+
"issuesClosed": {
|
|
54
|
+
"type": "array",
|
|
55
|
+
"items": {
|
|
56
|
+
"type": "string"
|
|
57
|
+
}
|
|
58
|
+
},
|
|
59
|
+
"progress": {
|
|
60
|
+
"type": "boolean"
|
|
61
|
+
},
|
|
62
|
+
"nextAction": {
|
|
63
|
+
"type": "string",
|
|
64
|
+
"minLength": 1
|
|
65
|
+
},
|
|
66
|
+
"status": {
|
|
67
|
+
"type": "string"
|
|
68
|
+
}
|
|
69
|
+
}
|
|
70
|
+
}
|