dsh-math-modeling-agent 0.1.1 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -1,3303 +0,0 @@
|
|
|
1
|
-
# MathModelingAgent DSH Plugin Implementation Plan
|
|
2
|
-
|
|
3
|
-
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
4
|
-
|
|
5
|
-
**Goal:** Build and verify a minimal, installable DSH bundle that provides evidence-driven mathematical modeling, independent auditing, and MCM/ICM final-panel scoring without requiring Python, Lean, or Wolfram at install time.
|
|
6
|
-
|
|
7
|
-
**Architecture:** Ship exactly two public Skills through the official `@deepseek-ai/dsh-skill-filesystem` bundle seam. Keep deterministic state, capability, Python-environment, and score arithmetic behind four small Node modules; keep mathematical judgment in selective Markdown references and require artifact-backed verification before any strong status.
|
|
8
|
-
|
|
9
|
-
**Tech Stack:** DSH bundle manifest and Cordis patch, Node.js ESM and built-in `node:test`, Markdown/YAML/JSON/JSONL, optional uv/Python, optional Lean/Lake, optional WolframScript.
|
|
10
|
-
|
|
11
|
-
---
|
|
12
|
-
|
|
13
|
-
## Scope decision
|
|
14
|
-
|
|
15
|
-
This remains one implementation plan because every task contributes to one installable package and one release gate. The MCM/ICM scorer is an internal mode of `math-modeling-audit`, not an independently deployed subsystem.
|
|
16
|
-
|
|
17
|
-
## Locked file map
|
|
18
|
-
|
|
19
|
-
- Create: `package.json` — npm metadata and `dsh.bundle.patch` declaration only.
|
|
20
|
-
- Create: `cordis.patch.yml` — mount the packaged Skill root through the official filesystem provider.
|
|
21
|
-
- Create: `.gitignore` — ignore only generated dependencies, packages, coverage, and run artifacts.
|
|
22
|
-
- Create: `LICENSE` — MIT license.
|
|
23
|
-
- Create: `README.md` — single concise bilingual user and contributor guide.
|
|
24
|
-
- Create: `skills/math-modeling-agent/SKILL.md` — sole solve/resume interface.
|
|
25
|
-
- Create: `skills/math-modeling-agent/references/*.md` — one authoritative file per internal policy.
|
|
26
|
-
- Create: `skills/math-modeling-agent/scripts/capability-probe.mjs` — read-only PATH capability discovery.
|
|
27
|
-
- Create: `skills/math-modeling-agent/scripts/python-environment.mjs` — run-local Python environment creation.
|
|
28
|
-
- Create: `skills/math-modeling-agent/scripts/run-state.mjs` — atomic state, transition journal, validation, and recovery.
|
|
29
|
-
- Create: `skills/math-modeling-agent/schemas/*.json` — versioned run, ledger, and attempt contracts.
|
|
30
|
-
- Create: `skills/math-modeling-agent/examples/*` — minimal and resumed golden runs only.
|
|
31
|
-
- Create: `skills/math-modeling-audit/SKILL.md` — independent audit and MCM/ICM judging interface.
|
|
32
|
-
- Create: `skills/math-modeling-audit/references/*.md` — verification, evidence, data/citation, and final-judge protocols.
|
|
33
|
-
- Create: `skills/math-modeling-audit/scripts/mcm-score.mjs` — deterministic 100-point arithmetic and cap enforcement.
|
|
34
|
-
- Create: `skills/math-modeling-audit/examples/*.md` — one generic audit and one MCM final-review example.
|
|
35
|
-
- Create: `tests/*.test.mjs` — one focused test file per deterministic module plus package integrity.
|
|
36
|
-
- Preserve: `docs/superpowers/specs/2026-08-22-math-modeling-dsh-plugin-design.md` — approved design source of truth.
|
|
37
|
-
|
|
38
|
-
### Task 1: Establish the installable DSH bundle shell
|
|
39
|
-
|
|
40
|
-
**Files:**
|
|
41
|
-
- Create: `tests/plugin-integrity.test.mjs`
|
|
42
|
-
- Create: `package.json`
|
|
43
|
-
- Create: `cordis.patch.yml`
|
|
44
|
-
- Create: `.gitignore`
|
|
45
|
-
- Create: `LICENSE`
|
|
46
|
-
|
|
47
|
-
- [ ] **Step 1: Write the failing bundle-integrity test**
|
|
48
|
-
|
|
49
|
-
Create `tests/plugin-integrity.test.mjs`:
|
|
50
|
-
|
|
51
|
-
~~~javascript
|
|
52
|
-
import assert from 'node:assert/strict'
|
|
53
|
-
import { readFile } from 'node:fs/promises'
|
|
54
|
-
import { dirname, resolve } from 'node:path'
|
|
55
|
-
import test from 'node:test'
|
|
56
|
-
import { fileURLToPath } from 'node:url'
|
|
57
|
-
|
|
58
|
-
const ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..')
|
|
59
|
-
|
|
60
|
-
async function read(relativePath) {
|
|
61
|
-
return readFile(resolve(ROOT, relativePath), 'utf8')
|
|
62
|
-
}
|
|
63
|
-
|
|
64
|
-
test('package declares a dependency-free DSH bundle', async () => {
|
|
65
|
-
const manifest = JSON.parse(await read('package.json'))
|
|
66
|
-
|
|
67
|
-
assert.equal(manifest.name, 'dsh-math-modeling-agent')
|
|
68
|
-
assert.equal(manifest.version, '0.1.0')
|
|
69
|
-
assert.deepEqual(manifest.dsh, {
|
|
70
|
-
bundle: { patch: './cordis.patch.yml' },
|
|
71
|
-
})
|
|
72
|
-
assert.deepEqual(manifest.files, [
|
|
73
|
-
'cordis.patch.yml',
|
|
74
|
-
'skills',
|
|
75
|
-
'README.md',
|
|
76
|
-
'LICENSE',
|
|
77
|
-
])
|
|
78
|
-
assert.equal(manifest.dependencies, undefined)
|
|
79
|
-
assert.equal(manifest.scripts.prepare, undefined)
|
|
80
|
-
})
|
|
81
|
-
|
|
82
|
-
test('Cordis patch mounts only the packaged Skill root', async () => {
|
|
83
|
-
const patch = await read('cordis.patch.yml')
|
|
84
|
-
|
|
85
|
-
assert.match(patch, /name: '@deepseek-ai\/dsh-skill-filesystem'/)
|
|
86
|
-
assert.match(patch, /providerName: dsh-math-modeling-agent/)
|
|
87
|
-
assert.match(patch, /includeDefaultRoots: false/)
|
|
88
|
-
assert.match(patch, /bundledSkillDir:/)
|
|
89
|
-
assert.match(patch, /new URL\('skills\/', baseUrl\)/)
|
|
90
|
-
assert.match(patch, /watch: false/)
|
|
91
|
-
assert.doesNotMatch(patch, /\.claude[\\/]plugins/i)
|
|
92
|
-
assert.doesNotMatch(patch, /C:[\/]Users[\/]/i)
|
|
93
|
-
})
|
|
94
|
-
~~~
|
|
95
|
-
|
|
96
|
-
- [ ] **Step 2: Run the test and verify RED**
|
|
97
|
-
|
|
98
|
-
Run:
|
|
99
|
-
|
|
100
|
-
~~~bash
|
|
101
|
-
node --test tests/plugin-integrity.test.mjs
|
|
102
|
-
~~~
|
|
103
|
-
|
|
104
|
-
Expected: FAIL with `ENOENT` for `package.json` or `cordis.patch.yml`.
|
|
105
|
-
|
|
106
|
-
- [ ] **Step 3: Create the minimal package manifest**
|
|
107
|
-
|
|
108
|
-
Create `package.json`:
|
|
109
|
-
|
|
110
|
-
~~~json
|
|
111
|
-
{
|
|
112
|
-
"name": "dsh-math-modeling-agent",
|
|
113
|
-
"version": "0.1.0",
|
|
114
|
-
"description": "Evidence-driven mathematical modeling and verification skills for DeepSeek Harness",
|
|
115
|
-
"type": "module",
|
|
116
|
-
"files": [
|
|
117
|
-
"cordis.patch.yml",
|
|
118
|
-
"skills",
|
|
119
|
-
"README.md",
|
|
120
|
-
"LICENSE"
|
|
121
|
-
],
|
|
122
|
-
"scripts": {
|
|
123
|
-
"test": "node --test",
|
|
124
|
-
"pack:check": "npm pack --dry-run --json"
|
|
125
|
-
},
|
|
126
|
-
"license": "MIT",
|
|
127
|
-
"repository": {
|
|
128
|
-
"type": "git",
|
|
129
|
-
"url": "git+https://github.com/yohanchen1/MathModelingAgent.git"
|
|
130
|
-
},
|
|
131
|
-
"keywords": [
|
|
132
|
-
"dsh",
|
|
133
|
-
"mathematical-modeling",
|
|
134
|
-
"verification",
|
|
135
|
-
"lean",
|
|
136
|
-
"wolfram",
|
|
137
|
-
"python"
|
|
138
|
-
],
|
|
139
|
-
"dsh": {
|
|
140
|
-
"bundle": {
|
|
141
|
-
"patch": "./cordis.patch.yml"
|
|
142
|
-
}
|
|
143
|
-
}
|
|
144
|
-
}
|
|
145
|
-
~~~
|
|
146
|
-
|
|
147
|
-
- [ ] **Step 4: Create the package-relative Cordis patch**
|
|
148
|
-
|
|
149
|
-
Create `cordis.patch.yml`:
|
|
150
|
-
|
|
151
|
-
~~~yaml
|
|
152
|
-
- insert:
|
|
153
|
-
- id: dsh-math-modeling-agent-skills
|
|
154
|
-
name: '@deepseek-ai/dsh-skill-filesystem'
|
|
155
|
-
config:
|
|
156
|
-
providerName: dsh-math-modeling-agent
|
|
157
|
-
includeDefaultRoots: false
|
|
158
|
-
bundledSkillDir: !!js "process.getBuiltinModule('node:url').fileURLToPath(new URL('skills/', baseUrl))"
|
|
159
|
-
watch: false
|
|
160
|
-
~~~
|
|
161
|
-
|
|
162
|
-
- [ ] **Step 5: Add the minimal ignore file and license**
|
|
163
|
-
|
|
164
|
-
Create `.gitignore`:
|
|
165
|
-
|
|
166
|
-
~~~gitignore
|
|
167
|
-
node_modules/
|
|
168
|
-
coverage/
|
|
169
|
-
*.tgz
|
|
170
|
-
math-modeling-runs/
|
|
171
|
-
.DS_Store
|
|
172
|
-
Thumbs.db
|
|
173
|
-
~~~
|
|
174
|
-
|
|
175
|
-
Create `LICENSE`:
|
|
176
|
-
|
|
177
|
-
~~~text
|
|
178
|
-
MIT License
|
|
179
|
-
|
|
180
|
-
Copyright (c) 2026 Yohan Chen
|
|
181
|
-
|
|
182
|
-
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
183
|
-
of this software and associated documentation files (the "Software"), to deal
|
|
184
|
-
in the Software without restriction, including without limitation the rights
|
|
185
|
-
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
186
|
-
copies of the Software, and to permit persons to whom the Software is
|
|
187
|
-
furnished to do so, subject to the following conditions:
|
|
188
|
-
|
|
189
|
-
The above copyright notice and this permission notice shall be included in all
|
|
190
|
-
copies or substantial portions of the Software.
|
|
191
|
-
|
|
192
|
-
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
193
|
-
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
194
|
-
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
195
|
-
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
196
|
-
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
197
|
-
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
198
|
-
SOFTWARE.
|
|
199
|
-
~~~
|
|
200
|
-
|
|
201
|
-
- [ ] **Step 6: Run the bundle test and verify GREEN**
|
|
202
|
-
|
|
203
|
-
Run:
|
|
204
|
-
|
|
205
|
-
~~~bash
|
|
206
|
-
node --test tests/plugin-integrity.test.mjs
|
|
207
|
-
~~~
|
|
208
|
-
|
|
209
|
-
Expected: 2 tests PASS, 0 failures.
|
|
210
|
-
|
|
211
|
-
- [ ] **Step 7: Commit the bundle shell**
|
|
212
|
-
|
|
213
|
-
~~~bash
|
|
214
|
-
git add package.json cordis.patch.yml .gitignore LICENSE tests/plugin-integrity.test.mjs
|
|
215
|
-
git commit -m "feat: add minimal dsh bundle shell"
|
|
216
|
-
~~~
|
|
217
|
-
|
|
218
|
-
### Task 2: Add read-only capability discovery
|
|
219
|
-
|
|
220
|
-
**Files:**
|
|
221
|
-
- Create: `tests/capability-probe.test.mjs`
|
|
222
|
-
- Create: `skills/math-modeling-agent/scripts/capability-probe.mjs`
|
|
223
|
-
|
|
224
|
-
- [ ] **Step 1: Write failing capability-probe tests**
|
|
225
|
-
|
|
226
|
-
Create `tests/capability-probe.test.mjs`:
|
|
227
|
-
|
|
228
|
-
~~~javascript
|
|
229
|
-
import assert from 'node:assert/strict'
|
|
230
|
-
import { chmod, mkdtemp, writeFile } from 'node:fs/promises'
|
|
231
|
-
import { tmpdir } from 'node:os'
|
|
232
|
-
import { delimiter, join } from 'node:path'
|
|
233
|
-
import test from 'node:test'
|
|
234
|
-
|
|
235
|
-
import {
|
|
236
|
-
findExecutable,
|
|
237
|
-
probeCapabilities,
|
|
238
|
-
} from '../skills/math-modeling-agent/scripts/capability-probe.mjs'
|
|
239
|
-
|
|
240
|
-
async function fakeExecutable(directory, baseName, platform) {
|
|
241
|
-
const name = platform === 'win32' ? `${baseName}.cmd` : baseName
|
|
242
|
-
const path = join(directory, name)
|
|
243
|
-
await writeFile(
|
|
244
|
-
path,
|
|
245
|
-
platform === 'win32' ? '@exit /b 0\r\n' : '#!/bin/sh\nexit 0\n',
|
|
246
|
-
)
|
|
247
|
-
if (platform !== 'win32') await chmod(path, 0o755)
|
|
248
|
-
return path
|
|
249
|
-
}
|
|
250
|
-
|
|
251
|
-
test('findExecutable returns the first PATH candidate', async () => {
|
|
252
|
-
const platform = process.platform
|
|
253
|
-
const directory = await mkdtemp(join(tmpdir(), 'mma-capability-'))
|
|
254
|
-
const expected = await fakeExecutable(directory, 'uv', platform)
|
|
255
|
-
const env = {
|
|
256
|
-
PATH: [directory, process.env.PATH ?? ''].join(delimiter),
|
|
257
|
-
PATHEXT: '.COM;.EXE;.BAT;.CMD',
|
|
258
|
-
}
|
|
259
|
-
|
|
260
|
-
assert.equal(findExecutable(['uv'], { env, platform }), expected)
|
|
261
|
-
})
|
|
262
|
-
|
|
263
|
-
test('probeCapabilities reports candidates without executing them', async () => {
|
|
264
|
-
const platform = process.platform
|
|
265
|
-
const directory = await mkdtemp(join(tmpdir(), 'mma-capability-'))
|
|
266
|
-
const uv = await fakeExecutable(directory, 'uv', platform)
|
|
267
|
-
const lean = await fakeExecutable(directory, 'lean', platform)
|
|
268
|
-
const env = {
|
|
269
|
-
PATH: directory,
|
|
270
|
-
PATHEXT: '.COM;.EXE;.BAT;.CMD',
|
|
271
|
-
}
|
|
272
|
-
|
|
273
|
-
const report = probeCapabilities({ env, platform })
|
|
274
|
-
|
|
275
|
-
assert.equal(report.schemaVersion, 1)
|
|
276
|
-
assert.equal(report.node.status, 'available')
|
|
277
|
-
assert.deepEqual(report.tools.uv, {
|
|
278
|
-
status: 'candidate',
|
|
279
|
-
executable: uv,
|
|
280
|
-
kind: 'uv',
|
|
281
|
-
reason: 'Found on PATH; execute through the DSH shell to verify version and usability.',
|
|
282
|
-
})
|
|
283
|
-
assert.equal(report.tools.lean.executable, lean)
|
|
284
|
-
assert.equal(report.tools.wolfram.status, 'unavailable')
|
|
285
|
-
})
|
|
286
|
-
~~~
|
|
287
|
-
|
|
288
|
-
- [ ] **Step 2: Run the test and verify RED**
|
|
289
|
-
|
|
290
|
-
Run:
|
|
291
|
-
|
|
292
|
-
~~~bash
|
|
293
|
-
node --test tests/capability-probe.test.mjs
|
|
294
|
-
~~~
|
|
295
|
-
|
|
296
|
-
Expected: FAIL with `ERR_MODULE_NOT_FOUND` for `capability-probe.mjs`.
|
|
297
|
-
|
|
298
|
-
- [ ] **Step 3: Implement PATH-only capability probing**
|
|
299
|
-
|
|
300
|
-
Create `skills/math-modeling-agent/scripts/capability-probe.mjs`:
|
|
301
|
-
|
|
302
|
-
~~~javascript
|
|
303
|
-
import { accessSync, constants, existsSync } from 'node:fs'
|
|
304
|
-
import { delimiter, extname, join } from 'node:path'
|
|
305
|
-
import { pathToFileURL } from 'node:url'
|
|
306
|
-
|
|
307
|
-
const REASON = 'Found on PATH; execute through the DSH shell to verify version and usability.'
|
|
308
|
-
|
|
309
|
-
const TOOL_CANDIDATES = {
|
|
310
|
-
uv: ['uv'],
|
|
311
|
-
python: ['python3', 'python', 'py'],
|
|
312
|
-
lean: ['lean'],
|
|
313
|
-
lake: ['lake'],
|
|
314
|
-
wolfram: ['wolframscript', 'WolframScript'],
|
|
315
|
-
}
|
|
316
|
-
|
|
317
|
-
function executableExtensions(env, platform) {
|
|
318
|
-
if (platform !== 'win32') return ['']
|
|
319
|
-
const raw = env.PATHEXT || '.COM;.EXE;.BAT;.CMD'
|
|
320
|
-
return ['', ...raw.split(';').filter(Boolean).map((value) => value.toLowerCase())]
|
|
321
|
-
}
|
|
322
|
-
|
|
323
|
-
function candidateNames(name, extensions, platform) {
|
|
324
|
-
if (platform !== 'win32' || extname(name)) return [name]
|
|
325
|
-
return extensions.map((extension) => `${name}${extension}`)
|
|
326
|
-
}
|
|
327
|
-
|
|
328
|
-
function isExecutable(path, platform) {
|
|
329
|
-
if (!existsSync(path)) return false
|
|
330
|
-
try {
|
|
331
|
-
accessSync(path, platform === 'win32' ? constants.F_OK : constants.X_OK)
|
|
332
|
-
return true
|
|
333
|
-
} catch {
|
|
334
|
-
return false
|
|
335
|
-
}
|
|
336
|
-
}
|
|
337
|
-
|
|
338
|
-
export function findExecutable(names, options = {}) {
|
|
339
|
-
const env = options.env ?? process.env
|
|
340
|
-
const platform = options.platform ?? process.platform
|
|
341
|
-
const directories = (env.PATH ?? '').split(delimiter).filter(Boolean)
|
|
342
|
-
const extensions = executableExtensions(env, platform)
|
|
343
|
-
|
|
344
|
-
for (const directory of directories) {
|
|
345
|
-
for (const name of names) {
|
|
346
|
-
for (const candidate of candidateNames(name, extensions, platform)) {
|
|
347
|
-
const path = join(directory, candidate)
|
|
348
|
-
if (isExecutable(path, platform)) return path
|
|
349
|
-
}
|
|
350
|
-
}
|
|
351
|
-
}
|
|
352
|
-
return undefined
|
|
353
|
-
}
|
|
354
|
-
|
|
355
|
-
function toolRecord(key, names, options) {
|
|
356
|
-
const executable = findExecutable(names, options)
|
|
357
|
-
if (!executable) {
|
|
358
|
-
return {
|
|
359
|
-
status: 'unavailable',
|
|
360
|
-
executable: null,
|
|
361
|
-
kind: key,
|
|
362
|
-
reason: 'No candidate executable was found on PATH.',
|
|
363
|
-
}
|
|
364
|
-
}
|
|
365
|
-
return {
|
|
366
|
-
status: 'candidate',
|
|
367
|
-
executable,
|
|
368
|
-
kind: key,
|
|
369
|
-
reason: REASON,
|
|
370
|
-
}
|
|
371
|
-
}
|
|
372
|
-
|
|
373
|
-
export function probeCapabilities(options = {}) {
|
|
374
|
-
const env = options.env ?? process.env
|
|
375
|
-
const platform = options.platform ?? process.platform
|
|
376
|
-
return {
|
|
377
|
-
schemaVersion: 1,
|
|
378
|
-
platform,
|
|
379
|
-
node: {
|
|
380
|
-
status: 'available',
|
|
381
|
-
executable: process.execPath,
|
|
382
|
-
version: process.version,
|
|
383
|
-
},
|
|
384
|
-
tools: Object.fromEntries(
|
|
385
|
-
Object.entries(TOOL_CANDIDATES).map(([key, names]) => [
|
|
386
|
-
key,
|
|
387
|
-
toolRecord(key, names, { env, platform }),
|
|
388
|
-
]),
|
|
389
|
-
),
|
|
390
|
-
}
|
|
391
|
-
}
|
|
392
|
-
|
|
393
|
-
function main() {
|
|
394
|
-
const pretty = process.argv.includes('--pretty')
|
|
395
|
-
process.stdout.write(
|
|
396
|
-
`${JSON.stringify(probeCapabilities(), null, pretty ? 2 : 0)}
|
|
397
|
-
`,
|
|
398
|
-
)
|
|
399
|
-
}
|
|
400
|
-
|
|
401
|
-
const invokedPath = process.argv[1]
|
|
402
|
-
if (invokedPath && import.meta.url === pathToFileURL(invokedPath).href) main()
|
|
403
|
-
~~~
|
|
404
|
-
|
|
405
|
-
- [ ] **Step 4: Run tests and inspect CLI JSON**
|
|
406
|
-
|
|
407
|
-
Run:
|
|
408
|
-
|
|
409
|
-
~~~bash
|
|
410
|
-
node --test tests/capability-probe.test.mjs
|
|
411
|
-
node skills/math-modeling-agent/scripts/capability-probe.mjs --pretty
|
|
412
|
-
~~~
|
|
413
|
-
|
|
414
|
-
Expected: 2 tests PASS; CLI emits valid JSON with Node available and external tools marked `candidate` or `unavailable`.
|
|
415
|
-
|
|
416
|
-
- [ ] **Step 5: Commit capability discovery**
|
|
417
|
-
|
|
418
|
-
~~~bash
|
|
419
|
-
git add tests/capability-probe.test.mjs skills/math-modeling-agent/scripts/capability-probe.mjs
|
|
420
|
-
git commit -m "feat: add deterministic capability discovery"
|
|
421
|
-
~~~
|
|
422
|
-
|
|
423
|
-
### Task 3: Add versioned run state, transitions, validation, and recovery
|
|
424
|
-
|
|
425
|
-
**Files:**
|
|
426
|
-
- Create: `skills/math-modeling-agent/schemas/run.schema.json`
|
|
427
|
-
- Create: `skills/math-modeling-agent/schemas/ledger.schema.json`
|
|
428
|
-
- Create: `skills/math-modeling-agent/schemas/attempt.schema.json`
|
|
429
|
-
- Create: `tests/run-state.test.mjs`
|
|
430
|
-
- Create: `tests/recovery.test.mjs`
|
|
431
|
-
- Create: `skills/math-modeling-agent/scripts/run-state.mjs`
|
|
432
|
-
|
|
433
|
-
- [ ] **Step 1: Write failing state and recovery tests**
|
|
434
|
-
|
|
435
|
-
Create `tests/run-state.test.mjs`:
|
|
436
|
-
|
|
437
|
-
~~~javascript
|
|
438
|
-
import assert from 'node:assert/strict'
|
|
439
|
-
import { mkdtemp, readFile, writeFile } from 'node:fs/promises'
|
|
440
|
-
import { tmpdir } from 'node:os'
|
|
441
|
-
import { join } from 'node:path'
|
|
442
|
-
import test from 'node:test'
|
|
443
|
-
|
|
444
|
-
import {
|
|
445
|
-
initRun,
|
|
446
|
-
transitionRun,
|
|
447
|
-
validateRun,
|
|
448
|
-
} from '../skills/math-modeling-agent/scripts/run-state.mjs'
|
|
449
|
-
|
|
450
|
-
async function newRoot() {
|
|
451
|
-
return mkdtemp(join(tmpdir(), 'mma-run-'))
|
|
452
|
-
}
|
|
453
|
-
|
|
454
|
-
test('initRun creates a valid TRIAGE checkpoint and ledger', async () => {
|
|
455
|
-
const root = await newRoot()
|
|
456
|
-
const run = await initRun(root, {
|
|
457
|
-
taskId: 'sample-task',
|
|
458
|
-
mode: 'standard',
|
|
459
|
-
})
|
|
460
|
-
|
|
461
|
-
assert.equal(run.status, 'TRIAGE')
|
|
462
|
-
assert.equal(run.eventSequence, 0)
|
|
463
|
-
assert.equal(run.currentAttempt, 0)
|
|
464
|
-
|
|
465
|
-
const ledger = JSON.parse(await readFile(join(root, 'ledger.json'), 'utf8'))
|
|
466
|
-
assert.deepEqual(ledger.claims, [])
|
|
467
|
-
assert.deepEqual(ledger.assumptions, [])
|
|
468
|
-
assert.deepEqual(ledger.candidates, [])
|
|
469
|
-
assert.equal((await validateRun(root)).valid, true)
|
|
470
|
-
})
|
|
471
|
-
|
|
472
|
-
test('transitionRun rejects ATTEMPT-to-SOLVED shortcuts', async () => {
|
|
473
|
-
const root = await newRoot()
|
|
474
|
-
await initRun(root, { taskId: 'shortcut-test', mode: 'standard' })
|
|
475
|
-
|
|
476
|
-
await assert.rejects(
|
|
477
|
-
transitionRun(root, {
|
|
478
|
-
to: 'SOLVED',
|
|
479
|
-
reason: 'Model says it is correct.',
|
|
480
|
-
evidenceIds: ['E-001'],
|
|
481
|
-
}),
|
|
482
|
-
/illegal transition TRIAGE -> SOLVED/,
|
|
483
|
-
)
|
|
484
|
-
})
|
|
485
|
-
|
|
486
|
-
test('legal transitions are evidence-linked and increment attempts', async () => {
|
|
487
|
-
const root = await newRoot()
|
|
488
|
-
await initRun(root, { taskId: 'legal-test', mode: 'standard' })
|
|
489
|
-
|
|
490
|
-
const sequence = [
|
|
491
|
-
['SCOPE_FROZEN', 'I-SCOPE'],
|
|
492
|
-
['INPUT_PROFILED', 'E-INPUT'],
|
|
493
|
-
['CLAIMS_REGISTERED', 'I-CLAIMS'],
|
|
494
|
-
['CANDIDATES_READY', 'I-CANDIDATES'],
|
|
495
|
-
['ATTEMPT', 'I-ATTEMPT'],
|
|
496
|
-
]
|
|
497
|
-
for (const [to, reference] of sequence) {
|
|
498
|
-
await transitionRun(root, {
|
|
499
|
-
to,
|
|
500
|
-
reason: `advance to ${to}`,
|
|
501
|
-
issueIds: [reference],
|
|
502
|
-
})
|
|
503
|
-
}
|
|
504
|
-
|
|
505
|
-
const status = JSON.parse(await readFile(join(root, 'run.json'), 'utf8'))
|
|
506
|
-
assert.equal(status.status, 'ATTEMPT')
|
|
507
|
-
assert.equal(status.currentAttempt, 1)
|
|
508
|
-
assert.equal(status.eventSequence, 5)
|
|
509
|
-
assert.equal((await validateRun(root)).valid, true)
|
|
510
|
-
})
|
|
511
|
-
|
|
512
|
-
test('SOLVED requires claims, passed obligations, and no open critical issues', async () => {
|
|
513
|
-
const root = await newRoot()
|
|
514
|
-
await initRun(root, { taskId: 'solved-gate', mode: 'standard' })
|
|
515
|
-
for (const [to, reference] of [
|
|
516
|
-
['SCOPE_FROZEN', 'I-SCOPE'],
|
|
517
|
-
['INPUT_PROFILED', 'E-INPUT'],
|
|
518
|
-
['CLAIMS_REGISTERED', 'I-CLAIMS'],
|
|
519
|
-
['CANDIDATES_READY', 'I-CANDIDATES'],
|
|
520
|
-
['ATTEMPT', 'I-ATTEMPT'],
|
|
521
|
-
['EXECUTE', 'E-EXECUTE'],
|
|
522
|
-
['VERIFY', 'E-VERIFY'],
|
|
523
|
-
]) {
|
|
524
|
-
await transitionRun(root, { to, reason: 'advance', issueIds: [reference] })
|
|
525
|
-
}
|
|
526
|
-
|
|
527
|
-
const ledgerPath = join(root, 'ledger.json')
|
|
528
|
-
const ledger = JSON.parse(await readFile(ledgerPath, 'utf8'))
|
|
529
|
-
ledger.claims = [{ id: 'C-001', text: 'x = 4' }]
|
|
530
|
-
ledger.obligations = [{ id: 'O-001', required: true, status: 'OPEN' }]
|
|
531
|
-
ledger.issues = []
|
|
532
|
-
await writeFile(ledgerPath, JSON.stringify(ledger, null, 2))
|
|
533
|
-
|
|
534
|
-
await assert.rejects(
|
|
535
|
-
transitionRun(root, { to: 'SOLVED', reason: 'done', evidenceIds: ['E-VERIFY'] }),
|
|
536
|
-
/required obligations remain open/,
|
|
537
|
-
)
|
|
538
|
-
|
|
539
|
-
ledger.obligations[0].status = 'PASS'
|
|
540
|
-
ledger.issues = [{ id: 'I-CRITICAL', severity: 'critical', status: 'OPEN' }]
|
|
541
|
-
await writeFile(ledgerPath, JSON.stringify(ledger, null, 2))
|
|
542
|
-
await assert.rejects(
|
|
543
|
-
transitionRun(root, { to: 'SOLVED', reason: 'done', evidenceIds: ['E-VERIFY'] }),
|
|
544
|
-
/critical issues remain open/,
|
|
545
|
-
)
|
|
546
|
-
|
|
547
|
-
ledger.issues[0].status = 'CLOSED'
|
|
548
|
-
await writeFile(ledgerPath, JSON.stringify(ledger, null, 2))
|
|
549
|
-
const solved = await transitionRun(root, {
|
|
550
|
-
to: 'SOLVED',
|
|
551
|
-
reason: 'all obligations passed',
|
|
552
|
-
evidenceIds: ['E-VERIFY'],
|
|
553
|
-
})
|
|
554
|
-
assert.equal(solved.status, 'SOLVED')
|
|
555
|
-
})
|
|
556
|
-
|
|
557
|
-
test('High-Assurance SOLVED requires an independent audit flag', async () => {
|
|
558
|
-
const root = await newRoot()
|
|
559
|
-
await initRun(root, { taskId: 'audit-gate', mode: 'high-assurance' })
|
|
560
|
-
for (const [to, reference] of [
|
|
561
|
-
['SCOPE_FROZEN', 'I-SCOPE'],
|
|
562
|
-
['INPUT_PROFILED', 'E-INPUT'],
|
|
563
|
-
['CLAIMS_REGISTERED', 'I-CLAIMS'],
|
|
564
|
-
['CANDIDATES_READY', 'I-CANDIDATES'],
|
|
565
|
-
['ATTEMPT', 'I-ATTEMPT'],
|
|
566
|
-
['EXECUTE', 'E-EXECUTE'],
|
|
567
|
-
['VERIFY', 'E-VERIFY'],
|
|
568
|
-
]) {
|
|
569
|
-
await transitionRun(root, { to, reason: 'advance', issueIds: [reference] })
|
|
570
|
-
}
|
|
571
|
-
|
|
572
|
-
const ledgerPath = join(root, 'ledger.json')
|
|
573
|
-
const ledger = JSON.parse(await readFile(ledgerPath, 'utf8'))
|
|
574
|
-
ledger.claims = [{ id: 'C-001', text: 'verified claim' }]
|
|
575
|
-
ledger.obligations = [{ id: 'O-001', required: true, status: 'PASS' }]
|
|
576
|
-
ledger.issues = []
|
|
577
|
-
ledger.scope.independentAuditPassed = false
|
|
578
|
-
await writeFile(ledgerPath, JSON.stringify(ledger, null, 2))
|
|
579
|
-
|
|
580
|
-
await assert.rejects(
|
|
581
|
-
transitionRun(root, { to: 'SOLVED', reason: 'done', evidenceIds: ['E-VERIFY'] }),
|
|
582
|
-
/independent audit/,
|
|
583
|
-
)
|
|
584
|
-
|
|
585
|
-
ledger.scope.independentAuditPassed = true
|
|
586
|
-
await writeFile(ledgerPath, JSON.stringify(ledger, null, 2))
|
|
587
|
-
const solved = await transitionRun(root, {
|
|
588
|
-
to: 'SOLVED',
|
|
589
|
-
reason: 'audit passed',
|
|
590
|
-
evidenceIds: ['E-AUDIT'],
|
|
591
|
-
})
|
|
592
|
-
assert.equal(solved.status, 'SOLVED')
|
|
593
|
-
})
|
|
594
|
-
~~~
|
|
595
|
-
|
|
596
|
-
Create `tests/recovery.test.mjs`:
|
|
597
|
-
|
|
598
|
-
~~~javascript
|
|
599
|
-
import assert from 'node:assert/strict'
|
|
600
|
-
import { mkdtemp, readFile, writeFile } from 'node:fs/promises'
|
|
601
|
-
import { tmpdir } from 'node:os'
|
|
602
|
-
import { join } from 'node:path'
|
|
603
|
-
import test from 'node:test'
|
|
604
|
-
|
|
605
|
-
import {
|
|
606
|
-
initRun,
|
|
607
|
-
recoverRun,
|
|
608
|
-
transitionRun,
|
|
609
|
-
validateRun,
|
|
610
|
-
} from '../skills/math-modeling-agent/scripts/run-state.mjs'
|
|
611
|
-
|
|
612
|
-
test('recoverRun restores the last journal snapshot without losing best candidate', async () => {
|
|
613
|
-
const root = await mkdtemp(join(tmpdir(), 'mma-recovery-'))
|
|
614
|
-
await initRun(root, { taskId: 'recovery-test', mode: 'high-assurance' })
|
|
615
|
-
await transitionRun(root, {
|
|
616
|
-
to: 'SCOPE_FROZEN',
|
|
617
|
-
reason: 'Scope accepted.',
|
|
618
|
-
issueIds: ['I-SCOPE'],
|
|
619
|
-
patch: { bestCandidateId: 'C-007' },
|
|
620
|
-
})
|
|
621
|
-
|
|
622
|
-
const corrupted = JSON.parse(await readFile(join(root, 'run.json'), 'utf8'))
|
|
623
|
-
corrupted.status = 'SOLVED'
|
|
624
|
-
corrupted.eventSequence = 999
|
|
625
|
-
corrupted.bestCandidateId = null
|
|
626
|
-
await writeFile(join(root, 'run.json'), JSON.stringify(corrupted, null, 2))
|
|
627
|
-
|
|
628
|
-
assert.equal((await validateRun(root)).valid, false)
|
|
629
|
-
const recovered = await recoverRun(root)
|
|
630
|
-
|
|
631
|
-
assert.equal(recovered.status, 'SCOPE_FROZEN')
|
|
632
|
-
assert.equal(recovered.eventSequence, 1)
|
|
633
|
-
assert.equal(recovered.bestCandidateId, 'C-007')
|
|
634
|
-
assert.equal((await validateRun(root)).valid, true)
|
|
635
|
-
})
|
|
636
|
-
~~~
|
|
637
|
-
|
|
638
|
-
- [ ] **Step 2: Run the tests and verify RED**
|
|
639
|
-
|
|
640
|
-
Run:
|
|
641
|
-
|
|
642
|
-
~~~bash
|
|
643
|
-
node --test tests/run-state.test.mjs tests/recovery.test.mjs
|
|
644
|
-
~~~
|
|
645
|
-
|
|
646
|
-
Expected: FAIL with `ERR_MODULE_NOT_FOUND` for `run-state.mjs`.
|
|
647
|
-
|
|
648
|
-
- [ ] **Step 3: Create the run schema**
|
|
649
|
-
|
|
650
|
-
Create `skills/math-modeling-agent/schemas/run.schema.json`:
|
|
651
|
-
|
|
652
|
-
~~~json
|
|
653
|
-
{
|
|
654
|
-
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
655
|
-
"$id": "https://github.com/yohanchen1/MathModelingAgent/schemas/run.schema.json",
|
|
656
|
-
"title": "Math Modeling Run",
|
|
657
|
-
"type": "object",
|
|
658
|
-
"additionalProperties": false,
|
|
659
|
-
"required": [
|
|
660
|
-
"schemaVersion",
|
|
661
|
-
"taskId",
|
|
662
|
-
"mode",
|
|
663
|
-
"status",
|
|
664
|
-
"eventSequence",
|
|
665
|
-
"currentAttempt",
|
|
666
|
-
"bestCandidateId",
|
|
667
|
-
"budget",
|
|
668
|
-
"createdAt",
|
|
669
|
-
"updatedAt"
|
|
670
|
-
],
|
|
671
|
-
"properties": {
|
|
672
|
-
"schemaVersion": { "const": 1 },
|
|
673
|
-
"taskId": { "type": "string", "pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$" },
|
|
674
|
-
"mode": { "enum": ["fast", "standard", "high-assurance"] },
|
|
675
|
-
"status": {
|
|
676
|
-
"enum": [
|
|
677
|
-
"TRIAGE", "SCOPE_FROZEN", "INPUT_PROFILED", "CLAIMS_REGISTERED",
|
|
678
|
-
"CANDIDATES_READY", "ATTEMPT", "EXECUTE", "VERIFY", "REVISE",
|
|
679
|
-
"RESEARCH", "FORK", "SOLVED", "PARTIAL", "CONDITIONAL",
|
|
680
|
-
"INCONCLUSIVE", "REFUTED", "INFEASIBLE", "UNIDENTIFIABLE",
|
|
681
|
-
"BLOCKED", "CANCELLED"
|
|
682
|
-
]
|
|
683
|
-
},
|
|
684
|
-
"eventSequence": { "type": "integer", "minimum": 0 },
|
|
685
|
-
"currentAttempt": { "type": "integer", "minimum": 0 },
|
|
686
|
-
"bestCandidateId": { "type": ["string", "null"] },
|
|
687
|
-
"budget": {
|
|
688
|
-
"type": "object",
|
|
689
|
-
"additionalProperties": false,
|
|
690
|
-
"required": ["attempts", "researchQueries", "computeSeconds"],
|
|
691
|
-
"properties": {
|
|
692
|
-
"attempts": { "type": "integer", "minimum": 1 },
|
|
693
|
-
"researchQueries": { "type": "integer", "minimum": 0 },
|
|
694
|
-
"computeSeconds": { "type": "integer", "minimum": 0 }
|
|
695
|
-
}
|
|
696
|
-
},
|
|
697
|
-
"createdAt": { "type": "string" },
|
|
698
|
-
"updatedAt": { "type": "string" }
|
|
699
|
-
}
|
|
700
|
-
}
|
|
701
|
-
~~~
|
|
702
|
-
|
|
703
|
-
- [ ] **Step 4: Create the ledger and attempt schemas**
|
|
704
|
-
|
|
705
|
-
Create `skills/math-modeling-agent/schemas/ledger.schema.json`:
|
|
706
|
-
|
|
707
|
-
~~~json
|
|
708
|
-
{
|
|
709
|
-
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
710
|
-
"$id": "https://github.com/yohanchen1/MathModelingAgent/schemas/ledger.schema.json",
|
|
711
|
-
"title": "Math Modeling Ledger",
|
|
712
|
-
"type": "object",
|
|
713
|
-
"additionalProperties": false,
|
|
714
|
-
"required": [
|
|
715
|
-
"schemaVersion", "taskId", "scope", "assumptions", "claims",
|
|
716
|
-
"obligations", "subproblems", "candidates", "issues"
|
|
717
|
-
],
|
|
718
|
-
"properties": {
|
|
719
|
-
"schemaVersion": { "const": 1 },
|
|
720
|
-
"taskId": { "type": "string" },
|
|
721
|
-
"scope": { "type": "object" },
|
|
722
|
-
"assumptions": { "type": "array" },
|
|
723
|
-
"claims": { "type": "array" },
|
|
724
|
-
"obligations": { "type": "array" },
|
|
725
|
-
"subproblems": { "type": "array" },
|
|
726
|
-
"candidates": { "type": "array" },
|
|
727
|
-
"issues": { "type": "array" }
|
|
728
|
-
}
|
|
729
|
-
}
|
|
730
|
-
~~~
|
|
731
|
-
|
|
732
|
-
Create `skills/math-modeling-agent/schemas/attempt.schema.json`:
|
|
733
|
-
|
|
734
|
-
~~~json
|
|
735
|
-
{
|
|
736
|
-
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
737
|
-
"$id": "https://github.com/yohanchen1/MathModelingAgent/schemas/attempt.schema.json",
|
|
738
|
-
"title": "Math Modeling Attempt",
|
|
739
|
-
"type": "object",
|
|
740
|
-
"additionalProperties": false,
|
|
741
|
-
"required": [
|
|
742
|
-
"schemaVersion", "attempt", "candidateId", "objective", "evidenceIds",
|
|
743
|
-
"closedObligationIds", "issuesOpened", "issuesClosed", "progress",
|
|
744
|
-
"nextAction", "status"
|
|
745
|
-
],
|
|
746
|
-
"properties": {
|
|
747
|
-
"schemaVersion": { "const": 1 },
|
|
748
|
-
"attempt": { "type": "integer", "minimum": 1 },
|
|
749
|
-
"candidateId": { "type": "string" },
|
|
750
|
-
"objective": { "type": "string", "minLength": 1 },
|
|
751
|
-
"evidenceIds": { "type": "array", "items": { "type": "string" } },
|
|
752
|
-
"closedObligationIds": { "type": "array", "items": { "type": "string" } },
|
|
753
|
-
"issuesOpened": { "type": "array", "items": { "type": "string" } },
|
|
754
|
-
"issuesClosed": { "type": "array", "items": { "type": "string" } },
|
|
755
|
-
"progress": { "type": "boolean" },
|
|
756
|
-
"nextAction": { "type": "string", "minLength": 1 },
|
|
757
|
-
"status": { "type": "string" }
|
|
758
|
-
}
|
|
759
|
-
}
|
|
760
|
-
~~~
|
|
761
|
-
|
|
762
|
-
- [ ] **Step 5: Implement the state Module**
|
|
763
|
-
|
|
764
|
-
Create `skills/math-modeling-agent/scripts/run-state.mjs`:
|
|
765
|
-
|
|
766
|
-
~~~javascript
|
|
767
|
-
import { appendFile, mkdir, readFile, rename, writeFile } from 'node:fs/promises'
|
|
768
|
-
import { randomUUID } from 'node:crypto'
|
|
769
|
-
import { dirname, resolve } from 'node:path'
|
|
770
|
-
import { pathToFileURL } from 'node:url'
|
|
771
|
-
|
|
772
|
-
const MODES = new Set(['fast', 'standard', 'high-assurance'])
|
|
773
|
-
const FINAL = new Set([
|
|
774
|
-
'SOLVED',
|
|
775
|
-
'PARTIAL',
|
|
776
|
-
'CONDITIONAL',
|
|
777
|
-
'INCONCLUSIVE',
|
|
778
|
-
'REFUTED',
|
|
779
|
-
'INFEASIBLE',
|
|
780
|
-
'UNIDENTIFIABLE',
|
|
781
|
-
'BLOCKED',
|
|
782
|
-
'CANCELLED',
|
|
783
|
-
])
|
|
784
|
-
|
|
785
|
-
const TRANSITIONS = {
|
|
786
|
-
TRIAGE: new Set(['SCOPE_FROZEN', 'BLOCKED', 'CANCELLED']),
|
|
787
|
-
SCOPE_FROZEN: new Set(['INPUT_PROFILED', 'BLOCKED', 'CANCELLED']),
|
|
788
|
-
INPUT_PROFILED: new Set(['CLAIMS_REGISTERED', 'BLOCKED', 'CANCELLED']),
|
|
789
|
-
CLAIMS_REGISTERED: new Set(['CANDIDATES_READY', 'BLOCKED', 'CANCELLED']),
|
|
790
|
-
CANDIDATES_READY: new Set(['ATTEMPT', 'BLOCKED', 'CANCELLED']),
|
|
791
|
-
ATTEMPT: new Set(['EXECUTE', 'BLOCKED', 'CANCELLED']),
|
|
792
|
-
EXECUTE: new Set(['VERIFY', 'BLOCKED', 'CANCELLED']),
|
|
793
|
-
VERIFY: new Set([
|
|
794
|
-
'REVISE',
|
|
795
|
-
'RESEARCH',
|
|
796
|
-
'FORK',
|
|
797
|
-
...FINAL,
|
|
798
|
-
]),
|
|
799
|
-
REVISE: new Set(['ATTEMPT', 'BLOCKED', 'CANCELLED']),
|
|
800
|
-
RESEARCH: new Set(['CANDIDATES_READY', 'BLOCKED', 'CANCELLED']),
|
|
801
|
-
FORK: new Set(['ATTEMPT', 'BLOCKED', 'CANCELLED']),
|
|
802
|
-
}
|
|
803
|
-
|
|
804
|
-
const DEFAULT_BUDGETS = {
|
|
805
|
-
fast: { attempts: 2, researchQueries: 0, computeSeconds: 60 },
|
|
806
|
-
standard: { attempts: 12, researchQueries: 12, computeSeconds: 1800 },
|
|
807
|
-
'high-assurance': { attempts: 24, researchQueries: 30, computeSeconds: 7200 },
|
|
808
|
-
}
|
|
809
|
-
|
|
810
|
-
function runPath(root) {
|
|
811
|
-
return resolve(root, 'run.json')
|
|
812
|
-
}
|
|
813
|
-
|
|
814
|
-
function ledgerPath(root) {
|
|
815
|
-
return resolve(root, 'ledger.json')
|
|
816
|
-
}
|
|
817
|
-
|
|
818
|
-
function eventsPath(root) {
|
|
819
|
-
return resolve(root, 'events.jsonl')
|
|
820
|
-
}
|
|
821
|
-
|
|
822
|
-
async function readJson(path) {
|
|
823
|
-
return JSON.parse(await readFile(path, 'utf8'))
|
|
824
|
-
}
|
|
825
|
-
|
|
826
|
-
async function atomicWriteJson(path, value) {
|
|
827
|
-
await mkdir(dirname(path), { recursive: true })
|
|
828
|
-
const temporary = `${path}.${process.pid}.${randomUUID()}.tmp`
|
|
829
|
-
await writeFile(temporary, `${JSON.stringify(value, null, 2)}\n`)
|
|
830
|
-
await rename(temporary, path)
|
|
831
|
-
}
|
|
832
|
-
|
|
833
|
-
async function appendEvent(root, event) {
|
|
834
|
-
await appendFile(eventsPath(root), `${JSON.stringify(event)}\n`)
|
|
835
|
-
}
|
|
836
|
-
|
|
837
|
-
function assertTaskId(taskId) {
|
|
838
|
-
if (!/^[a-z0-9]+(?:-[a-z0-9]+)*$/.test(taskId)) {
|
|
839
|
-
throw new TypeError('taskId must be kebab-case')
|
|
840
|
-
}
|
|
841
|
-
}
|
|
842
|
-
|
|
843
|
-
function assertReferences(evidenceIds, issueIds) {
|
|
844
|
-
if (![...(evidenceIds ?? []), ...(issueIds ?? [])].length) {
|
|
845
|
-
throw new TypeError('every transition requires at least one evidence or issue reference')
|
|
846
|
-
}
|
|
847
|
-
}
|
|
848
|
-
|
|
849
|
-
function assertSolvedGate(to, run, ledger) {
|
|
850
|
-
if (to !== 'SOLVED') return
|
|
851
|
-
if (!Array.isArray(ledger.claims) || ledger.claims.length === 0) {
|
|
852
|
-
throw new Error('SOLVED requires at least one registered claim')
|
|
853
|
-
}
|
|
854
|
-
const openObligations = ledger.obligations.filter(
|
|
855
|
-
(item) => item.required !== false && item.status !== 'PASS',
|
|
856
|
-
)
|
|
857
|
-
if (openObligations.length) throw new Error('required obligations remain open')
|
|
858
|
-
|
|
859
|
-
const openCriticalIssues = ledger.issues.filter(
|
|
860
|
-
(item) => item.severity === 'critical' && item.status !== 'CLOSED',
|
|
861
|
-
)
|
|
862
|
-
if (openCriticalIssues.length) throw new Error('critical issues remain open')
|
|
863
|
-
|
|
864
|
-
if (run.mode === 'high-assurance' && ledger.scope?.independentAuditPassed !== true) {
|
|
865
|
-
throw new Error('High-Assurance SOLVED requires an independent audit')
|
|
866
|
-
}
|
|
867
|
-
}
|
|
868
|
-
|
|
869
|
-
function assertRun(run) {
|
|
870
|
-
if (run.schemaVersion !== 1) throw new TypeError('unsupported run schemaVersion')
|
|
871
|
-
assertTaskId(run.taskId)
|
|
872
|
-
if (!MODES.has(run.mode)) throw new TypeError('invalid run mode')
|
|
873
|
-
if (!Number.isInteger(run.eventSequence) || run.eventSequence < 0) {
|
|
874
|
-
throw new TypeError('invalid eventSequence')
|
|
875
|
-
}
|
|
876
|
-
if (!Number.isInteger(run.currentAttempt) || run.currentAttempt < 0) {
|
|
877
|
-
throw new TypeError('invalid currentAttempt')
|
|
878
|
-
}
|
|
879
|
-
}
|
|
880
|
-
|
|
881
|
-
export async function initRun(root, input) {
|
|
882
|
-
assertTaskId(input.taskId)
|
|
883
|
-
if (!MODES.has(input.mode)) throw new TypeError('invalid run mode')
|
|
884
|
-
|
|
885
|
-
const now = new Date().toISOString()
|
|
886
|
-
const run = {
|
|
887
|
-
schemaVersion: 1,
|
|
888
|
-
taskId: input.taskId,
|
|
889
|
-
mode: input.mode,
|
|
890
|
-
status: 'TRIAGE',
|
|
891
|
-
eventSequence: 0,
|
|
892
|
-
currentAttempt: 0,
|
|
893
|
-
bestCandidateId: null,
|
|
894
|
-
budget: input.budget ?? DEFAULT_BUDGETS[input.mode],
|
|
895
|
-
createdAt: now,
|
|
896
|
-
updatedAt: now,
|
|
897
|
-
}
|
|
898
|
-
const ledger = {
|
|
899
|
-
schemaVersion: 1,
|
|
900
|
-
taskId: input.taskId,
|
|
901
|
-
scope: {},
|
|
902
|
-
assumptions: [],
|
|
903
|
-
claims: [],
|
|
904
|
-
obligations: [],
|
|
905
|
-
subproblems: [],
|
|
906
|
-
candidates: [],
|
|
907
|
-
issues: [],
|
|
908
|
-
}
|
|
909
|
-
|
|
910
|
-
await mkdir(resolve(root), { recursive: true })
|
|
911
|
-
await atomicWriteJson(ledgerPath(root), ledger)
|
|
912
|
-
await appendEvent(root, {
|
|
913
|
-
schemaVersion: 1,
|
|
914
|
-
sequence: 0,
|
|
915
|
-
type: 'RUN_INITIALIZED',
|
|
916
|
-
snapshot: run,
|
|
917
|
-
})
|
|
918
|
-
await atomicWriteJson(runPath(root), run)
|
|
919
|
-
return run
|
|
920
|
-
}
|
|
921
|
-
|
|
922
|
-
export async function transitionRun(root, input) {
|
|
923
|
-
const run = await readJson(runPath(root))
|
|
924
|
-
const ledger = await readJson(ledgerPath(root))
|
|
925
|
-
assertRun(run)
|
|
926
|
-
if (FINAL.has(run.status)) throw new Error(`run is terminal: ${run.status}`)
|
|
927
|
-
if (!TRANSITIONS[run.status]?.has(input.to)) {
|
|
928
|
-
throw new Error(`illegal transition ${run.status} -> ${input.to}`)
|
|
929
|
-
}
|
|
930
|
-
assertReferences(input.evidenceIds, input.issueIds)
|
|
931
|
-
assertSolvedGate(input.to, run, ledger)
|
|
932
|
-
|
|
933
|
-
const next = {
|
|
934
|
-
...run,
|
|
935
|
-
...(input.patch ?? {}),
|
|
936
|
-
status: input.to,
|
|
937
|
-
eventSequence: run.eventSequence + 1,
|
|
938
|
-
currentAttempt: run.currentAttempt + (input.to === 'ATTEMPT' ? 1 : 0),
|
|
939
|
-
updatedAt: new Date().toISOString(),
|
|
940
|
-
}
|
|
941
|
-
assertRun(next)
|
|
942
|
-
|
|
943
|
-
const event = {
|
|
944
|
-
schemaVersion: 1,
|
|
945
|
-
sequence: next.eventSequence,
|
|
946
|
-
type: 'STATUS_TRANSITION',
|
|
947
|
-
from: run.status,
|
|
948
|
-
to: next.status,
|
|
949
|
-
reason: input.reason,
|
|
950
|
-
evidenceIds: input.evidenceIds ?? [],
|
|
951
|
-
issueIds: input.issueIds ?? [],
|
|
952
|
-
snapshot: next,
|
|
953
|
-
}
|
|
954
|
-
await appendEvent(root, event)
|
|
955
|
-
await atomicWriteJson(runPath(root), next)
|
|
956
|
-
return next
|
|
957
|
-
}
|
|
958
|
-
|
|
959
|
-
export async function loadEvents(root) {
|
|
960
|
-
const raw = await readFile(eventsPath(root), 'utf8')
|
|
961
|
-
return raw.split(/\r?\n/).filter(Boolean).map((line) => JSON.parse(line))
|
|
962
|
-
}
|
|
963
|
-
|
|
964
|
-
export async function validateRun(root) {
|
|
965
|
-
const diagnostics = []
|
|
966
|
-
let run
|
|
967
|
-
let ledger
|
|
968
|
-
let events
|
|
969
|
-
try {
|
|
970
|
-
run = await readJson(runPath(root))
|
|
971
|
-
assertRun(run)
|
|
972
|
-
} catch (error) {
|
|
973
|
-
diagnostics.push(`run.json: ${error.message}`)
|
|
974
|
-
}
|
|
975
|
-
try {
|
|
976
|
-
ledger = await readJson(ledgerPath(root))
|
|
977
|
-
if (ledger.schemaVersion !== 1) throw new TypeError('unsupported schemaVersion')
|
|
978
|
-
} catch (error) {
|
|
979
|
-
diagnostics.push(`ledger.json: ${error.message}`)
|
|
980
|
-
}
|
|
981
|
-
try {
|
|
982
|
-
events = await loadEvents(root)
|
|
983
|
-
events.forEach((event, index) => {
|
|
984
|
-
if (event.sequence !== index) throw new TypeError(`non-monotonic sequence at ${index}`)
|
|
985
|
-
})
|
|
986
|
-
} catch (error) {
|
|
987
|
-
diagnostics.push(`events.jsonl: ${error.message}`)
|
|
988
|
-
}
|
|
989
|
-
|
|
990
|
-
const last = events?.at(-1)
|
|
991
|
-
if (run && ledger && run.taskId !== ledger.taskId) {
|
|
992
|
-
diagnostics.push('run and ledger taskId differ')
|
|
993
|
-
}
|
|
994
|
-
if (run && last && run.eventSequence !== last.sequence) {
|
|
995
|
-
diagnostics.push('run snapshot is not aligned with the event journal')
|
|
996
|
-
}
|
|
997
|
-
if (run && last && JSON.stringify(run) !== JSON.stringify(last.snapshot)) {
|
|
998
|
-
diagnostics.push('run snapshot differs from the last journal snapshot')
|
|
999
|
-
}
|
|
1000
|
-
|
|
1001
|
-
return { valid: diagnostics.length === 0, diagnostics }
|
|
1002
|
-
}
|
|
1003
|
-
|
|
1004
|
-
export async function recoverRun(root) {
|
|
1005
|
-
const events = await loadEvents(root)
|
|
1006
|
-
const last = events.at(-1)
|
|
1007
|
-
if (!last?.snapshot) throw new Error('event journal has no recoverable snapshot')
|
|
1008
|
-
assertRun(last.snapshot)
|
|
1009
|
-
await atomicWriteJson(runPath(root), last.snapshot)
|
|
1010
|
-
return last.snapshot
|
|
1011
|
-
}
|
|
1012
|
-
|
|
1013
|
-
export async function readRun(root) {
|
|
1014
|
-
const run = await readJson(runPath(root))
|
|
1015
|
-
assertRun(run)
|
|
1016
|
-
return run
|
|
1017
|
-
}
|
|
1018
|
-
|
|
1019
|
-
function usage() {
|
|
1020
|
-
return 'usage: run-state.mjs <init|transition|validate|recover|status> <run-root> [arguments]'
|
|
1021
|
-
}
|
|
1022
|
-
|
|
1023
|
-
async function main() {
|
|
1024
|
-
const [command, root, ...args] = process.argv.slice(2)
|
|
1025
|
-
if (!command || !root) throw new Error(usage())
|
|
1026
|
-
|
|
1027
|
-
let result
|
|
1028
|
-
if (command === 'init') {
|
|
1029
|
-
const [taskId, mode = 'standard'] = args
|
|
1030
|
-
result = await initRun(root, { taskId, mode })
|
|
1031
|
-
} else if (command === 'transition') {
|
|
1032
|
-
const [to, reason, references = ''] = args
|
|
1033
|
-
result = await transitionRun(root, {
|
|
1034
|
-
to,
|
|
1035
|
-
reason,
|
|
1036
|
-
issueIds: references.split(',').filter(Boolean),
|
|
1037
|
-
})
|
|
1038
|
-
} else if (command === 'validate') {
|
|
1039
|
-
result = await validateRun(root)
|
|
1040
|
-
if (!result.valid) process.exitCode = 1
|
|
1041
|
-
} else if (command === 'recover') {
|
|
1042
|
-
result = await recoverRun(root)
|
|
1043
|
-
} else if (command === 'status') {
|
|
1044
|
-
result = await readRun(root)
|
|
1045
|
-
} else {
|
|
1046
|
-
throw new Error(usage())
|
|
1047
|
-
}
|
|
1048
|
-
process.stdout.write(`${JSON.stringify(result, null, 2)}
|
|
1049
|
-
`)
|
|
1050
|
-
}
|
|
1051
|
-
|
|
1052
|
-
const invokedPath = process.argv[1]
|
|
1053
|
-
if (invokedPath && import.meta.url === pathToFileURL(invokedPath).href) {
|
|
1054
|
-
main().catch((error) => {
|
|
1055
|
-
process.stderr.write(`${error.message}
|
|
1056
|
-
`)
|
|
1057
|
-
process.exitCode = 1
|
|
1058
|
-
})
|
|
1059
|
-
}
|
|
1060
|
-
~~~
|
|
1061
|
-
|
|
1062
|
-
- [ ] **Step 6: Run state and recovery tests**
|
|
1063
|
-
|
|
1064
|
-
Run:
|
|
1065
|
-
|
|
1066
|
-
~~~bash
|
|
1067
|
-
node --test tests/run-state.test.mjs tests/recovery.test.mjs
|
|
1068
|
-
~~~
|
|
1069
|
-
|
|
1070
|
-
Expected: 6 tests PASS, 0 failures.
|
|
1071
|
-
|
|
1072
|
-
- [ ] **Step 7: Commit the state Module**
|
|
1073
|
-
|
|
1074
|
-
~~~bash
|
|
1075
|
-
git add skills/math-modeling-agent/schemas skills/math-modeling-agent/scripts/run-state.mjs tests/run-state.test.mjs tests/recovery.test.mjs
|
|
1076
|
-
git commit -m "feat: add resumable modeling run state"
|
|
1077
|
-
~~~
|
|
1078
|
-
|
|
1079
|
-
### Task 4: Add isolated Python environment creation
|
|
1080
|
-
|
|
1081
|
-
**Files:**
|
|
1082
|
-
- Create: `tests/python-environment.test.mjs`
|
|
1083
|
-
- Create: `skills/math-modeling-agent/scripts/python-environment.mjs`
|
|
1084
|
-
|
|
1085
|
-
- [ ] **Step 1: Write failing Python-environment tests**
|
|
1086
|
-
|
|
1087
|
-
Create `tests/python-environment.test.mjs`:
|
|
1088
|
-
|
|
1089
|
-
~~~javascript
|
|
1090
|
-
import assert from 'node:assert/strict'
|
|
1091
|
-
import { mkdtemp, readFile, writeFile } from 'node:fs/promises'
|
|
1092
|
-
import { tmpdir } from 'node:os'
|
|
1093
|
-
import { join } from 'node:path'
|
|
1094
|
-
import test from 'node:test'
|
|
1095
|
-
|
|
1096
|
-
import {
|
|
1097
|
-
createPythonEnvironment,
|
|
1098
|
-
validatePackageSpec,
|
|
1099
|
-
} from '../skills/math-modeling-agent/scripts/python-environment.mjs'
|
|
1100
|
-
|
|
1101
|
-
test('validatePackageSpec rejects URLs and VCS dependencies', () => {
|
|
1102
|
-
assert.throws(() => validatePackageSpec('git+https://example.com/pkg.git'), /unsafe package spec/)
|
|
1103
|
-
assert.throws(() => validatePackageSpec('../local.whl'), /unsafe package spec/)
|
|
1104
|
-
assert.equal(validatePackageSpec('numpy==2.2.0'), 'numpy==2.2.0')
|
|
1105
|
-
assert.equal(validatePackageSpec('scipy'), 'scipy')
|
|
1106
|
-
})
|
|
1107
|
-
|
|
1108
|
-
test('uv path creates only a run-local environment and lock', async () => {
|
|
1109
|
-
const root = await mkdtemp(join(tmpdir(), 'mma-python-'))
|
|
1110
|
-
const commands = []
|
|
1111
|
-
const fakeRun = async (command, args, options = {}) => {
|
|
1112
|
-
commands.push({ command, args, options })
|
|
1113
|
-
if (options.stdoutFile) await writeFile(options.stdoutFile, 'numpy==2.2.0\n')
|
|
1114
|
-
return 0
|
|
1115
|
-
}
|
|
1116
|
-
const capabilities = {
|
|
1117
|
-
tools: {
|
|
1118
|
-
uv: { status: 'candidate', executable: '/tools/uv', kind: 'uv' },
|
|
1119
|
-
python: { status: 'unavailable', executable: null, kind: 'python' },
|
|
1120
|
-
},
|
|
1121
|
-
}
|
|
1122
|
-
|
|
1123
|
-
const manifest = await createPythonEnvironment({
|
|
1124
|
-
runDir: root,
|
|
1125
|
-
packages: ['numpy==2.2.0'],
|
|
1126
|
-
capabilities,
|
|
1127
|
-
runCommand: fakeRun,
|
|
1128
|
-
})
|
|
1129
|
-
|
|
1130
|
-
assert.equal(manifest.status, 'ready')
|
|
1131
|
-
assert.equal(manifest.adapter, 'uv')
|
|
1132
|
-
assert.match(manifest.environmentDir, /\.python-env$/)
|
|
1133
|
-
assert.equal(commands[0].command, '/tools/uv')
|
|
1134
|
-
assert.deepEqual(commands[0].args.slice(0, 2), ['venv', '--seed'])
|
|
1135
|
-
assert.equal(await readFile(join(root, 'requirements.lock'), 'utf8'), 'numpy==2.2.0\n')
|
|
1136
|
-
})
|
|
1137
|
-
|
|
1138
|
-
test('missing runtime returns a blocker instead of modifying the system', async () => {
|
|
1139
|
-
const root = await mkdtemp(join(tmpdir(), 'mma-python-'))
|
|
1140
|
-
const manifest = await createPythonEnvironment({
|
|
1141
|
-
runDir: root,
|
|
1142
|
-
packages: [],
|
|
1143
|
-
capabilities: {
|
|
1144
|
-
tools: {
|
|
1145
|
-
uv: { status: 'unavailable', executable: null },
|
|
1146
|
-
python: { status: 'unavailable', executable: null },
|
|
1147
|
-
},
|
|
1148
|
-
},
|
|
1149
|
-
runCommand: async () => {
|
|
1150
|
-
throw new Error('must not run')
|
|
1151
|
-
},
|
|
1152
|
-
})
|
|
1153
|
-
|
|
1154
|
-
assert.equal(manifest.status, 'blocked')
|
|
1155
|
-
assert.equal(manifest.reason, 'No uv or Python runtime candidate is available.')
|
|
1156
|
-
})
|
|
1157
|
-
~~~
|
|
1158
|
-
|
|
1159
|
-
- [ ] **Step 2: Run the test and verify RED**
|
|
1160
|
-
|
|
1161
|
-
Run:
|
|
1162
|
-
|
|
1163
|
-
~~~bash
|
|
1164
|
-
node --test tests/python-environment.test.mjs
|
|
1165
|
-
~~~
|
|
1166
|
-
|
|
1167
|
-
Expected: FAIL with `ERR_MODULE_NOT_FOUND` for `python-environment.mjs`.
|
|
1168
|
-
|
|
1169
|
-
- [ ] **Step 3: Implement the isolated environment Module**
|
|
1170
|
-
|
|
1171
|
-
Create `skills/math-modeling-agent/scripts/python-environment.mjs`:
|
|
1172
|
-
|
|
1173
|
-
~~~javascript
|
|
1174
|
-
import { closeSync, openSync } from 'node:fs'
|
|
1175
|
-
import { mkdir, rename, writeFile } from 'node:fs/promises'
|
|
1176
|
-
import { randomUUID } from 'node:crypto'
|
|
1177
|
-
import { basename, dirname, resolve } from 'node:path'
|
|
1178
|
-
import { spawnSync } from 'node:child_process'
|
|
1179
|
-
import { pathToFileURL } from 'node:url'
|
|
1180
|
-
|
|
1181
|
-
const PACKAGE_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._-]*(?:==[A-Za-z0-9][A-Za-z0-9._+-]*)?$/
|
|
1182
|
-
|
|
1183
|
-
export function validatePackageSpec(value) {
|
|
1184
|
-
if (!PACKAGE_PATTERN.test(value)) throw new TypeError(`unsafe package spec: ${value}`)
|
|
1185
|
-
return value
|
|
1186
|
-
}
|
|
1187
|
-
|
|
1188
|
-
function environmentPython(environmentDir) {
|
|
1189
|
-
return process.platform === 'win32'
|
|
1190
|
-
? resolve(environmentDir, 'Scripts', 'python.exe')
|
|
1191
|
-
: resolve(environmentDir, 'bin', 'python')
|
|
1192
|
-
}
|
|
1193
|
-
|
|
1194
|
-
async function atomicWriteJson(path, value) {
|
|
1195
|
-
await mkdir(dirname(path), { recursive: true })
|
|
1196
|
-
const temporary = `${path}.${process.pid}.${randomUUID()}.tmp`
|
|
1197
|
-
await writeFile(temporary, `${JSON.stringify(value, null, 2)}\n`)
|
|
1198
|
-
await rename(temporary, path)
|
|
1199
|
-
}
|
|
1200
|
-
|
|
1201
|
-
async function defaultRun(command, args, options = {}) {
|
|
1202
|
-
let descriptor
|
|
1203
|
-
try {
|
|
1204
|
-
descriptor = options.stdoutFile ? openSync(options.stdoutFile, 'w') : undefined
|
|
1205
|
-
const result = spawnSync(command, args, {
|
|
1206
|
-
cwd: options.cwd,
|
|
1207
|
-
shell: false,
|
|
1208
|
-
stdio: descriptor === undefined
|
|
1209
|
-
? 'inherit'
|
|
1210
|
-
: ['ignore', descriptor, 'inherit'],
|
|
1211
|
-
})
|
|
1212
|
-
return result.status ?? 1
|
|
1213
|
-
} finally {
|
|
1214
|
-
if (descriptor !== undefined) closeSync(descriptor)
|
|
1215
|
-
}
|
|
1216
|
-
}
|
|
1217
|
-
|
|
1218
|
-
function assertSuccess(status, label) {
|
|
1219
|
-
if (status !== 0) throw new Error(`${label} failed with exit code ${status}`)
|
|
1220
|
-
}
|
|
1221
|
-
|
|
1222
|
-
function pythonVenvArgs(record, environmentDir) {
|
|
1223
|
-
const name = basename(record.executable).toLowerCase()
|
|
1224
|
-
if (name === 'py' || name === 'py.exe') return ['-3', '-m', 'venv', environmentDir]
|
|
1225
|
-
return ['-m', 'venv', environmentDir]
|
|
1226
|
-
}
|
|
1227
|
-
|
|
1228
|
-
export async function createPythonEnvironment(input) {
|
|
1229
|
-
const runDir = resolve(input.runDir)
|
|
1230
|
-
const environmentDir = resolve(runDir, '.python-env')
|
|
1231
|
-
const lockPath = resolve(runDir, 'requirements.lock')
|
|
1232
|
-
const manifestPath = resolve(runDir, 'python-environment.json')
|
|
1233
|
-
const packages = input.packages.map(validatePackageSpec)
|
|
1234
|
-
const runCommand = input.runCommand ?? defaultRun
|
|
1235
|
-
const uv = input.capabilities.tools?.uv
|
|
1236
|
-
const python = input.capabilities.tools?.python
|
|
1237
|
-
const commands = []
|
|
1238
|
-
|
|
1239
|
-
await mkdir(runDir, { recursive: true })
|
|
1240
|
-
|
|
1241
|
-
if (uv?.status === 'candidate' && uv.executable) {
|
|
1242
|
-
const createArgs = ['venv', '--seed', environmentDir]
|
|
1243
|
-
commands.push({ command: uv.executable, args: createArgs })
|
|
1244
|
-
assertSuccess(await runCommand(uv.executable, createArgs, { cwd: runDir }), 'uv venv')
|
|
1245
|
-
|
|
1246
|
-
const pythonPath = environmentPython(environmentDir)
|
|
1247
|
-
if (packages.length) {
|
|
1248
|
-
const installArgs = ['pip', 'install', '--python', pythonPath, ...packages]
|
|
1249
|
-
commands.push({ command: uv.executable, args: installArgs })
|
|
1250
|
-
assertSuccess(await runCommand(uv.executable, installArgs, { cwd: runDir }), 'uv pip install')
|
|
1251
|
-
}
|
|
1252
|
-
const freezeArgs = ['-m', 'pip', 'freeze']
|
|
1253
|
-
commands.push({ command: pythonPath, args: freezeArgs, stdoutFile: lockPath })
|
|
1254
|
-
assertSuccess(
|
|
1255
|
-
await runCommand(pythonPath, freezeArgs, { cwd: runDir, stdoutFile: lockPath }),
|
|
1256
|
-
'pip freeze',
|
|
1257
|
-
)
|
|
1258
|
-
|
|
1259
|
-
const manifest = {
|
|
1260
|
-
schemaVersion: 1,
|
|
1261
|
-
status: 'ready',
|
|
1262
|
-
adapter: 'uv',
|
|
1263
|
-
environmentDir,
|
|
1264
|
-
python: pythonPath,
|
|
1265
|
-
requestedPackages: packages,
|
|
1266
|
-
lockPath,
|
|
1267
|
-
commands,
|
|
1268
|
-
}
|
|
1269
|
-
await atomicWriteJson(manifestPath, manifest)
|
|
1270
|
-
return manifest
|
|
1271
|
-
}
|
|
1272
|
-
|
|
1273
|
-
if (python?.status === 'candidate' && python.executable) {
|
|
1274
|
-
const createArgs = pythonVenvArgs(python, environmentDir)
|
|
1275
|
-
commands.push({ command: python.executable, args: createArgs })
|
|
1276
|
-
assertSuccess(
|
|
1277
|
-
await runCommand(python.executable, createArgs, { cwd: runDir }),
|
|
1278
|
-
'python venv',
|
|
1279
|
-
)
|
|
1280
|
-
|
|
1281
|
-
const pythonPath = environmentPython(environmentDir)
|
|
1282
|
-
if (packages.length) {
|
|
1283
|
-
const installArgs = ['-m', 'pip', 'install', ...packages]
|
|
1284
|
-
commands.push({ command: pythonPath, args: installArgs })
|
|
1285
|
-
assertSuccess(
|
|
1286
|
-
await runCommand(pythonPath, installArgs, { cwd: runDir }),
|
|
1287
|
-
'pip install',
|
|
1288
|
-
)
|
|
1289
|
-
}
|
|
1290
|
-
const freezeArgs = ['-m', 'pip', 'freeze']
|
|
1291
|
-
commands.push({ command: pythonPath, args: freezeArgs, stdoutFile: lockPath })
|
|
1292
|
-
assertSuccess(
|
|
1293
|
-
await runCommand(pythonPath, freezeArgs, { cwd: runDir, stdoutFile: lockPath }),
|
|
1294
|
-
'pip freeze',
|
|
1295
|
-
)
|
|
1296
|
-
|
|
1297
|
-
const manifest = {
|
|
1298
|
-
schemaVersion: 1,
|
|
1299
|
-
status: 'ready',
|
|
1300
|
-
adapter: 'venv',
|
|
1301
|
-
environmentDir,
|
|
1302
|
-
python: pythonPath,
|
|
1303
|
-
requestedPackages: packages,
|
|
1304
|
-
lockPath,
|
|
1305
|
-
commands,
|
|
1306
|
-
}
|
|
1307
|
-
await atomicWriteJson(manifestPath, manifest)
|
|
1308
|
-
return manifest
|
|
1309
|
-
}
|
|
1310
|
-
|
|
1311
|
-
const manifest = {
|
|
1312
|
-
schemaVersion: 1,
|
|
1313
|
-
status: 'blocked',
|
|
1314
|
-
adapter: null,
|
|
1315
|
-
environmentDir,
|
|
1316
|
-
python: null,
|
|
1317
|
-
requestedPackages: packages,
|
|
1318
|
-
lockPath: null,
|
|
1319
|
-
commands,
|
|
1320
|
-
reason: 'No uv or Python runtime candidate is available.',
|
|
1321
|
-
}
|
|
1322
|
-
await atomicWriteJson(manifestPath, manifest)
|
|
1323
|
-
return manifest
|
|
1324
|
-
}
|
|
1325
|
-
|
|
1326
|
-
function parseArguments(argv) {
|
|
1327
|
-
const result = { packages: [] }
|
|
1328
|
-
for (let index = 0; index < argv.length; index += 1) {
|
|
1329
|
-
const value = argv[index]
|
|
1330
|
-
if (value === '--run-dir') result.runDir = argv[++index]
|
|
1331
|
-
else if (value === '--capabilities') result.capabilitiesPath = argv[++index]
|
|
1332
|
-
else if (value === '--package') result.packages.push(argv[++index])
|
|
1333
|
-
else throw new Error(`unknown argument: ${value}`)
|
|
1334
|
-
}
|
|
1335
|
-
if (!result.runDir || !result.capabilitiesPath) {
|
|
1336
|
-
throw new Error('usage: python-environment.mjs --run-dir <path> --capabilities <json> [--package <spec>]')
|
|
1337
|
-
}
|
|
1338
|
-
return result
|
|
1339
|
-
}
|
|
1340
|
-
|
|
1341
|
-
async function main() {
|
|
1342
|
-
const args = parseArguments(process.argv.slice(2))
|
|
1343
|
-
const { readFile } = await import('node:fs/promises')
|
|
1344
|
-
const capabilities = JSON.parse(await readFile(args.capabilitiesPath, 'utf8'))
|
|
1345
|
-
const manifest = await createPythonEnvironment({
|
|
1346
|
-
runDir: args.runDir,
|
|
1347
|
-
packages: args.packages,
|
|
1348
|
-
capabilities,
|
|
1349
|
-
})
|
|
1350
|
-
process.stdout.write(`${JSON.stringify(manifest, null, 2)}
|
|
1351
|
-
`)
|
|
1352
|
-
if (manifest.status === 'blocked') process.exitCode = 2
|
|
1353
|
-
}
|
|
1354
|
-
|
|
1355
|
-
const invokedPath = process.argv[1]
|
|
1356
|
-
if (invokedPath && import.meta.url === pathToFileURL(invokedPath).href) {
|
|
1357
|
-
main().catch((error) => {
|
|
1358
|
-
process.stderr.write(`${error.message}
|
|
1359
|
-
`)
|
|
1360
|
-
process.exitCode = 1
|
|
1361
|
-
})
|
|
1362
|
-
}
|
|
1363
|
-
~~~
|
|
1364
|
-
|
|
1365
|
-
- [ ] **Step 4: Run Python-environment tests**
|
|
1366
|
-
|
|
1367
|
-
Run:
|
|
1368
|
-
|
|
1369
|
-
~~~bash
|
|
1370
|
-
node --test tests/python-environment.test.mjs
|
|
1371
|
-
~~~
|
|
1372
|
-
|
|
1373
|
-
Expected: 3 tests PASS, 0 failures.
|
|
1374
|
-
|
|
1375
|
-
- [ ] **Step 5: Commit isolated Python setup**
|
|
1376
|
-
|
|
1377
|
-
~~~bash
|
|
1378
|
-
git add tests/python-environment.test.mjs skills/math-modeling-agent/scripts/python-environment.mjs
|
|
1379
|
-
git commit -m "feat: add isolated python environment setup"
|
|
1380
|
-
~~~
|
|
1381
|
-
|
|
1382
|
-
|
|
1383
|
-
### Task 5: Create the deep math-modeling Skill and intake protocol
|
|
1384
|
-
|
|
1385
|
-
**Files:**
|
|
1386
|
-
- Modify: `tests/plugin-integrity.test.mjs`
|
|
1387
|
-
- Create: `skills/math-modeling-agent/SKILL.md`
|
|
1388
|
-
- Create: `skills/math-modeling-agent/references/workflow.md`
|
|
1389
|
-
- Create: `skills/math-modeling-agent/references/math-grill.md`
|
|
1390
|
-
- Create: `skills/math-modeling-agent/references/problem-types.md`
|
|
1391
|
-
|
|
1392
|
-
- [ ] **Step 1: Extend the integrity test to require exactly two valid Skills and real resources**
|
|
1393
|
-
|
|
1394
|
-
In `tests/plugin-integrity.test.mjs`, replace:
|
|
1395
|
-
|
|
1396
|
-
~~~javascript
|
|
1397
|
-
import { readFile } from 'node:fs/promises'
|
|
1398
|
-
~~~
|
|
1399
|
-
|
|
1400
|
-
with:
|
|
1401
|
-
|
|
1402
|
-
~~~javascript
|
|
1403
|
-
import { access, readFile, readdir } from 'node:fs/promises'
|
|
1404
|
-
~~~
|
|
1405
|
-
|
|
1406
|
-
Append this block:
|
|
1407
|
-
|
|
1408
|
-
~~~javascript
|
|
1409
|
-
function parseFrontmatter(source) {
|
|
1410
|
-
const match = /^---\r?\n([\s\S]*?)\r?\n---\r?\n([\s\S]*)$/.exec(source)
|
|
1411
|
-
assert.ok(match, 'SKILL.md must contain YAML frontmatter')
|
|
1412
|
-
const metadata = Object.fromEntries(
|
|
1413
|
-
match[1]
|
|
1414
|
-
.split(/\r?\n/)
|
|
1415
|
-
.filter(Boolean)
|
|
1416
|
-
.map((line) => {
|
|
1417
|
-
const separator = line.indexOf(':')
|
|
1418
|
-
assert.ok(separator > 0, 'invalid frontmatter line: ' + line)
|
|
1419
|
-
return [line.slice(0, separator).trim(), line.slice(separator + 1).trim()]
|
|
1420
|
-
}),
|
|
1421
|
-
)
|
|
1422
|
-
return { metadata, body: match[2] }
|
|
1423
|
-
}
|
|
1424
|
-
|
|
1425
|
-
test('package exposes exactly two kebab-case Skills', async () => {
|
|
1426
|
-
const skillRoot = resolve(ROOT, 'skills')
|
|
1427
|
-
const entries = (await readdir(skillRoot, { withFileTypes: true }))
|
|
1428
|
-
.filter((entry) => entry.isDirectory())
|
|
1429
|
-
.map((entry) => entry.name)
|
|
1430
|
-
.sort()
|
|
1431
|
-
|
|
1432
|
-
assert.deepEqual(entries, ['math-modeling-agent', 'math-modeling-audit'])
|
|
1433
|
-
|
|
1434
|
-
for (const name of entries) {
|
|
1435
|
-
const source = await read('skills/' + name + '/SKILL.md')
|
|
1436
|
-
const { metadata } = parseFrontmatter(source)
|
|
1437
|
-
assert.equal(metadata.name, name)
|
|
1438
|
-
assert.match(metadata.description, /This skill should be used when/i)
|
|
1439
|
-
}
|
|
1440
|
-
})
|
|
1441
|
-
|
|
1442
|
-
test('every resource named by a Skill exists within that Skill bundle', async () => {
|
|
1443
|
-
for (const name of ['math-modeling-agent', 'math-modeling-audit']) {
|
|
1444
|
-
const source = await read('skills/' + name + '/SKILL.md')
|
|
1445
|
-
const { body } = parseFrontmatter(source)
|
|
1446
|
-
const resources = [
|
|
1447
|
-
...body.matchAll(/`((?:references|scripts|schemas)\/[a-z0-9._/-]+)`/g),
|
|
1448
|
-
].map((match) => match[1])
|
|
1449
|
-
|
|
1450
|
-
assert.ok(resources.length > 0, name + ' must name its selective resources')
|
|
1451
|
-
for (const resource of resources) {
|
|
1452
|
-
await access(resolve(ROOT, 'skills', name, resource))
|
|
1453
|
-
}
|
|
1454
|
-
}
|
|
1455
|
-
})
|
|
1456
|
-
|
|
1457
|
-
test('packaged files contain no developer-specific paths', async () => {
|
|
1458
|
-
const files = [
|
|
1459
|
-
'cordis.patch.yml',
|
|
1460
|
-
'skills/math-modeling-agent/SKILL.md',
|
|
1461
|
-
'skills/math-modeling-audit/SKILL.md',
|
|
1462
|
-
]
|
|
1463
|
-
for (const file of files) {
|
|
1464
|
-
const source = await read(file)
|
|
1465
|
-
assert.doesNotMatch(source, /C:\\Users\\16605|\.claude[\\/]plugins/i)
|
|
1466
|
-
}
|
|
1467
|
-
})
|
|
1468
|
-
~~~
|
|
1469
|
-
|
|
1470
|
-
- [ ] **Step 2: Run the integrity test and verify RED**
|
|
1471
|
-
|
|
1472
|
-
Run:
|
|
1473
|
-
|
|
1474
|
-
~~~bash
|
|
1475
|
-
node --test tests/plugin-integrity.test.mjs
|
|
1476
|
-
~~~
|
|
1477
|
-
|
|
1478
|
-
Expected: FAIL because the two Skill bundles and their resources do not exist.
|
|
1479
|
-
|
|
1480
|
-
- [ ] **Step 3: Create the main Skill interface**
|
|
1481
|
-
|
|
1482
|
-
Create `skills/math-modeling-agent/SKILL.md`:
|
|
1483
|
-
|
|
1484
|
-
~~~markdown
|
|
1485
|
-
---
|
|
1486
|
-
name: math-modeling-agent
|
|
1487
|
-
description: This skill should be used when the user asks to "建立数学模型", "解决数学建模题", "继续上次建模", "寻找新的建模方向", "solve this modeling problem", or needs an evidence-backed mathematical model with computation, research, verification, per-attempt reports, and resumable state.
|
|
1488
|
-
whenToUse: Use for open-ended mathematical modeling, prediction, optimization, estimation, simulation, mechanism, decision, or multi-part contest problems; do not use for auditing an already completed artifact without changing it.
|
|
1489
|
-
user-invocable: true
|
|
1490
|
-
---
|
|
1491
|
-
|
|
1492
|
-
# Math Modeling Agent
|
|
1493
|
-
|
|
1494
|
-
## Goal
|
|
1495
|
-
|
|
1496
|
-
Move one mathematical problem from scoped intake to an evidence-backed conclusion or a precise resumable scientific status.
|
|
1497
|
-
|
|
1498
|
-
## Interface
|
|
1499
|
-
|
|
1500
|
-
Accept problem text, attachments, or an existing run directory. Do not ask the user to choose internal phases, Python/Lean/Wolfram adapters, file names, or state fields.
|
|
1501
|
-
|
|
1502
|
-
Return a concise current status, supported claims, unresolved obligations, attempt delta, artifact path, and next action or final report.
|
|
1503
|
-
|
|
1504
|
-
## Default workflow
|
|
1505
|
-
|
|
1506
|
-
1. Route the task to Fast, Standard, or High-Assurance using `references/problem-types.md`.
|
|
1507
|
-
2. Read available inputs before asking questions; apply the one-question math grill in `references/math-grill.md`.
|
|
1508
|
-
3. Create or validate run state with `scripts/run-state.mjs` and follow `references/state-recovery.md`.
|
|
1509
|
-
4. Profile inputs and build the subproblem DAG using `references/data-subproblems.md`.
|
|
1510
|
-
5. Register assumptions, claims, and verification obligations from `references/claims-evidence.md`.
|
|
1511
|
-
6. Build a baseline and materially different candidates using `references/modeling-methodology.md`.
|
|
1512
|
-
7. Probe tools with `scripts/capability-probe.mjs`; create a run-local Python environment with `scripts/python-environment.mjs` only when computation is required.
|
|
1513
|
-
8. Execute, verify, critique, and revise according to `references/workflow.md` and `references/tool-policy.md`.
|
|
1514
|
-
9. Escalate evidence gaps or stalled directions through `references/research-breakthrough.md`.
|
|
1515
|
-
10. Write only incremental attempt and terminal reports defined by `references/report-contract.md`.
|
|
1516
|
-
|
|
1517
|
-
## Invariants
|
|
1518
|
-
|
|
1519
|
-
- ATTEMPT never transitions directly to SOLVED.
|
|
1520
|
-
- Tool success, model confidence, paper count, or an analyzer adjective is not evidence of correctness.
|
|
1521
|
-
- Every status transition cites evidence or issue IDs.
|
|
1522
|
-
- Every material claim has explicit verification obligations.
|
|
1523
|
-
- Upstream artifact changes make dependent downstream results stale.
|
|
1524
|
-
- Failed tools or providers never erase the best valid candidate or raw output.
|
|
1525
|
-
- Private chain-of-thought is never stored as evidence.
|
|
1526
|
-
- Continue only when a round closes an obligation, adds reproducible evidence, removes a blocker, tightens uncertainty, or refutes a candidate.
|
|
1527
|
-
|
|
1528
|
-
## Tool degradation
|
|
1529
|
-
|
|
1530
|
-
Python is recommended but not required for installation. Lean and Wolfram are optional. If a required tool is unavailable, weaken the claim and report the missing obligation; never pretend execution or formal verification occurred.
|
|
1531
|
-
|
|
1532
|
-
## Boundaries
|
|
1533
|
-
|
|
1534
|
-
Do not review an existing paper as a final judge; use `math-modeling-audit` for that durable job. Do not install Lean or Wolfram. Do not send private raw data to literature search. Do not execute commands derived from problem text.
|
|
1535
|
-
~~~
|
|
1536
|
-
|
|
1537
|
-
- [ ] **Step 4: Create the state-machine workflow reference**
|
|
1538
|
-
|
|
1539
|
-
Create `skills/math-modeling-agent/references/workflow.md`:
|
|
1540
|
-
|
|
1541
|
-
~~~markdown
|
|
1542
|
-
# Modeling Workflow
|
|
1543
|
-
|
|
1544
|
-
## Modes
|
|
1545
|
-
|
|
1546
|
-
- Fast: explicit low-risk calculation; at most two clarifications; direct inverse, substitution, domain, or unit check.
|
|
1547
|
-
- Standard: normal modeling; input profile, baseline, candidates, execution, sensitivity, and reproducibility.
|
|
1548
|
-
- High-Assurance: publication, proof, global optimum, causal, safety, guarantee, or high-stakes use; require independent audit.
|
|
1549
|
-
|
|
1550
|
-
## States
|
|
1551
|
-
|
|
1552
|
-
TRIAGE → SCOPE_FROZEN → INPUT_PROFILED → CLAIMS_REGISTERED → CANDIDATES_READY → ATTEMPT → EXECUTE → VERIFY.
|
|
1553
|
-
|
|
1554
|
-
VERIFY may move to REVISE, RESEARCH, FORK, an independent audit, or one terminal status:
|
|
1555
|
-
SOLVED, PARTIAL, CONDITIONAL, INCONCLUSIVE, REFUTED, INFEASIBLE, UNIDENTIFIABLE, BLOCKED, CANCELLED.
|
|
1556
|
-
|
|
1557
|
-
## Round contract
|
|
1558
|
-
|
|
1559
|
-
Each round has one objective and records:
|
|
1560
|
-
|
|
1561
|
-
- candidate and assumption delta;
|
|
1562
|
-
- actual command or derivation artifact;
|
|
1563
|
-
- new evidence IDs;
|
|
1564
|
-
- obligations closed;
|
|
1565
|
-
- issues opened and closed;
|
|
1566
|
-
- whether auditable progress occurred;
|
|
1567
|
-
- budget used;
|
|
1568
|
-
- one next action.
|
|
1569
|
-
|
|
1570
|
-
## Progress test
|
|
1571
|
-
|
|
1572
|
-
Count progress only when a round closes an obligation, adds reproducible evidence, refutes a candidate, tightens a bound or uncertainty interval, removes a blocker, or weakens an unsupported claim correctly.
|
|
1573
|
-
|
|
1574
|
-
Do not count rewording, same-parameter reruns, training-only gains, longer prose, tool exit success, or unverified search titles.
|
|
1575
|
-
|
|
1576
|
-
After two no-progress rounds, run a stagnation review. A third no-progress round requires a material fork, user decision, or resumable non-SOLVED status.
|
|
1577
|
-
|
|
1578
|
-
## SOLVED gate
|
|
1579
|
-
|
|
1580
|
-
SOLVED requires frozen scope, all required obligations passed, critical adversarial checks passed, reproducibility material present, limitations stated, and—under High-Assurance—an independent artifact-only audit.
|
|
1581
|
-
~~~
|
|
1582
|
-
|
|
1583
|
-
- [ ] **Step 5: Create the mathematical grilling reference**
|
|
1584
|
-
|
|
1585
|
-
Create `skills/math-modeling-agent/references/math-grill.md`:
|
|
1586
|
-
|
|
1587
|
-
~~~markdown
|
|
1588
|
-
# Math-Grill Protocol
|
|
1589
|
-
|
|
1590
|
-
Ask one question at a time. Read the problem and attachments first. Ask only questions whose answer can change the model, evidence obligation, or deliverable.
|
|
1591
|
-
|
|
1592
|
-
## Minimum routing questions
|
|
1593
|
-
|
|
1594
|
-
1. What output is required: number, proof, model, prediction, optimum, or decision?
|
|
1595
|
-
2. Does data exist, and what is its source?
|
|
1596
|
-
3. Is the requested claim proof, global optimality, causality, safety, or a useful approximation?
|
|
1597
|
-
4. What error, confidence, or decision risk is acceptable?
|
|
1598
|
-
5. Is the result for practice, competition, publication, production, or high-stakes use?
|
|
1599
|
-
|
|
1600
|
-
## Selective modeling fields
|
|
1601
|
-
|
|
1602
|
-
Collect only applicable fields:
|
|
1603
|
-
|
|
1604
|
-
- background and practical objective;
|
|
1605
|
-
- explicit and hidden subproblems;
|
|
1606
|
-
- data, label source, units, quality, time, and group structure;
|
|
1607
|
-
- decision/state variables and parameters;
|
|
1608
|
-
- objective or evaluation metric;
|
|
1609
|
-
- equality, inequality, integer, nonnegative, logical, initial, and boundary constraints;
|
|
1610
|
-
- assumptions and their source category;
|
|
1611
|
-
- acceptance criteria and final deliverable.
|
|
1612
|
-
|
|
1613
|
-
## Rules
|
|
1614
|
-
|
|
1615
|
-
- Infer before asking.
|
|
1616
|
-
- Ask the highest-impact unknown first.
|
|
1617
|
-
- Mark irrelevant fields “not applicable” with a reason.
|
|
1618
|
-
- Turn “自行判断” into an explicit modeling assumption with risk and sensitivity plan.
|
|
1619
|
-
- Do not freeze scope while a missing answer can change the problem type or claim strength.
|
|
1620
|
-
- Do not force a proof problem to invent data or an exploratory analysis to invent an objective function.
|
|
1621
|
-
~~~
|
|
1622
|
-
|
|
1623
|
-
- [ ] **Step 6: Create the problem-type routing reference**
|
|
1624
|
-
|
|
1625
|
-
Create `skills/math-modeling-agent/references/problem-types.md`:
|
|
1626
|
-
|
|
1627
|
-
~~~markdown
|
|
1628
|
-
# Problem Types and Obligations
|
|
1629
|
-
|
|
1630
|
-
| Type | Primary output | Mandatory checks |
|
|
1631
|
-
|---|---|---|
|
|
1632
|
-
| Symbolic calculation | exact expression or roots | domain, equivalence, substitution, singular cases |
|
|
1633
|
-
| Theorem/proposition | proof or counterexample | quantifiers, assumptions, boundary cases, proof completeness |
|
|
1634
|
-
| Deterministic numerical | approximation | residual, convergence, stability, discretization error |
|
|
1635
|
-
| Feasibility | satisfying assignment | every constraint, tolerance, certificate |
|
|
1636
|
-
| Optimization | local/global solution | objective fidelity, constraints, bounds, optimality gap/certificate |
|
|
1637
|
-
| Statistical estimation | estimate and uncertainty | sampling assumptions, interval, residual diagnostics |
|
|
1638
|
-
| Prediction/classification | out-of-sample performance | leakage-safe split, baseline, calibration, uncertainty |
|
|
1639
|
-
| Causal inference | identified effect | estimand, temporal order, confounding, sensitivity |
|
|
1640
|
-
| Stochastic/risk | distribution or risk measure | distribution assumptions, Monte Carlo error, tail behavior |
|
|
1641
|
-
| Inverse problem | recovered parameter/structure | identifiability, conditioning, regularization |
|
|
1642
|
-
| Mechanism/system | dynamic behavior | structural validity, units, calibration, initial/boundary conditions |
|
|
1643
|
-
| Decision analysis | recommendation | preference/weight source, constraints, sensitivity, regret |
|
|
1644
|
-
| Exploratory analysis | hypotheses/patterns | no causal upgrade, multiplicity, robustness |
|
|
1645
|
-
| Literature synthesis | supported method/fact | source verification, applicability, conflicting evidence |
|
|
1646
|
-
|
|
1647
|
-
Use multiple labels when necessary, but choose one primary output and one explicit assurance mode.
|
|
1648
|
-
~~~
|
|
1649
|
-
|
|
1650
|
-
- [ ] **Step 7: Run integrity test and verify expected partial RED**
|
|
1651
|
-
|
|
1652
|
-
Run:
|
|
1653
|
-
|
|
1654
|
-
~~~bash
|
|
1655
|
-
node --test tests/plugin-integrity.test.mjs
|
|
1656
|
-
~~~
|
|
1657
|
-
|
|
1658
|
-
Expected: Skill-count test still FAILS because `math-modeling-audit` is not created; all `math-modeling-agent` resource checks PASS.
|
|
1659
|
-
|
|
1660
|
-
- [ ] **Step 8: Commit the main Skill interface and intake protocol**
|
|
1661
|
-
|
|
1662
|
-
~~~bash
|
|
1663
|
-
git add tests/plugin-integrity.test.mjs skills/math-modeling-agent/SKILL.md skills/math-modeling-agent/references/workflow.md skills/math-modeling-agent/references/math-grill.md skills/math-modeling-agent/references/problem-types.md
|
|
1664
|
-
git commit -m "feat: add mathematical modeling skill interface"
|
|
1665
|
-
~~~
|
|
1666
|
-
|
|
1667
|
-
|
|
1668
|
-
|
|
1669
|
-
### Task 6: Add modeling, evidence, research, tool, report, and recovery references
|
|
1670
|
-
|
|
1671
|
-
**Files:**
|
|
1672
|
-
- Create: `skills/math-modeling-agent/references/claims-evidence.md`
|
|
1673
|
-
- Create: `skills/math-modeling-agent/references/modeling-methodology.md`
|
|
1674
|
-
- Create: `skills/math-modeling-agent/references/data-subproblems.md`
|
|
1675
|
-
- Create: `skills/math-modeling-agent/references/research-breakthrough.md`
|
|
1676
|
-
- Create: `skills/math-modeling-agent/references/tool-policy.md`
|
|
1677
|
-
- Create: `skills/math-modeling-agent/references/state-recovery.md`
|
|
1678
|
-
- Create: `skills/math-modeling-agent/references/report-contract.md`
|
|
1679
|
-
- Create: `skills/math-modeling-agent/examples/minimal-run/README.md`
|
|
1680
|
-
- Create: `skills/math-modeling-agent/examples/resumed-run/README.md`
|
|
1681
|
-
|
|
1682
|
-
- [ ] **Step 1: Create the claim/evidence contract**
|
|
1683
|
-
|
|
1684
|
-
Create `skills/math-modeling-agent/references/claims-evidence.md`:
|
|
1685
|
-
|
|
1686
|
-
~~~markdown
|
|
1687
|
-
# Claims and Evidence
|
|
1688
|
-
|
|
1689
|
-
## Claim record
|
|
1690
|
-
|
|
1691
|
-
Store claim ID, exact wording, type, scope/quantifiers, assumptions, risk, verification obligations, evidence IDs, countercheck, status, and limitations.
|
|
1692
|
-
|
|
1693
|
-
## Assumption record
|
|
1694
|
-
|
|
1695
|
-
Store statement, scope, source category, risk, sensitivity plan, validation status, and affected claims. Allowed source categories are problem, data, theory, verified literature, domain instruction, and modeling simplification.
|
|
1696
|
-
|
|
1697
|
-
## Evidence record
|
|
1698
|
-
|
|
1699
|
-
Store evidence ID, method, tool, timestamp, covered claim IDs, input/output hashes, command/workdir/environment, exit code, output artifacts, tolerance, limitations, and level:
|
|
1700
|
-
DERIVED, EXECUTED, VERIFIED, INDEPENDENTLY_VERIFIED, EXTERNALLY_VALIDATED, or NOT_CHECKED.
|
|
1701
|
-
|
|
1702
|
-
## Obligation patterns
|
|
1703
|
-
|
|
1704
|
-
- Numeric value: independent recomputation, residual/error bound, domain, and boundary check.
|
|
1705
|
-
- Feasibility: all constraints and justified tolerance on original scale.
|
|
1706
|
-
- Global optimum: convexity/KKT/duality, exact search, or certified bound; otherwise say local or best found.
|
|
1707
|
-
- Uniqueness: proof or explicit non-uniqueness disclaimer.
|
|
1708
|
-
- Prediction: untouched test/external validation, leakage-safe split, baseline, uncertainty, and calibration.
|
|
1709
|
-
- Causal effect: estimand, identification, temporal order, confounding, and sensitivity.
|
|
1710
|
-
- Robustness: parameter, data, seed, specification, and scenario perturbation.
|
|
1711
|
-
- Theorem: readable or kernel-checked proof plus formalization-fidelity audit.
|
|
1712
|
-
- Literature support: resolvable source, exact location, excerpt, retrieval date, and mapped claim.
|
|
1713
|
-
- Reproducibility: data/code/config versions, lock, seed, tolerance, commands, and clean rerun.
|
|
1714
|
-
|
|
1715
|
-
Evidence strength may not be weaker than claim strength. Weaken unsupported claims; never weaken the verification rule.
|
|
1716
|
-
~~~
|
|
1717
|
-
|
|
1718
|
-
- [ ] **Step 2: Create the modeling-methodology reference**
|
|
1719
|
-
|
|
1720
|
-
Create `skills/math-modeling-agent/references/modeling-methodology.md`:
|
|
1721
|
-
|
|
1722
|
-
~~~markdown
|
|
1723
|
-
# Modeling Methodology
|
|
1724
|
-
|
|
1725
|
-
## Mechanism first
|
|
1726
|
-
|
|
1727
|
-
Map real objects to variables, mechanism to equations, reality constraints to mathematical constraints, and requested decisions to objective/evaluation criteria. Never choose a named method first and force the problem into it.
|
|
1728
|
-
|
|
1729
|
-
## Candidate portfolio
|
|
1730
|
-
|
|
1731
|
-
Always begin with a defensible baseline. Add a more complex candidate only when it targets a documented baseline failure. Candidates must differ in mechanism, information, assumptions, or verification route—not wording.
|
|
1732
|
-
|
|
1733
|
-
For each candidate record:
|
|
1734
|
-
|
|
1735
|
-
- stable ID and parent/fork;
|
|
1736
|
-
- target subproblem and claims;
|
|
1737
|
-
- mechanism and equations;
|
|
1738
|
-
- assumptions and parameter sources;
|
|
1739
|
-
- input requirements;
|
|
1740
|
-
- expected information gain;
|
|
1741
|
-
- verification obligations;
|
|
1742
|
-
- stop/abandon conditions;
|
|
1743
|
-
- complexity, interpretability, and compute cost.
|
|
1744
|
-
|
|
1745
|
-
## Selection
|
|
1746
|
-
|
|
1747
|
-
Prefer the simplest candidate that meets evidence and accuracy requirements. Compare verification coverage, critical issues, empirical performance, robustness, complexity, interpretability, cost, and reproducibility. Do not reward model count, deep learning, metaheuristics, AHP, TOPSIS, entropy weights, or Monte Carlo by name.
|
|
1748
|
-
|
|
1749
|
-
## Core checks
|
|
1750
|
-
|
|
1751
|
-
- units and dimensions;
|
|
1752
|
-
- limiting and boundary cases;
|
|
1753
|
-
- known special cases;
|
|
1754
|
-
- constraint completeness;
|
|
1755
|
-
- identifiability and conditioning;
|
|
1756
|
-
- convergence and numerical stability;
|
|
1757
|
-
- data leakage and post-hoc parameters;
|
|
1758
|
-
- baseline and ablation;
|
|
1759
|
-
- sensitivity of conclusion-changing parameters;
|
|
1760
|
-
- model-result-conclusion traceability.
|
|
1761
|
-
~~~
|
|
1762
|
-
|
|
1763
|
-
- [ ] **Step 3: Create input profiling and subproblem dependency guidance**
|
|
1764
|
-
|
|
1765
|
-
Create `skills/math-modeling-agent/references/data-subproblems.md`:
|
|
1766
|
-
|
|
1767
|
-
~~~markdown
|
|
1768
|
-
# Inputs, Data, and Subproblems
|
|
1769
|
-
|
|
1770
|
-
## Input manifest
|
|
1771
|
-
|
|
1772
|
-
For each accessible artifact record stable ID, cryptographic hash, source, path, type, size, encoding, readability, and extraction provenance. For tables record sheets, rows, columns, inferred types, units, missingness, duplicates, ranges, anomalies, label source, time structure, group structure, and leakage risk.
|
|
1773
|
-
|
|
1774
|
-
Do not silently ignore unsupported files. Mark them unreadable and state the minimum representation needed.
|
|
1775
|
-
|
|
1776
|
-
## Leakage audit
|
|
1777
|
-
|
|
1778
|
-
Before model selection, identify target labels, future information, repeated entities, grouped observations, temporal ordering, and preprocessing fit scope. Fit imputation, scaling, feature selection, dimensionality reduction, and thresholds only inside training folds. Use time or group splits when ordinary random splits leak information.
|
|
1779
|
-
|
|
1780
|
-
## Subproblem DAG
|
|
1781
|
-
|
|
1782
|
-
Each subproblem records ID, exact task, required inputs, produced claims/artifacts, dependencies, acceptance criteria, and status. Downstream work references upstream IDs rather than copied prose.
|
|
1783
|
-
|
|
1784
|
-
When an upstream artifact changes, mark every dependent output stale. If an upstream node is blocked, downstream work must choose exactly one state: blocked, provisional with explicit condition, or alternative independent path.
|
|
1785
|
-
~~~
|
|
1786
|
-
|
|
1787
|
-
- [ ] **Step 4: Create research and breakthrough escalation guidance**
|
|
1788
|
-
|
|
1789
|
-
Create `skills/math-modeling-agent/references/research-breakthrough.md`:
|
|
1790
|
-
|
|
1791
|
-
~~~markdown
|
|
1792
|
-
# Research and Breakthrough
|
|
1793
|
-
|
|
1794
|
-
Research only when it closes a named evidence gap, validates a parameter/assumption, supplies an applicable method, or explains a failure.
|
|
1795
|
-
|
|
1796
|
-
## Source record
|
|
1797
|
-
|
|
1798
|
-
Record author, title, year, venue, DOI/stable URL, version, retrieval date, exact page/section/theorem/table, excerpt, mapped claim, source quality, applicability, and support strength. Distinguish full text, abstract only, metadata only, secondary source, conflicting evidence, and citation needed.
|
|
1799
|
-
|
|
1800
|
-
## Method matrix
|
|
1801
|
-
|
|
1802
|
-
For each candidate method record applicability, assumptions, data needs, implementation cost, verification route, reason to try, and reason to reject.
|
|
1803
|
-
|
|
1804
|
-
## Isolated campaign
|
|
1805
|
-
|
|
1806
|
-
For genuinely hard multi-direction problems, give isolated subagents independent briefs and prevent cross-talk. Require a counterexample family, the first unproved step, honest evidence level, and direction-exhaustion discipline. The coordinator reads final reports and adjudicates.
|
|
1807
|
-
|
|
1808
|
-
## Wall memo
|
|
1809
|
-
|
|
1810
|
-
A direction may be abandoned only when refuted, blocked by a precise missing tool/theory/information, or after natural variants are exhausted. Write:
|
|
1811
|
-
|
|
1812
|
-
- target and direction;
|
|
1813
|
-
- paths tried and evidence;
|
|
1814
|
-
- exact wall and wall type;
|
|
1815
|
-
- breakthrough condition;
|
|
1816
|
-
- restart checklist.
|
|
1817
|
-
|
|
1818
|
-
On a new tool, model, source, or resumed request, scan wall memos before restarting from zero.
|
|
1819
|
-
~~~
|
|
1820
|
-
|
|
1821
|
-
- [ ] **Step 5: Create tool adapter policy**
|
|
1822
|
-
|
|
1823
|
-
Create `skills/math-modeling-agent/references/tool-policy.md`:
|
|
1824
|
-
|
|
1825
|
-
~~~markdown
|
|
1826
|
-
# Tool Policy
|
|
1827
|
-
|
|
1828
|
-
## Capability discovery
|
|
1829
|
-
|
|
1830
|
-
Run `scripts/capability-probe.mjs`; treat PATH hits as candidates, then verify version and usability through the available DSH shell. Record unavailable, candidate, available, or unknown with reason.
|
|
1831
|
-
|
|
1832
|
-
## Python
|
|
1833
|
-
|
|
1834
|
-
Use `scripts/python-environment.mjs`. Prefer uv, then an existing Python runtime plus venv. Keep the environment under the run directory. Install only required normalized PyPI package names; VCS URLs, local wheels, or arbitrary indexes require explicit user approval. Record lock, seed, command, workdir, timeout, stdout/stderr, exit code, and generated files.
|
|
1835
|
-
|
|
1836
|
-
## Lean
|
|
1837
|
-
|
|
1838
|
-
Never auto-install. Use only for formalizable obligations. Record Lean/Lake/library versions, axioms/imports, proof artifact, absence of sorry/admit, and a separate natural-language-to-formal-statement fidelity check.
|
|
1839
|
-
|
|
1840
|
-
## Wolfram
|
|
1841
|
-
|
|
1842
|
-
Never auto-install. Verify executable and license usability. Use for applicable symbolic, exact, or high-precision claims and state the exact coverage boundary.
|
|
1843
|
-
|
|
1844
|
-
## Search, PDF, and images
|
|
1845
|
-
|
|
1846
|
-
Use available web, PDF, vision/OCR, or local Python readers. Do not send private raw data in search queries. Missing capabilities lower evidence level rather than causing fabricated execution.
|
|
1847
|
-
~~~
|
|
1848
|
-
|
|
1849
|
-
- [ ] **Step 6: Create state and recovery guidance**
|
|
1850
|
-
|
|
1851
|
-
Create `skills/math-modeling-agent/references/state-recovery.md`:
|
|
1852
|
-
|
|
1853
|
-
~~~markdown
|
|
1854
|
-
# State and Recovery
|
|
1855
|
-
|
|
1856
|
-
Default run root: `math-modeling-runs/<task-id>/`.
|
|
1857
|
-
|
|
1858
|
-
Required durable files:
|
|
1859
|
-
|
|
1860
|
-
- run.json: atomic current snapshot;
|
|
1861
|
-
- ledger.json: scope, assumptions, claims, obligations, subproblems, candidates, and issues;
|
|
1862
|
-
- events.jsonl: append-only transition journal;
|
|
1863
|
-
- problem-brief.md and inputs.json;
|
|
1864
|
-
- attempts/<number>/report.md plus code/artifacts only when they exist;
|
|
1865
|
-
- research/sources.jsonl and methods.md only when research occurs;
|
|
1866
|
-
- walls/ only when a direction is abandoned;
|
|
1867
|
-
- reproducibility.json and final-report.md for terminal output.
|
|
1868
|
-
|
|
1869
|
-
Use `scripts/run-state.mjs` for initialization, transition, validation, and recovery. A transition without evidence/issue references is invalid. Provider, parser, or verifier failure preserves raw output and the best candidate. Resume by validating state, recovering from the last journal snapshot when needed, then skipping hash-stable completed work.
|
|
1870
|
-
|
|
1871
|
-
Failure classes include data missing/schema/leakage, tool unavailable/dependency/license, runtime/timeout/numerical instability, model/assumption/parameter, verification/formalization, citation/research, parser/provider, no progress, and budget exhausted.
|
|
1872
|
-
~~~
|
|
1873
|
-
|
|
1874
|
-
- [ ] **Step 7: Create report contracts**
|
|
1875
|
-
|
|
1876
|
-
Create `skills/math-modeling-agent/references/report-contract.md`:
|
|
1877
|
-
|
|
1878
|
-
~~~markdown
|
|
1879
|
-
# Report Contract
|
|
1880
|
-
|
|
1881
|
-
## Attempt report
|
|
1882
|
-
|
|
1883
|
-
Every attempt report contains exactly:
|
|
1884
|
-
|
|
1885
|
-
1. one round objective;
|
|
1886
|
-
2. candidate/direction;
|
|
1887
|
-
3. assumption delta;
|
|
1888
|
-
4. claims changed;
|
|
1889
|
-
5. actual execution or derivation artifacts;
|
|
1890
|
-
6. new evidence IDs;
|
|
1891
|
-
7. obligations closed;
|
|
1892
|
-
8. issues opened and closed;
|
|
1893
|
-
9. delta from prior attempt;
|
|
1894
|
-
10. valid-progress decision;
|
|
1895
|
-
11. budget usage;
|
|
1896
|
-
12. current status;
|
|
1897
|
-
13. one next action.
|
|
1898
|
-
|
|
1899
|
-
Do not copy full prior solutions or private reasoning.
|
|
1900
|
-
|
|
1901
|
-
## Terminal report
|
|
1902
|
-
|
|
1903
|
-
State the frozen question, final scientific status, answer/recommendation, verified claims and evidence links, conditional/unresolved claims, assumptions and sensitivity, candidate comparison, data and parameter sources, validation, failure cases, limitations, reproducibility commands, and resume conditions when non-SOLVED.
|
|
1904
|
-
|
|
1905
|
-
## Chat summary
|
|
1906
|
-
|
|
1907
|
-
After each round show only attempt number, direction, tool execution status, obligations passed/failed, most important failure, artifact path, and next action.
|
|
1908
|
-
~~~
|
|
1909
|
-
|
|
1910
|
-
- [ ] **Step 8: Create two minimal examples**
|
|
1911
|
-
|
|
1912
|
-
Create `skills/math-modeling-agent/examples/minimal-run/README.md`:
|
|
1913
|
-
|
|
1914
|
-
~~~markdown
|
|
1915
|
-
# Minimal Run Example
|
|
1916
|
-
|
|
1917
|
-
Problem: solve 2x + 3 = 11.
|
|
1918
|
-
|
|
1919
|
-
Mode: Fast.
|
|
1920
|
-
|
|
1921
|
-
Claim C-001: x = 4.
|
|
1922
|
-
|
|
1923
|
-
Evidence E-001: algebraic derivation.
|
|
1924
|
-
Evidence E-002: substitution 2(4) + 3 = 11.
|
|
1925
|
-
|
|
1926
|
-
Status: SOLVED.
|
|
1927
|
-
|
|
1928
|
-
No Python, Lean, Wolfram, research directory, or empty artifact directory is required.
|
|
1929
|
-
~~~
|
|
1930
|
-
|
|
1931
|
-
Create `skills/math-modeling-agent/examples/resumed-run/README.md`:
|
|
1932
|
-
|
|
1933
|
-
~~~markdown
|
|
1934
|
-
# Resumed Run Example
|
|
1935
|
-
|
|
1936
|
-
A Standard optimization run reached VERIFY with candidate C-003 and evidence E-008 before interruption. `run.json` was corrupted after the journal append.
|
|
1937
|
-
|
|
1938
|
-
Resume sequence:
|
|
1939
|
-
|
|
1940
|
-
1. run `node scripts/run-state.mjs validate <run-root>`;
|
|
1941
|
-
2. run `node scripts/run-state.mjs recover <run-root>`;
|
|
1942
|
-
3. confirm C-003 remains the best candidate;
|
|
1943
|
-
4. rerun only the incomplete verifier;
|
|
1944
|
-
5. append a new evidence-linked transition.
|
|
1945
|
-
|
|
1946
|
-
Previously hash-stable Python outputs are not recomputed.
|
|
1947
|
-
~~~
|
|
1948
|
-
|
|
1949
|
-
- [ ] **Step 9: Verify every main-Skill resource exists**
|
|
1950
|
-
|
|
1951
|
-
Run:
|
|
1952
|
-
|
|
1953
|
-
~~~bash
|
|
1954
|
-
node --test tests/plugin-integrity.test.mjs
|
|
1955
|
-
~~~
|
|
1956
|
-
|
|
1957
|
-
Expected: only the missing `math-modeling-audit` bundle keeps the Skill-count test red; no missing main-Skill resource errors.
|
|
1958
|
-
|
|
1959
|
-
- [ ] **Step 10: Commit modeling references and examples**
|
|
1960
|
-
|
|
1961
|
-
~~~bash
|
|
1962
|
-
git add skills/math-modeling-agent/references skills/math-modeling-agent/examples
|
|
1963
|
-
git commit -m "docs: add modeling evidence and recovery protocols"
|
|
1964
|
-
~~~
|
|
1965
|
-
|
|
1966
|
-
|
|
1967
|
-
|
|
1968
|
-
### Task 7: Add the independent audit Skill
|
|
1969
|
-
|
|
1970
|
-
**Files:**
|
|
1971
|
-
- Create: `skills/math-modeling-audit/SKILL.md`
|
|
1972
|
-
- Create: `skills/math-modeling-audit/references/verification-protocol.md`
|
|
1973
|
-
- Create: `skills/math-modeling-audit/references/evidence-levels.md`
|
|
1974
|
-
- Create: `skills/math-modeling-audit/references/data-citation-audit.md`
|
|
1975
|
-
- Create: `skills/math-modeling-audit/examples/audit-report.md`
|
|
1976
|
-
|
|
1977
|
-
- [ ] **Step 1: Create the audit Skill interface**
|
|
1978
|
-
|
|
1979
|
-
Create `skills/math-modeling-audit/SKILL.md`:
|
|
1980
|
-
|
|
1981
|
-
~~~markdown
|
|
1982
|
-
---
|
|
1983
|
-
name: math-modeling-audit
|
|
1984
|
-
description: This skill should be used when the user asks to "独立核验数学模型", "检查推导", "寻找反例", "审计建模报告", "verify this model", or needs an artifact-only adversarial audit of mathematical claims, data, code, results, or citations.
|
|
1985
|
-
whenToUse: Use for existing artifacts and papers; do not take over open-ended model construction or silently rewrite the audited work.
|
|
1986
|
-
user-invocable: true
|
|
1987
|
-
---
|
|
1988
|
-
|
|
1989
|
-
# Math Modeling Audit
|
|
1990
|
-
|
|
1991
|
-
## Goal
|
|
1992
|
-
|
|
1993
|
-
Independently decide which claims in an existing artifact pass, fail, or remain inconclusive, and why.
|
|
1994
|
-
|
|
1995
|
-
## Interface
|
|
1996
|
-
|
|
1997
|
-
Accept a model, derivation, code/result artifact, paper, or MathModelingAgent run. Optionally accept the original problem, target claims, and assurance level.
|
|
1998
|
-
|
|
1999
|
-
Return per-claim verdicts, evidence levels, counterexamples, reproducibility findings, and residual risk.
|
|
2000
|
-
|
|
2001
|
-
## Workflow
|
|
2002
|
-
|
|
2003
|
-
1. Freeze the artifact set and record review coverage.
|
|
2004
|
-
2. Reconstruct claims, assumptions, obligations, and evidence without trusting the author’s summary.
|
|
2005
|
-
3. Apply `references/verification-protocol.md` and `references/evidence-levels.md`.
|
|
2006
|
-
4. Audit data, parameters, leakage, citations, and reproducibility with `references/data-citation-audit.md`.
|
|
2007
|
-
5. Recompute applicable formulas and results through independent tools.
|
|
2008
|
-
6. Seek boundary cases, counterexamples, alternative explanations, and simpler models.
|
|
2009
|
-
7. Output PASS, FAIL, or INCONCLUSIVE per claim; do not silently repair the source.
|
|
2010
|
-
|
|
2011
|
-
## Invariants
|
|
2012
|
-
|
|
2013
|
-
- Paper prose and solver success are not verification artifacts.
|
|
2014
|
-
- Missing evidence is reported, never supplied on the author’s behalf.
|
|
2015
|
-
- Lean verifies the formal statement, not its natural-language fidelity or data pipeline.
|
|
2016
|
-
|
|
2017
|
-
## Boundaries
|
|
2018
|
-
|
|
2019
|
-
Do not edit the audited artifact. Do not reward complexity or presentation without evidence.
|
|
2020
|
-
~~~
|
|
2021
|
-
|
|
2022
|
-
- [ ] **Step 2: Create the generic verification protocol**
|
|
2023
|
-
|
|
2024
|
-
Create `skills/math-modeling-audit/references/verification-protocol.md`:
|
|
2025
|
-
|
|
2026
|
-
~~~markdown
|
|
2027
|
-
# Independent Verification Protocol
|
|
2028
|
-
|
|
2029
|
-
## Freeze inputs
|
|
2030
|
-
|
|
2031
|
-
Hash the exact artifact set, problem statement, data, code, configuration, and environment evidence. Record anything unavailable.
|
|
2032
|
-
|
|
2033
|
-
## Rebuild the claim map
|
|
2034
|
-
|
|
2035
|
-
For each material claim record exact text, source location, assumptions, required evidence, supplied evidence, and independent countercheck.
|
|
2036
|
-
|
|
2037
|
-
## Attack order
|
|
2038
|
-
|
|
2039
|
-
1. task coverage and proxy substitution;
|
|
2040
|
-
2. units, dimensions, domain, bounds, and constraints;
|
|
2041
|
-
3. derivation and implementation consistency;
|
|
2042
|
-
4. data leakage, labels, splits, preprocessing, and post-hoc parameters;
|
|
2043
|
-
5. baseline, uncertainty, sensitivity, and external validity;
|
|
2044
|
-
6. reproducibility and citation truth;
|
|
2045
|
-
7. counterexamples, failure cases, and simpler alternatives.
|
|
2046
|
-
|
|
2047
|
-
## Verdicts
|
|
2048
|
-
|
|
2049
|
-
- PASS: required obligation is supported by reproducible evidence.
|
|
2050
|
-
- FAIL: a contradiction, counterexample, invalid method, or failed reproduction defeats the claim.
|
|
2051
|
-
- INCONCLUSIVE: evidence is insufficient or unavailable.
|
|
2052
|
-
|
|
2053
|
-
Never convert INCONCLUSIVE to PASS because the approach looks reasonable.
|
|
2054
|
-
~~~
|
|
2055
|
-
|
|
2056
|
-
- [ ] **Step 3: Create evidence-level guidance**
|
|
2057
|
-
|
|
2058
|
-
Create `skills/math-modeling-audit/references/evidence-levels.md`:
|
|
2059
|
-
|
|
2060
|
-
~~~markdown
|
|
2061
|
-
# Evidence Levels
|
|
2062
|
-
|
|
2063
|
-
- NOT_CHECKED: no independent check.
|
|
2064
|
-
- DERIVED: transparent reasoning exists but was not executed.
|
|
2065
|
-
- EXECUTED: a tool ran and produced an artifact; correctness is not implied.
|
|
2066
|
-
- VERIFIED: the artifact passed its stated obligation and counterchecks.
|
|
2067
|
-
- INDEPENDENTLY_VERIFIED: a separate method/context reproduced or proved the claim.
|
|
2068
|
-
- EXTERNALLY_VALIDATED: independent real data, known theorem, or primary source supports applicability.
|
|
2069
|
-
|
|
2070
|
-
Evidence strength must match claim strength. A training score cannot support generalization; one returned optimizer point cannot support global optimality; a formal proof cannot support unformalized real-world assumptions.
|
|
2071
|
-
~~~
|
|
2072
|
-
|
|
2073
|
-
- [ ] **Step 4: Create data and citation audit guidance**
|
|
2074
|
-
|
|
2075
|
-
Create `skills/math-modeling-audit/references/data-citation-audit.md`:
|
|
2076
|
-
|
|
2077
|
-
~~~markdown
|
|
2078
|
-
# Data and Citation Audit
|
|
2079
|
-
|
|
2080
|
-
## Data
|
|
2081
|
-
|
|
2082
|
-
Map each key dataset to source, time, geography, sample size, unit, labels, preprocessing, split, and consuming model. Check duplicates, selection/survivor bias, time/future leakage, group leakage, normalization/feature selection outside training folds, and repeated test-set tuning.
|
|
2083
|
-
|
|
2084
|
-
## Parameters
|
|
2085
|
-
|
|
2086
|
-
Map each key parameter to value, unit, source, setting time, estimation/calibration route, allowed range, sensitivity, and whether it was changed after seeing results. Unsupported conclusion-sensitive values are Achilles’ heels.
|
|
2087
|
-
|
|
2088
|
-
## Citations
|
|
2089
|
-
|
|
2090
|
-
Verify author, title, year, venue, DOI/stable URL, version, retrieval date, exact page/section/theorem/table, excerpt, mapped claim, and whether evidence is full text, abstract only, metadata only, secondary, conflicting, or citation needed.
|
|
2091
|
-
|
|
2092
|
-
Never invent a DOI, page, quotation, source, data set, or annual contest rule.
|
|
2093
|
-
~~~
|
|
2094
|
-
|
|
2095
|
-
- [ ] **Step 5: Create the generic audit example**
|
|
2096
|
-
|
|
2097
|
-
Create `skills/math-modeling-audit/examples/audit-report.md`:
|
|
2098
|
-
|
|
2099
|
-
~~~markdown
|
|
2100
|
-
# Audit Example
|
|
2101
|
-
|
|
2102
|
-
## Claim C-004
|
|
2103
|
-
|
|
2104
|
-
Claim: “The returned point is the global optimum.”
|
|
2105
|
-
|
|
2106
|
-
Evidence supplied: one successful SLSQP run.
|
|
2107
|
-
|
|
2108
|
-
Verdict: INCONCLUSIVE.
|
|
2109
|
-
|
|
2110
|
-
Reason: solver success supports a stationary/best-found point only. The artifact contains no convexity proof, certified bound, multi-start/global search, or optimality gap.
|
|
2111
|
-
|
|
2112
|
-
Required upgrade: weaken the claim to “best found local solution” or add an appropriate global-optimality certificate.
|
|
2113
|
-
~~~
|
|
2114
|
-
|
|
2115
|
-
- [ ] **Step 6: Run the complete package integrity test**
|
|
2116
|
-
|
|
2117
|
-
Run:
|
|
2118
|
-
|
|
2119
|
-
~~~bash
|
|
2120
|
-
node --test tests/plugin-integrity.test.mjs
|
|
2121
|
-
~~~
|
|
2122
|
-
|
|
2123
|
-
Expected: all package and Skill integrity tests PASS; catalog contains exactly `math-modeling-agent` and `math-modeling-audit`.
|
|
2124
|
-
|
|
2125
|
-
- [ ] **Step 7: Commit the independent audit Skill**
|
|
2126
|
-
|
|
2127
|
-
~~~bash
|
|
2128
|
-
git add skills/math-modeling-audit tests/plugin-integrity.test.mjs
|
|
2129
|
-
git commit -m "feat: add independent mathematical audit skill"
|
|
2130
|
-
~~~
|
|
2131
|
-
|
|
2132
|
-
|
|
2133
|
-
|
|
2134
|
-
### Task 8: Add deterministic MCM/ICM score arithmetic and cap enforcement
|
|
2135
|
-
|
|
2136
|
-
**Files:**
|
|
2137
|
-
- Create: `tests/mcm-score.test.mjs`
|
|
2138
|
-
- Create: `skills/math-modeling-audit/scripts/mcm-score.mjs`
|
|
2139
|
-
|
|
2140
|
-
- [ ] **Step 1: Write failing scorer tests**
|
|
2141
|
-
|
|
2142
|
-
Create `tests/mcm-score.test.mjs`:
|
|
2143
|
-
|
|
2144
|
-
~~~javascript
|
|
2145
|
-
import assert from 'node:assert/strict'
|
|
2146
|
-
import test from 'node:test'
|
|
2147
|
-
|
|
2148
|
-
import {
|
|
2149
|
-
RUBRIC,
|
|
2150
|
-
scoreReview,
|
|
2151
|
-
} from '../skills/math-modeling-audit/scripts/mcm-score.mjs'
|
|
2152
|
-
|
|
2153
|
-
function perfectScores() {
|
|
2154
|
-
return Object.fromEntries(
|
|
2155
|
-
Object.entries(RUBRIC).map(([id, maximum]) => [
|
|
2156
|
-
id,
|
|
2157
|
-
{ score: maximum, evidence: ['paper p.1'] },
|
|
2158
|
-
]),
|
|
2159
|
-
)
|
|
2160
|
-
}
|
|
2161
|
-
|
|
2162
|
-
function allOutstandingGates(value = true) {
|
|
2163
|
-
return {
|
|
2164
|
-
taskComplete: value,
|
|
2165
|
-
noMajorError: value,
|
|
2166
|
-
coreValidated: value,
|
|
2167
|
-
meaningfulSensitivity: value,
|
|
2168
|
-
parametersSupported: value,
|
|
2169
|
-
summaryStrong: value,
|
|
2170
|
-
nonTemplateModeling: value,
|
|
2171
|
-
closedLoop: value,
|
|
2172
|
-
reproducible: value,
|
|
2173
|
-
memorableContribution: value,
|
|
2174
|
-
}
|
|
2175
|
-
}
|
|
2176
|
-
|
|
2177
|
-
function input(overrides = {}) {
|
|
2178
|
-
return {
|
|
2179
|
-
schemaVersion: 1,
|
|
2180
|
-
scores: perfectScores(),
|
|
2181
|
-
caps: [],
|
|
2182
|
-
awardCeilings: [],
|
|
2183
|
-
outstandingGates: allOutstandingGates(),
|
|
2184
|
-
disqualificationRisk: null,
|
|
2185
|
-
...overrides,
|
|
2186
|
-
}
|
|
2187
|
-
}
|
|
2188
|
-
|
|
2189
|
-
test('perfect supported review scores 100 and remains Outstanding', () => {
|
|
2190
|
-
const result = scoreReview(input())
|
|
2191
|
-
assert.equal(result.rawScore, 100)
|
|
2192
|
-
assert.equal(result.cappedScore, 100)
|
|
2193
|
-
assert.equal(result.finalBand, 'Outstanding Candidate')
|
|
2194
|
-
assert.equal(result.outstandingGatePassed, true)
|
|
2195
|
-
})
|
|
2196
|
-
|
|
2197
|
-
test('unvalidated core model applies the 84 cap', () => {
|
|
2198
|
-
const result = scoreReview(input({
|
|
2199
|
-
caps: [{ code: 'CORE_UNVALIDATED', evidence: 'No holdout or external validation.' }],
|
|
2200
|
-
}))
|
|
2201
|
-
assert.equal(result.strictestCap, 84)
|
|
2202
|
-
assert.equal(result.cappedScore, 84)
|
|
2203
|
-
assert.equal(result.finalBand, 'Meritorious')
|
|
2204
|
-
})
|
|
2205
|
-
|
|
2206
|
-
test('multiple numeric caps apply the strictest one', () => {
|
|
2207
|
-
const result = scoreReview(input({
|
|
2208
|
-
caps: [
|
|
2209
|
-
{ code: 'NOT_REPRODUCIBLE', evidence: 'Algorithm settings absent.' },
|
|
2210
|
-
{ code: 'DATA_LEAKAGE', evidence: 'Target used during feature construction.' },
|
|
2211
|
-
],
|
|
2212
|
-
}))
|
|
2213
|
-
assert.equal(result.strictestCap, 69)
|
|
2214
|
-
assert.equal(result.cappedScore, 69)
|
|
2215
|
-
assert.equal(result.finalBand, 'Honorable Mention')
|
|
2216
|
-
})
|
|
2217
|
-
|
|
2218
|
-
test('missing core task removes Outstanding and Finalist eligibility', () => {
|
|
2219
|
-
const result = scoreReview(input({
|
|
2220
|
-
awardCeilings: [{ code: 'MISSING_CORE_TASK', evidence: 'Problem 3 has no answer.' }],
|
|
2221
|
-
}))
|
|
2222
|
-
assert.equal(result.cappedScore, 100)
|
|
2223
|
-
assert.equal(result.awardCeiling, 'Meritorious')
|
|
2224
|
-
assert.equal(result.finalBand, 'Meritorious')
|
|
2225
|
-
})
|
|
2226
|
-
|
|
2227
|
-
test('disqualification risk suppresses ordinary score requirements and output', () => {
|
|
2228
|
-
const result = scoreReview({
|
|
2229
|
-
schemaVersion: 1,
|
|
2230
|
-
scores: {},
|
|
2231
|
-
caps: [],
|
|
2232
|
-
awardCeilings: [],
|
|
2233
|
-
outstandingGates: {},
|
|
2234
|
-
disqualificationRisk: {
|
|
2235
|
-
code: 'FABRICATED_REFERENCE',
|
|
2236
|
-
evidence: 'The cited DOI does not resolve and the title cannot be found.',
|
|
2237
|
-
},
|
|
2238
|
-
})
|
|
2239
|
-
assert.equal(result.disqualification, true)
|
|
2240
|
-
assert.equal(result.rawScore, null)
|
|
2241
|
-
assert.equal(result.categoryScores, null)
|
|
2242
|
-
assert.equal(result.outstandingGatePassed, null)
|
|
2243
|
-
assert.equal(result.finalBand, 'Disqualification Risk')
|
|
2244
|
-
})
|
|
2245
|
-
|
|
2246
|
-
test('every subscore and cap requires evidence', () => {
|
|
2247
|
-
const scores = perfectScores()
|
|
2248
|
-
scores['1.summary'] = { score: 4, evidence: [] }
|
|
2249
|
-
assert.throws(() => scoreReview(input({ scores })), /evidence/)
|
|
2250
|
-
assert.throws(
|
|
2251
|
-
() => scoreReview(input({ caps: [{ code: 'MATH_ERROR', evidence: '' }] })),
|
|
2252
|
-
/evidence/,
|
|
2253
|
-
)
|
|
2254
|
-
})
|
|
2255
|
-
~~~
|
|
2256
|
-
|
|
2257
|
-
- [ ] **Step 2: Run the scorer test and verify RED**
|
|
2258
|
-
|
|
2259
|
-
Run:
|
|
2260
|
-
|
|
2261
|
-
~~~bash
|
|
2262
|
-
node --test tests/mcm-score.test.mjs
|
|
2263
|
-
~~~
|
|
2264
|
-
|
|
2265
|
-
Expected: FAIL with `ERR_MODULE_NOT_FOUND` for `mcm-score.mjs`.
|
|
2266
|
-
|
|
2267
|
-
- [ ] **Step 3: Implement fixed rubric, cap, ceiling, and disqualification logic**
|
|
2268
|
-
|
|
2269
|
-
Create `skills/math-modeling-audit/scripts/mcm-score.mjs`:
|
|
2270
|
-
|
|
2271
|
-
~~~javascript
|
|
2272
|
-
import { readFile } from 'node:fs/promises'
|
|
2273
|
-
import { pathToFileURL } from 'node:url'
|
|
2274
|
-
|
|
2275
|
-
export const RUBRIC = {
|
|
2276
|
-
'1.summary': 4,
|
|
2277
|
-
'1.understanding': 3,
|
|
2278
|
-
'1.assumptions': 3,
|
|
2279
|
-
'2.data-source': 3,
|
|
2280
|
-
'2.preprocessing': 2,
|
|
2281
|
-
'2.bias-leakage': 2,
|
|
2282
|
-
'2.parameter-calibration': 3,
|
|
2283
|
-
'2.consistency': 2,
|
|
2284
|
-
'3.mechanism': 5,
|
|
2285
|
-
'3.innovation': 4,
|
|
2286
|
-
'3.formulation': 4,
|
|
2287
|
-
'3.internal-consistency': 3,
|
|
2288
|
-
'3.complexity-explainability': 3,
|
|
2289
|
-
'3.multi-problem-unity': 3,
|
|
2290
|
-
'4.derivation': 4,
|
|
2291
|
-
'4.algorithm-numerics': 4,
|
|
2292
|
-
'4.reproducibility': 3,
|
|
2293
|
-
'4.complexity-convergence-optimality': 3,
|
|
2294
|
-
'4.sanity': 2,
|
|
2295
|
-
'5.task-completion': 4,
|
|
2296
|
-
'5.numerical-consistency': 4,
|
|
2297
|
-
'5.baselines': 3,
|
|
2298
|
-
'5.sensitivity': 4,
|
|
2299
|
-
'5.robustness-uncertainty': 4,
|
|
2300
|
-
'5.independent-validation': 3,
|
|
2301
|
-
'5.failure-cases': 2,
|
|
2302
|
-
'6.conclusions': 3,
|
|
2303
|
-
'6.actionability': 2,
|
|
2304
|
-
'6.limitations': 2,
|
|
2305
|
-
'6.transferability': 1,
|
|
2306
|
-
'7.structure': 2,
|
|
2307
|
-
'7.figures': 2,
|
|
2308
|
-
'7.language': 1,
|
|
2309
|
-
'7.notation': 1,
|
|
2310
|
-
'7.citations': 1,
|
|
2311
|
-
'7.page-efficiency': 1,
|
|
2312
|
-
}
|
|
2313
|
-
|
|
2314
|
-
const CAP_VALUES = {
|
|
2315
|
-
MATH_ERROR: 69,
|
|
2316
|
-
CORE_UNVALIDATED: 84,
|
|
2317
|
-
UNSUPPORTED_SENSITIVE_PARAMETERS: 79,
|
|
2318
|
-
NOT_REPRODUCIBLE: 79,
|
|
2319
|
-
DATA_LEAKAGE: 69,
|
|
2320
|
-
CONCLUSION_CONFLICT: 74,
|
|
2321
|
-
NO_QUANTITATIVE_RESULTS: 59,
|
|
2322
|
-
}
|
|
2323
|
-
|
|
2324
|
-
const CEILING_VALUES = {
|
|
2325
|
-
MISSING_CORE_TASK: 'Meritorious',
|
|
2326
|
-
}
|
|
2327
|
-
|
|
2328
|
-
const OUTSTANDING_GATES = [
|
|
2329
|
-
'taskComplete',
|
|
2330
|
-
'noMajorError',
|
|
2331
|
-
'coreValidated',
|
|
2332
|
-
'meaningfulSensitivity',
|
|
2333
|
-
'parametersSupported',
|
|
2334
|
-
'summaryStrong',
|
|
2335
|
-
'nonTemplateModeling',
|
|
2336
|
-
'closedLoop',
|
|
2337
|
-
'reproducible',
|
|
2338
|
-
'memorableContribution',
|
|
2339
|
-
]
|
|
2340
|
-
|
|
2341
|
-
const BAND_ORDER = [
|
|
2342
|
-
'Unsuccessful Risk',
|
|
2343
|
-
'Successful Participant',
|
|
2344
|
-
'Honorable Mention',
|
|
2345
|
-
'Meritorious',
|
|
2346
|
-
'Finalist',
|
|
2347
|
-
'Outstanding Candidate',
|
|
2348
|
-
]
|
|
2349
|
-
|
|
2350
|
-
function requireEvidence(value, label) {
|
|
2351
|
-
const items = Array.isArray(value) ? value : [value]
|
|
2352
|
-
if (!items.length || items.some((item) => typeof item !== 'string' || !item.trim())) {
|
|
2353
|
-
throw new TypeError(label + ' requires evidence')
|
|
2354
|
-
}
|
|
2355
|
-
}
|
|
2356
|
-
|
|
2357
|
-
function scoreBand(score) {
|
|
2358
|
-
if (score >= 93) return 'Outstanding Candidate'
|
|
2359
|
-
if (score >= 87) return 'Finalist'
|
|
2360
|
-
if (score >= 78) return 'Meritorious'
|
|
2361
|
-
if (score >= 68) return 'Honorable Mention'
|
|
2362
|
-
if (score >= 55) return 'Successful Participant'
|
|
2363
|
-
return 'Unsuccessful Risk'
|
|
2364
|
-
}
|
|
2365
|
-
|
|
2366
|
-
function lowerBand(left, right) {
|
|
2367
|
-
return BAND_ORDER[Math.min(BAND_ORDER.indexOf(left), BAND_ORDER.indexOf(right))]
|
|
2368
|
-
}
|
|
2369
|
-
|
|
2370
|
-
function validateScores(scores) {
|
|
2371
|
-
const expected = Object.keys(RUBRIC).sort()
|
|
2372
|
-
const actual = Object.keys(scores ?? {}).sort()
|
|
2373
|
-
if (JSON.stringify(actual) !== JSON.stringify(expected)) {
|
|
2374
|
-
throw new TypeError('scores must contain every rubric subcriterion exactly once')
|
|
2375
|
-
}
|
|
2376
|
-
|
|
2377
|
-
let total = 0
|
|
2378
|
-
const categories = {}
|
|
2379
|
-
for (const [id, maximum] of Object.entries(RUBRIC)) {
|
|
2380
|
-
const entry = scores[id]
|
|
2381
|
-
if (!Number.isFinite(entry.score) || entry.score < 0 || entry.score > maximum) {
|
|
2382
|
-
throw new TypeError(id + ' score must be between 0 and ' + maximum)
|
|
2383
|
-
}
|
|
2384
|
-
requireEvidence(entry.evidence, id)
|
|
2385
|
-
total += entry.score
|
|
2386
|
-
const category = id.split('.')[0]
|
|
2387
|
-
categories[category] = (categories[category] ?? 0) + entry.score
|
|
2388
|
-
}
|
|
2389
|
-
return { total, categories }
|
|
2390
|
-
}
|
|
2391
|
-
|
|
2392
|
-
function validateRubricTotal() {
|
|
2393
|
-
const maximum = Object.values(RUBRIC).reduce((sum, value) => sum + value, 0)
|
|
2394
|
-
if (maximum !== 100) throw new TypeError('rubric maxima must total 100')
|
|
2395
|
-
}
|
|
2396
|
-
|
|
2397
|
-
function validateOutstandingGates(gates) {
|
|
2398
|
-
for (const gate of OUTSTANDING_GATES) {
|
|
2399
|
-
if (typeof gates?.[gate] !== 'boolean') {
|
|
2400
|
-
throw new TypeError('missing Outstanding gate: ' + gate)
|
|
2401
|
-
}
|
|
2402
|
-
}
|
|
2403
|
-
return OUTSTANDING_GATES.every((gate) => gates[gate])
|
|
2404
|
-
}
|
|
2405
|
-
|
|
2406
|
-
export function scoreReview(input) {
|
|
2407
|
-
if (input.schemaVersion !== 1) throw new TypeError('unsupported schemaVersion')
|
|
2408
|
-
validateRubricTotal()
|
|
2409
|
-
|
|
2410
|
-
if (input.disqualificationRisk) {
|
|
2411
|
-
requireEvidence(input.disqualificationRisk.evidence, 'disqualification risk')
|
|
2412
|
-
return {
|
|
2413
|
-
schemaVersion: 1,
|
|
2414
|
-
disqualification: true,
|
|
2415
|
-
disqualificationRisk: input.disqualificationRisk,
|
|
2416
|
-
categoryScores: null,
|
|
2417
|
-
rawScore: null,
|
|
2418
|
-
caps: [],
|
|
2419
|
-
strictestCap: null,
|
|
2420
|
-
cappedScore: null,
|
|
2421
|
-
awardCeiling: null,
|
|
2422
|
-
outstandingGatePassed: null,
|
|
2423
|
-
finalBand: 'Disqualification Risk',
|
|
2424
|
-
}
|
|
2425
|
-
}
|
|
2426
|
-
|
|
2427
|
-
const { total: rawScore, categories } = validateScores(input.scores)
|
|
2428
|
-
const outstandingGatePassed = validateOutstandingGates(input.outstandingGates)
|
|
2429
|
-
|
|
2430
|
-
const caps = (input.caps ?? []).map((entry) => {
|
|
2431
|
-
if (!(entry.code in CAP_VALUES)) throw new TypeError('unknown cap code: ' + entry.code)
|
|
2432
|
-
requireEvidence(entry.evidence, 'cap ' + entry.code)
|
|
2433
|
-
return { ...entry, maximum: CAP_VALUES[entry.code] }
|
|
2434
|
-
})
|
|
2435
|
-
const strictestCap = caps.length
|
|
2436
|
-
? Math.min(...caps.map((entry) => entry.maximum))
|
|
2437
|
-
: 100
|
|
2438
|
-
const cappedScore = Math.min(rawScore, strictestCap)
|
|
2439
|
-
|
|
2440
|
-
const ceilings = (input.awardCeilings ?? []).map((entry) => {
|
|
2441
|
-
if (!(entry.code in CEILING_VALUES)) {
|
|
2442
|
-
throw new TypeError('unknown award ceiling code: ' + entry.code)
|
|
2443
|
-
}
|
|
2444
|
-
requireEvidence(entry.evidence, 'award ceiling ' + entry.code)
|
|
2445
|
-
return { ...entry, maximumBand: CEILING_VALUES[entry.code] }
|
|
2446
|
-
})
|
|
2447
|
-
|
|
2448
|
-
let awardCeiling = ceilings.length
|
|
2449
|
-
? ceilings
|
|
2450
|
-
.map((entry) => entry.maximumBand)
|
|
2451
|
-
.reduce((current, value) => lowerBand(current, value), 'Outstanding Candidate')
|
|
2452
|
-
: 'Outstanding Candidate'
|
|
2453
|
-
if (!outstandingGatePassed) awardCeiling = lowerBand(awardCeiling, 'Finalist')
|
|
2454
|
-
|
|
2455
|
-
const rawBand = scoreBand(rawScore)
|
|
2456
|
-
const cappedBand = scoreBand(cappedScore)
|
|
2457
|
-
const finalBand = lowerBand(cappedBand, awardCeiling)
|
|
2458
|
-
|
|
2459
|
-
return {
|
|
2460
|
-
schemaVersion: 1,
|
|
2461
|
-
disqualification: false,
|
|
2462
|
-
disqualificationRisk: null,
|
|
2463
|
-
categoryScores: categories,
|
|
2464
|
-
rawScore,
|
|
2465
|
-
caps,
|
|
2466
|
-
strictestCap,
|
|
2467
|
-
cappedScore,
|
|
2468
|
-
rawBand,
|
|
2469
|
-
cappedBand,
|
|
2470
|
-
awardCeiling,
|
|
2471
|
-
outstandingGatePassed,
|
|
2472
|
-
finalBand,
|
|
2473
|
-
}
|
|
2474
|
-
}
|
|
2475
|
-
|
|
2476
|
-
async function main() {
|
|
2477
|
-
const path = process.argv[2]
|
|
2478
|
-
if (!path) throw new Error('usage: mcm-score.mjs <review-score.json>')
|
|
2479
|
-
const input = JSON.parse(await readFile(path, 'utf8'))
|
|
2480
|
-
process.stdout.write(JSON.stringify(scoreReview(input), null, 2) + '\n')
|
|
2481
|
-
}
|
|
2482
|
-
|
|
2483
|
-
const invokedPath = process.argv[1]
|
|
2484
|
-
if (invokedPath && import.meta.url === pathToFileURL(invokedPath).href) {
|
|
2485
|
-
main().catch((error) => {
|
|
2486
|
-
process.stderr.write(error.message + '\n')
|
|
2487
|
-
process.exitCode = 1
|
|
2488
|
-
})
|
|
2489
|
-
}
|
|
2490
|
-
~~~
|
|
2491
|
-
|
|
2492
|
-
- [ ] **Step 4: Run scorer tests**
|
|
2493
|
-
|
|
2494
|
-
Run:
|
|
2495
|
-
|
|
2496
|
-
~~~bash
|
|
2497
|
-
node --test tests/mcm-score.test.mjs
|
|
2498
|
-
~~~
|
|
2499
|
-
|
|
2500
|
-
Expected: 6 tests PASS, 0 failures.
|
|
2501
|
-
|
|
2502
|
-
- [ ] **Step 5: Commit deterministic scoring**
|
|
2503
|
-
|
|
2504
|
-
~~~bash
|
|
2505
|
-
git add tests/mcm-score.test.mjs skills/math-modeling-audit/scripts/mcm-score.mjs
|
|
2506
|
-
git commit -m "feat: add deterministic mcm icm scoring"
|
|
2507
|
-
~~~
|
|
2508
|
-
|
|
2509
|
-
|
|
2510
|
-
|
|
2511
|
-
### Task 9: Add the MCM/ICM final-panel rubric and golden output
|
|
2512
|
-
|
|
2513
|
-
**Files:**
|
|
2514
|
-
- Modify: `skills/math-modeling-audit/SKILL.md`
|
|
2515
|
-
- Create: `skills/math-modeling-audit/references/mcm-icm-final-judge.md`
|
|
2516
|
-
- Create: `skills/math-modeling-audit/examples/mcm-final-review.md`
|
|
2517
|
-
|
|
2518
|
-
- [ ] **Step 1: Extend the audit Skill only after the scorer exists**
|
|
2519
|
-
|
|
2520
|
-
In `skills/math-modeling-audit/SKILL.md`, replace the frontmatter description with:
|
|
2521
|
-
|
|
2522
|
-
~~~yaml
|
|
2523
|
-
description: This skill should be used when the user asks to "独立核验数学模型", "检查推导", "寻找反例", "审计建模报告", "按 MCM/ICM 终审框架打分", "verify this model", or needs an artifact-only adversarial audit of mathematical claims, data, code, results, citations, or a competition paper.
|
|
2524
|
-
~~~
|
|
2525
|
-
|
|
2526
|
-
Replace the Interface paragraphs with:
|
|
2527
|
-
|
|
2528
|
-
~~~markdown
|
|
2529
|
-
Accept a paper, model, derivation, code/result artifact, or MathModelingAgent run. Optionally accept the original problem, competition year/code, target claims, and assurance level.
|
|
2530
|
-
|
|
2531
|
-
Return per-claim verdicts, evidence levels, counterexamples, reproducibility findings, residual risk, and—only for MCM/ICM judging intent—the fixed final-panel report.
|
|
2532
|
-
~~~
|
|
2533
|
-
|
|
2534
|
-
Replace workflow Step 7 with:
|
|
2535
|
-
|
|
2536
|
-
~~~markdown
|
|
2537
|
-
7. Output PASS, FAIL, or INCONCLUSIVE per claim; do not silently repair the source.
|
|
2538
|
-
8. For MCM/ICM scoring intent only, load `references/mcm-icm-final-judge.md`, validate arithmetic with `scripts/mcm-score.mjs`, and preserve its fourteen-section output.
|
|
2539
|
-
~~~
|
|
2540
|
-
|
|
2541
|
-
After the Lean invariant, add:
|
|
2542
|
-
|
|
2543
|
-
~~~markdown
|
|
2544
|
-
- A score never feeds back into the solver’s SOLVED gate.
|
|
2545
|
-
- Disqualification risk stops ordinary award scoring.
|
|
2546
|
-
~~~
|
|
2547
|
-
|
|
2548
|
-
Replace the Boundaries paragraph with:
|
|
2549
|
-
|
|
2550
|
-
~~~markdown
|
|
2551
|
-
Do not edit the audited artifact. Do not reward complexity or presentation without evidence. Do not load the large MCM/ICM rubric for generic audits.
|
|
2552
|
-
~~~
|
|
2553
|
-
|
|
2554
|
-
- [ ] **Step 2: Run integrity test and verify RED for newly named resources**
|
|
2555
|
-
|
|
2556
|
-
Run:
|
|
2557
|
-
|
|
2558
|
-
~~~bash
|
|
2559
|
-
node --test tests/plugin-integrity.test.mjs
|
|
2560
|
-
~~~
|
|
2561
|
-
|
|
2562
|
-
Expected: FAIL because `references/mcm-icm-final-judge.md` does not exist yet; the already implemented scorer path exists.
|
|
2563
|
-
|
|
2564
|
-
- [ ] **Step 3: Create the complete final-judge reference**
|
|
2565
|
-
|
|
2566
|
-
Create `skills/math-modeling-audit/references/mcm-icm-final-judge.md`:
|
|
2567
|
-
|
|
2568
|
-
~~~markdown
|
|
2569
|
-
# MCM/ICM Final Judging Panel
|
|
2570
|
-
|
|
2571
|
-
This is a simulated 100-point final-panel framework, not a COMAP official quantitative scorecard. Review as if the paper is competing for Outstanding Winner or Finalist. Do not coach, reward effort, or supply missing evidence.
|
|
2572
|
-
|
|
2573
|
-
## Non-negotiable principles
|
|
2574
|
-
|
|
2575
|
-
- Do not assume the model is correct.
|
|
2576
|
-
- Complexity, machine learning, optimization, neural networks, Monte Carlo, AHP, TOPSIS, entropy weights, or named algorithms earn no automatic innovation credit.
|
|
2577
|
-
- Reject formulas without mechanism, results without traceable computation, figures without conclusions, and formalistic sensitivity analysis that misses critical parameters.
|
|
2578
|
-
- Distinguish plausible from validated and a number from a credible modeling result.
|
|
2579
|
-
- Cite page, section, equation, table, figure, or exact value whenever possible.
|
|
2580
|
-
- If evidence is absent, write “论文未提供证据”.
|
|
2581
|
-
|
|
2582
|
-
## Stage 1: veto and award caps
|
|
2583
|
-
|
|
2584
|
-
Classify each item as 通过, 警告, 严重问题, or 一票否决风险:
|
|
2585
|
-
|
|
2586
|
-
1. every subproblem is answered;
|
|
2587
|
-
2. no omission, off-target answer, or proxy substituted for the requested metric;
|
|
2588
|
-
3. no critical mathematical formula error;
|
|
2589
|
-
4. no dimensional, unit, or order-of-magnitude error;
|
|
2590
|
-
5. no key result violates constraints;
|
|
2591
|
-
6. summary, body, tables, and figures agree numerically;
|
|
2592
|
-
7. conclusions follow from model and computation;
|
|
2593
|
-
8. critical parameters have a source or defensible estimate;
|
|
2594
|
-
9. critical model and algorithm are reproducible;
|
|
2595
|
-
10. no data/future leakage or train-test mixing;
|
|
2596
|
-
11. sources, data, and references are traceable and non-fabricated;
|
|
2597
|
-
12. external methods, algorithms, images, and data are cited;
|
|
2598
|
-
13. applicable annual COMAP page, anonymity, submission, citation, and AI/disclosure rules are satisfied;
|
|
2599
|
-
14. core prediction/optimization/decision results are validated;
|
|
2600
|
-
15. model structure fits the real mechanism or the mismatch is discussed.
|
|
2601
|
-
|
|
2602
|
-
Apply evidence-linked caps:
|
|
2603
|
-
|
|
2604
|
-
- missing core subtask: Outstanding and Finalist normally unavailable;
|
|
2605
|
-
- conclusion-changing main-model math error: cap 69;
|
|
2606
|
-
- completely unvalidated core prediction/optimization/decision model: cap 84;
|
|
2607
|
-
- unsupported highly sensitive core parameter: cap 79;
|
|
2608
|
-
- irreproducible critical result: cap 79;
|
|
2609
|
-
- clear leakage or answer-information training: cap 69;
|
|
2610
|
-
- main conclusion conflicts with calculation/figure: cap 74;
|
|
2611
|
-
- almost no quantitative result: cap 59;
|
|
2612
|
-
- suspected fabricated data/reference, plagiarism, or serious rule violation: stop ordinary scoring and report Disqualification Risk.
|
|
2613
|
-
|
|
2614
|
-
## Stage 2: 100-point scoring
|
|
2615
|
-
|
|
2616
|
-
### 1. Problem understanding, decomposition, and summary — 10
|
|
2617
|
-
|
|
2618
|
-
#### 1.1 Summary Sheet — 4
|
|
2619
|
-
|
|
2620
|
-
Check whether a 60-second reader learns the problem, models, method per subproblem, core quantitative results, recommendation, validation, and numbers consistent with the body. “Good performance” without numbers is weak evidence.
|
|
2621
|
-
|
|
2622
|
-
#### 1.2 Understanding and decomposition — 3
|
|
2623
|
-
|
|
2624
|
-
Check every explicit task, hidden constraint, practical objective, stakeholder, subproblem relation, and whether the paper solved an easier substitute problem.
|
|
2625
|
-
|
|
2626
|
-
#### 1.3 Assumptions, definitions, boundaries — 3
|
|
2627
|
-
|
|
2628
|
-
Check completeness, necessity, realism, simplification rationale, applicability, distinction among fact/assumption/setting, and support from data, literature, or sensitivity.
|
|
2629
|
-
|
|
2630
|
-
### 2. Data, evidence, and parameters — 12
|
|
2631
|
-
|
|
2632
|
-
#### 2.1 Source credibility — 3
|
|
2633
|
-
|
|
2634
|
-
For every key dataset record name, source, time, sample size, unit, and consuming model. Check temporal/spatial match, conflicts, update frequency, and traceability.
|
|
2635
|
-
|
|
2636
|
-
#### 2.2 Cleaning and preprocessing — 2
|
|
2637
|
-
|
|
2638
|
-
Check missing values, outliers, scaling, interpolation, smoothing, trend creation, method explanation, and before-after comparison.
|
|
2639
|
-
|
|
2640
|
-
#### 2.3 Representativeness, bias, leakage — 2
|
|
2641
|
-
|
|
2642
|
-
Check selection, survivor, temporal, spatial, and entity bias; train/validation/test separation; future and target leakage.
|
|
2643
|
-
|
|
2644
|
-
#### 2.4 Parameter determination and calibration — 3
|
|
2645
|
-
|
|
2646
|
-
Every key parameter must come from data estimation, literature, theory, calibration, inverse optimization, justified expert input, or an explicitly tested scenario. Unsupported values such as “let alpha=0.5” are penalized.
|
|
2647
|
-
|
|
2648
|
-
#### 2.5 Data-parameter-model consistency — 2
|
|
2649
|
-
|
|
2650
|
-
Check unit, temporal, and spatial consistency; physical meaning; whether data supports variables; and whether variables were invented only to fit a method.
|
|
2651
|
-
|
|
2652
|
-
### 3. Model formulation — 22
|
|
2653
|
-
|
|
2654
|
-
#### 3.1 Mechanism-to-mathematics mapping — 5
|
|
2655
|
-
|
|
2656
|
-
Check whether variables, equations, probability structure, objective, and constraints correspond to the real mechanism rather than mathematical assembly.
|
|
2657
|
-
|
|
2658
|
-
#### 3.2 Valuable innovation — 4
|
|
2659
|
-
|
|
2660
|
-
Credit mechanism design, justified coupling, structural adaptation, meaningful new indicators, evidence-based correction, or a genuinely useful solver. Do not credit stacking, deep learning by name, jargon, or unnecessary parameters. Ask whether a simpler model gives essentially the same answer.
|
|
2661
|
-
|
|
2662
|
-
#### 3.3 Variables, objective, constraints, equations — 4
|
|
2663
|
-
|
|
2664
|
-
Check every symbol, objective fidelity, complete real constraints, mathematical correctness, units, initial conditions, and boundary conditions.
|
|
2665
|
-
|
|
2666
|
-
#### 3.4 Assumption and internal consistency — 3
|
|
2667
|
-
|
|
2668
|
-
Check contradictions among assumptions/equations, changing parameter meanings, reused symbols, and circular logic.
|
|
2669
|
-
|
|
2670
|
-
#### 3.5 Complexity and explainability — 3
|
|
2671
|
-
|
|
2672
|
-
Check necessary complexity, overfitting, removable modules, interpretability, and fit to a four-day contest.
|
|
2673
|
-
|
|
2674
|
-
#### 3.6 Multi-subproblem unity — 3
|
|
2675
|
-
|
|
2676
|
-
Check whether earlier results genuinely support later tasks and whether the paper forms problem → core model → extension → verification → decision rather than a model collage.
|
|
2677
|
-
|
|
2678
|
-
### 4. Mathematical solution, algorithm, reproducibility — 16
|
|
2679
|
-
|
|
2680
|
-
#### 4.1 Derivation correctness — 4
|
|
2681
|
-
|
|
2682
|
-
Audit probability, calculus, differential/difference equations, matrices, statistics, normalization, weights, and recurrences. State whether an error changes the conclusion.
|
|
2683
|
-
|
|
2684
|
-
#### 4.2 Algorithms and numerical methods — 4
|
|
2685
|
-
|
|
2686
|
-
Check method suitability, initial values, stop/convergence conditions, numerical stability, hyperparameters, seeds, and local-versus-global claims.
|
|
2687
|
-
|
|
2688
|
-
#### 4.3 Reproducibility — 3
|
|
2689
|
-
|
|
2690
|
-
Check inputs, parameters, steps, outputs, code/pseudocode, and figure-to-computation traceability.
|
|
2691
|
-
|
|
2692
|
-
#### 4.4 Complexity, convergence, optimality — 3
|
|
2693
|
-
|
|
2694
|
-
Check time/space complexity, convergence, grid convergence, optimality proof/gap, baselines for heuristics, and multi-start tests.
|
|
2695
|
-
|
|
2696
|
-
#### 4.5 Sanity checks — 2
|
|
2697
|
-
|
|
2698
|
-
Check extremes, magnitude, bounds, units, sums/probabilities, resource capacities, nonnegative time, and common-sense violations.
|
|
2699
|
-
|
|
2700
|
-
### 5. Results, validation, robustness, credibility — 24
|
|
2701
|
-
|
|
2702
|
-
#### 5.1 Task completion — 4
|
|
2703
|
-
|
|
2704
|
-
Build requirement → model → output → conclusion mapping. A method without the requested number, ranking, strategy, prediction, or decision is incomplete.
|
|
2705
|
-
|
|
2706
|
-
#### 5.2 Numerical correctness and internal consistency — 4
|
|
2707
|
-
|
|
2708
|
-
Audit the three to ten most important results: value, unit, page, model, input, computation path, reproducibility, plausibility, validation, and PASS/WARNING/FAIL. Compare summary, body, tables, figures, and repeated values.
|
|
2709
|
-
|
|
2710
|
-
#### 5.3 Baseline and controls — 3
|
|
2711
|
-
|
|
2712
|
-
Check naive, traditional, historical, official, current, random, or literature baselines. A standalone 91% result has no comparative meaning.
|
|
2713
|
-
|
|
2714
|
-
#### 5.4 Sensitivity — 4
|
|
2715
|
-
|
|
2716
|
-
Identify truly critical parameters first. Check realistic perturbation, quantitative output change, thresholds, rank reversal, and failure. Ask how far a parameter moves before the conclusion changes.
|
|
2717
|
-
|
|
2718
|
-
#### 5.5 Robustness and uncertainty — 4
|
|
2719
|
-
|
|
2720
|
-
Check parameter, data, measurement, random, structural, and scenario uncertainty; Monte Carlo, bootstrap, confidence/prediction intervals, worst case, or robust optimization. Distinguish exact-looking numbers from credible intervals.
|
|
2721
|
-
|
|
2722
|
-
#### 5.6 Independent/external validation — 3
|
|
2723
|
-
|
|
2724
|
-
Prefer holdout, time extrapolation, backtest, real case, independent source, literature result, known theorem, or simulation-to-reality comparison. Training-data self-validation is weak.
|
|
2725
|
-
|
|
2726
|
-
#### 5.7 Failure scenarios and counterexamples — 2
|
|
2727
|
-
|
|
2728
|
-
Check extreme parameters, missing data, network failure, demand shock, environmental/policy change, and black swans. Outstanding papers know when they fail.
|
|
2729
|
-
|
|
2730
|
-
### 6. Conclusions, practical meaning, generalization — 8
|
|
2731
|
-
|
|
2732
|
-
#### 6.1 Model-grounded conclusions — 3
|
|
2733
|
-
|
|
2734
|
-
Trace data → model → result → conclusion. Penalize correlation written as causation, unlimited extrapolation, or precise policy advice without computation.
|
|
2735
|
-
|
|
2736
|
-
#### 6.2 Actionable recommendations — 2
|
|
2737
|
-
|
|
2738
|
-
Identify actor, action, timing, cost, risk, resources, and implementation constraints.
|
|
2739
|
-
|
|
2740
|
-
#### 6.3 Concrete limitations — 2
|
|
2741
|
-
|
|
2742
|
-
Name the most dangerous assumption, missing data, sensitive parameter, failure condition, and bias direction—not “more work is needed”.
|
|
2743
|
-
|
|
2744
|
-
#### 6.4 Transferability — 1
|
|
2745
|
-
|
|
2746
|
-
Assess extension across region, time, scale, network, and policy scenario.
|
|
2747
|
-
|
|
2748
|
-
### 7. Writing, figures, professional presentation — 8
|
|
2749
|
-
|
|
2750
|
-
#### 7.1 Structure — 2
|
|
2751
|
-
|
|
2752
|
-
Check logical service of sections, repetition, result-before-model reasoning, jumps, and clear section purpose.
|
|
2753
|
-
|
|
2754
|
-
#### 7.2 Figures — 2
|
|
2755
|
-
|
|
2756
|
-
Check necessity, axes, units, legend, self-contained caption, resolution, font, color, independent readability, and whether the figure supports a conclusion.
|
|
2757
|
-
|
|
2758
|
-
#### 7.3 Language — 1
|
|
2759
|
-
|
|
2760
|
-
Check concise, professional, accurate language without empty AI-style prose.
|
|
2761
|
-
|
|
2762
|
-
#### 7.4 Notation and formulas — 1
|
|
2763
|
-
|
|
2764
|
-
Check prior definition, consistency, numbering, units, and absence of decorative equations.
|
|
2765
|
-
|
|
2766
|
-
#### 7.5 Citations — 1
|
|
2767
|
-
|
|
2768
|
-
Check data, image, algorithm, and literature references and correspondence between text and bibliography.
|
|
2769
|
-
|
|
2770
|
-
#### 7.6 Page efficiency — 1
|
|
2771
|
-
|
|
2772
|
-
Check wasted space, unnecessary contents pages, code in body, hidden key reasoning, and information density under the page limit.
|
|
2773
|
-
|
|
2774
|
-
## Stage 3: model-by-model autopsy
|
|
2775
|
-
|
|
2776
|
-
For every M1, M2, ... report:
|
|
2777
|
-
|
|
2778
|
-
- problem solved, inputs, outputs, core equations, parameters, assumptions;
|
|
2779
|
-
- why chosen and mechanism fit;
|
|
2780
|
-
- mathematical correctness and parameter interpretability;
|
|
2781
|
-
- solver and result;
|
|
2782
|
-
- validation, sensitivity, robustness;
|
|
2783
|
-
- strengths, most serious defect, simpler alternative, marginal value;
|
|
2784
|
-
- score /10 and retain/simplify/restructure/untrustworthy.
|
|
2785
|
-
|
|
2786
|
-
Conclude unified system or model collage.
|
|
2787
|
-
|
|
2788
|
-
## Stage 4: key-result audit
|
|
2789
|
-
|
|
2790
|
-
Use:
|
|
2791
|
-
|
|
2792
|
-
| ID | Result | Source | Unit | Model | Reproducible | Magnitude | Validated | Importance | Verdict |
|
|
2793
|
-
|---|---|---|---|---|---|---|---|---|---|
|
|
2794
|
-
|
|
2795
|
-
Then answer:
|
|
2796
|
-
|
|
2797
|
-
1. most credible result;
|
|
2798
|
-
2. most fragile result;
|
|
2799
|
-
3. result most dependent on manual parameters;
|
|
2800
|
-
4. result most exposed to data failure;
|
|
2801
|
-
5. result whose failure collapses the paper;
|
|
2802
|
-
6. precision illusion;
|
|
2803
|
-
7. all constraints satisfied;
|
|
2804
|
-
8. better solution possibly missed;
|
|
2805
|
-
9. stability under ±5%, ±10%, ±20% input perturbations;
|
|
2806
|
-
10. reason to trust for real decisions.
|
|
2807
|
-
|
|
2808
|
-
## Stage 5: type-specific module
|
|
2809
|
-
|
|
2810
|
-
- MCM A: ODE/PDE/difference equations, initial/boundary conditions, dimensions, discretization, grid convergence, stability, parameter estimation, physical meaning.
|
|
2811
|
-
- MCM B: graph/combinatorial structure, dynamic/integer programming, correctness, complexity, optimality, scalability.
|
|
2812
|
-
- MCM C: quality, leakage, splits, feature engineering, variable selection, multicollinearity, overfitting, baselines, metrics, calibration, uncertainty, interpretation.
|
|
2813
|
-
- ICM D: network construction, node/edge meaning, weights, metrics, objective, constraints, optimization, scale, perturbation stability.
|
|
2814
|
-
- ICM E: boundary, time scale, lifecycle, environmental-economic-social tradeoff, weights, scenarios, uncertainty, long-run and unintended consequences.
|
|
2815
|
-
- ICM F: correlation/causality, behavior, stakeholders, fairness, cost, incentives, execution, scenarios, unintended and heterogeneous effects.
|
|
2816
|
-
|
|
2817
|
-
## Stage 6: award judgment
|
|
2818
|
-
|
|
2819
|
-
Reference bands:
|
|
2820
|
-
|
|
2821
|
-
- 93–100 Outstanding Candidate;
|
|
2822
|
-
- 87–92 Finalist;
|
|
2823
|
-
- 78–86 Meritorious;
|
|
2824
|
-
- 68–77 Honorable Mention;
|
|
2825
|
-
- 55–67 Successful Participant;
|
|
2826
|
-
- below 55 Unsuccessful Risk.
|
|
2827
|
-
|
|
2828
|
-
Outstanding additionally requires complete tasks, no major math/logic error, credible core validation, meaningful sensitivity/robustness, sourced key parameters, strong Summary Sheet, non-template modeling, closed model-result-conclusion loop, reproducibility, and one memorable contribution. High score without gates is Finalist or below.
|
|
2829
|
-
|
|
2830
|
-
## Required final output
|
|
2831
|
-
|
|
2832
|
-
Use exactly these top-level sections and fields.
|
|
2833
|
-
|
|
2834
|
-
### 一、60 秒终审印象
|
|
2835
|
-
|
|
2836
|
-
Paper objective; core methods; core results; strongest point; most serious first-impression problem; continue reading: 是/勉强/否.
|
|
2837
|
-
|
|
2838
|
-
### 二、一票否决与奖项封顶检查
|
|
2839
|
-
|
|
2840
|
-
For each risk: risk; evidence; severity; cap triggered; cap level.
|
|
2841
|
-
|
|
2842
|
-
### 三、总体评价与最终得分
|
|
2843
|
-
|
|
2844
|
-
Composite score /100; raw score; capped score; predicted award; Outstanding probability; Finalist-or-higher probability; core advantages; fatal weaknesses.
|
|
2845
|
-
|
|
2846
|
-
### 四、100 分分项评分
|
|
2847
|
-
|
|
2848
|
-
| 一级指标 | 二级指标 | 得分 | 满分 | 评委意见 |
|
|
2849
|
-
|---|---|---:|---:|---|
|
|
2850
|
-
|
|
2851
|
-
Every row needs evidence.
|
|
2852
|
-
|
|
2853
|
-
### 五、题目要求覆盖矩阵
|
|
2854
|
-
|
|
2855
|
-
| 题目要求 | 是否完成 | 使用模型 | 最终结果 | 验证情况 | 判定 |
|
|
2856
|
-
|---|---|---|---|---|---|
|
|
2857
|
-
|
|
2858
|
-
Find omissions and proxy answers.
|
|
2859
|
-
|
|
2860
|
-
### 六、模型逐个尸检
|
|
2861
|
-
|
|
2862
|
-
Review M1, M2, M3 separately using the autopsy template.
|
|
2863
|
-
|
|
2864
|
-
### 七、建模结果专项审计
|
|
2865
|
-
|
|
2866
|
-
Provide R1, R2, ... table and judge correctness, credibility, robustness, reproducibility, and practical meaning.
|
|
2867
|
-
|
|
2868
|
-
### 八、数据与参数审计
|
|
2869
|
-
|
|
2870
|
-
| 参数 | 数值 | 来源 | 是否有依据 | 敏感性 | 风险 |
|
|
2871
|
-
|---|---:|---|---|---|---|
|
|
2872
|
-
|
|
2873
|
-
Name Achilles’ heel parameters.
|
|
2874
|
-
|
|
2875
|
-
### 九、灵敏度、稳健性与验证专项评价
|
|
2876
|
-
|
|
2877
|
-
State what authors did, omitted, whether existing analysis is effective or formalistic, and validation needed for Finalist/Outstanding.
|
|
2878
|
-
|
|
2879
|
-
### 十、数学与计算正确性抽查
|
|
2880
|
-
|
|
2881
|
-
Check at least three formulas, three numbers, one algorithm, one figure, and one final conclusion when available; recompute when possible.
|
|
2882
|
-
|
|
2883
|
-
### 十一、图表与写作评审
|
|
2884
|
-
|
|
2885
|
-
Name best/worst/delete/add figures, hardest and most wasteful sections, and whether Summary Sheet is final-round quality.
|
|
2886
|
-
|
|
2887
|
-
### 十二、Outstanding Winner 差距分析
|
|
2888
|
-
|
|
2889
|
-
Explain why another paper wins; missing decisive evidence; current award level; three blockers to Finalist; three blockers to Outstanding; genuine innovation; removable model.
|
|
2890
|
-
|
|
2891
|
-
### 十三、修改优先级
|
|
2892
|
-
|
|
2893
|
-
P0 no high award without fix; P1 Meritorious vs Finalist; P2 Finalist vs Outstanding; P3 polish. Every item states problem, severity, fix, expected gain, and low/medium/high cost.
|
|
2894
|
-
|
|
2895
|
-
### 十四、评委最终裁决
|
|
2896
|
-
|
|
2897
|
-
One-sentence summary; most likely award; recommend Final Round yes/no; Head Judge defense yes/no; reason; final verdict.
|
|
2898
|
-
|
|
2899
|
-
Do not inflate for encouragement. Use “技术复杂度高于证据强度” or “presentation quality exceeds modeling quality” when supported. The final question is how credible the conclusions are and why they belong among the best papers.
|
|
2900
|
-
~~~
|
|
2901
|
-
|
|
2902
|
-
- [ ] **Step 4: Create a complete fourteen-section golden example**
|
|
2903
|
-
|
|
2904
|
-
Create `skills/math-modeling-audit/examples/mcm-final-review.md`:
|
|
2905
|
-
|
|
2906
|
-
~~~markdown
|
|
2907
|
-
# MCM/ICM Final Review Example
|
|
2908
|
-
|
|
2909
|
-
## 一、60 秒终审印象
|
|
2910
|
-
|
|
2911
|
-
- 这篇论文试图解决什么:多阶段资源配置。
|
|
2912
|
-
- 核心方法:线性规划与情景分析。
|
|
2913
|
-
- 核心结果:方案 A 的报告成本最低,但未提供可复现求解日志。
|
|
2914
|
-
- 最大亮点:约束映射清楚。
|
|
2915
|
-
- 第一眼最严重的问题:核心优化没有独立验证。
|
|
2916
|
-
- 评委是否愿意继续认真阅读:勉强。
|
|
2917
|
-
|
|
2918
|
-
## 二、一票否决与奖项封顶检查
|
|
2919
|
-
|
|
2920
|
-
- 风险:核心优化模型完全没有验证。
|
|
2921
|
-
- 证据:论文第 12 页仅报告求解器 success,无基线、gap 或复算。
|
|
2922
|
-
- 严重程度:严重问题。
|
|
2923
|
-
- 是否触发封顶:是。
|
|
2924
|
-
- 封顶等级:84。
|
|
2925
|
-
|
|
2926
|
-
## 三、总体评价与最终得分
|
|
2927
|
-
|
|
2928
|
-
- 综合得分:84/100。
|
|
2929
|
-
- 原始得分:90/100。
|
|
2930
|
-
- 封顶后得分:84/100。
|
|
2931
|
-
- 获奖等级预测:Meritorious。
|
|
2932
|
-
- Outstanding 概率:2%。
|
|
2933
|
-
- Finalist 及以上概率:18%。
|
|
2934
|
-
- 核心优势:机制与约束清楚。
|
|
2935
|
-
- 致命弱点:结果证据不足。
|
|
2936
|
-
|
|
2937
|
-
## 四、100 分分项评分
|
|
2938
|
-
|
|
2939
|
-
| 一级指标 | 二级指标 | 得分 | 满分 | 评委意见 |
|
|
2940
|
-
|---|---|---:|---:|---|
|
|
2941
|
-
| 结果与验证 | 独立验证 | 0 | 3 | 第 12 页未提供独立证据。 |
|
|
2942
|
-
|
|
2943
|
-
## 五、题目要求覆盖矩阵
|
|
2944
|
-
|
|
2945
|
-
| 题目要求 | 是否完成 | 使用模型 | 最终结果 | 验证情况 | 判定 |
|
|
2946
|
-
|---|---|---|---|---|---|
|
|
2947
|
-
| 最优配置 | 表面完成 | 线性规划 | 方案 A | 未验证 | WARNING |
|
|
2948
|
-
|
|
2949
|
-
## 六、模型逐个尸检
|
|
2950
|
-
|
|
2951
|
-
M1 线性规划:可简化;模型评分 7/10;缺少最优性与复现证据。
|
|
2952
|
-
|
|
2953
|
-
## 七、建模结果专项审计
|
|
2954
|
-
|
|
2955
|
-
| ID | Result | Source | Unit | Model | Reproducible | Magnitude | Validated | Importance | Verdict |
|
|
2956
|
-
|---|---|---|---|---|---|---|---|---|---|
|
|
2957
|
-
| R1 | 方案 A 成本最低 | p.12 | USD | M1 | 否 | 合理 | 否 | 核心 | WARNING |
|
|
2958
|
-
|
|
2959
|
-
## 八、数据与参数审计
|
|
2960
|
-
|
|
2961
|
-
| 参数 | 数值 | 来源 | 是否有依据 | 敏感性 | 风险 |
|
|
2962
|
-
|---|---:|---|---|---|---|
|
|
2963
|
-
| 惩罚系数 | 0.5 | 无 | 否 | 未分析 | 高 |
|
|
2964
|
-
|
|
2965
|
-
## 九、灵敏度、稳健性与验证专项评价
|
|
2966
|
-
|
|
2967
|
-
作者只扰动非关键需求参数;未分析惩罚系数和可行域变化,因此现有分析形式主义。
|
|
2968
|
-
|
|
2969
|
-
## 十、数学与计算正确性抽查
|
|
2970
|
-
|
|
2971
|
-
材料不足以完成三项数值复算;该缺口保留为未验证,不补造结果。
|
|
2972
|
-
|
|
2973
|
-
## 十一、图表与写作评审
|
|
2974
|
-
|
|
2975
|
-
最好图为图 3;最差且应删除图为无单位的图 7;应增加约束 slack 与基线对比图。
|
|
2976
|
-
|
|
2977
|
-
## 十二、Outstanding Winner 差距分析
|
|
2978
|
-
|
|
2979
|
-
真正的 Outstanding 会提供最优性、基线、敏感性和复现证据;本稿技术复杂度高于证据强度。
|
|
2980
|
-
|
|
2981
|
-
## 十三、修改优先级
|
|
2982
|
-
|
|
2983
|
-
P0:补充可复现求解、约束检查和独立复算;预计提升高;成本中。
|
|
2984
|
-
|
|
2985
|
-
## 十四、评委最终裁决
|
|
2986
|
-
|
|
2987
|
-
- 一句话总结:模型合理但核心结果未被证明可信。
|
|
2988
|
-
- 最可能奖项:Meritorious。
|
|
2989
|
-
- 是否推荐进入 Final Round:否。
|
|
2990
|
-
- 如果我是 Head Judge,我是否会为这篇论文辩护:否。
|
|
2991
|
-
- 理由:缺少决定性结果证据。
|
|
2992
|
-
- 最终 verdict:Meritorious,不能因表达漂亮上调。
|
|
2993
|
-
~~~
|
|
2994
|
-
|
|
2995
|
-
- [ ] **Step 5: Re-run package integrity and scorer tests**
|
|
2996
|
-
|
|
2997
|
-
Run:
|
|
2998
|
-
|
|
2999
|
-
~~~bash
|
|
3000
|
-
node --test tests/plugin-integrity.test.mjs tests/mcm-score.test.mjs
|
|
3001
|
-
~~~
|
|
3002
|
-
|
|
3003
|
-
Expected: all tests PASS and every MCM/ICM resource named by the audit Skill exists.
|
|
3004
|
-
|
|
3005
|
-
- [ ] **Step 6: Commit the final-judge rubric and example**
|
|
3006
|
-
|
|
3007
|
-
~~~bash
|
|
3008
|
-
git add skills/math-modeling-audit/SKILL.md skills/math-modeling-audit/references/mcm-icm-final-judge.md skills/math-modeling-audit/examples/mcm-final-review.md
|
|
3009
|
-
git commit -m "feat: add mcm icm final judging mode"
|
|
3010
|
-
~~~
|
|
3011
|
-
|
|
3012
|
-
|
|
3013
|
-
|
|
3014
|
-
### Task 10: Write the single bilingual README and lock user-facing behavior
|
|
3015
|
-
|
|
3016
|
-
**Files:**
|
|
3017
|
-
- Modify: `tests/plugin-integrity.test.mjs`
|
|
3018
|
-
- Create: `README.md`
|
|
3019
|
-
|
|
3020
|
-
- [ ] **Step 1: Add failing README contract checks**
|
|
3021
|
-
|
|
3022
|
-
Append to `tests/plugin-integrity.test.mjs`:
|
|
3023
|
-
|
|
3024
|
-
~~~javascript
|
|
3025
|
-
test('README documents verified installation, degradation, scoring, and uninstall', async () => {
|
|
3026
|
-
const readme = await read('README.md')
|
|
3027
|
-
assert.match(readme, /dsh plugin --profile web add github:yohanchen1\/MathModelingAgent#v0\.1\.0/)
|
|
3028
|
-
assert.match(readme, /dsh --profile web --dump-config/)
|
|
3029
|
-
assert.match(readme, /restart|重启/i)
|
|
3030
|
-
assert.match(readme, /Python.*optional|Python.*可选/is)
|
|
3031
|
-
assert.match(readme, /Lean.*optional|Lean.*可选/is)
|
|
3032
|
-
assert.match(readme, /Wolfram.*optional|Wolfram.*可选/is)
|
|
3033
|
-
assert.match(readme, /MCM\/ICM/)
|
|
3034
|
-
assert.match(readme, /math-modeling-runs/)
|
|
3035
|
-
assert.match(readme, /dsh plugin --profile web remove dsh-math-modeling-agent/)
|
|
3036
|
-
})
|
|
3037
|
-
~~~
|
|
3038
|
-
|
|
3039
|
-
- [ ] **Step 2: Run integrity test and verify RED**
|
|
3040
|
-
|
|
3041
|
-
Run:
|
|
3042
|
-
|
|
3043
|
-
~~~bash
|
|
3044
|
-
node --test tests/plugin-integrity.test.mjs
|
|
3045
|
-
~~~
|
|
3046
|
-
|
|
3047
|
-
Expected: FAIL with `ENOENT` for `README.md`.
|
|
3048
|
-
|
|
3049
|
-
- [ ] **Step 3: Create the concise bilingual README**
|
|
3050
|
-
|
|
3051
|
-
Create `README.md`:
|
|
3052
|
-
|
|
3053
|
-
~~~markdown
|
|
3054
|
-
# MathModelingAgent for DSH / 数学建模智能体
|
|
3055
|
-
|
|
3056
|
-
An installable DeepSeek Harness bundle for evidence-driven mathematical modeling, computation, research, independent verification, and MCM/ICM final-panel review.
|
|
3057
|
-
|
|
3058
|
-
一个可一键安装的 DSH 数学建模 Bundle:渐进式澄清问题、按需调用 Python/Lean/Wolfram、搜索文献与替代方法、记录每轮试错,并以证据而不是模型自信决定结论状态。
|
|
3059
|
-
|
|
3060
|
-
## Install / 安装
|
|
3061
|
-
|
|
3062
|
-
Pinned GitHub release / 固定 GitHub 版本:
|
|
3063
|
-
|
|
3064
|
-
~~~bash
|
|
3065
|
-
dsh plugin --profile web add github:yohanchen1/MathModelingAgent#v0.1.0
|
|
3066
|
-
~~~
|
|
3067
|
-
|
|
3068
|
-
After the npm release / npm 发布后:
|
|
3069
|
-
|
|
3070
|
-
~~~bash
|
|
3071
|
-
dsh plugin --profile web add dsh-math-modeling-agent
|
|
3072
|
-
~~~
|
|
3073
|
-
|
|
3074
|
-
Verify the composed profile / 验证配置:
|
|
3075
|
-
|
|
3076
|
-
~~~bash
|
|
3077
|
-
dsh --profile web --dump-config
|
|
3078
|
-
~~~
|
|
3079
|
-
|
|
3080
|
-
Restart the running DSH host after installation, then start the Web profile:
|
|
3081
|
-
安装后需重启当前 DSH host,再启动 Web profile:
|
|
3082
|
-
|
|
3083
|
-
~~~bash
|
|
3084
|
-
dsh web
|
|
3085
|
-
~~~
|
|
3086
|
-
|
|
3087
|
-
## Use / 使用
|
|
3088
|
-
|
|
3089
|
-
Describe the task naturally. Internal phases and tools are selected automatically.
|
|
3090
|
-
直接描述任务,不需要记住 Skill 名称或选择内部流程。
|
|
3091
|
-
|
|
3092
|
-
Examples / 示例:
|
|
3093
|
-
|
|
3094
|
-
- “建立并验证这个优化问题的数学模型,附件中有 Excel 数据。”
|
|
3095
|
-
- “Continue the modeling run in math-modeling-runs/supply-plan.”
|
|
3096
|
-
- “按 MCM/ICM 终审 100 分框架审查这篇论文,先做封顶检查。”
|
|
3097
|
-
|
|
3098
|
-
## What happens / 工作流程
|
|
3099
|
-
|
|
3100
|
-
1. Classify the problem and choose Fast, Standard, or High-Assurance.
|
|
3101
|
-
2. Ask one high-impact mathematical-modeling question at a time.
|
|
3102
|
-
3. Profile inputs and register assumptions, claims, and verification obligations.
|
|
3103
|
-
4. Build a baseline and materially different candidate models.
|
|
3104
|
-
5. Execute and verify with available tools.
|
|
3105
|
-
6. Search literature or fork directions when evidence is missing or a route fails.
|
|
3106
|
-
7. Save an incremental report for every attempt.
|
|
3107
|
-
8. Finish with SOLVED, PARTIAL, CONDITIONAL, INCONCLUSIVE, REFUTED, INFEASIBLE, UNIDENTIFIABLE, BLOCKED, or CANCELLED.
|
|
3108
|
-
|
|
3109
|
-
## Tools / 工具
|
|
3110
|
-
|
|
3111
|
-
The bundle itself requires no mathematical runtime.
|
|
3112
|
-
插件安装本身不要求任何数学运行环境。
|
|
3113
|
-
|
|
3114
|
-
- Python is optional but recommended. When available, the agent creates a run-local uv/venv environment and installs only required packages.
|
|
3115
|
-
- Lean is optional and is never auto-installed.
|
|
3116
|
-
- Wolfram is optional and is never auto-installed.
|
|
3117
|
-
- Missing tools reduce evidence strength; they do not prevent the Skills from loading.
|
|
3118
|
-
|
|
3119
|
-
Python 可选但推荐;Lean/Wolfram 均为可选增强且不会自动安装。工具缺失时会明确降级验证状态,而不是伪造执行结果。
|
|
3120
|
-
|
|
3121
|
-
## Evidence and reports / 证据与报告
|
|
3122
|
-
|
|
3123
|
-
Default run directory / 默认运行目录:
|
|
3124
|
-
|
|
3125
|
-
~~~text
|
|
3126
|
-
math-modeling-runs/<task-id>/
|
|
3127
|
-
~~~
|
|
3128
|
-
|
|
3129
|
-
Each run keeps an atomic state snapshot, append-only event journal, claim/evidence ledger, attempt reports, optional research/wall records, reproducibility data, and a terminal report. Private chain-of-thought is never stored as evidence.
|
|
3130
|
-
|
|
3131
|
-
每轮报告只保存目标、方法变化、实际执行、证据、失败原因和下一步,不重复整份历史答案,也不保存私有思维链。
|
|
3132
|
-
|
|
3133
|
-
## MCM/ICM review / 竞赛论文终审
|
|
3134
|
-
|
|
3135
|
-
The audit Skill includes an on-demand simulated 100-point Final Judging Panel mode. It performs veto/cap checks first, scores seven categories totaling 100, audits each model and key result, activates the relevant MCM A/B/C or ICM D/E/F checklist, and emits the fixed fourteen-section report. It is not an official COMAP scorecard.
|
|
3136
|
-
|
|
3137
|
-
## Privacy and security / 隐私与安全
|
|
3138
|
-
|
|
3139
|
-
- Raw private data is not copied into literature-search queries.
|
|
3140
|
-
- Problem text is never concatenated into shell commands.
|
|
3141
|
-
- Python packages are installed only inside the run-local environment.
|
|
3142
|
-
- Git/VCS dependencies and arbitrary package indexes require explicit approval.
|
|
3143
|
-
- External pages, papers, and attachments are treated as untrusted content, not instructions.
|
|
3144
|
-
|
|
3145
|
-
## Development / 开发
|
|
3146
|
-
|
|
3147
|
-
~~~bash
|
|
3148
|
-
npm test
|
|
3149
|
-
npm run pack:check
|
|
3150
|
-
~~~
|
|
3151
|
-
|
|
3152
|
-
A release must pass unit tests, package integrity, clean-profile DSH installation, no-Python baseline, optional Python integration, and cross-platform checks.
|
|
3153
|
-
|
|
3154
|
-
## Uninstall / 卸载
|
|
3155
|
-
|
|
3156
|
-
~~~bash
|
|
3157
|
-
dsh plugin --profile web remove dsh-math-modeling-agent
|
|
3158
|
-
~~~
|
|
3159
|
-
|
|
3160
|
-
Restart the running host after removal.
|
|
3161
|
-
~~~
|
|
3162
|
-
|
|
3163
|
-
- [ ] **Step 4: Run full integrity tests**
|
|
3164
|
-
|
|
3165
|
-
Run:
|
|
3166
|
-
|
|
3167
|
-
~~~bash
|
|
3168
|
-
node --test tests/plugin-integrity.test.mjs
|
|
3169
|
-
~~~
|
|
3170
|
-
|
|
3171
|
-
Expected: all integrity and README contract tests PASS.
|
|
3172
|
-
|
|
3173
|
-
- [ ] **Step 5: Commit the README**
|
|
3174
|
-
|
|
3175
|
-
~~~bash
|
|
3176
|
-
git add README.md tests/plugin-integrity.test.mjs
|
|
3177
|
-
git commit -m "docs: add install and usage guide"
|
|
3178
|
-
~~~
|
|
3179
|
-
|
|
3180
|
-
### Task 11: Run full verification and package smoke tests
|
|
3181
|
-
|
|
3182
|
-
**Files:**
|
|
3183
|
-
- Verify only: all production, test, design, and plan files
|
|
3184
|
-
|
|
3185
|
-
- [ ] **Step 1: Run all deterministic tests**
|
|
3186
|
-
|
|
3187
|
-
Run:
|
|
3188
|
-
|
|
3189
|
-
~~~bash
|
|
3190
|
-
npm test
|
|
3191
|
-
~~~
|
|
3192
|
-
|
|
3193
|
-
Expected: all tests PASS, 0 failures.
|
|
3194
|
-
|
|
3195
|
-
- [ ] **Step 2: Verify package contents without building**
|
|
3196
|
-
|
|
3197
|
-
Run:
|
|
3198
|
-
|
|
3199
|
-
~~~bash
|
|
3200
|
-
npm run pack:check
|
|
3201
|
-
~~~
|
|
3202
|
-
|
|
3203
|
-
Expected: exit 0; package contains `package.json`, `cordis.patch.yml`, `README.md`, `LICENSE`, both Skill bundles, every named reference/script/schema/example, and no tests, legacy Python, logs, data, or design documents.
|
|
3204
|
-
|
|
3205
|
-
- [ ] **Step 3: Verify local DSH installation on Windows via package tarball**
|
|
3206
|
-
|
|
3207
|
-
Verified finding: `dsh plugin --profile smoke add .` installs the directory basename as a plain dependency and never activates the bundle. Always install by package name (tarball, GitHub spec, or npm).
|
|
3208
|
-
|
|
3209
|
-
Run from the plugin root in PowerShell:
|
|
3210
|
-
|
|
3211
|
-
~~~powershell
|
|
3212
|
-
$oldHome = $env:DSH_HOME
|
|
3213
|
-
$packDir = Join-Path $env:TEMP ("mma-pack-" + [guid]::NewGuid().ToString("N"))
|
|
3214
|
-
New-Item -ItemType Directory -Path $packDir | Out-Null
|
|
3215
|
-
try {
|
|
3216
|
-
npm pack --pack-destination $packDir | Out-Null
|
|
3217
|
-
$tgz = Get-ChildItem -LiteralPath $packDir -Filter '*.tgz' | Select-Object -First 1
|
|
3218
|
-
$tempHome = Join-Path $env:TEMP ("dsh-mma-smoke-" + [guid]::NewGuid().ToString("N"))
|
|
3219
|
-
$env:DSH_HOME = $tempHome
|
|
3220
|
-
dsh plugin --profile smoke add (Join-Path $packDir $tgz.Name)
|
|
3221
|
-
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
|
|
3222
|
-
|
|
3223
|
-
$dump = dsh --profile smoke --dump-config
|
|
3224
|
-
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
|
|
3225
|
-
$dump | Select-String 'dsh-math-modeling-agent-skills'
|
|
3226
|
-
$dump | Select-String 'bundledSkillDir'
|
|
3227
|
-
|
|
3228
|
-
dsh plugin --profile smoke remove dsh-math-modeling-agent
|
|
3229
|
-
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
|
|
3230
|
-
} finally {
|
|
3231
|
-
$env:DSH_HOME = $oldHome
|
|
3232
|
-
}
|
|
3233
|
-
~~~
|
|
3234
|
-
|
|
3235
|
-
Expected: the profile records `dsh-math-modeling-agent` in `dsh.profile.bundles`, dump contains the Skill provider row and `bundledSkillDir`, removal succeeds, and the user’s normal `DSH_HOME` is untouched.
|
|
3236
|
-
|
|
3237
|
-
- [ ] **Step 4: Verify local DSH installation on Linux/macOS CI**
|
|
3238
|
-
|
|
3239
|
-
Run:
|
|
3240
|
-
|
|
3241
|
-
~~~bash
|
|
3242
|
-
old_home="${DSH_HOME-}"
|
|
3243
|
-
pack_dir="$(mktemp -d)"
|
|
3244
|
-
temp_home="$(mktemp -d)"
|
|
3245
|
-
npm pack --pack-destination "$pack_dir" >/dev/null
|
|
3246
|
-
export DSH_HOME="$temp_home"
|
|
3247
|
-
dsh plugin --profile smoke add "$pack_dir"/dsh-math-modeling-agent-0.1.0.tgz
|
|
3248
|
-
dsh --profile smoke --dump-config | grep 'dsh-math-modeling-agent-skills'
|
|
3249
|
-
dsh --profile smoke --dump-config | grep 'bundledSkillDir'
|
|
3250
|
-
dsh plugin --profile smoke remove dsh-math-modeling-agent
|
|
3251
|
-
export DSH_HOME="$old_home"
|
|
3252
|
-
~~~
|
|
3253
|
-
|
|
3254
|
-
Expected: the same install/dump/remove behavior as Windows.
|
|
3255
|
-
|
|
3256
|
-
- [ ] **Step 5: Run static repository checks**
|
|
3257
|
-
|
|
3258
|
-
Run:
|
|
3259
|
-
|
|
3260
|
-
~~~bash
|
|
3261
|
-
git diff --check
|
|
3262
|
-
git status --short
|
|
3263
|
-
git grep -n -E 'C:\\Users\\16605|\.claude[/\\]plugins|GEMINI_API_KEY|google-generativeai|run_logs|thinking_log' -- ':!docs/superpowers/**'
|
|
3264
|
-
~~~
|
|
3265
|
-
|
|
3266
|
-
Expected:
|
|
3267
|
-
|
|
3268
|
-
- `git diff --check` emits nothing;
|
|
3269
|
-
- `git status --short` contains only the implementation-plan file if it was not committed before execution;
|
|
3270
|
-
- the grep command returns no production matches.
|
|
3271
|
-
|
|
3272
|
-
- [ ] **Step 6: Review the final file inventory for minimality**
|
|
3273
|
-
|
|
3274
|
-
Run:
|
|
3275
|
-
|
|
3276
|
-
~~~bash
|
|
3277
|
-
git ls-files
|
|
3278
|
-
~~~
|
|
3279
|
-
|
|
3280
|
-
Expected: only the files declared by the approved spec plus design/plan documentation. No empty directories, duplicated protocols, lockfile without dependencies, generated package tarball, coverage output, legacy code, or temporary artifacts.
|
|
3281
|
-
|
|
3282
|
-
- [ ] **Step 7: Commit any final verified documentation-only corrections**
|
|
3283
|
-
|
|
3284
|
-
If verification required a factual README or reference correction, make only that correction, rerun Steps 1–6, then commit:
|
|
3285
|
-
|
|
3286
|
-
~~~bash
|
|
3287
|
-
git add README.md skills
|
|
3288
|
-
git commit -m "docs: correct verified plugin guidance"
|
|
3289
|
-
~~~
|
|
3290
|
-
|
|
3291
|
-
If no correction was required, do not create an empty commit.
|
|
3292
|
-
|
|
3293
|
-
## Plan self-review checklist
|
|
3294
|
-
|
|
3295
|
-
- [ ] Every approved spec section maps to at least one task above.
|
|
3296
|
-
- [ ] Exactly two public Skills are created.
|
|
3297
|
-
- [ ] The four deterministic Modules have stable interfaces and tests.
|
|
3298
|
-
- [ ] Python, Lean, and Wolfram remain optional at install time.
|
|
3299
|
-
- [ ] Every strong status is evidence-gated.
|
|
3300
|
-
- [ ] MCM/ICM subtotals equal 100 and caps are deterministic.
|
|
3301
|
-
- [ ] No task uses placeholders, “similar to”, or unspecified error handling.
|
|
3302
|
-
- [ ] Every code-changing step shows exact content or an exact replacement block.
|
|
3303
|
-
- [ ] Every task ends with a focused Conventional Commit.
|