@surlymochan/evalid 0.5.2 → 0.5.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,14 @@ All notable changes to this project are documented here. Detailed notes live in
5
5
 
6
6
  ## [Unreleased]
7
7
 
8
+ ## [0.5.3] - 2026-11-07
9
+
10
+ ### Fixed
11
+ - **Bench openswarm planner timeout raised 300s → 600s (#41)**: the 300s cap was calibrated on DeepSeek, whose planner finished in 87–244s on the 6 pilot instances. With MiniMax M3.1-Flash-Preview at high thinking the planner exceeds 300s on all 6 pilot2 cells: killed mid-work (rc 143) before writing the task file, so the executor never ran and all 6 cells recorded false negatives (wall ~301s). 600s keeps planner + executor inside the CAP_S 1200s cell budget; the executor budget is derived from the remaining time.
12
+ - **Bench openswarm planner prompt tightened to read-only planning (#42)**: the 600s validation cell (pylint-8898) showed the planner spending 13 turns / 23 tools exploring (openswarm lane serve/assign probes, full-disk find for pylint wheels) and getting killed (rc 143) before writing the task file — raising the budget alone cannot fix that. The planner now reads only the skill text and the task, runs no shell command, does not reproduce / explore / list / search, launches no other program, writes the task file + verify command, then stops. The 600s planner timeout from #41 is unchanged. Test evidence: new `bench/swebench/test_planner_prompt.py` (7 tests, stdlib-only via ast so CI can run it without pandas/swebench) + node wrapper `test/bench-planner-prompt.test.js`; bench Python suite 35/35, full `npm test` green.
13
+
14
+ Acceptance debt, stated plainly: the bench-number acceptance for P1-2 / P1-3 / P2-1 / P2-3 still awaits the MiniMax full-matrix batch (15 instances × 4 arms × reps 3) — no bench numbers are claimed in this release.
15
+
8
16
  ## [0.5.2] - 2026-11-07
9
17
 
10
18
  ### Fixed
package/RELEASE_NOTES.md CHANGED
@@ -1,5 +1,15 @@
1
1
  # Release Notes
2
2
 
3
+ ## 0.5.3 - Bench planner stops timing out and stops wandering
4
+
5
+ Patch release. Two bench-harness fixes from the MiniMax pilot, both pinned by tests rather than by a live run.
6
+
7
+ **Openswarm planner timeout 300s → 600s (#41).** The 300s cap was calibrated on DeepSeek, whose planner finished in 87–244s on the 6 pilot instances. With MiniMax M3.1-Flash-Preview at high thinking the planner exceeds 300s on all 6 pilot2 cells: killed mid-work (rc 143) before writing the task file, so the executor never ran and all 6 cells recorded false negatives (wall ~301s). 600s keeps planner + executor inside the CAP_S 1200s cell budget; the executor budget is derived from the remaining time.
8
+
9
+ **Openswarm planner prompt tightened to read-only planning (#42).** The 600s validation cell (pylint-8898) showed the planner spending 13 turns / 23 tools exploring (openswarm lane serve/assign probes, full-disk find for pylint wheels) and getting killed (rc 143) before writing the task file — raising the budget alone cannot fix that. The planner now reads only the skill text and the task, runs no shell command, does not reproduce / explore / list / search, launches no other program, writes the task file + verify command, then stops. The 600s planner timeout from #41 is unchanged. Test evidence (from the PR): new `bench/swebench/test_planner_prompt.py` (7 tests, stdlib-only via ast so CI can run it without pandas/swebench) + node wrapper `test/bench-planner-prompt.test.js`; bench Python suite 35/35, full `npm test` green.
10
+
11
+ Acceptance debt, stated plainly: the bench-number acceptance for P1-2, P1-3, P2-1 and P2-3 — the wall-clock/token/resolved lines from issue #5 — still awaits the MiniMax full-matrix batch (15 instances × 4 arms × reps 3), which is running. No bench numbers are claimed in this release.
12
+
3
13
  ## 0.5.2 - Bench grades stop silently zeroing; the 15-instance set is validated
4
14
 
5
15
  Patch release. One bench-harness fix that removes a silent false-negative from grading, plus the P2-2 validation close-out from the issue #5 backlog. Both are pinned by tests and by a validation log, not by a live run.
@@ -1,6 +1,6 @@
1
1
  # evalid Launch Dossier
2
2
 
3
- Generated: 2026-11-06T19:58:36.504Z
3
+ Generated: 2026-11-07T05:49:01.426Z
4
4
  Mode: post_publish
5
5
  Verdict: conditionally ready: engine and adoption evidence are strong, registry publication remains required for a splash launch
6
6
  Launch ready: not yet
@@ -117,7 +117,7 @@ Evidence:
117
117
  - Adoption records: 1; status: pass
118
118
  - Goal audit: pass; goal: Keep evalid credible as an open-source agent delivery runtime: repo-owned workflow contracts, executable verification gates, durable project context, and honest benchmark evidence.
119
119
  - Package evidence: missing; source: registry_json
120
- - Package name/version: @surlymochan/evalid@0.5.2
120
+ - Package name/version: @surlymochan/evalid@0.5.3
121
121
 
122
122
  ## Passing Evidence
123
123
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@surlymochan/evalid",
3
- "version": "0.5.2",
3
+ "version": "0.5.3",
4
4
  "description": "Claimed != verified: a runtime verification layer for coding agents. It seals each test run and replays it from a clean snapshot, with a zero-judge gate pipeline (spec clarification, code review, TDD proof) and the xflow CLI.",
5
5
  "keywords": [
6
6
  "ai-agents",