devlyn-cli 2.10.1 → 3.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +4 -9
- package/CLAUDE.md +11 -15
- package/README.md +31 -7
- package/benchmark/auto-resolve/results/20260507-f10-f11-tier1-full-pipeline/headroom-gate.json +25 -0
- package/benchmark/auto-resolve/results/20260507-f10-f11-tier1-full-pipeline/headroom-gate.md +10 -0
- package/benchmark/auto-resolve/results/20260508-f22-exact-error-headroom/headroom-gate.json +18 -0
- package/benchmark/auto-resolve/results/20260508-f22-exact-error-headroom/headroom-gate.md +9 -0
- package/benchmark/auto-resolve/results/20260508-f26-headroom/headroom-gate.json +18 -0
- package/benchmark/auto-resolve/results/20260508-f26-headroom/headroom-gate.md +9 -0
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/F16-cli-quote-tax-rules/l2_risk_probes/result.json +28 -0
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/F23-cli-fulfillment-wave/l2_risk_probes/result.json +28 -0
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/F25-cli-cart-promotion-rules/l2_risk_probes/result.json +30 -0
- package/benchmark/auto-resolve/results/20260511-f12-webhook-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260511-f12-webhook-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260511-f15-concurrency-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260511-f15-concurrency-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/F21-cli-scheduler-priority/l2_risk_probes/result.json +32 -0
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/full-pipeline-pair-gate.json +44 -0
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/full-pipeline-pair-gate.md +16 -0
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/headroom-gate.json +18 -0
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/headroom-gate.md +9 -0
- package/benchmark/auto-resolve/results/20260511-f3-http-error-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260511-f3-http-error-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260512-f2-medium-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260512-f2-medium-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260512-f31-seat-rebalance-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260512-f31-seat-rebalance-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260512-f32-subscription-renewal-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260512-f32-subscription-renewal-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260512-f4-web-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260512-f4-web-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260512-f5-fixloop-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260512-f5-fixloop-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260512-f6-checksum-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260512-f6-checksum-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260512-f7-scope-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260512-f7-scope-headroom/headroom-gate.md +15 -0
- package/benchmark/auto-resolve/results/20260512-f9-e2e-headroom/headroom-gate.json +28 -0
- package/benchmark/auto-resolve/results/20260512-f9-e2e-headroom/headroom-gate.md +15 -0
- package/bin/devlyn.js +6 -1
- package/config/skills/_shared/.ruff.toml +2 -0
- package/config/skills/_shared/adapters/claude.md +9 -1
- package/config/skills/_shared/adapters/codex.md +5 -0
- package/config/skills/_shared/adapters/grok.md +4 -2
- package/config/skills/_shared/archive_run.py +738 -56
- package/config/skills/_shared/codex-config.md +19 -4
- package/config/skills/_shared/codex-monitored.sh +81 -12
- package/config/skills/_shared/collect-codex-findings.py +311 -128
- package/config/skills/_shared/engine-preflight.md +8 -7
- package/config/skills/_shared/expected.schema.json +61 -0
- package/config/skills/_shared/finish-gate.py +12 -11
- package/config/skills/_shared/grok-anchor-guard.py +18 -1
- package/config/skills/_shared/invocation-receipt.py +1041 -0
- package/config/skills/_shared/judge-output-parser.py +200 -0
- package/config/skills/_shared/judge-role-evidence.py +273 -0
- package/config/skills/_shared/process-evidence.py +909 -0
- package/config/skills/_shared/resolve-bootstrap.py +776 -84
- package/config/skills/_shared/resolve-stop-hook.py +33 -1
- package/config/skills/_shared/role-config.py +552 -0
- package/config/skills/_shared/run-bounded.py +1 -1
- package/config/skills/_shared/spec-verify-check.py +749 -181
- package/config/skills/_shared/state-phase-write.py +1838 -58
- package/config/skills/_shared/terminal-claim-check.py +15 -0
- package/config/skills/_shared/verify-merge-findings.py +958 -51
- package/config/skills/devlyn:engines/SKILL.md +8 -16
- package/config/skills/devlyn:ideate/references/spec-template.md +1 -0
- package/config/skills/devlyn:queue/SKILL.md +7 -4
- package/config/skills/devlyn:resolve/SKILL.md +91 -33
- package/config/skills/devlyn:resolve/references/outer-loop.md +15 -0
- package/config/skills/devlyn:resolve/references/phases/build-gate.md +31 -2
- package/config/skills/devlyn:resolve/references/phases/implement.md +3 -2
- package/config/skills/devlyn:resolve/references/phases/verify.md +120 -84
- package/config/skills/devlyn:resolve/references/state-schema.md +22 -8
- package/package.json +5 -1
- package/scripts/lint-skills.sh +314 -285
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
{
|
|
2
|
+
"run_id": "20260511-f3-http-error-headroom",
|
|
3
|
+
"rule": "at least 1 candidate fixtures; each must satisfy bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, with both baseline arms evidence-complete",
|
|
4
|
+
"verdict": "FAIL",
|
|
5
|
+
"fixtures_total": 1,
|
|
6
|
+
"fixtures_passed": 0,
|
|
7
|
+
"min_fixtures": 1,
|
|
8
|
+
"bare_max": 60,
|
|
9
|
+
"solo_max": 80,
|
|
10
|
+
"min_bare_headroom_required": 5,
|
|
11
|
+
"min_solo_headroom_required": 5,
|
|
12
|
+
"fixture_count_ok": true,
|
|
13
|
+
"avg_bare_headroom": -37.0,
|
|
14
|
+
"min_bare_headroom": -37,
|
|
15
|
+
"avg_solo_headroom": -19.0,
|
|
16
|
+
"min_solo_headroom": -19,
|
|
17
|
+
"rows": [
|
|
18
|
+
{
|
|
19
|
+
"fixture": "F3-backend-contract-risk",
|
|
20
|
+
"status": "FAIL",
|
|
21
|
+
"bare_score": 97,
|
|
22
|
+
"solo_score": 99,
|
|
23
|
+
"bare_headroom": -37,
|
|
24
|
+
"solo_headroom": -19,
|
|
25
|
+
"reason": "bare score 97 > 60; solo_claude score 99 > 80"
|
|
26
|
+
}
|
|
27
|
+
]
|
|
28
|
+
}
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Headroom Gate — 20260511-f3-http-error-headroom
|
|
2
|
+
|
|
3
|
+
Verdict: **FAIL**
|
|
4
|
+
|
|
5
|
+
Fixtures passed: 0/1 (minimum required: 1)
|
|
6
|
+
|
|
7
|
+
Rule: at least 1 fixtures; bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, both baseline arms evidence-complete.
|
|
8
|
+
Average bare headroom: -37.0
|
|
9
|
+
Minimum bare headroom: -37
|
|
10
|
+
Average solo headroom: -19.0
|
|
11
|
+
Minimum solo headroom: -19
|
|
12
|
+
|
|
13
|
+
| Fixture | Bare | Bare headroom | Solo | Solo headroom | Status | Reason |
|
|
14
|
+
|---|---:|---:|---:|---:|---|---|
|
|
15
|
+
| F3-backend-contract-risk | 97 | -37 | 99 | -19 | FAIL | bare score 97 > 60; solo_claude score 99 > 80 |
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
{
|
|
2
|
+
"run_id": "20260512-f2-medium-headroom",
|
|
3
|
+
"rule": "at least 1 candidate fixtures; each must satisfy bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, with both baseline arms evidence-complete",
|
|
4
|
+
"verdict": "FAIL",
|
|
5
|
+
"fixtures_total": 1,
|
|
6
|
+
"fixtures_passed": 0,
|
|
7
|
+
"min_fixtures": 1,
|
|
8
|
+
"bare_max": 60,
|
|
9
|
+
"solo_max": 80,
|
|
10
|
+
"min_bare_headroom_required": 5,
|
|
11
|
+
"min_solo_headroom_required": 5,
|
|
12
|
+
"fixture_count_ok": true,
|
|
13
|
+
"avg_bare_headroom": -23.0,
|
|
14
|
+
"min_bare_headroom": -23,
|
|
15
|
+
"avg_solo_headroom": -15.0,
|
|
16
|
+
"min_solo_headroom": -15,
|
|
17
|
+
"rows": [
|
|
18
|
+
{
|
|
19
|
+
"fixture": "F2-cli-medium-subcommand",
|
|
20
|
+
"status": "FAIL",
|
|
21
|
+
"bare_score": 83,
|
|
22
|
+
"solo_score": 95,
|
|
23
|
+
"bare_headroom": -23,
|
|
24
|
+
"solo_headroom": -15,
|
|
25
|
+
"reason": "bare score 83 > 60; solo_claude score 95 > 80"
|
|
26
|
+
}
|
|
27
|
+
]
|
|
28
|
+
}
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Headroom Gate — 20260512-f2-medium-headroom
|
|
2
|
+
|
|
3
|
+
Verdict: **FAIL**
|
|
4
|
+
|
|
5
|
+
Fixtures passed: 0/1 (minimum required: 1)
|
|
6
|
+
|
|
7
|
+
Rule: at least 1 fixtures; bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, both baseline arms evidence-complete.
|
|
8
|
+
Average bare headroom: -23.0
|
|
9
|
+
Minimum bare headroom: -23
|
|
10
|
+
Average solo headroom: -15.0
|
|
11
|
+
Minimum solo headroom: -15
|
|
12
|
+
|
|
13
|
+
| Fixture | Bare | Bare headroom | Solo | Solo headroom | Status | Reason |
|
|
14
|
+
|---|---:|---:|---:|---:|---|---|
|
|
15
|
+
| F2-cli-medium-subcommand | 83 | -23 | 95 | -15 | FAIL | bare score 83 > 60; solo_claude score 95 > 80 |
|
package/benchmark/auto-resolve/results/20260512-f31-seat-rebalance-headroom/headroom-gate.json
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
{
|
|
2
|
+
"run_id": "20260512-f31-seat-rebalance-headroom",
|
|
3
|
+
"rule": "at least 1 candidate fixtures; each must satisfy bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, with both baseline arms evidence-complete",
|
|
4
|
+
"verdict": "FAIL",
|
|
5
|
+
"fixtures_total": 1,
|
|
6
|
+
"fixtures_passed": 0,
|
|
7
|
+
"min_fixtures": 1,
|
|
8
|
+
"bare_max": 60,
|
|
9
|
+
"solo_max": 80,
|
|
10
|
+
"min_bare_headroom_required": 5,
|
|
11
|
+
"min_solo_headroom_required": 5,
|
|
12
|
+
"fixture_count_ok": true,
|
|
13
|
+
"avg_bare_headroom": 27.0,
|
|
14
|
+
"min_bare_headroom": 27,
|
|
15
|
+
"avg_solo_headroom": -18.0,
|
|
16
|
+
"min_solo_headroom": -18,
|
|
17
|
+
"rows": [
|
|
18
|
+
{
|
|
19
|
+
"fixture": "F31-cli-seat-rebalance",
|
|
20
|
+
"status": "FAIL",
|
|
21
|
+
"bare_score": 33,
|
|
22
|
+
"solo_score": 98,
|
|
23
|
+
"bare_headroom": 27,
|
|
24
|
+
"solo_headroom": -18,
|
|
25
|
+
"reason": "solo_claude score 98 > 80; bare judge disqualifier; bare result disqualifier; bare verify disqualifier"
|
|
26
|
+
}
|
|
27
|
+
]
|
|
28
|
+
}
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Headroom Gate — 20260512-f31-seat-rebalance-headroom
|
|
2
|
+
|
|
3
|
+
Verdict: **FAIL**
|
|
4
|
+
|
|
5
|
+
Fixtures passed: 0/1 (minimum required: 1)
|
|
6
|
+
|
|
7
|
+
Rule: at least 1 fixtures; bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, both baseline arms evidence-complete.
|
|
8
|
+
Average bare headroom: 27.0
|
|
9
|
+
Minimum bare headroom: 27
|
|
10
|
+
Average solo headroom: -18.0
|
|
11
|
+
Minimum solo headroom: -18
|
|
12
|
+
|
|
13
|
+
| Fixture | Bare | Bare headroom | Solo | Solo headroom | Status | Reason |
|
|
14
|
+
|---|---:|---:|---:|---:|---|---|
|
|
15
|
+
| F31-cli-seat-rebalance | 33 | 27 | 98 | -18 | FAIL | solo_claude score 98 > 80; bare judge disqualifier; bare result disqualifier; bare verify disqualifier |
|
package/benchmark/auto-resolve/results/20260512-f32-subscription-renewal-headroom/headroom-gate.json
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
{
|
|
2
|
+
"run_id": "20260512-f32-subscription-renewal-headroom",
|
|
3
|
+
"rule": "at least 1 candidate fixtures; each must satisfy bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, with both baseline arms evidence-complete",
|
|
4
|
+
"verdict": "FAIL",
|
|
5
|
+
"fixtures_total": 1,
|
|
6
|
+
"fixtures_passed": 0,
|
|
7
|
+
"min_fixtures": 1,
|
|
8
|
+
"bare_max": 60,
|
|
9
|
+
"solo_max": 80,
|
|
10
|
+
"min_bare_headroom_required": 5,
|
|
11
|
+
"min_solo_headroom_required": 5,
|
|
12
|
+
"fixture_count_ok": true,
|
|
13
|
+
"avg_bare_headroom": 27.0,
|
|
14
|
+
"min_bare_headroom": 27,
|
|
15
|
+
"avg_solo_headroom": -18.0,
|
|
16
|
+
"min_solo_headroom": -18,
|
|
17
|
+
"rows": [
|
|
18
|
+
{
|
|
19
|
+
"fixture": "F32-cli-subscription-renewal",
|
|
20
|
+
"status": "FAIL",
|
|
21
|
+
"bare_score": 33,
|
|
22
|
+
"solo_score": 98,
|
|
23
|
+
"bare_headroom": 27,
|
|
24
|
+
"solo_headroom": -18,
|
|
25
|
+
"reason": "solo_claude score 98 > 80"
|
|
26
|
+
}
|
|
27
|
+
]
|
|
28
|
+
}
|
package/benchmark/auto-resolve/results/20260512-f32-subscription-renewal-headroom/headroom-gate.md
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Headroom Gate — 20260512-f32-subscription-renewal-headroom
|
|
2
|
+
|
|
3
|
+
Verdict: **FAIL**
|
|
4
|
+
|
|
5
|
+
Fixtures passed: 0/1 (minimum required: 1)
|
|
6
|
+
|
|
7
|
+
Rule: at least 1 fixtures; bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, both baseline arms evidence-complete.
|
|
8
|
+
Average bare headroom: 27.0
|
|
9
|
+
Minimum bare headroom: 27
|
|
10
|
+
Average solo headroom: -18.0
|
|
11
|
+
Minimum solo headroom: -18
|
|
12
|
+
|
|
13
|
+
| Fixture | Bare | Bare headroom | Solo | Solo headroom | Status | Reason |
|
|
14
|
+
|---|---:|---:|---:|---:|---|---|
|
|
15
|
+
| F32-cli-subscription-renewal | 33 | 27 | 98 | -18 | FAIL | solo_claude score 98 > 80 |
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
{
|
|
2
|
+
"run_id": "20260512-f4-web-headroom",
|
|
3
|
+
"rule": "at least 1 candidate fixtures; each must satisfy bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, with both baseline arms evidence-complete",
|
|
4
|
+
"verdict": "FAIL",
|
|
5
|
+
"fixtures_total": 1,
|
|
6
|
+
"fixtures_passed": 0,
|
|
7
|
+
"min_fixtures": 1,
|
|
8
|
+
"bare_max": 60,
|
|
9
|
+
"solo_max": 80,
|
|
10
|
+
"min_bare_headroom_required": 5,
|
|
11
|
+
"min_solo_headroom_required": 5,
|
|
12
|
+
"fixture_count_ok": true,
|
|
13
|
+
"avg_bare_headroom": -10.0,
|
|
14
|
+
"min_bare_headroom": -10,
|
|
15
|
+
"avg_solo_headroom": -12.0,
|
|
16
|
+
"min_solo_headroom": -12,
|
|
17
|
+
"rows": [
|
|
18
|
+
{
|
|
19
|
+
"fixture": "F4-web-browser-design",
|
|
20
|
+
"status": "FAIL",
|
|
21
|
+
"bare_score": 70,
|
|
22
|
+
"solo_score": 92,
|
|
23
|
+
"bare_headroom": -10,
|
|
24
|
+
"solo_headroom": -12,
|
|
25
|
+
"reason": "bare score 70 > 60; solo_claude score 92 > 80; bare judge disqualifier; bare result disqualifier; bare verify disqualifier"
|
|
26
|
+
}
|
|
27
|
+
]
|
|
28
|
+
}
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Headroom Gate — 20260512-f4-web-headroom
|
|
2
|
+
|
|
3
|
+
Verdict: **FAIL**
|
|
4
|
+
|
|
5
|
+
Fixtures passed: 0/1 (minimum required: 1)
|
|
6
|
+
|
|
7
|
+
Rule: at least 1 fixtures; bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, both baseline arms evidence-complete.
|
|
8
|
+
Average bare headroom: -10.0
|
|
9
|
+
Minimum bare headroom: -10
|
|
10
|
+
Average solo headroom: -12.0
|
|
11
|
+
Minimum solo headroom: -12
|
|
12
|
+
|
|
13
|
+
| Fixture | Bare | Bare headroom | Solo | Solo headroom | Status | Reason |
|
|
14
|
+
|---|---:|---:|---:|---:|---|---|
|
|
15
|
+
| F4-web-browser-design | 70 | -10 | 92 | -12 | FAIL | bare score 70 > 60; solo_claude score 92 > 80; bare judge disqualifier; bare result disqualifier; bare verify disqualifier |
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
{
|
|
2
|
+
"run_id": "20260512-f5-fixloop-headroom",
|
|
3
|
+
"rule": "at least 1 candidate fixtures; each must satisfy bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, with both baseline arms evidence-complete",
|
|
4
|
+
"verdict": "FAIL",
|
|
5
|
+
"fixtures_total": 1,
|
|
6
|
+
"fixtures_passed": 0,
|
|
7
|
+
"min_fixtures": 1,
|
|
8
|
+
"bare_max": 60,
|
|
9
|
+
"solo_max": 80,
|
|
10
|
+
"min_bare_headroom_required": 5,
|
|
11
|
+
"min_solo_headroom_required": 5,
|
|
12
|
+
"fixture_count_ok": true,
|
|
13
|
+
"avg_bare_headroom": -39.0,
|
|
14
|
+
"min_bare_headroom": -39,
|
|
15
|
+
"avg_solo_headroom": -19.0,
|
|
16
|
+
"min_solo_headroom": -19,
|
|
17
|
+
"rows": [
|
|
18
|
+
{
|
|
19
|
+
"fixture": "F5-fix-loop-red-green",
|
|
20
|
+
"status": "FAIL",
|
|
21
|
+
"bare_score": 99,
|
|
22
|
+
"solo_score": 99,
|
|
23
|
+
"bare_headroom": -39,
|
|
24
|
+
"solo_headroom": -19,
|
|
25
|
+
"reason": "bare score 99 > 60; solo_claude score 99 > 80"
|
|
26
|
+
}
|
|
27
|
+
]
|
|
28
|
+
}
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Headroom Gate — 20260512-f5-fixloop-headroom
|
|
2
|
+
|
|
3
|
+
Verdict: **FAIL**
|
|
4
|
+
|
|
5
|
+
Fixtures passed: 0/1 (minimum required: 1)
|
|
6
|
+
|
|
7
|
+
Rule: at least 1 fixtures; bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, both baseline arms evidence-complete.
|
|
8
|
+
Average bare headroom: -39.0
|
|
9
|
+
Minimum bare headroom: -39
|
|
10
|
+
Average solo headroom: -19.0
|
|
11
|
+
Minimum solo headroom: -19
|
|
12
|
+
|
|
13
|
+
| Fixture | Bare | Bare headroom | Solo | Solo headroom | Status | Reason |
|
|
14
|
+
|---|---:|---:|---:|---:|---|---|
|
|
15
|
+
| F5-fix-loop-red-green | 99 | -39 | 99 | -19 | FAIL | bare score 99 > 60; solo_claude score 99 > 80 |
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
{
|
|
2
|
+
"run_id": "20260512-f6-checksum-headroom",
|
|
3
|
+
"rule": "at least 1 candidate fixtures; each must satisfy bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, with both baseline arms evidence-complete",
|
|
4
|
+
"verdict": "FAIL",
|
|
5
|
+
"fixtures_total": 1,
|
|
6
|
+
"fixtures_passed": 0,
|
|
7
|
+
"min_fixtures": 1,
|
|
8
|
+
"bare_max": 60,
|
|
9
|
+
"solo_max": 80,
|
|
10
|
+
"min_bare_headroom_required": 5,
|
|
11
|
+
"min_solo_headroom_required": 5,
|
|
12
|
+
"fixture_count_ok": true,
|
|
13
|
+
"avg_bare_headroom": -37.0,
|
|
14
|
+
"min_bare_headroom": -37,
|
|
15
|
+
"avg_solo_headroom": -16.0,
|
|
16
|
+
"min_solo_headroom": -16,
|
|
17
|
+
"rows": [
|
|
18
|
+
{
|
|
19
|
+
"fixture": "F6-dep-audit-native-module",
|
|
20
|
+
"status": "FAIL",
|
|
21
|
+
"bare_score": 97,
|
|
22
|
+
"solo_score": 96,
|
|
23
|
+
"bare_headroom": -37,
|
|
24
|
+
"solo_headroom": -16,
|
|
25
|
+
"reason": "bare score 97 > 60; solo_claude score 96 > 80"
|
|
26
|
+
}
|
|
27
|
+
]
|
|
28
|
+
}
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Headroom Gate — 20260512-f6-checksum-headroom
|
|
2
|
+
|
|
3
|
+
Verdict: **FAIL**
|
|
4
|
+
|
|
5
|
+
Fixtures passed: 0/1 (minimum required: 1)
|
|
6
|
+
|
|
7
|
+
Rule: at least 1 fixtures; bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, both baseline arms evidence-complete.
|
|
8
|
+
Average bare headroom: -37.0
|
|
9
|
+
Minimum bare headroom: -37
|
|
10
|
+
Average solo headroom: -16.0
|
|
11
|
+
Minimum solo headroom: -16
|
|
12
|
+
|
|
13
|
+
| Fixture | Bare | Bare headroom | Solo | Solo headroom | Status | Reason |
|
|
14
|
+
|---|---:|---:|---:|---:|---|---|
|
|
15
|
+
| F6-dep-audit-native-module | 97 | -37 | 96 | -16 | FAIL | bare score 97 > 60; solo_claude score 96 > 80 |
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
{
|
|
2
|
+
"run_id": "20260512-f7-scope-headroom",
|
|
3
|
+
"rule": "at least 1 candidate fixtures; each must satisfy bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, with both baseline arms evidence-complete",
|
|
4
|
+
"verdict": "FAIL",
|
|
5
|
+
"fixtures_total": 1,
|
|
6
|
+
"fixtures_passed": 0,
|
|
7
|
+
"min_fixtures": 1,
|
|
8
|
+
"bare_max": 60,
|
|
9
|
+
"solo_max": 80,
|
|
10
|
+
"min_bare_headroom_required": 5,
|
|
11
|
+
"min_solo_headroom_required": 5,
|
|
12
|
+
"fixture_count_ok": true,
|
|
13
|
+
"avg_bare_headroom": -39.0,
|
|
14
|
+
"min_bare_headroom": -39,
|
|
15
|
+
"avg_solo_headroom": -20.0,
|
|
16
|
+
"min_solo_headroom": -20,
|
|
17
|
+
"rows": [
|
|
18
|
+
{
|
|
19
|
+
"fixture": "F7-out-of-scope-trap",
|
|
20
|
+
"status": "FAIL",
|
|
21
|
+
"bare_score": 99,
|
|
22
|
+
"solo_score": 100,
|
|
23
|
+
"bare_headroom": -39,
|
|
24
|
+
"solo_headroom": -20,
|
|
25
|
+
"reason": "bare score 99 > 60; solo_claude score 100 > 80"
|
|
26
|
+
}
|
|
27
|
+
]
|
|
28
|
+
}
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Headroom Gate — 20260512-f7-scope-headroom
|
|
2
|
+
|
|
3
|
+
Verdict: **FAIL**
|
|
4
|
+
|
|
5
|
+
Fixtures passed: 0/1 (minimum required: 1)
|
|
6
|
+
|
|
7
|
+
Rule: at least 1 fixtures; bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, both baseline arms evidence-complete.
|
|
8
|
+
Average bare headroom: -39.0
|
|
9
|
+
Minimum bare headroom: -39
|
|
10
|
+
Average solo headroom: -20.0
|
|
11
|
+
Minimum solo headroom: -20
|
|
12
|
+
|
|
13
|
+
| Fixture | Bare | Bare headroom | Solo | Solo headroom | Status | Reason |
|
|
14
|
+
|---|---:|---:|---:|---:|---|---|
|
|
15
|
+
| F7-out-of-scope-trap | 99 | -39 | 100 | -20 | FAIL | bare score 99 > 60; solo_claude score 100 > 80 |
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
{
|
|
2
|
+
"run_id": "20260512-f9-e2e-headroom",
|
|
3
|
+
"rule": "at least 1 candidate fixtures; each must satisfy bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, with both baseline arms evidence-complete",
|
|
4
|
+
"verdict": "FAIL",
|
|
5
|
+
"fixtures_total": 1,
|
|
6
|
+
"fixtures_passed": 0,
|
|
7
|
+
"min_fixtures": 1,
|
|
8
|
+
"bare_max": 60,
|
|
9
|
+
"solo_max": 80,
|
|
10
|
+
"min_bare_headroom_required": 5,
|
|
11
|
+
"min_solo_headroom_required": 5,
|
|
12
|
+
"fixture_count_ok": true,
|
|
13
|
+
"avg_bare_headroom": 0.0,
|
|
14
|
+
"min_bare_headroom": 0,
|
|
15
|
+
"avg_solo_headroom": -10.0,
|
|
16
|
+
"min_solo_headroom": -10,
|
|
17
|
+
"rows": [
|
|
18
|
+
{
|
|
19
|
+
"fixture": "F9-e2e-ideate-to-resolve",
|
|
20
|
+
"status": "FAIL",
|
|
21
|
+
"bare_score": 60,
|
|
22
|
+
"solo_score": 90,
|
|
23
|
+
"bare_headroom": 0,
|
|
24
|
+
"solo_headroom": -10,
|
|
25
|
+
"reason": "bare headroom 0 < 5; solo_claude score 90 > 80; bare judge disqualifier"
|
|
26
|
+
}
|
|
27
|
+
]
|
|
28
|
+
}
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Headroom Gate — 20260512-f9-e2e-headroom
|
|
2
|
+
|
|
3
|
+
Verdict: **FAIL**
|
|
4
|
+
|
|
5
|
+
Fixtures passed: 0/1 (minimum required: 1)
|
|
6
|
+
|
|
7
|
+
Rule: at least 1 fixtures; bare <= 60 with headroom >= 5, solo_claude <= 80 with headroom >= 5, both baseline arms evidence-complete.
|
|
8
|
+
Average bare headroom: 0.0
|
|
9
|
+
Minimum bare headroom: 0
|
|
10
|
+
Average solo headroom: -10.0
|
|
11
|
+
Minimum solo headroom: -10
|
|
12
|
+
|
|
13
|
+
| Fixture | Bare | Bare headroom | Solo | Solo headroom | Status | Reason |
|
|
14
|
+
|---|---:|---:|---:|---:|---|---|
|
|
15
|
+
| F9-e2e-ideate-to-resolve | 60 | 0 | 90 | -10 | FAIL | bare headroom 0 < 5; solo_claude score 90 > 80; bare judge disqualifier |
|
package/bin/devlyn.js
CHANGED
|
@@ -865,6 +865,11 @@ function installClaudeCore() {
|
|
|
865
865
|
settings.env.CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS = '1';
|
|
866
866
|
settingsChanged = true;
|
|
867
867
|
}
|
|
868
|
+
const bashMaxTimeoutMs = Number.parseInt(settings.env.BASH_MAX_TIMEOUT_MS, 10);
|
|
869
|
+
if (!Number.isFinite(bashMaxTimeoutMs) || bashMaxTimeoutMs < 3600000) {
|
|
870
|
+
settings.env.BASH_MAX_TIMEOUT_MS = '3600000';
|
|
871
|
+
settingsChanged = true;
|
|
872
|
+
}
|
|
868
873
|
if (!hasOwnSetting('hooks')) {
|
|
869
874
|
settings.hooks = {};
|
|
870
875
|
settingsChanged = true;
|
|
@@ -893,7 +898,7 @@ function installClaudeCore() {
|
|
|
893
898
|
}
|
|
894
899
|
if (settingsChanged) {
|
|
895
900
|
fs.writeFileSync(settingsPath, JSON.stringify(settings, null, 2) + '\n');
|
|
896
|
-
log(' → settings.json (agent teams + pipeline permissions + Stop hook)', 'dim');
|
|
901
|
+
log(' → settings.json (agent teams + one-hour Bash max + pipeline permissions + Stop hook)', 'dim');
|
|
897
902
|
}
|
|
898
903
|
|
|
899
904
|
// Configure global Claude Code settings (~/.claude/settings.json)
|
|
@@ -24,7 +24,7 @@ two targeted probes; the invoking phase sets `--effort`, pair-JUDGE uses
|
|
|
24
24
|
```bash
|
|
25
25
|
python3 "$DEVLYN_SHARED_DIR/run-bounded.py" 600 -- claude -p "<judge prompt>" \
|
|
26
26
|
--permission-mode dontAsk \
|
|
27
|
-
--
|
|
27
|
+
--tools "Read,Grep,Glob" --allowedTools "Read,Grep,Glob" \
|
|
28
28
|
--setting-sources project --strict-mcp-config --mcp-config '{"mcpServers":{}}' \
|
|
29
29
|
--effort medium \
|
|
30
30
|
> .devlyn/claude-judge.stdout 2> .devlyn/claude-judge.stderr
|
|
@@ -85,3 +85,11 @@ You interpret instructions literally and explicitly. The official guide is expli
|
|
|
85
85
|
3. **Overengineering**: do NOT add files, abstractions, error handling, validation, or "future flexibility" beyond what the spec asks. A bug fix doesn't need surrounding cleanup. The right complexity is the minimum needed for the current task.
|
|
86
86
|
|
|
87
87
|
You do NOT need stronger imperatives ("CRITICAL!", "YOU MUST!") to follow rules. Normal phrasing is sufficient.
|
|
88
|
+
|
|
89
|
+
## Explicit role capability
|
|
90
|
+
|
|
91
|
+
`role-config.py` validates explicit CLI judge effort against this bounded native capability declaration. Source: Claude Code2.1.263 built-in Fable5.1 metadata (native SHA ef5d2909c8af49f31ab6d5487e90316777bc2fac170adfe8160716caa8aaf4f9, byte157907500; effort/max/xhigh gates159068038/159068443/159068839). This is option support, not model fitness. Unknown version/model effort support fails with actionable unsupported-role-option. Model-only requests pass the exact ID to the native CLI without this effort table and still require matching native result evidence; engine-only defaults remain available. Native Agent workers have no validated exact-model/effort transport here, so explicit worker fields are rejected instead of switching channels.
|
|
92
|
+
|
|
93
|
+
<!-- devlyn-effort 2.1.263 claude-fable-5-1 low,medium,high,xhigh,max -->
|
|
94
|
+
|
|
95
|
+
For an explicit CLI judge profile, use the resolved `--model`/`--effort` in the bounded read-only invocation and add `--output-format json`. Follow the canonical resolve `JUDGE_STEM` round-scoped argv/prompt/raw capture and `judge-role-evidence.py` derivation contract. Only its exact derived result goes to the findings collector; never feed the JSON wrapper directly. Omitted options keep that role’s existing defaults.
|
|
@@ -14,6 +14,11 @@ Your default is efficient, direct, task-oriented. The canonical body specifies t
|
|
|
14
14
|
|
|
15
15
|
Resolve the request in the fewest useful tool loops without sacrificing correctness. For retrieval tasks: start with one broad search using short discriminative keywords; make another retrieval call only when the top results don't answer the core question or a required fact / parameter / source is missing. For tool-heavy tasks, start with a brief preamble: a one-line acknowledgment of the request and the first step you'll take.
|
|
16
16
|
|
|
17
|
+
For VERIFY JUDGE retrieval, batch related reads. After sealed parity proves a
|
|
18
|
+
canonical file and its `.agents` mirror are byte-identical, inspect the
|
|
19
|
+
canonical file only. Open self-test bodies or MECHANICAL raw streams only for a
|
|
20
|
+
named clause that remains unresolved after the broad pass.
|
|
21
|
+
|
|
17
22
|
## Validation pattern
|
|
18
23
|
|
|
19
24
|
Validation is concrete commands and tools, not self-belief. When the canonical body lists verification commands, execute them and trust their output. Do not substitute your judgment for a deterministic check the harness has provided. When validation tools are available (test runners, lint, type-check, the harness's `spec-verify-check.py`), run them before declaring success. The minimum evidence sufficient to answer correctly, cited precisely — then stop.
|
|
@@ -44,7 +44,7 @@ python3 "$DEVLYN_SHARED_DIR/run-bounded.py" 600 -- grok -p "<judge prompt>" \
|
|
|
44
44
|
--disallowed-tools "Agent,use_tool,search_tool" \
|
|
45
45
|
--allow 'Bash(<the bare anchor>)' \
|
|
46
46
|
--reasoning-effort medium \
|
|
47
|
-
--output-format json \
|
|
47
|
+
--output-format streaming-messages-json \
|
|
48
48
|
> "$REPO/.devlyn/grok-judge.stdout" 2> "$REPO/.devlyn/grok-judge.stderr"
|
|
49
49
|
```
|
|
50
50
|
|
|
@@ -96,4 +96,6 @@ python3 "$DEVLYN_SHARED_DIR/collect-codex-findings.py" \
|
|
|
96
96
|
```
|
|
97
97
|
|
|
98
98
|
On exit 124, write `.devlyn/verify.pair.timeout.json` with
|
|
99
|
-
`{"engine": "grok", "budget_seconds": 600}` before merge.
|
|
99
|
+
`{"engine": "grok", "budget_seconds": 600}` before merge. A budget abort can
|
|
100
|
+
leave the stream truncated before its terminal `result` record; that capture
|
|
101
|
+
stays `TIMEOUT` at merge, never a pair PASS.
|