devlyn-cli 3.2.0 → 3.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +2 -2
- package/CLAUDE.md +2 -2
- package/README.md +22 -113
- package/bin/devlyn.js +13 -225
- package/bin/instruction-templates.json +752 -1
- package/bin/instructions.js +101 -17
- package/config/skills/_shared/adapters/grok.md +1 -2
- package/config/skills/_shared/archive_run.py +15 -3
- package/config/skills/_shared/judge-role-evidence.py +21 -1
- package/config/skills/_shared/runtime-principles.md +1 -1
- package/config/skills/_shared/spec-verify-check.py +0 -653
- package/config/skills/_shared/state-phase-write.py +371 -30
- package/config/skills/_shared/task-complete.py +635 -143
- package/config/skills/_shared/terminal-claim-check.py +229 -4
- package/config/skills/_shared/verify-merge-findings.py +93 -2
- package/config/skills/devlyn:ideate/SKILL.md +3 -4
- package/config/skills/devlyn:ideate/references/elicitation.md +2 -35
- package/config/skills/devlyn:ideate/references/from-spec-mode.md +3 -18
- package/config/skills/devlyn:ideate/references/project-mode.md +1 -16
- package/config/skills/devlyn:ideate/references/spec-template.md +0 -2
- package/config/skills/devlyn:queue/SKILL.md +6 -4
- package/config/skills/devlyn:resolve/SKILL.md +16 -25
- package/config/skills/devlyn:resolve/references/free-form-mode.md +0 -14
- package/config/skills/devlyn:resolve/references/outer-loop.md +1 -1
- package/config/skills/devlyn:resolve/references/phases/build-gate.md +1 -1
- package/config/skills/devlyn:resolve/references/phases/cleanup.md +3 -3
- package/config/skills/devlyn:resolve/references/phases/implement.md +1 -1
- package/config/skills/devlyn:resolve/references/phases/probe-derive.md +1 -16
- package/config/skills/devlyn:resolve/references/phases/verify.md +4 -9
- package/config/skills/devlyn:resolve/references/state-schema.md +9 -7
- package/config/skills/devlyn:resolve/references/task-completion.md +46 -37
- package/{config/skills → optional-skills}/code-health-standards/SKILL.md +4 -13
- package/{config/skills → optional-skills}/code-review-standards/SKILL.md +4 -12
- package/{config/skills → optional-skills}/root-cause-analysis/SKILL.md +4 -12
- package/{config/skills → optional-skills}/ui-implementation-standards/SKILL.md +4 -14
- package/package.json +1 -57
- package/benchmark/auto-resolve/BENCHMARK-DESIGN.md +0 -289
- package/benchmark/auto-resolve/BENCHMARK-RESULTS.md +0 -341
- package/benchmark/auto-resolve/README.md +0 -696
- package/benchmark/auto-resolve/RUBRIC.md +0 -177
- package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/NOTES.md +0 -34
- package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/expected.json +0 -68
- package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/setup.sh +0 -4
- package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/spec.md +0 -44
- package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/task.ko.txt +0 -8
- package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/task.txt +0 -8
- package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/NOTES.md +0 -68
- package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/expected.json +0 -60
- package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/setup.sh +0 -17
- package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/spec.md +0 -51
- package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/verifiers/invalid.js +0 -29
- package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/verifiers/parallel.js +0 -50
- package/benchmark/auto-resolve/fixtures/F11-batch-import-all-or-nothing/NOTES.md +0 -75
- package/benchmark/auto-resolve/fixtures/F11-batch-import-all-or-nothing/expected.json +0 -52
- package/benchmark/auto-resolve/fixtures/F11-batch-import-all-or-nothing/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F11-batch-import-all-or-nothing/setup.sh +0 -171
- package/benchmark/auto-resolve/fixtures/F11-batch-import-all-or-nothing/spec.md +0 -50
- package/benchmark/auto-resolve/fixtures/F11-batch-import-all-or-nothing/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/F12-webhook-raw-body-signature/NOTES.md +0 -89
- package/benchmark/auto-resolve/fixtures/F12-webhook-raw-body-signature/expected.json +0 -74
- package/benchmark/auto-resolve/fixtures/F12-webhook-raw-body-signature/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F12-webhook-raw-body-signature/setup.sh +0 -251
- package/benchmark/auto-resolve/fixtures/F12-webhook-raw-body-signature/spec.md +0 -57
- package/benchmark/auto-resolve/fixtures/F12-webhook-raw-body-signature/task.txt +0 -13
- package/benchmark/auto-resolve/fixtures/F12-webhook-raw-body-signature/verifiers/replay-malformed-body.js +0 -64
- package/benchmark/auto-resolve/fixtures/F15-frozen-diff-race-review/NOTES.md +0 -104
- package/benchmark/auto-resolve/fixtures/F15-frozen-diff-race-review/expected.json +0 -46
- package/benchmark/auto-resolve/fixtures/F15-frozen-diff-race-review/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F15-frozen-diff-race-review/setup.sh +0 -336
- package/benchmark/auto-resolve/fixtures/F15-frozen-diff-race-review/spec.md +0 -51
- package/benchmark/auto-resolve/fixtures/F15-frozen-diff-race-review/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/NOTES.md +0 -38
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/expected.json +0 -64
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/setup.sh +0 -32
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/spec.md +0 -63
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/verifiers/exact-success.js +0 -54
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/verifiers/no-hardcoded-pricing.js +0 -47
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/verifiers/stock-error.js +0 -45
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/NOTES.md +0 -57
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/expected-pair-plan-registry.json +0 -170
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/expected.json +0 -84
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/metadata.json +0 -21
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/pair-plan.sample-fail.json +0 -214
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/pair-plan.sample-pass.json +0 -223
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/setup.sh +0 -5
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/spec.md +0 -55
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/task.txt +0 -14
- package/benchmark/auto-resolve/fixtures/F21-cli-scheduler-priority/NOTES.md +0 -39
- package/benchmark/auto-resolve/fixtures/F21-cli-scheduler-priority/expected.json +0 -62
- package/benchmark/auto-resolve/fixtures/F21-cli-scheduler-priority/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F21-cli-scheduler-priority/setup.sh +0 -2
- package/benchmark/auto-resolve/fixtures/F21-cli-scheduler-priority/spec.md +0 -67
- package/benchmark/auto-resolve/fixtures/F21-cli-scheduler-priority/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F21-cli-scheduler-priority/verifiers/error-order.js +0 -55
- package/benchmark/auto-resolve/fixtures/F21-cli-scheduler-priority/verifiers/priority-blocked.js +0 -48
- package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/NOTES.md +0 -35
- package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/expected.json +0 -56
- package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/setup.sh +0 -2
- package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/spec.md +0 -64
- package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/verifiers/conflicting-duplicate.js +0 -34
- package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/verifiers/idempotent-close.js +0 -41
- package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/NOTES.md +0 -39
- package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/expected.json +0 -56
- package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/setup.sh +0 -2
- package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/spec.md +0 -76
- package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/verifiers/priority-rollback.js +0 -64
- package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/verifiers/single-warehouse-fefo.js +0 -66
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/NOTES.md +0 -40
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/expected.json +0 -66
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/setup.sh +0 -36
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/spec.md +0 -71
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/verifiers/catalog-source.js +0 -57
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/verifiers/exact-success.js +0 -63
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/verifiers/stock-error.js +0 -34
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/NOTES.md +0 -31
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/expected.json +0 -68
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/setup.sh +0 -17
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/spec.md +0 -68
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/verifiers/conflicting-duplicate.js +0 -29
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/verifiers/exact-payout.js +0 -58
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/verifiers/rules-source.js +0 -56
- package/benchmark/auto-resolve/fixtures/F3-backend-contract-risk/NOTES.md +0 -35
- package/benchmark/auto-resolve/fixtures/F3-backend-contract-risk/expected-pair-plan-registry.json +0 -162
- package/benchmark/auto-resolve/fixtures/F3-backend-contract-risk/expected.json +0 -67
- package/benchmark/auto-resolve/fixtures/F3-backend-contract-risk/metadata.json +0 -19
- package/benchmark/auto-resolve/fixtures/F3-backend-contract-risk/setup.sh +0 -4
- package/benchmark/auto-resolve/fixtures/F3-backend-contract-risk/spec.md +0 -55
- package/benchmark/auto-resolve/fixtures/F3-backend-contract-risk/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/F31-cli-seat-rebalance/NOTES.md +0 -34
- package/benchmark/auto-resolve/fixtures/F31-cli-seat-rebalance/expected.json +0 -57
- package/benchmark/auto-resolve/fixtures/F31-cli-seat-rebalance/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F31-cli-seat-rebalance/setup.sh +0 -2
- package/benchmark/auto-resolve/fixtures/F31-cli-seat-rebalance/spec.md +0 -67
- package/benchmark/auto-resolve/fixtures/F31-cli-seat-rebalance/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F31-cli-seat-rebalance/verifiers/duplicate-event-error.js +0 -35
- package/benchmark/auto-resolve/fixtures/F31-cli-seat-rebalance/verifiers/priority-transfer-rollback.js +0 -53
- package/benchmark/auto-resolve/fixtures/F32-cli-subscription-renewal/NOTES.md +0 -38
- package/benchmark/auto-resolve/fixtures/F32-cli-subscription-renewal/expected.json +0 -57
- package/benchmark/auto-resolve/fixtures/F32-cli-subscription-renewal/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F32-cli-subscription-renewal/setup.sh +0 -2
- package/benchmark/auto-resolve/fixtures/F32-cli-subscription-renewal/spec.md +0 -70
- package/benchmark/auto-resolve/fixtures/F32-cli-subscription-renewal/task.txt +0 -3
- package/benchmark/auto-resolve/fixtures/F32-cli-subscription-renewal/verifiers/duplicate-renewal-error.js +0 -42
- package/benchmark/auto-resolve/fixtures/F32-cli-subscription-renewal/verifiers/priority-credit-rollback.js +0 -70
- package/benchmark/auto-resolve/fixtures/F4-web-browser-design/NOTES.md +0 -47
- package/benchmark/auto-resolve/fixtures/F4-web-browser-design/expected.json +0 -57
- package/benchmark/auto-resolve/fixtures/F4-web-browser-design/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F4-web-browser-design/setup.sh +0 -6
- package/benchmark/auto-resolve/fixtures/F4-web-browser-design/spec.md +0 -48
- package/benchmark/auto-resolve/fixtures/F4-web-browser-design/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/F5-fix-loop-red-green/NOTES.md +0 -45
- package/benchmark/auto-resolve/fixtures/F5-fix-loop-red-green/expected.json +0 -65
- package/benchmark/auto-resolve/fixtures/F5-fix-loop-red-green/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F5-fix-loop-red-green/setup.sh +0 -55
- package/benchmark/auto-resolve/fixtures/F5-fix-loop-red-green/spec.md +0 -48
- package/benchmark/auto-resolve/fixtures/F5-fix-loop-red-green/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F6-dep-audit-native-module/NOTES.md +0 -43
- package/benchmark/auto-resolve/fixtures/F6-dep-audit-native-module/expected.json +0 -77
- package/benchmark/auto-resolve/fixtures/F6-dep-audit-native-module/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F6-dep-audit-native-module/setup.sh +0 -4
- package/benchmark/auto-resolve/fixtures/F6-dep-audit-native-module/spec.md +0 -48
- package/benchmark/auto-resolve/fixtures/F6-dep-audit-native-module/task.txt +0 -10
- package/benchmark/auto-resolve/fixtures/F7-out-of-scope-trap/NOTES.md +0 -57
- package/benchmark/auto-resolve/fixtures/F7-out-of-scope-trap/expected.json +0 -76
- package/benchmark/auto-resolve/fixtures/F7-out-of-scope-trap/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F7-out-of-scope-trap/setup.sh +0 -36
- package/benchmark/auto-resolve/fixtures/F7-out-of-scope-trap/spec.md +0 -45
- package/benchmark/auto-resolve/fixtures/F7-out-of-scope-trap/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F8-known-limit-ambiguous/NOTES.md +0 -53
- package/benchmark/auto-resolve/fixtures/F8-known-limit-ambiguous/expected.json +0 -63
- package/benchmark/auto-resolve/fixtures/F8-known-limit-ambiguous/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F8-known-limit-ambiguous/setup.sh +0 -4
- package/benchmark/auto-resolve/fixtures/F8-known-limit-ambiguous/spec.md +0 -48
- package/benchmark/auto-resolve/fixtures/F8-known-limit-ambiguous/task.txt +0 -1
- package/benchmark/auto-resolve/fixtures/F9-e2e-ideate-to-resolve/NOTES.md +0 -105
- package/benchmark/auto-resolve/fixtures/F9-e2e-ideate-to-resolve/expected.json +0 -74
- package/benchmark/auto-resolve/fixtures/F9-e2e-ideate-to-resolve/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F9-e2e-ideate-to-resolve/setup.sh +0 -28
- package/benchmark/auto-resolve/fixtures/F9-e2e-ideate-to-resolve/spec.md +0 -59
- package/benchmark/auto-resolve/fixtures/F9-e2e-ideate-to-resolve/task.txt +0 -5
- package/benchmark/auto-resolve/fixtures/SCHEMA.md +0 -188
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/NOTES.md +0 -37
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/RETIRED.md +0 -13
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/expected.json +0 -56
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/setup.sh +0 -18
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/spec.md +0 -69
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/verifiers/exact-proration.js +0 -48
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/verifiers/rules-source-and-conflict.js +0 -79
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/NOTES.md +0 -54
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/RETIRED.md +0 -7
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/expected.json +0 -67
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/setup.sh +0 -2
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/spec.md +0 -67
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/task.txt +0 -5
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/verifiers/policy-precedence.js +0 -72
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/verifiers/validation-and-immutability.js +0 -43
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/verifiers/validation-boundary.js +0 -116
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/NOTES.md +0 -35
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/RETIRED.md +0 -12
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/expected.json +0 -58
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/setup.sh +0 -2
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/spec.md +0 -73
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/task.txt +0 -17
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/verifiers/mixed-idempotent-settlement.js +0 -53
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/verifiers/rejection-boundaries.js +0 -74
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/NOTES.md +0 -28
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/RETIRED.md +0 -15
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/expected.json +0 -57
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/setup.sh +0 -50
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/spec.md +0 -59
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/verifiers/ordered-settlement.js +0 -88
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/verifiers/reject-invalid.js +0 -45
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/NOTES.md +0 -30
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/RETIRED.md +0 -16
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/expected.json +0 -64
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/setup.sh +0 -21
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/spec.md +0 -58
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/verifiers/dup-op.js +0 -54
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/verifiers/priority-apply.js +0 -82
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/verifiers/rollback.js +0 -77
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/NOTES.md +0 -99
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/RETIRED.md +0 -18
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/expected.json +0 -64
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/setup.sh +0 -4
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/spec.md +0 -61
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/verifiers/correctness-small.js +0 -135
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/verifiers/scale.js +0 -100
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/verifiers/validation.js +0 -75
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/NOTES.md +0 -93
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/RETIRED.md +0 -16
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/expected.json +0 -65
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/setup.sh +0 -18
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/spec.md +0 -66
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/verifiers/correctness-small.js +0 -84
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/verifiers/scale.js +0 -184
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/verifiers/validation.js +0 -74
- package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/NOTES.md +0 -60
- package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/RETIRED.md +0 -29
- package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/expected.json +0 -73
- package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/setup.sh +0 -28
- package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/spec.md +0 -58
- package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/task.txt +0 -5
- package/benchmark/auto-resolve/fixtures/test-repo/README.md +0 -12
- package/benchmark/auto-resolve/fixtures/test-repo/bin/cli.js +0 -63
- package/benchmark/auto-resolve/fixtures/test-repo/package-lock.json +0 -823
- package/benchmark/auto-resolve/fixtures/test-repo/package.json +0 -22
- package/benchmark/auto-resolve/fixtures/test-repo/playwright.config.js +0 -17
- package/benchmark/auto-resolve/fixtures/test-repo/server/index.js +0 -37
- package/benchmark/auto-resolve/fixtures/test-repo/tests/cli.test.js +0 -25
- package/benchmark/auto-resolve/fixtures/test-repo/tests/server.test.js +0 -58
- package/benchmark/auto-resolve/fixtures/test-repo/web/index.html +0 -37
- package/benchmark/auto-resolve/results/20260507-f10-f11-tier1-full-pipeline/headroom-gate.json +0 -25
- package/benchmark/auto-resolve/results/20260507-f10-f11-tier1-full-pipeline/headroom-gate.md +0 -10
- package/benchmark/auto-resolve/results/20260508-f22-exact-error-headroom/headroom-gate.json +0 -18
- package/benchmark/auto-resolve/results/20260508-f22-exact-error-headroom/headroom-gate.md +0 -9
- package/benchmark/auto-resolve/results/20260508-f26-headroom/headroom-gate.json +0 -18
- package/benchmark/auto-resolve/results/20260508-f26-headroom/headroom-gate.md +0 -9
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/F16-cli-quote-tax-rules/l2_risk_probes/result.json +0 -28
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/F23-cli-fulfillment-wave/l2_risk_probes/result.json +0 -28
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/F25-cli-cart-promotion-rules/l2_risk_probes/result.json +0 -30
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/full-pipeline-pair-gate.json +0 -82
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/full-pipeline-pair-gate.md +0 -18
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/headroom-gate.json +0 -46
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/headroom-gate.md +0 -17
- package/benchmark/auto-resolve/results/20260511-f12-webhook-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260511-f12-webhook-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260511-f15-concurrency-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260511-f15-concurrency-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/F21-cli-scheduler-priority/l2_risk_probes/result.json +0 -32
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/full-pipeline-pair-gate.json +0 -44
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/full-pipeline-pair-gate.md +0 -16
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/headroom-gate.json +0 -18
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/headroom-gate.md +0 -9
- package/benchmark/auto-resolve/results/20260511-f3-http-error-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260511-f3-http-error-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260512-f2-medium-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260512-f2-medium-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260512-f31-seat-rebalance-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260512-f31-seat-rebalance-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260512-f32-subscription-renewal-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260512-f32-subscription-renewal-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260512-f4-web-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260512-f4-web-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260512-f5-fixloop-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260512-f5-fixloop-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260512-f6-checksum-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260512-f6-checksum-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260512-f7-scope-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260512-f7-scope-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260512-f9-e2e-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260512-f9-e2e-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/run-real-benchmark.md +0 -303
- package/benchmark/auto-resolve/scripts/audit-headroom-rejections.py +0 -441
- package/benchmark/auto-resolve/scripts/audit-pair-evidence.py +0 -1256
- package/benchmark/auto-resolve/scripts/build-pair-eligible-manifest.py +0 -306
- package/benchmark/auto-resolve/scripts/check-f9-artifacts.py +0 -268
- package/benchmark/auto-resolve/scripts/collect-swebench-predictions.py +0 -108
- package/benchmark/auto-resolve/scripts/compile-report.py +0 -493
- package/benchmark/auto-resolve/scripts/fetch-swebench-instances.py +0 -129
- package/benchmark/auto-resolve/scripts/frozen-verify-gate.py +0 -434
- package/benchmark/auto-resolve/scripts/full-pipeline-pair-gate.py +0 -612
- package/benchmark/auto-resolve/scripts/headroom-gate.py +0 -378
- package/benchmark/auto-resolve/scripts/iter-0033c-compare.py +0 -683
- package/benchmark/auto-resolve/scripts/iter-0033c-l1-summary.py +0 -97
- package/benchmark/auto-resolve/scripts/judge-opus-pass.sh +0 -542
- package/benchmark/auto-resolve/scripts/judge.sh +0 -565
- package/benchmark/auto-resolve/scripts/oracle-scope-tier-a.py +0 -256
- package/benchmark/auto-resolve/scripts/oracle-scope-tier-b.py +0 -287
- package/benchmark/auto-resolve/scripts/oracle-test-fidelity.py +0 -328
- package/benchmark/auto-resolve/scripts/pair-candidate-frontier.py +0 -469
- package/benchmark/auto-resolve/scripts/pair-plan-idgen.py +0 -401
- package/benchmark/auto-resolve/scripts/pair-plan-lint.py +0 -475
- package/benchmark/auto-resolve/scripts/pair-rejected-fixtures.sh +0 -103
- package/benchmark/auto-resolve/scripts/pair_evidence_contract.py +0 -270
- package/benchmark/auto-resolve/scripts/prepare-swebench-frozen-case.py +0 -273
- package/benchmark/auto-resolve/scripts/prepare-swebench-frozen-corpus.py +0 -148
- package/benchmark/auto-resolve/scripts/prepare-swebench-solver-worktree.py +0 -210
- package/benchmark/auto-resolve/scripts/recent-benchmark-summary.py +0 -232
- package/benchmark/auto-resolve/scripts/run-fixture.sh +0 -1024
- package/benchmark/auto-resolve/scripts/run-frozen-verify-pair.sh +0 -683
- package/benchmark/auto-resolve/scripts/run-full-pipeline-pair-candidate.sh +0 -458
- package/benchmark/auto-resolve/scripts/run-headroom-candidate.sh +0 -336
- package/benchmark/auto-resolve/scripts/run-iter-0033c.sh +0 -208
- package/benchmark/auto-resolve/scripts/run-suite.sh +0 -251
- package/benchmark/auto-resolve/scripts/run-swebench-frozen-corpus.sh +0 -310
- package/benchmark/auto-resolve/scripts/run-swebench-solver-batch.sh +0 -257
- package/benchmark/auto-resolve/scripts/ship-gate.py +0 -391
- package/benchmark/auto-resolve/scripts/solo-ceiling-avoidance.py +0 -53
- package/benchmark/auto-resolve/scripts/solo-headroom-hypothesis.py +0 -77
- package/benchmark/auto-resolve/scripts/swebench-frozen-matrix.py +0 -478
- package/benchmark/auto-resolve/scripts/test-audit-headroom-rejections.sh +0 -288
- package/benchmark/auto-resolve/scripts/test-audit-pair-evidence.sh +0 -1672
- package/benchmark/auto-resolve/scripts/test-benchmark-arg-parsing.sh +0 -1015
- package/benchmark/auto-resolve/scripts/test-build-pair-eligible-manifest.sh +0 -491
- package/benchmark/auto-resolve/scripts/test-check-f9-artifacts.sh +0 -91
- package/benchmark/auto-resolve/scripts/test-frozen-verify-gate.sh +0 -517
- package/benchmark/auto-resolve/scripts/test-full-pipeline-pair-gate.sh +0 -610
- package/benchmark/auto-resolve/scripts/test-headroom-gate.sh +0 -401
- package/benchmark/auto-resolve/scripts/test-iter-0033c-compare.sh +0 -525
- package/benchmark/auto-resolve/scripts/test-iter-0033c-l1-summary.sh +0 -254
- package/benchmark/auto-resolve/scripts/test-lint-fixtures.sh +0 -580
- package/benchmark/auto-resolve/scripts/test-pair-candidate-frontier.sh +0 -591
- package/benchmark/auto-resolve/scripts/test-run-full-pipeline-pair-candidate.sh +0 -497
- package/benchmark/auto-resolve/scripts/test-run-headroom-candidate.sh +0 -401
- package/benchmark/auto-resolve/scripts/test-run-swebench-solver-batch.sh +0 -111
- package/benchmark/auto-resolve/scripts/test-ship-gate.sh +0 -1189
- package/benchmark/auto-resolve/scripts/test-swebench-frozen-case.sh +0 -1221
- package/benchmark/auto-resolve/shadow-fixtures/S1-cli-lang-flag/NOTES.md +0 -28
- package/benchmark/auto-resolve/shadow-fixtures/S1-cli-lang-flag/expected.json +0 -63
- package/benchmark/auto-resolve/shadow-fixtures/S1-cli-lang-flag/metadata.json +0 -10
- package/benchmark/auto-resolve/shadow-fixtures/S1-cli-lang-flag/setup.sh +0 -3
- package/benchmark/auto-resolve/shadow-fixtures/S1-cli-lang-flag/spec.md +0 -47
- package/benchmark/auto-resolve/shadow-fixtures/S1-cli-lang-flag/task.txt +0 -1
- package/benchmark/auto-resolve/shadow-fixtures/S2-cli-inventory-reservation/NOTES.md +0 -34
- package/benchmark/auto-resolve/shadow-fixtures/S2-cli-inventory-reservation/expected.json +0 -53
- package/benchmark/auto-resolve/shadow-fixtures/S2-cli-inventory-reservation/metadata.json +0 -10
- package/benchmark/auto-resolve/shadow-fixtures/S2-cli-inventory-reservation/setup.sh +0 -3
- package/benchmark/auto-resolve/shadow-fixtures/S2-cli-inventory-reservation/spec.md +0 -50
- package/benchmark/auto-resolve/shadow-fixtures/S2-cli-inventory-reservation/task.txt +0 -1
- package/benchmark/auto-resolve/shadow-fixtures/S2-cli-inventory-reservation/verifiers/duplicate-order-error.js +0 -27
- package/benchmark/auto-resolve/shadow-fixtures/S2-cli-inventory-reservation/verifiers/priority-stock-reservation.js +0 -44
- package/benchmark/auto-resolve/shadow-fixtures/S3-cli-ticket-assignment/NOTES.md +0 -34
- package/benchmark/auto-resolve/shadow-fixtures/S3-cli-ticket-assignment/expected.json +0 -55
- package/benchmark/auto-resolve/shadow-fixtures/S3-cli-ticket-assignment/metadata.json +0 -10
- package/benchmark/auto-resolve/shadow-fixtures/S3-cli-ticket-assignment/setup.sh +0 -3
- package/benchmark/auto-resolve/shadow-fixtures/S3-cli-ticket-assignment/spec.md +0 -52
- package/benchmark/auto-resolve/shadow-fixtures/S3-cli-ticket-assignment/task.txt +0 -1
- package/benchmark/auto-resolve/shadow-fixtures/S3-cli-ticket-assignment/verifiers/duplicate-ticket-error.js +0 -29
- package/benchmark/auto-resolve/shadow-fixtures/S3-cli-ticket-assignment/verifiers/priority-agent-assignment.js +0 -48
- package/benchmark/auto-resolve/shadow-fixtures/S4-cli-return-routing/NOTES.md +0 -34
- package/benchmark/auto-resolve/shadow-fixtures/S4-cli-return-routing/expected.json +0 -55
- package/benchmark/auto-resolve/shadow-fixtures/S4-cli-return-routing/metadata.json +0 -10
- package/benchmark/auto-resolve/shadow-fixtures/S4-cli-return-routing/setup.sh +0 -3
- package/benchmark/auto-resolve/shadow-fixtures/S4-cli-return-routing/spec.md +0 -55
- package/benchmark/auto-resolve/shadow-fixtures/S4-cli-return-routing/task.txt +0 -1
- package/benchmark/auto-resolve/shadow-fixtures/S4-cli-return-routing/verifiers/duplicate-return-error.js +0 -43
- package/benchmark/auto-resolve/shadow-fixtures/S4-cli-return-routing/verifiers/priority-return-routing.js +0 -70
- package/benchmark/auto-resolve/shadow-fixtures/S5-cli-credit-grant-ledger/NOTES.md +0 -37
- package/benchmark/auto-resolve/shadow-fixtures/S5-cli-credit-grant-ledger/expected.json +0 -54
- package/benchmark/auto-resolve/shadow-fixtures/S5-cli-credit-grant-ledger/metadata.json +0 -10
- package/benchmark/auto-resolve/shadow-fixtures/S5-cli-credit-grant-ledger/setup.sh +0 -3
- package/benchmark/auto-resolve/shadow-fixtures/S5-cli-credit-grant-ledger/spec.md +0 -59
- package/benchmark/auto-resolve/shadow-fixtures/S5-cli-credit-grant-ledger/task.txt +0 -1
- package/benchmark/auto-resolve/shadow-fixtures/S5-cli-credit-grant-ledger/verifiers/credit-ledger-priority.js +0 -98
- package/benchmark/auto-resolve/shadow-fixtures/S5-cli-credit-grant-ledger/verifiers/duplicate-charge-error.js +0 -38
- package/benchmark/auto-resolve/shadow-fixtures/S6-cli-refund-window-ledger/NOTES.md +0 -36
- package/benchmark/auto-resolve/shadow-fixtures/S6-cli-refund-window-ledger/expected.json +0 -56
- package/benchmark/auto-resolve/shadow-fixtures/S6-cli-refund-window-ledger/metadata.json +0 -10
- package/benchmark/auto-resolve/shadow-fixtures/S6-cli-refund-window-ledger/setup.sh +0 -3
- package/benchmark/auto-resolve/shadow-fixtures/S6-cli-refund-window-ledger/spec.md +0 -59
- package/benchmark/auto-resolve/shadow-fixtures/S6-cli-refund-window-ledger/task.txt +0 -1
- package/benchmark/auto-resolve/shadow-fixtures/S6-cli-refund-window-ledger/verifiers/duplicate-refund-error.js +0 -41
- package/benchmark/auto-resolve/shadow-fixtures/S6-cli-refund-window-ledger/verifiers/priority-refund-ledger.js +0 -65
- package/config/skills/_shared/pair-plan-schema.md +0 -298
- package/scripts/lint-fixtures.sh +0 -352
- package/scripts/lint-shadow-fixtures.sh +0 -58
- package/scripts/lint-skills.sh +0 -4265
- package/scripts/test-owner-phases.py +0 -193
|
@@ -9,6 +9,7 @@ import json
|
|
|
9
9
|
import os
|
|
10
10
|
import pathlib
|
|
11
11
|
import re
|
|
12
|
+
import runpy
|
|
12
13
|
import sys
|
|
13
14
|
import tempfile
|
|
14
15
|
from dataclasses import dataclass
|
|
@@ -34,8 +35,6 @@ HALT_WITNESS_PHASES = {
|
|
|
34
35
|
"plan-empty": "plan",
|
|
35
36
|
"risk-halt": "plan",
|
|
36
37
|
"implement-empty": "implement",
|
|
37
|
-
"build-gate-exhausted": "build_gate",
|
|
38
|
-
"verify-exhausted": "verify",
|
|
39
38
|
}
|
|
40
39
|
|
|
41
40
|
|
|
@@ -130,6 +129,30 @@ def terminal_halt_witness(phases: dict[str, object]) -> tuple[str, str] | None:
|
|
|
130
129
|
return target, reason
|
|
131
130
|
|
|
132
131
|
|
|
132
|
+
def exhausted_origin(state: dict[str, object]) -> str | None:
|
|
133
|
+
writer = runpy.run_path(str(pathlib.Path(__file__).with_name("state-phase-write.py")))
|
|
134
|
+
predecessor = writer["repair_predecessor"](state)
|
|
135
|
+
if predecessor is None:
|
|
136
|
+
return None
|
|
137
|
+
name, entry = predecessor
|
|
138
|
+
if entry.get("completed_at") is None:
|
|
139
|
+
return None
|
|
140
|
+
if name == "verify":
|
|
141
|
+
merged = entry.get("merged")
|
|
142
|
+
return "verify" if entry.get("verdict") == "NEEDS_WORK" and isinstance(merged, dict) and merged.get("verdict") == "NEEDS_WORK" else None
|
|
143
|
+
if name == "cleanup":
|
|
144
|
+
return name if entry.get("verdict") == "FAIL" and entry.get("execution_kind") == "orchestrator_commands" else None
|
|
145
|
+
if name == "build_gate":
|
|
146
|
+
return name if entry.get("verdict") == "FAIL" else None
|
|
147
|
+
progress = entry.get("exec")
|
|
148
|
+
if not writer["valid_phase_gate_progress"](progress):
|
|
149
|
+
return None
|
|
150
|
+
if (entry.get("verdict") == "FAIL"
|
|
151
|
+
and progress["statuses"][progress["current"] - 1] == "FAIL"):
|
|
152
|
+
return "phase_gate"
|
|
153
|
+
return None
|
|
154
|
+
|
|
155
|
+
|
|
133
156
|
def classify_state_bytes(
|
|
134
157
|
root: pathlib.Path,
|
|
135
158
|
state_path: pathlib.Path,
|
|
@@ -206,6 +229,39 @@ def classify_state_bytes(
|
|
|
206
229
|
archive_valid = (
|
|
207
230
|
archived and state_path.parent.name == run_id
|
|
208
231
|
) or archive_state.is_file()
|
|
232
|
+
report_verdict = final_report.get("verdict") if isinstance(final_report, dict) else None
|
|
233
|
+
rounds = state.get("rounds")
|
|
234
|
+
schema_v3 = state.get("version") == "3.0"
|
|
235
|
+
has_v3_repair_round = schema_v3 and any(
|
|
236
|
+
isinstance(phases.get(name), dict) and "round" in phases[name]
|
|
237
|
+
for name in ("implement", "build_gate", "cleanup", "verify")
|
|
238
|
+
)
|
|
239
|
+
try:
|
|
240
|
+
origin = exhausted_origin(state) if schema_v3 and (rounds is not None or has_v3_repair_round) else None
|
|
241
|
+
except SystemExit:
|
|
242
|
+
return malformed("repair predecessor has invalid round", run_id), state
|
|
243
|
+
exhausted = (isinstance(rounds, dict) and type(rounds.get("global")) is int
|
|
244
|
+
and type(rounds.get("max_rounds")) is int and rounds["global"] == rounds["max_rounds"]
|
|
245
|
+
and rounds["max_rounds"] >= 1)
|
|
246
|
+
claims_exhaustion = schema_v3 and (report_verdict == "BLOCKED:repair-budget-exhausted" or (
|
|
247
|
+
report_verdict == "NEEDS_WORK" and state.get("mode") != "verify-only"
|
|
248
|
+
and (rounds is not None or origin is not None)
|
|
249
|
+
))
|
|
250
|
+
if claims_exhaustion:
|
|
251
|
+
expected = "NEEDS_WORK" if origin == "verify" else "BLOCKED:repair-budget-exhausted"
|
|
252
|
+
blocked_source = any(
|
|
253
|
+
isinstance(entry, dict) and entry.get("verdict") == "BLOCKED"
|
|
254
|
+
for name, entry in phases.items() if name != "final_report"
|
|
255
|
+
)
|
|
256
|
+
if not final_completed or not exhausted or origin is None or report_verdict != expected or blocked_source:
|
|
257
|
+
return incomplete("final_report", "repair exhaustion witness invalid", run_id), state
|
|
258
|
+
if not archive_valid:
|
|
259
|
+
return incomplete("archive", "repair exhaustion report not archived", run_id), state
|
|
260
|
+
return Classification("CLEAN", None, f"repair budget exhausted at {origin}", run_id, False), state
|
|
261
|
+
if (origin is not None and verdict is not None
|
|
262
|
+
and not (state.get("mode") == "verify-only" and origin == "verify" and report_verdict == "NEEDS_WORK")
|
|
263
|
+
and not (isinstance(report_verdict, str) and report_verdict.startswith("BLOCKED:"))):
|
|
264
|
+
return incomplete("final_report", "current repair exhaustion cannot use stale VERIFY", run_id), state
|
|
209
265
|
if verdict is not None:
|
|
210
266
|
if not final_completed:
|
|
211
267
|
return incomplete(
|
|
@@ -396,6 +452,177 @@ def self_test() -> int:
|
|
|
396
452
|
},
|
|
397
453
|
}
|
|
398
454
|
|
|
455
|
+
def exhausted_case(origin: str) -> dict[str, object]:
|
|
456
|
+
span = {"started_at": "2026-07-20T00:00:00Z", "completed_at": "2026-07-20T00:01:00Z"}
|
|
457
|
+
phases = {"verify": {**span, "round": 0, "verdict": "PASS"}}
|
|
458
|
+
if origin == "phase_gate":
|
|
459
|
+
phases["implement"] = {**span, "round": 1, "verdict": "FAIL",
|
|
460
|
+
"exec": {"total": 2, "current": 2, "statuses": ["PASS", "FAIL"]}}
|
|
461
|
+
elif origin == "verify":
|
|
462
|
+
phases["verify"] = {**span, "round": 1, "verdict": "NEEDS_WORK",
|
|
463
|
+
"merged": {"verdict": "NEEDS_WORK"}}
|
|
464
|
+
else:
|
|
465
|
+
phases[origin] = {**span, "round": 1, "verdict": "FAIL"}
|
|
466
|
+
if origin == "cleanup":
|
|
467
|
+
phases[origin]["execution_kind"] = "orchestrator_commands"
|
|
468
|
+
phases["final_report"] = {**span, "round": 0,
|
|
469
|
+
"verdict": "NEEDS_WORK" if origin == "verify" else "BLOCKED:repair-budget-exhausted"}
|
|
470
|
+
return {"version": "3.0", "run_id": "repair-" + origin,
|
|
471
|
+
"rounds": {"global": 1, "max_rounds": 1},
|
|
472
|
+
"phases": phases}
|
|
473
|
+
|
|
474
|
+
for origin in ("build_gate", "cleanup", "verify", "phase_gate"):
|
|
475
|
+
state = exhausted_case(origin)
|
|
476
|
+
path = base / ".devlyn" / "runs" / state["run_id"] / "pipeline.state.json"
|
|
477
|
+
raw = json.dumps(state).encode()
|
|
478
|
+
result, _ = classify_state_bytes(base, path, raw, archived=True)
|
|
479
|
+
assert result.status == "CLEAN" and not result.incomplete, (origin, result)
|
|
480
|
+
tests += 1
|
|
481
|
+
result, _ = classify_state_bytes(base, path, raw, archived=False)
|
|
482
|
+
assert result.status == "INCOMPLETE:archive", (origin, result)
|
|
483
|
+
tests += 1
|
|
484
|
+
for bad in (True, 2):
|
|
485
|
+
forged = json.loads(raw)
|
|
486
|
+
forged["rounds"]["global"] = bad
|
|
487
|
+
result, _ = classify_state_bytes(base, path, json.dumps(forged).encode(), archived=True)
|
|
488
|
+
assert result.status != "CLEAN", (origin, bad, result)
|
|
489
|
+
tests += 1
|
|
490
|
+
opened = json.loads(raw)
|
|
491
|
+
name = "implement" if origin == "phase_gate" else origin
|
|
492
|
+
opened["phases"][name]["completed_at"] = None
|
|
493
|
+
result, _ = classify_state_bytes(base, path, json.dumps(opened).encode(), archived=True)
|
|
494
|
+
assert result.incomplete, (origin, result)
|
|
495
|
+
tests += 1
|
|
496
|
+
opened = json.loads(raw)
|
|
497
|
+
opened["phases"][name]["history"] = [{"started_at": "2026-07-19T00:00:00Z", "completed_at": None}]
|
|
498
|
+
result, _ = classify_state_bytes(base, path, json.dumps(opened).encode(), archived=True)
|
|
499
|
+
assert result.incomplete, (origin, result)
|
|
500
|
+
tests += 1
|
|
501
|
+
opened = json.loads(raw)
|
|
502
|
+
opened["phases"]["final_report"]["completed_at"] = None
|
|
503
|
+
result, _ = classify_state_bytes(base, path, json.dumps(opened).encode(), archived=True)
|
|
504
|
+
assert result.incomplete, (origin, result)
|
|
505
|
+
tests += 1
|
|
506
|
+
if origin != "verify":
|
|
507
|
+
stale = json.loads(raw)
|
|
508
|
+
stale["phases"]["final_report"]["verdict"] = "PASS"
|
|
509
|
+
result, _ = classify_state_bytes(base, path, json.dumps(stale).encode(), archived=True)
|
|
510
|
+
assert result.status == "INCOMPLETE:final_report", (origin, result)
|
|
511
|
+
tests += 1
|
|
512
|
+
if origin == "phase_gate":
|
|
513
|
+
forged = json.loads(raw)
|
|
514
|
+
forged["phases"]["implement"]["exec"]["total"] = 1
|
|
515
|
+
result, _ = classify_state_bytes(base, path, json.dumps(forged).encode(), archived=True)
|
|
516
|
+
assert result.status == "INCOMPLETE:final_report", result
|
|
517
|
+
tests += 1
|
|
518
|
+
blocked = json.loads(raw)
|
|
519
|
+
blocked["phases"]["surface_close"] = {"started_at": "2026-07-19T00:00:00Z",
|
|
520
|
+
"completed_at": "2026-07-19T00:01:00Z", "verdict": "BLOCKED"}
|
|
521
|
+
result, _ = classify_state_bytes(base, path, json.dumps(blocked).encode(), archived=True)
|
|
522
|
+
assert result.status == "INCOMPLETE:final_report", (origin, result)
|
|
523
|
+
tests += 1
|
|
524
|
+
for missing in (True, False):
|
|
525
|
+
without_rounds = json.loads(raw)
|
|
526
|
+
if missing:
|
|
527
|
+
del without_rounds["rounds"]
|
|
528
|
+
else:
|
|
529
|
+
without_rounds["rounds"] = None
|
|
530
|
+
without_rounds["phases"]["final_report"]["verdict"] = (
|
|
531
|
+
"NEEDS_WORK" if origin == "verify" else "PASS"
|
|
532
|
+
)
|
|
533
|
+
result, _ = classify_state_bytes(
|
|
534
|
+
base, path, json.dumps(without_rounds).encode(), archived=True,
|
|
535
|
+
)
|
|
536
|
+
assert result.status == "INCOMPLETE:final_report", (origin, missing, result)
|
|
537
|
+
tests += 1
|
|
538
|
+
|
|
539
|
+
def test_higher_priority_terminal_floor() -> None:
|
|
540
|
+
state = exhausted_case("build_gate")
|
|
541
|
+
state["rounds"] = {"global": 4, "max_rounds": 4}
|
|
542
|
+
state["phases"]["build_gate"]["round"] = 4
|
|
543
|
+
state["phases"]["verify"]["round"] = 3
|
|
544
|
+
state["phases"]["final_report"]["verdict"] = "BLOCKED:finish-gate-unclean"
|
|
545
|
+
path = base / ".devlyn" / "runs" / state["run_id"] / "pipeline.state.json"
|
|
546
|
+
result, _ = classify_state_bytes(base, path, json.dumps(state).encode(), archived=True)
|
|
547
|
+
assert result.status == "CLEAN", result
|
|
548
|
+
|
|
549
|
+
def test_invalid_counters_reject_stale_verify() -> None:
|
|
550
|
+
for global_ in (True, 5):
|
|
551
|
+
state = exhausted_case("build_gate")
|
|
552
|
+
state["rounds"] = {"global": global_, "max_rounds": 4}
|
|
553
|
+
state["phases"]["build_gate"]["round"] = 4
|
|
554
|
+
state["phases"]["verify"]["round"] = 3
|
|
555
|
+
state["phases"]["final_report"]["verdict"] = "PASS"
|
|
556
|
+
path = base / ".devlyn" / "runs" / state["run_id"] / "pipeline.state.json"
|
|
557
|
+
result, _ = classify_state_bytes(base, path, json.dumps(state).encode(), archived=True)
|
|
558
|
+
assert result.status != "CLEAN", (global_, result)
|
|
559
|
+
|
|
560
|
+
def test_invalid_phase_gate_witness() -> None:
|
|
561
|
+
state = exhausted_case("phase_gate")
|
|
562
|
+
state["phases"]["implement"]["exec"] = {
|
|
563
|
+
"total": 2, "current": 1, "statuses": ["FAIL", "UNKNOWN"],
|
|
564
|
+
}
|
|
565
|
+
writer = runpy.run_path(str(pathlib.Path(__file__).with_name("state-phase-write.py")))
|
|
566
|
+
try:
|
|
567
|
+
writer["repair_admission"](state, "implement", 2, None)
|
|
568
|
+
except SystemExit as exc:
|
|
569
|
+
assert str(exc) == "BLOCKED:repair-edge-invalid", exc
|
|
570
|
+
else:
|
|
571
|
+
raise AssertionError("writer admitted malformed phase gate")
|
|
572
|
+
path = base / ".devlyn" / "runs" / state["run_id"] / "pipeline.state.json"
|
|
573
|
+
result, _ = classify_state_bytes(base, path, json.dumps(state).encode(), archived=True)
|
|
574
|
+
assert result.status == "INCOMPLETE:final_report", result
|
|
575
|
+
|
|
576
|
+
def test_verify_only_direct_report_and_stale_verify() -> None:
|
|
577
|
+
state = exhausted_case("verify")
|
|
578
|
+
state["mode"] = "verify-only"
|
|
579
|
+
state["rounds"] = {"global": 0, "max_rounds": 4}
|
|
580
|
+
state["phases"]["verify"]["round"] = 0
|
|
581
|
+
path = base / ".devlyn" / "runs" / state["run_id"] / "pipeline.state.json"
|
|
582
|
+
result, _ = classify_state_bytes(base, path, json.dumps(state).encode(), archived=True)
|
|
583
|
+
assert result.status == "CLEAN", result
|
|
584
|
+
for missing in (True, False):
|
|
585
|
+
direct = json.loads(json.dumps(state))
|
|
586
|
+
if missing:
|
|
587
|
+
del direct["rounds"]
|
|
588
|
+
else:
|
|
589
|
+
direct["rounds"] = None
|
|
590
|
+
result, _ = classify_state_bytes(base, path, json.dumps(direct).encode(), archived=True)
|
|
591
|
+
assert result.status == "CLEAN", (missing, result)
|
|
592
|
+
ordinary = json.loads(json.dumps(direct))
|
|
593
|
+
ordinary.pop("mode")
|
|
594
|
+
ordinary["phases"]["verify"]["verdict"] = "PASS"
|
|
595
|
+
ordinary["phases"]["verify"]["merged"]["verdict"] = "PASS"
|
|
596
|
+
ordinary["phases"]["final_report"]["verdict"] = "PASS"
|
|
597
|
+
result, _ = classify_state_bytes(base, path, json.dumps(ordinary).encode(), archived=True)
|
|
598
|
+
assert result.status == "CLEAN", (missing, result)
|
|
599
|
+
stale = exhausted_case("build_gate")
|
|
600
|
+
stale["mode"] = "verify-only"
|
|
601
|
+
stale["rounds"] = {"global": 0, "max_rounds": 4}
|
|
602
|
+
stale["phases"]["final_report"]["verdict"] = "PASS"
|
|
603
|
+
path = base / ".devlyn" / "runs" / stale["run_id"] / "pipeline.state.json"
|
|
604
|
+
result, _ = classify_state_bytes(base, path, json.dumps(stale).encode(), archived=True)
|
|
605
|
+
assert result.status == "INCOMPLETE:final_report", result
|
|
606
|
+
|
|
607
|
+
def test_pre_v3_exhaustion_archive_compatibility() -> None:
|
|
608
|
+
for version, expected in (("2.0", "CLEAN"), ("3.0", "INCOMPLETE:final_report")):
|
|
609
|
+
state = exhausted_case("verify")
|
|
610
|
+
state["version"] = version
|
|
611
|
+
state["rounds"] = {"global": 4, "max_rounds": 4}
|
|
612
|
+
state["phases"]["verify"]["round"] = 4
|
|
613
|
+
del state["phases"]["verify"]["merged"]
|
|
614
|
+
root = base / f"exhausted-archive-{version}"
|
|
615
|
+
write_archived_state(root, state)
|
|
616
|
+
result = classify(root)
|
|
617
|
+
assert result.status == expected, (version, result)
|
|
618
|
+
|
|
619
|
+
test_higher_priority_terminal_floor()
|
|
620
|
+
test_invalid_counters_reject_stale_verify()
|
|
621
|
+
test_invalid_phase_gate_witness()
|
|
622
|
+
test_verify_only_direct_report_and_stale_verify()
|
|
623
|
+
test_pre_v3_exhaustion_archive_compatibility()
|
|
624
|
+
tests += 11
|
|
625
|
+
|
|
399
626
|
root = base / "set-quantification"
|
|
400
627
|
write_archived_state(root, f23_state)
|
|
401
628
|
write_archived_state(root, fs1_state)
|
|
@@ -458,8 +685,6 @@ def self_test() -> int:
|
|
|
458
685
|
("plan-empty", "plan", "PASS"),
|
|
459
686
|
("risk-halt", "plan", "PASS"),
|
|
460
687
|
("implement-empty", "implement", "PASS"),
|
|
461
|
-
("build-gate-exhausted", "build_gate", "FAIL"),
|
|
462
|
-
("verify-exhausted", "verify", "NEEDS_WORK"),
|
|
463
688
|
("fresh-context-unavailable", "implement", "BLOCKED"),
|
|
464
689
|
("codex-unavailable", "plan", "BLOCKED"),
|
|
465
690
|
)
|
|
@@ -1471,6 +1471,91 @@ def write_state(devlyn: pathlib.Path, summary: dict[str, Any]) -> None:
|
|
|
1471
1471
|
def self_test() -> int:
|
|
1472
1472
|
import subprocess
|
|
1473
1473
|
|
|
1474
|
+
with tempfile.TemporaryDirectory() as tmp:
|
|
1475
|
+
work = pathlib.Path(tmp).resolve()
|
|
1476
|
+
devlyn = work / ".devlyn"
|
|
1477
|
+
devlyn.mkdir()
|
|
1478
|
+
role = JUDGE_ROLE_EVIDENCE["ROLE"]
|
|
1479
|
+
(devlyn / "engines.json").write_bytes(role["encoded"]({
|
|
1480
|
+
"roles": {"primary_judge": {"engine": "claude", "model": "fixture-claude-model", "effort": "high"}},
|
|
1481
|
+
}))
|
|
1482
|
+
resolution = role["resolve"](work, "claude", no_pair=True, available=lambda engine: True)
|
|
1483
|
+
state = {"run_id": "merge-relative-role", "engine": "claude", "role_resolution": resolution,
|
|
1484
|
+
"rounds": {"global": 0, "max_rounds": 1},
|
|
1485
|
+
"phases": {"verify": {"engine": "claude", "round": 0, "verdict": None,
|
|
1486
|
+
"started_at": "2026-01-01T00:00:00Z", "completed_at": "2026-01-01T00:01:00Z"}}}
|
|
1487
|
+
state_path = devlyn / "pipeline.state.json"
|
|
1488
|
+
state_path.write_text(json.dumps(state), encoding="utf-8")
|
|
1489
|
+
stem = "claude-judge.r0"
|
|
1490
|
+
prompt_path = devlyn / (stem + ".prompt")
|
|
1491
|
+
prompt = b"review"
|
|
1492
|
+
prompt_path.write_bytes(prompt)
|
|
1493
|
+
command = ["claude", "-p", "--model", "fixture-claude-model", "--effort", "high",
|
|
1494
|
+
"--permission-mode", "dontAsk", "--tools", "Read,Grep,Glob",
|
|
1495
|
+
"--allowedTools", "Read,Grep,Glob", "--setting-sources", "project",
|
|
1496
|
+
"--output-format", "json", "--strict-mcp-config", "--mcp-config", '{"mcpServers":{}}']
|
|
1497
|
+
argv = ["python3", "run-bounded.py", "600", "--stdin-file", str(prompt_path),
|
|
1498
|
+
"--record-transport", "--", *command]
|
|
1499
|
+
argv_path = devlyn / (stem + ".argv.json")
|
|
1500
|
+
argv_path.write_bytes(role["encoded"](argv))
|
|
1501
|
+
(devlyn / (stem + ".stderr")).write_bytes(b"")
|
|
1502
|
+
(devlyn / (stem + ".output.json")).write_bytes(role["encoded"]({
|
|
1503
|
+
"type": "result", "subtype": "success", "is_error": False, "stop_reason": "end_turn",
|
|
1504
|
+
"session_id": "fixture-claude", "result": "PASS", "modelUsage": {"fixture-claude-model": {}},
|
|
1505
|
+
}))
|
|
1506
|
+
transport = {"schema_version": 1, "transport": "stdin-file",
|
|
1507
|
+
"prompt": {"path": str(prompt_path), "sha256": role["digest"](prompt), "bytes": len(prompt)},
|
|
1508
|
+
"command": command, "argv": command, "timeout_sec": 600, "isolated": False,
|
|
1509
|
+
"status": "completed", "exit_code": 0}
|
|
1510
|
+
(devlyn / (stem + ".prompt.transport.json")).write_text(json.dumps(transport), encoding="utf-8")
|
|
1511
|
+
evidence, derived = JUDGE_ROLE_EVIDENCE["describe"](devlyn, state, "primary_judge", 0)
|
|
1512
|
+
(devlyn / (stem + ".role-evidence.json")).write_bytes(role["encoded"](evidence))
|
|
1513
|
+
(devlyn / (stem + ".stdout")).write_bytes(derived)
|
|
1514
|
+
(devlyn / "claude-judge.stdout").write_bytes(derived)
|
|
1515
|
+
(devlyn / "verify-mechanical.findings.jsonl").write_text("", encoding="utf-8")
|
|
1516
|
+
(devlyn / "verify.findings.jsonl").write_text("", encoding="utf-8")
|
|
1517
|
+
script = str(pathlib.Path(__file__).resolve())
|
|
1518
|
+
def merge_at(directory):
|
|
1519
|
+
return subprocess.run([sys.executable, script, "--devlyn-dir", directory, "--write-state"],
|
|
1520
|
+
cwd=work, capture_output=True, text=True)
|
|
1521
|
+
relative = merge_at(".devlyn")
|
|
1522
|
+
assert relative.returncode == 0, relative.stderr
|
|
1523
|
+
relative_state = json.loads(state_path.read_text())
|
|
1524
|
+
assert relative_state["phases"]["verify"]["verdict"] == "PASS", relative.stdout
|
|
1525
|
+
state_path.write_text(json.dumps(state), encoding="utf-8")
|
|
1526
|
+
absolute = merge_at(str(devlyn))
|
|
1527
|
+
assert absolute.returncode == 0, absolute.stderr
|
|
1528
|
+
absolute_state = json.loads(state_path.read_text())
|
|
1529
|
+
assert absolute_state["phases"]["verify"]["role_evidence"] == relative_state["phases"]["verify"]["role_evidence"]
|
|
1530
|
+
argv[4] = str(work / "wrong.prompt")
|
|
1531
|
+
argv_path.write_bytes(role["encoded"](argv))
|
|
1532
|
+
state_path.write_text(json.dumps(state), encoding="utf-8")
|
|
1533
|
+
wrong = merge_at(".devlyn")
|
|
1534
|
+
assert wrong.returncode == 0, wrong.stderr
|
|
1535
|
+
blocked_state = json.loads(state_path.read_text())
|
|
1536
|
+
assert blocked_state["phases"]["verify"]["verdict"] == "BLOCKED"
|
|
1537
|
+
writer = runpy.run_path(str(pathlib.Path(__file__).with_name("state-phase-write.py")))
|
|
1538
|
+
try:
|
|
1539
|
+
writer["do_spawn"](blocked_state, "implement", 1, "verify", None, None, None)
|
|
1540
|
+
except SystemExit as exc:
|
|
1541
|
+
assert str(exc) == "BLOCKED:repair-edge-invalid"
|
|
1542
|
+
else:
|
|
1543
|
+
raise AssertionError("BLOCKED VERIFY entered product repair")
|
|
1544
|
+
assert blocked_state["rounds"]["global"] == 0
|
|
1545
|
+
argv[4] = str(prompt_path)
|
|
1546
|
+
argv_path.write_bytes(role["encoded"](argv))
|
|
1547
|
+
finding = {"id": "fixture-high", "rule_id": "fixture.binding", "severity": "HIGH",
|
|
1548
|
+
"confidence": "high", "file": "source.txt", "line": 1,
|
|
1549
|
+
"message": "required behavior is missing", "criterion_ref": "fixture"}
|
|
1550
|
+
(devlyn / "verify.findings.jsonl").write_text(json.dumps(finding) + "\n", encoding="utf-8")
|
|
1551
|
+
state_path.write_text(json.dumps(state), encoding="utf-8")
|
|
1552
|
+
needs_work = merge_at(".devlyn")
|
|
1553
|
+
assert needs_work.returncode == 0, needs_work.stderr
|
|
1554
|
+
repair_state = json.loads(state_path.read_text())
|
|
1555
|
+
assert repair_state["phases"]["verify"]["verdict"] == "NEEDS_WORK"
|
|
1556
|
+
writer["do_spawn"](repair_state, "implement", 1, "verify", None, None, None)
|
|
1557
|
+
assert repair_state["rounds"]["global"] == 1
|
|
1558
|
+
|
|
1474
1559
|
try:
|
|
1475
1560
|
loads_strict_json('{"verdict":"PASS","verdict":"BLOCKED"}')
|
|
1476
1561
|
except ValueError as exc:
|
|
@@ -3424,10 +3509,16 @@ def main() -> int:
|
|
|
3424
3509
|
if not devlyn.is_dir():
|
|
3425
3510
|
sys.stderr.write(f"error: {devlyn} is not a directory\n")
|
|
3426
3511
|
return 1
|
|
3512
|
+
if args.write_state:
|
|
3513
|
+
lock = runpy.run_path(str(pathlib.Path(__file__).with_name("platform-support.py")))["file_lock"]
|
|
3514
|
+
with lock(devlyn.resolve() / "pipeline.state.lock", blocking=True):
|
|
3515
|
+
findings, source_verdicts = read_findings(devlyn)
|
|
3516
|
+
summary = write_outputs(devlyn, findings, source_verdicts)
|
|
3517
|
+
write_state(devlyn, summary)
|
|
3518
|
+
print(json.dumps(summary, sort_keys=True))
|
|
3519
|
+
return 0
|
|
3427
3520
|
findings, source_verdicts = read_findings(devlyn)
|
|
3428
3521
|
summary = write_outputs(devlyn, findings, source_verdicts)
|
|
3429
|
-
if args.write_state:
|
|
3430
|
-
write_state(devlyn, summary)
|
|
3431
3522
|
print(json.dumps(summary, sort_keys=True))
|
|
3432
3523
|
return 0
|
|
3433
3524
|
|
|
@@ -110,8 +110,8 @@ Structural lint (inline check, no script needed):
|
|
|
110
110
|
After lint passes:
|
|
111
111
|
1. Write `<spec-dir>/<id>-<slug>/spec.md` (the spec).
|
|
112
112
|
2. Generate `<spec-dir>/<id>-<slug>/spec.expected.json` following `references/spec-template.md` § "Sibling file: `spec.expected.json`" for constraint coverage, diff scope and guard controls.
|
|
113
|
-
3. Run `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check <spec-path>` to validate the actual spec's carrier shape and supported `complexity` frontmatter. Sibling `spec.expected.json` takes precedence over the legacy inline `## Verification` JSON carrier
|
|
114
|
-
4. Run `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check-expected <expected-path>` to validate sibling `spec.expected.json` against `_shared/expected.schema.json` plus sibling spec `complexity` frontmatter
|
|
113
|
+
3. Run `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check <spec-path>` to validate the actual spec's carrier shape and supported `complexity` frontmatter. Sibling `spec.expected.json` takes precedence over the legacy inline `## Verification` JSON carrier. If exit 2, fix the file named in the error and re-run.
|
|
114
|
+
4. Run `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check-expected <expected-path>` to validate sibling `spec.expected.json` against `_shared/expected.schema.json` plus sibling spec `complexity` frontmatter. If exit 2, fix the JSON/frontmatter and re-run.
|
|
115
115
|
5. Print: `spec ready — /devlyn:resolve --spec <spec-path>`.
|
|
116
116
|
|
|
117
117
|
## PHASE 1Q: QUICK MODE
|
|
@@ -123,7 +123,6 @@ Single-turn assume-and-confirm. Prompt body: see `references/elicitation.md` §
|
|
|
123
123
|
3. User responds with "go" / "fix X" / "no, different".
|
|
124
124
|
4. On "go": write spec + spec.expected.json + lint + announce.
|
|
125
125
|
5. On "fix X": apply correction, re-show, ask again. Maximum 3 correction rounds before escalating to default mode.
|
|
126
|
-
6. Exception: for benchmark, risk-probe, or pair-evidence goals, do not infer a solo-headroom hypothesis. Ask for the actionable hypothesis first; if unavailable, exit with `spec not ready — solo-headroom hypothesis required`. For new unmeasured benchmark, shadow-fixture, golden-fixture, risk-probe, or pair-evidence candidates, also do not infer solo ceiling avoidance; ask for the concrete difference from rejected or solo-saturated controls such as `S2`-`S6`, and exit with `spec not ready — solo ceiling avoidance required` if unavailable.
|
|
127
126
|
|
|
128
127
|
## PHASE 1F: FROM-SPEC MODE
|
|
129
128
|
|
|
@@ -136,7 +135,7 @@ Prompt body: `references/from-spec-mode.md`.
|
|
|
136
135
|
5. Generate `spec.expected.json` if absent, following the same template carrier guidance as default mode.
|
|
137
136
|
6. Write the normalized spec back to `<spec-dir>/<id>-<slug>/` (preserves original at `<path>` untouched unless user passes `--in-place`).
|
|
138
137
|
7. Run both lint checks: `--check <spec-path>` and `--check-expected <expected-path>`.
|
|
139
|
-
8. Lint pass → announce. Lint fail → surface the unfixable issue and exit non-zero.
|
|
138
|
+
8. Lint pass → announce. Lint fail → surface the unfixable issue and exit non-zero.
|
|
140
139
|
|
|
141
140
|
## PHASE 1P: PROJECT MODE
|
|
142
141
|
|
|
@@ -40,27 +40,6 @@ For most coding tasks, the under-specified blanks are:
|
|
|
40
40
|
auth/error priority, or exact output shape, ask for one concrete compound
|
|
41
41
|
scenario that exercises the interaction end-to-end instead of accepting only
|
|
42
42
|
isolated happy-path checks.
|
|
43
|
-
8. **Pair-candidate headroom**: when the user is creating a benchmark, risk
|
|
44
|
-
probe, or pair-evidence candidate, ask for one solo-headroom hypothesis in
|
|
45
|
-
actionable form: the spec must literally contain `solo-headroom hypothesis`,
|
|
46
|
-
`solo_claude`, `miss`, and a backticked observable command while naming the
|
|
47
|
-
visible behavior a capable `solo_claude` baseline should miss; the backticked
|
|
48
|
-
line itself must contain `miss` and be framed as the command/observable that exposes it. If the
|
|
49
|
-
answer is only "the task is hard", rework the candidate before spending provider
|
|
50
|
-
calls. Do not write a benchmark/risk-probe/pair-evidence spec until this
|
|
51
|
-
hypothesis is actionable; if the user cannot provide it, stop with
|
|
52
|
-
`spec not ready — solo-headroom hypothesis required` and ask them to return
|
|
53
|
-
with the visible behavior `solo_claude` is expected to miss.
|
|
54
|
-
9. **Solo ceiling avoidance**: for a new unmeasured benchmark, shadow-fixture,
|
|
55
|
-
golden-fixture, risk-probe, or pair-evidence candidate, ask how this candidate
|
|
56
|
-
differs from rejected or solo-saturated controls such as `S2`-`S6`. The note
|
|
57
|
-
must literally contain `solo ceiling avoidance`, mention `solo_claude`, and
|
|
58
|
-
name the concrete difference expected to preserve `solo_claude` headroom.
|
|
59
|
-
Benchmark fixture directories put this in `NOTES.md` as
|
|
60
|
-
`## Solo ceiling avoidance`; ordinary specs keep it in `## Verification`
|
|
61
|
-
next to the solo-headroom hypothesis. Do not write or measure the candidate
|
|
62
|
-
if this answer is missing; stop with
|
|
63
|
-
`spec not ready — solo ceiling avoidance required`.
|
|
64
43
|
|
|
65
44
|
Walk through these in roughly this order. Skip the ones already clear from the user's initial text.
|
|
66
45
|
</missing_decisions_to_surface>
|
|
@@ -94,8 +73,8 @@ Before declaring the spec ready, verify structurally:
|
|
|
94
73
|
If the lint fails, fix the missing piece (ask one focused question if needed) before announcing.
|
|
95
74
|
|
|
96
75
|
After lint passes, run both mechanical checks:
|
|
97
|
-
1. `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check <spec-path>` validates the spec's verification carrier shape
|
|
98
|
-
2. `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check-expected <expected-path>` validates sibling `spec.expected.json` against `_shared/expected.schema.json` plus sibling spec `complexity` frontmatter
|
|
76
|
+
1. `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check <spec-path>` validates the spec's verification carrier shape and supported `complexity` frontmatter.
|
|
77
|
+
2. `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check-expected <expected-path>` validates sibling `spec.expected.json` against `_shared/expected.schema.json` plus sibling spec `complexity` frontmatter.
|
|
99
78
|
|
|
100
79
|
If either exits 2: read the stderr message, fix the malformed carrier or JSON, and re-run the failed command. Both commands must exit 0 before ready.
|
|
101
80
|
</lint>
|
|
@@ -120,18 +99,6 @@ When `--quick` is set:
|
|
|
120
99
|
4. On "go": write the spec + spec.expected.json, run both lint checks, announce.
|
|
121
100
|
5. On "fix X": apply correction, re-present, ask again. Maximum 3 correction rounds before escalating to default mode.
|
|
122
101
|
|
|
123
|
-
Exception: quick mode must not infer a solo-headroom hypothesis for benchmark,
|
|
124
|
-
risk-probe, or pair-evidence goals. If the one-line goal lacks the actionable
|
|
125
|
-
`solo-headroom hypothesis` / `solo_claude` / `miss` / backticked-command
|
|
126
|
-
contract, ask exactly one focused follow-up for that hypothesis before showing a
|
|
127
|
-
draft; if the user cannot provide it, exit with
|
|
128
|
-
`spec not ready — solo-headroom hypothesis required`. For a new unmeasured
|
|
129
|
-
benchmark, shadow-fixture, golden-fixture, risk-probe, or pair-evidence
|
|
130
|
-
candidate, quick mode also must not infer the `solo ceiling avoidance` note; ask
|
|
131
|
-
for the concrete difference from rejected or solo-saturated controls such as
|
|
132
|
-
`S2`-`S6`, and exit with `spec not ready — solo ceiling avoidance required` if
|
|
133
|
-
the user cannot provide it.
|
|
134
|
-
|
|
135
102
|
Quick mode trades thoroughness for speed. Use it for trivial-medium tasks where the user has a clear-enough goal that one round of inference + correction is sufficient.
|
|
136
103
|
|
|
137
104
|
## Anti-patterns
|
|
@@ -37,8 +37,8 @@ You must NOT:
|
|
|
37
37
|
3. For each missing/malformed piece: apply the smallest allowed fix.
|
|
38
38
|
4. Write the normalized spec. Default location: `<spec-dir>/<id>-<slug>/spec.md`. With `--in-place` flag: write to `<path>` directly (overwrites the original).
|
|
39
39
|
5. Generate or fix `spec.expected.json` per the rules above. Same dir as the spec.
|
|
40
|
-
6. Run `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check <spec-path>` to validate the actual spec's carrier shape and supported `complexity` frontmatter. Sibling `spec.expected.json` takes precedence over the legacy inline `## Verification` JSON carrier
|
|
41
|
-
7. Run `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check-expected <expected-path>` to validate sibling `spec.expected.json` plus sibling spec `complexity` frontmatter
|
|
40
|
+
6. Run `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check <spec-path>` to validate the actual spec's carrier shape and supported `complexity` frontmatter. Sibling `spec.expected.json` takes precedence over the legacy inline `## Verification` JSON carrier. If exit 2, fix the file named in the error and re-run.
|
|
41
|
+
7. Run `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check-expected <expected-path>` to validate sibling `spec.expected.json` plus sibling spec `complexity` frontmatter.
|
|
42
42
|
8. If lint still fails after allowed fixes (e.g. Requirements section is empty in the source), surface the issue and exit non-zero — do NOT invent Requirements.
|
|
43
43
|
9. If the preserved Requirements combine state mutation with ordering/priority,
|
|
44
44
|
idempotency, auth/error priority, or exact output shape but the Verification
|
|
@@ -46,27 +46,12 @@ You must NOT:
|
|
|
46
46
|
content. Add a final warning that `/devlyn:resolve` may need default-mode
|
|
47
47
|
ideation or a stronger Verification section before pair-relevant risks are
|
|
48
48
|
measurable.
|
|
49
|
-
10. If the source is a benchmark, risk probe, or pair-evidence candidate and it
|
|
50
|
-
lacks an actionable solo-headroom hypothesis, do not invent one. Add a final
|
|
51
|
-
warning that the candidate may be solo-saturated until Context or
|
|
52
|
-
Verification literally contains `solo-headroom hypothesis`, `solo_claude`,
|
|
53
|
-
`miss`, and a backticked observable command while naming the visible
|
|
54
|
-
behavior a capable `solo_claude` baseline is expected to miss; the
|
|
55
|
-
backticked line itself must contain `miss` and be framed as the
|
|
56
|
-
command/observable that exposes it. Do not call the normalized spec pair-evidence ready.
|
|
57
|
-
11. If the source is a new unmeasured benchmark, shadow-fixture, golden-fixture,
|
|
58
|
-
risk-probe, or pair-evidence candidate and it lacks a solo ceiling avoidance
|
|
59
|
-
note, do not invent one. Add a final warning that the candidate may replay
|
|
60
|
-
rejected or solo-saturated controls until Context, Verification, or fixture
|
|
61
|
-
`NOTES.md` literally contains `solo ceiling avoidance`, mentions
|
|
62
|
-
`solo_claude`, and names a concrete difference from rejected controls such
|
|
63
|
-
as `S2`-`S6`. Do not call the normalized spec pair-evidence ready.
|
|
64
49
|
</flow>
|
|
65
50
|
|
|
66
51
|
<output>
|
|
67
52
|
Same as default mode: `<spec-dir>/<id>-<slug>/spec.md` + `<spec-dir>/<id>-<slug>/spec.expected.json`.
|
|
68
53
|
|
|
69
|
-
Final announcement: `spec normalized — /devlyn:resolve --spec <spec-path>`. If the spec was lint-passing with no changes needed, announce: `spec already canonical — /devlyn:resolve --spec <spec-path>`. If step 9 applies, append: `warning: Verification may need one compound end-to-end scenario before pair-relevant risks are measurable`.
|
|
54
|
+
Final announcement: `spec normalized — /devlyn:resolve --spec <spec-path>`. If the spec was lint-passing with no changes needed, announce: `spec already canonical — /devlyn:resolve --spec <spec-path>`. If step 9 applies, append: `warning: Verification may need one compound end-to-end scenario before pair-relevant risks are measurable`.
|
|
70
55
|
|
|
71
56
|
If lint failed unfixably: print the specific failure, exit non-zero. Do not write a partial output.
|
|
72
57
|
</output>
|
|
@@ -15,21 +15,6 @@ The user wants to build a project, not a single feature. Your job is to elicit t
|
|
|
15
15
|
auth/error priority, or exact output shape, its per-feature Verification must
|
|
16
16
|
include one compound end-to-end scenario; do not hide the interaction in
|
|
17
17
|
project-level prose.
|
|
18
|
-
6. When a feature is intended as a benchmark, risk probe, or pair-evidence
|
|
19
|
-
candidate, its per-feature Verification must include a solo-headroom
|
|
20
|
-
hypothesis. The feature spec must literally contain
|
|
21
|
-
`solo-headroom hypothesis`, `solo_claude`, `miss`, and a backticked
|
|
22
|
-
observable command while naming the visible behavior a capable
|
|
23
|
-
`solo_claude` baseline is expected to miss; the backticked line itself must
|
|
24
|
-
contain `miss` and be framed as the command/observable that exposes it. Do not defer that to
|
|
25
|
-
project-level prose, and rework the feature spec if the hypothesis is only
|
|
26
|
-
"the task is hard".
|
|
27
|
-
7. When a feature is a new unmeasured benchmark, shadow-fixture, golden-fixture,
|
|
28
|
-
risk-probe, or pair-evidence candidate, its per-feature Verification must also include a solo ceiling avoidance note. The feature spec must literally
|
|
29
|
-
contain `solo ceiling avoidance`, mention `solo_claude`, and name a concrete
|
|
30
|
-
difference from rejected or solo-saturated controls such as `S2`-`S6`. Do not
|
|
31
|
-
defer that to project-level prose; benchmark fixture directories mirror the
|
|
32
|
-
same note in `NOTES.md` as `## Solo ceiling avoidance`.
|
|
33
18
|
</conversation_rules>
|
|
34
19
|
|
|
35
20
|
<decomposition_rules>
|
|
@@ -82,7 +67,7 @@ Anything binding all features (e.g. "no new top-level dependencies", "all CLI ou
|
|
|
82
67
|
- `<spec-dir>/<id-N>/spec.md` for each feature (per `references/spec-template.md`).
|
|
83
68
|
- `<spec-dir>/<id-N>/spec.expected.json` for each feature (per `_shared/expected.schema.json`).
|
|
84
69
|
|
|
85
|
-
Each per-feature spec is structurally lint-validated using `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check <spec-path>`, including supported `complexity` frontmatter
|
|
70
|
+
Each per-feature spec is structurally lint-validated using `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check <spec-path>`, including supported `complexity` frontmatter, and each sibling expected contract plus sibling spec `complexity` frontmatter is validated using `python3 "$DEVLYN_SHARED_DIR/spec-verify-check.py" --check-expected <expected-path>`.
|
|
86
71
|
|
|
87
72
|
Final announcement: `project ready — N specs at <spec-dir>/. Start with /devlyn:resolve --spec <first-spec-path>`.
|
|
88
73
|
</output>
|
|
@@ -118,6 +118,4 @@ Substantive (ideate's job during elicitation):
|
|
|
118
118
|
- Avoid formatter-sensitive single-line adjacency checks — prefer a semantic test (runtime, DB, or CLI check), or grep one stable line-local anchor and assert required context separately.
|
|
119
119
|
- Frontmatter `complexity` reflects verification shape: `high` when the spec combines state mutation with ordering/priority, idempotency, auth/error priority, rollback/failure handling, or exact output shape; `medium` for ordinary multi-step work; `trivial` only for a single localized behavior.
|
|
120
120
|
- When Requirements combine state mutation, ordering/priority, idempotency, auth/error priority, or exact output shape, Verification includes at least one compound scenario that exercises the interaction end-to-end; isolated happy paths are insufficient.
|
|
121
|
-
- For benchmark, risk-probe, or pair-evidence specs, include a solo-headroom hypothesis inside `## Verification`: the artifact must literally contain `solo-headroom hypothesis`, `solo_claude`, `miss`, and a backticked observable command while naming the visible behavior a capable `solo_claude` baseline is expected to miss; the backticked line itself must contain `miss`, be framed as the command/observable that exposes it, and match a `spec.expected.json.verification_commands[].cmd` entry. For regenerated pair evidence, this hypothesis is the source for VERIFY's canonical `spec.solo_headroom_hypothesis` trigger reason and must pass `benchmark audit --require-hypothesis-trigger`. If the hypothesis is only "the task is hard", rework the spec before measurement.
|
|
122
|
-
- For new unmeasured benchmark, shadow-fixture, golden-fixture, risk-probe, or pair-evidence candidates, include a solo ceiling avoidance note before measurement: the artifact must literally contain `solo ceiling avoidance`, mention `solo_claude`, and name a concrete difference from rejected or solo-saturated controls such as `S2`-`S6`. If the note cannot say why the candidate should preserve `solo_claude` headroom, rework the candidate instead of spending provider calls. Benchmark fixture directories put this in `NOTES.md` as `## Solo ceiling avoidance`; ordinary specs keep the note in `## Verification` next to the solo-headroom hypothesis.
|
|
123
121
|
- Spec text is plain language — no jargon walls, no "for future flexibility" hedging.
|
|
@@ -23,18 +23,20 @@ DEVLYN_SHARED_DIR="$(cd "$DEVLYN_SKILL_DIR/../_shared" && pwd)"
|
|
|
23
23
|
Before processing the first item, read and obey
|
|
24
24
|
`../devlyn:resolve/references/outer-loop.md`. Its scoped-commit order is binding.
|
|
25
25
|
|
|
26
|
+
Queue state is base's `docs/specs/queue.md` plus owned receipts: for each `$(git rev-parse --git-common-dir)/devlyn-completion/*/receipt.json`, read `git show <ref>:docs/specs/queue.md` from its `recovery_ref` if that ref exists (it is never deleted), else its `branch` if it still exists; an item unmarked on base but `[x]`/`[F]` there counts as that mark. Two different terminal marks for one item stop for inspection. Status and drain selection both use this view.
|
|
27
|
+
|
|
26
28
|
## No args — status
|
|
27
29
|
|
|
28
|
-
Read `docs/specs/queue.md`
|
|
30
|
+
Read the queue state above (absent `docs/specs/queue.md` → report "queue empty — nothing staged" and how to add). Print pending `[ ]`, done `[x]`, and blocked `[F]` counts, the next item up, and one usage line per subcommand.
|
|
29
31
|
|
|
30
32
|
## Subcommands
|
|
31
33
|
|
|
32
34
|
- `add <intent text>` — append `- [ ] <intent>` to `docs/specs/queue.md` (create the file with its header if missing). If the intent came out of a conversation that already produced a spec, link it: `- [ ] (spec: docs/specs/<id>/spec.md) <intent>`.
|
|
33
35
|
- `drain` — serial drain per the project-instructions contract. For each pending item, in order:
|
|
34
|
-
1. Allocate the owner's absent task branch per `../devlyn:resolve/references/task-completion.md
|
|
35
|
-
2.
|
|
36
|
+
1. Allocate the owner's absent task branch with `--worktree <absent path>` per `../devlyn:resolve/references/task-completion.md`, and work in that worktree; then spec it if unspecced (the queue entry is the user's go-ahead). Unattended assumptions may only take scope-narrowing, reversible, non-user-visible defaults; material ambiguity (user-visible behavior, data/state semantics, new files/scripts/flags, implementation surface) → mark `[F] needs-review: <question>`, commit that queue transition, and continue.
|
|
37
|
+
2. Bring only the current queue-item delta and linked spec bundle from the queue view into the task worktree, which starts from the fetched remote base with no other local commits, commit that scoped owner baseline, then run `/devlyn:resolve --spec <path>` hands-free.
|
|
36
38
|
After every resolve invocation, run `python3 "$DEVLYN_SHARED_DIR/terminal-claim-check.py" .`; exit 79 marks `[F] FAILED-INCOMPLETE` from the predicate, never from the session self-report.
|
|
37
|
-
3. Outer loop on the terminal verdict: PASS → mark `[x]`. Findings-backed verdicts (NEEDS_WORK,
|
|
39
|
+
3. Outer loop on the terminal verdict: PASS → mark `[x]`. Findings-backed verdicts (NEEDS_WORK, BLOCKED:repair-budget-exhausted) → amend the spec, commit that scoped amendment, then re-run — at most 3 outer iterations. Infrastructure / invalid-input / engine-availability / implement-empty BLOCKED verdicts are not spec-amendable → mark `[F] <verdict>` immediately.
|
|
38
40
|
4. Commit each terminal `[x]` / `[F]` queue transition before advancing. For successfully archived normal runs, invoke owner completion once after that declared queue-file-only commit; bind it separately from the verified source. Honor local-only/no-push. Failed products never publish. Retain PR/pending workspaces and use a separate owned branch from base for the next item; unavailable safe placement stops the drain with remaining items pending. A product-blocked item otherwise never halts the queue.
|
|
39
41
|
5. When the queue is drained (or the session must stop), emit the drain report: per-item product verdict and separate delivery status/URL/resume, every logged assumption, and the commit range produced.
|
|
40
42
|
|