devlyn-cli 3.2.1 → 3.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +2 -2
- package/CLAUDE.md +2 -2
- package/README.md +12 -102
- package/bin/devlyn.js +13 -225
- package/bin/instruction-templates.json +6 -0
- package/config/skills/_shared/adapters/grok.md +1 -2
- package/config/skills/_shared/archive_run.py +15 -3
- package/config/skills/_shared/judge-role-evidence.py +21 -1
- package/config/skills/_shared/runtime-principles.md +1 -1
- package/config/skills/_shared/spec-verify-check.py +0 -653
- package/config/skills/_shared/state-phase-write.py +351 -25
- package/config/skills/_shared/task-complete.py +635 -143
- package/config/skills/_shared/terminal-claim-check.py +229 -4
- package/config/skills/_shared/verify-merge-findings.py +93 -2
- package/config/skills/devlyn:ideate/SKILL.md +3 -4
- package/config/skills/devlyn:ideate/references/elicitation.md +2 -35
- package/config/skills/devlyn:ideate/references/from-spec-mode.md +3 -18
- package/config/skills/devlyn:ideate/references/project-mode.md +1 -16
- package/config/skills/devlyn:ideate/references/spec-template.md +0 -2
- package/config/skills/devlyn:queue/SKILL.md +6 -4
- package/config/skills/devlyn:resolve/SKILL.md +16 -25
- package/config/skills/devlyn:resolve/references/free-form-mode.md +0 -14
- package/config/skills/devlyn:resolve/references/outer-loop.md +1 -1
- package/config/skills/devlyn:resolve/references/phases/build-gate.md +1 -1
- package/config/skills/devlyn:resolve/references/phases/cleanup.md +3 -3
- package/config/skills/devlyn:resolve/references/phases/implement.md +1 -1
- package/config/skills/devlyn:resolve/references/phases/probe-derive.md +1 -16
- package/config/skills/devlyn:resolve/references/phases/verify.md +4 -9
- package/config/skills/devlyn:resolve/references/state-schema.md +9 -7
- package/config/skills/devlyn:resolve/references/task-completion.md +46 -37
- package/{config/skills → optional-skills}/code-health-standards/SKILL.md +4 -13
- package/{config/skills → optional-skills}/code-review-standards/SKILL.md +4 -12
- package/{config/skills → optional-skills}/root-cause-analysis/SKILL.md +4 -12
- package/{config/skills → optional-skills}/ui-implementation-standards/SKILL.md +4 -14
- package/package.json +1 -57
- package/benchmark/auto-resolve/BENCHMARK-DESIGN.md +0 -289
- package/benchmark/auto-resolve/BENCHMARK-RESULTS.md +0 -341
- package/benchmark/auto-resolve/README.md +0 -696
- package/benchmark/auto-resolve/RUBRIC.md +0 -177
- package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/NOTES.md +0 -34
- package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/expected.json +0 -68
- package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/setup.sh +0 -4
- package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/spec.md +0 -44
- package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/task.ko.txt +0 -8
- package/benchmark/auto-resolve/fixtures/F1-cli-trivial-flag/task.txt +0 -8
- package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/NOTES.md +0 -68
- package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/expected.json +0 -60
- package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/setup.sh +0 -17
- package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/spec.md +0 -51
- package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/verifiers/invalid.js +0 -29
- package/benchmark/auto-resolve/fixtures/F10-persist-write-collision/verifiers/parallel.js +0 -50
- package/benchmark/auto-resolve/fixtures/F11-batch-import-all-or-nothing/NOTES.md +0 -75
- package/benchmark/auto-resolve/fixtures/F11-batch-import-all-or-nothing/expected.json +0 -52
- package/benchmark/auto-resolve/fixtures/F11-batch-import-all-or-nothing/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F11-batch-import-all-or-nothing/setup.sh +0 -171
- package/benchmark/auto-resolve/fixtures/F11-batch-import-all-or-nothing/spec.md +0 -50
- package/benchmark/auto-resolve/fixtures/F11-batch-import-all-or-nothing/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/F12-webhook-raw-body-signature/NOTES.md +0 -89
- package/benchmark/auto-resolve/fixtures/F12-webhook-raw-body-signature/expected.json +0 -74
- package/benchmark/auto-resolve/fixtures/F12-webhook-raw-body-signature/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F12-webhook-raw-body-signature/setup.sh +0 -251
- package/benchmark/auto-resolve/fixtures/F12-webhook-raw-body-signature/spec.md +0 -57
- package/benchmark/auto-resolve/fixtures/F12-webhook-raw-body-signature/task.txt +0 -13
- package/benchmark/auto-resolve/fixtures/F12-webhook-raw-body-signature/verifiers/replay-malformed-body.js +0 -64
- package/benchmark/auto-resolve/fixtures/F15-frozen-diff-race-review/NOTES.md +0 -104
- package/benchmark/auto-resolve/fixtures/F15-frozen-diff-race-review/expected.json +0 -46
- package/benchmark/auto-resolve/fixtures/F15-frozen-diff-race-review/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F15-frozen-diff-race-review/setup.sh +0 -336
- package/benchmark/auto-resolve/fixtures/F15-frozen-diff-race-review/spec.md +0 -51
- package/benchmark/auto-resolve/fixtures/F15-frozen-diff-race-review/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/NOTES.md +0 -38
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/expected.json +0 -64
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/setup.sh +0 -32
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/spec.md +0 -63
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/verifiers/exact-success.js +0 -54
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/verifiers/no-hardcoded-pricing.js +0 -47
- package/benchmark/auto-resolve/fixtures/F16-cli-quote-tax-rules/verifiers/stock-error.js +0 -45
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/NOTES.md +0 -57
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/expected-pair-plan-registry.json +0 -170
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/expected.json +0 -84
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/metadata.json +0 -21
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/pair-plan.sample-fail.json +0 -214
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/pair-plan.sample-pass.json +0 -223
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/setup.sh +0 -5
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/spec.md +0 -55
- package/benchmark/auto-resolve/fixtures/F2-cli-medium-subcommand/task.txt +0 -14
- package/benchmark/auto-resolve/fixtures/F21-cli-scheduler-priority/NOTES.md +0 -39
- package/benchmark/auto-resolve/fixtures/F21-cli-scheduler-priority/expected.json +0 -62
- package/benchmark/auto-resolve/fixtures/F21-cli-scheduler-priority/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F21-cli-scheduler-priority/setup.sh +0 -2
- package/benchmark/auto-resolve/fixtures/F21-cli-scheduler-priority/spec.md +0 -67
- package/benchmark/auto-resolve/fixtures/F21-cli-scheduler-priority/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F21-cli-scheduler-priority/verifiers/error-order.js +0 -55
- package/benchmark/auto-resolve/fixtures/F21-cli-scheduler-priority/verifiers/priority-blocked.js +0 -48
- package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/NOTES.md +0 -35
- package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/expected.json +0 -56
- package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/setup.sh +0 -2
- package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/spec.md +0 -64
- package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/verifiers/conflicting-duplicate.js +0 -34
- package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/verifiers/idempotent-close.js +0 -41
- package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/NOTES.md +0 -39
- package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/expected.json +0 -56
- package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/setup.sh +0 -2
- package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/spec.md +0 -76
- package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/verifiers/priority-rollback.js +0 -64
- package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/verifiers/single-warehouse-fefo.js +0 -66
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/NOTES.md +0 -40
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/expected.json +0 -66
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/setup.sh +0 -36
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/spec.md +0 -71
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/verifiers/catalog-source.js +0 -57
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/verifiers/exact-success.js +0 -63
- package/benchmark/auto-resolve/fixtures/F25-cli-cart-promotion-rules/verifiers/stock-error.js +0 -34
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/NOTES.md +0 -31
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/expected.json +0 -68
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/setup.sh +0 -17
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/spec.md +0 -68
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/verifiers/conflicting-duplicate.js +0 -29
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/verifiers/exact-payout.js +0 -58
- package/benchmark/auto-resolve/fixtures/F26-cli-payout-ledger-rules/verifiers/rules-source.js +0 -56
- package/benchmark/auto-resolve/fixtures/F3-backend-contract-risk/NOTES.md +0 -35
- package/benchmark/auto-resolve/fixtures/F3-backend-contract-risk/expected-pair-plan-registry.json +0 -162
- package/benchmark/auto-resolve/fixtures/F3-backend-contract-risk/expected.json +0 -67
- package/benchmark/auto-resolve/fixtures/F3-backend-contract-risk/metadata.json +0 -19
- package/benchmark/auto-resolve/fixtures/F3-backend-contract-risk/setup.sh +0 -4
- package/benchmark/auto-resolve/fixtures/F3-backend-contract-risk/spec.md +0 -55
- package/benchmark/auto-resolve/fixtures/F3-backend-contract-risk/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/F31-cli-seat-rebalance/NOTES.md +0 -34
- package/benchmark/auto-resolve/fixtures/F31-cli-seat-rebalance/expected.json +0 -57
- package/benchmark/auto-resolve/fixtures/F31-cli-seat-rebalance/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F31-cli-seat-rebalance/setup.sh +0 -2
- package/benchmark/auto-resolve/fixtures/F31-cli-seat-rebalance/spec.md +0 -67
- package/benchmark/auto-resolve/fixtures/F31-cli-seat-rebalance/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F31-cli-seat-rebalance/verifiers/duplicate-event-error.js +0 -35
- package/benchmark/auto-resolve/fixtures/F31-cli-seat-rebalance/verifiers/priority-transfer-rollback.js +0 -53
- package/benchmark/auto-resolve/fixtures/F32-cli-subscription-renewal/NOTES.md +0 -38
- package/benchmark/auto-resolve/fixtures/F32-cli-subscription-renewal/expected.json +0 -57
- package/benchmark/auto-resolve/fixtures/F32-cli-subscription-renewal/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F32-cli-subscription-renewal/setup.sh +0 -2
- package/benchmark/auto-resolve/fixtures/F32-cli-subscription-renewal/spec.md +0 -70
- package/benchmark/auto-resolve/fixtures/F32-cli-subscription-renewal/task.txt +0 -3
- package/benchmark/auto-resolve/fixtures/F32-cli-subscription-renewal/verifiers/duplicate-renewal-error.js +0 -42
- package/benchmark/auto-resolve/fixtures/F32-cli-subscription-renewal/verifiers/priority-credit-rollback.js +0 -70
- package/benchmark/auto-resolve/fixtures/F4-web-browser-design/NOTES.md +0 -47
- package/benchmark/auto-resolve/fixtures/F4-web-browser-design/expected.json +0 -57
- package/benchmark/auto-resolve/fixtures/F4-web-browser-design/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F4-web-browser-design/setup.sh +0 -6
- package/benchmark/auto-resolve/fixtures/F4-web-browser-design/spec.md +0 -48
- package/benchmark/auto-resolve/fixtures/F4-web-browser-design/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/F5-fix-loop-red-green/NOTES.md +0 -45
- package/benchmark/auto-resolve/fixtures/F5-fix-loop-red-green/expected.json +0 -65
- package/benchmark/auto-resolve/fixtures/F5-fix-loop-red-green/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F5-fix-loop-red-green/setup.sh +0 -55
- package/benchmark/auto-resolve/fixtures/F5-fix-loop-red-green/spec.md +0 -48
- package/benchmark/auto-resolve/fixtures/F5-fix-loop-red-green/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F6-dep-audit-native-module/NOTES.md +0 -43
- package/benchmark/auto-resolve/fixtures/F6-dep-audit-native-module/expected.json +0 -77
- package/benchmark/auto-resolve/fixtures/F6-dep-audit-native-module/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F6-dep-audit-native-module/setup.sh +0 -4
- package/benchmark/auto-resolve/fixtures/F6-dep-audit-native-module/spec.md +0 -48
- package/benchmark/auto-resolve/fixtures/F6-dep-audit-native-module/task.txt +0 -10
- package/benchmark/auto-resolve/fixtures/F7-out-of-scope-trap/NOTES.md +0 -57
- package/benchmark/auto-resolve/fixtures/F7-out-of-scope-trap/expected.json +0 -76
- package/benchmark/auto-resolve/fixtures/F7-out-of-scope-trap/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F7-out-of-scope-trap/setup.sh +0 -36
- package/benchmark/auto-resolve/fixtures/F7-out-of-scope-trap/spec.md +0 -45
- package/benchmark/auto-resolve/fixtures/F7-out-of-scope-trap/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/F8-known-limit-ambiguous/NOTES.md +0 -53
- package/benchmark/auto-resolve/fixtures/F8-known-limit-ambiguous/expected.json +0 -63
- package/benchmark/auto-resolve/fixtures/F8-known-limit-ambiguous/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F8-known-limit-ambiguous/setup.sh +0 -4
- package/benchmark/auto-resolve/fixtures/F8-known-limit-ambiguous/spec.md +0 -48
- package/benchmark/auto-resolve/fixtures/F8-known-limit-ambiguous/task.txt +0 -1
- package/benchmark/auto-resolve/fixtures/F9-e2e-ideate-to-resolve/NOTES.md +0 -105
- package/benchmark/auto-resolve/fixtures/F9-e2e-ideate-to-resolve/expected.json +0 -74
- package/benchmark/auto-resolve/fixtures/F9-e2e-ideate-to-resolve/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/F9-e2e-ideate-to-resolve/setup.sh +0 -28
- package/benchmark/auto-resolve/fixtures/F9-e2e-ideate-to-resolve/spec.md +0 -59
- package/benchmark/auto-resolve/fixtures/F9-e2e-ideate-to-resolve/task.txt +0 -5
- package/benchmark/auto-resolve/fixtures/SCHEMA.md +0 -188
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/NOTES.md +0 -37
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/RETIRED.md +0 -13
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/expected.json +0 -56
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/setup.sh +0 -18
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/spec.md +0 -69
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/task.txt +0 -7
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/verifiers/exact-proration.js +0 -48
- package/benchmark/auto-resolve/fixtures/retired/F27-cli-subscription-proration/verifiers/rules-source-and-conflict.js +0 -79
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/NOTES.md +0 -54
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/RETIRED.md +0 -7
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/expected.json +0 -67
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/setup.sh +0 -2
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/spec.md +0 -67
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/task.txt +0 -5
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/verifiers/policy-precedence.js +0 -72
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/verifiers/validation-and-immutability.js +0 -43
- package/benchmark/auto-resolve/fixtures/retired/F28-cli-return-authorization/verifiers/validation-boundary.js +0 -116
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/NOTES.md +0 -35
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/RETIRED.md +0 -12
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/expected.json +0 -58
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/setup.sh +0 -2
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/spec.md +0 -73
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/task.txt +0 -17
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/verifiers/mixed-idempotent-settlement.js +0 -53
- package/benchmark/auto-resolve/fixtures/retired/F30-cli-credit-hold-settlement/verifiers/rejection-boundaries.js +0 -74
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/NOTES.md +0 -28
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/RETIRED.md +0 -15
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/expected.json +0 -57
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/setup.sh +0 -50
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/spec.md +0 -59
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/verifiers/ordered-settlement.js +0 -88
- package/benchmark/auto-resolve/fixtures/retired/F34-cli-settle-inflight/verifiers/reject-invalid.js +0 -45
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/NOTES.md +0 -30
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/RETIRED.md +0 -16
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/expected.json +0 -64
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/setup.sh +0 -21
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/spec.md +0 -58
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/verifiers/dup-op.js +0 -54
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/verifiers/priority-apply.js +0 -82
- package/benchmark/auto-resolve/fixtures/retired/F35-cli-apply-journal/verifiers/rollback.js +0 -77
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/NOTES.md +0 -99
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/RETIRED.md +0 -18
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/expected.json +0 -64
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/setup.sh +0 -4
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/spec.md +0 -61
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/verifiers/correctness-small.js +0 -135
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/verifiers/scale.js +0 -100
- package/benchmark/auto-resolve/fixtures/retired/F36-cli-session-admission/verifiers/validation.js +0 -75
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/NOTES.md +0 -93
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/RETIRED.md +0 -16
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/expected.json +0 -65
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/setup.sh +0 -18
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/spec.md +0 -66
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/task.txt +0 -9
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/verifiers/correctness-small.js +0 -84
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/verifiers/scale.js +0 -184
- package/benchmark/auto-resolve/fixtures/retired/F37-cli-rule-lookup/verifiers/validation.js +0 -74
- package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/NOTES.md +0 -60
- package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/RETIRED.md +0 -29
- package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/expected.json +0 -73
- package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/metadata.json +0 -10
- package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/setup.sh +0 -28
- package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/spec.md +0 -58
- package/benchmark/auto-resolve/fixtures/retired/F9-e2e-ideate-to-preflight/task.txt +0 -5
- package/benchmark/auto-resolve/fixtures/test-repo/README.md +0 -12
- package/benchmark/auto-resolve/fixtures/test-repo/bin/cli.js +0 -63
- package/benchmark/auto-resolve/fixtures/test-repo/package-lock.json +0 -823
- package/benchmark/auto-resolve/fixtures/test-repo/package.json +0 -22
- package/benchmark/auto-resolve/fixtures/test-repo/playwright.config.js +0 -17
- package/benchmark/auto-resolve/fixtures/test-repo/server/index.js +0 -37
- package/benchmark/auto-resolve/fixtures/test-repo/tests/cli.test.js +0 -25
- package/benchmark/auto-resolve/fixtures/test-repo/tests/server.test.js +0 -58
- package/benchmark/auto-resolve/fixtures/test-repo/web/index.html +0 -37
- package/benchmark/auto-resolve/results/20260507-f10-f11-tier1-full-pipeline/headroom-gate.json +0 -25
- package/benchmark/auto-resolve/results/20260507-f10-f11-tier1-full-pipeline/headroom-gate.md +0 -10
- package/benchmark/auto-resolve/results/20260508-f22-exact-error-headroom/headroom-gate.json +0 -18
- package/benchmark/auto-resolve/results/20260508-f22-exact-error-headroom/headroom-gate.md +0 -9
- package/benchmark/auto-resolve/results/20260508-f26-headroom/headroom-gate.json +0 -18
- package/benchmark/auto-resolve/results/20260508-f26-headroom/headroom-gate.md +0 -9
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/F16-cli-quote-tax-rules/l2_risk_probes/result.json +0 -28
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/F23-cli-fulfillment-wave/l2_risk_probes/result.json +0 -28
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/F25-cli-cart-promotion-rules/l2_risk_probes/result.json +0 -30
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/full-pipeline-pair-gate.json +0 -82
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/full-pipeline-pair-gate.md +0 -18
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/headroom-gate.json +0 -46
- package/benchmark/auto-resolve/results/20260510-f16-f23-f25-combined-proof/headroom-gate.md +0 -17
- package/benchmark/auto-resolve/results/20260511-f12-webhook-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260511-f12-webhook-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260511-f15-concurrency-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260511-f15-concurrency-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/F21-cli-scheduler-priority/l2_risk_probes/result.json +0 -32
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/full-pipeline-pair-gate.json +0 -44
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/full-pipeline-pair-gate.md +0 -16
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/headroom-gate.json +0 -18
- package/benchmark/auto-resolve/results/20260511-f21-current-riskprobes-v1/headroom-gate.md +0 -9
- package/benchmark/auto-resolve/results/20260511-f3-http-error-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260511-f3-http-error-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260512-f2-medium-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260512-f2-medium-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260512-f31-seat-rebalance-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260512-f31-seat-rebalance-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260512-f32-subscription-renewal-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260512-f32-subscription-renewal-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260512-f4-web-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260512-f4-web-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260512-f5-fixloop-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260512-f5-fixloop-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260512-f6-checksum-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260512-f6-checksum-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260512-f7-scope-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260512-f7-scope-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/results/20260512-f9-e2e-headroom/headroom-gate.json +0 -28
- package/benchmark/auto-resolve/results/20260512-f9-e2e-headroom/headroom-gate.md +0 -15
- package/benchmark/auto-resolve/run-real-benchmark.md +0 -303
- package/benchmark/auto-resolve/scripts/audit-headroom-rejections.py +0 -441
- package/benchmark/auto-resolve/scripts/audit-pair-evidence.py +0 -1256
- package/benchmark/auto-resolve/scripts/build-pair-eligible-manifest.py +0 -306
- package/benchmark/auto-resolve/scripts/check-f9-artifacts.py +0 -268
- package/benchmark/auto-resolve/scripts/collect-swebench-predictions.py +0 -108
- package/benchmark/auto-resolve/scripts/compile-report.py +0 -493
- package/benchmark/auto-resolve/scripts/fetch-swebench-instances.py +0 -129
- package/benchmark/auto-resolve/scripts/frozen-verify-gate.py +0 -434
- package/benchmark/auto-resolve/scripts/full-pipeline-pair-gate.py +0 -612
- package/benchmark/auto-resolve/scripts/headroom-gate.py +0 -378
- package/benchmark/auto-resolve/scripts/iter-0033c-compare.py +0 -683
- package/benchmark/auto-resolve/scripts/iter-0033c-l1-summary.py +0 -97
- package/benchmark/auto-resolve/scripts/judge-opus-pass.sh +0 -542
- package/benchmark/auto-resolve/scripts/judge.sh +0 -565
- package/benchmark/auto-resolve/scripts/oracle-scope-tier-a.py +0 -256
- package/benchmark/auto-resolve/scripts/oracle-scope-tier-b.py +0 -287
- package/benchmark/auto-resolve/scripts/oracle-test-fidelity.py +0 -328
- package/benchmark/auto-resolve/scripts/pair-candidate-frontier.py +0 -469
- package/benchmark/auto-resolve/scripts/pair-plan-idgen.py +0 -401
- package/benchmark/auto-resolve/scripts/pair-plan-lint.py +0 -475
- package/benchmark/auto-resolve/scripts/pair-rejected-fixtures.sh +0 -103
- package/benchmark/auto-resolve/scripts/pair_evidence_contract.py +0 -270
- package/benchmark/auto-resolve/scripts/prepare-swebench-frozen-case.py +0 -273
- package/benchmark/auto-resolve/scripts/prepare-swebench-frozen-corpus.py +0 -148
- package/benchmark/auto-resolve/scripts/prepare-swebench-solver-worktree.py +0 -210
- package/benchmark/auto-resolve/scripts/recent-benchmark-summary.py +0 -232
- package/benchmark/auto-resolve/scripts/run-fixture.sh +0 -1024
- package/benchmark/auto-resolve/scripts/run-frozen-verify-pair.sh +0 -683
- package/benchmark/auto-resolve/scripts/run-full-pipeline-pair-candidate.sh +0 -458
- package/benchmark/auto-resolve/scripts/run-headroom-candidate.sh +0 -336
- package/benchmark/auto-resolve/scripts/run-iter-0033c.sh +0 -208
- package/benchmark/auto-resolve/scripts/run-suite.sh +0 -251
- package/benchmark/auto-resolve/scripts/run-swebench-frozen-corpus.sh +0 -310
- package/benchmark/auto-resolve/scripts/run-swebench-solver-batch.sh +0 -257
- package/benchmark/auto-resolve/scripts/ship-gate.py +0 -391
- package/benchmark/auto-resolve/scripts/solo-ceiling-avoidance.py +0 -53
- package/benchmark/auto-resolve/scripts/solo-headroom-hypothesis.py +0 -77
- package/benchmark/auto-resolve/scripts/swebench-frozen-matrix.py +0 -478
- package/benchmark/auto-resolve/scripts/test-audit-headroom-rejections.sh +0 -288
- package/benchmark/auto-resolve/scripts/test-audit-pair-evidence.sh +0 -1672
- package/benchmark/auto-resolve/scripts/test-benchmark-arg-parsing.sh +0 -1015
- package/benchmark/auto-resolve/scripts/test-build-pair-eligible-manifest.sh +0 -491
- package/benchmark/auto-resolve/scripts/test-check-f9-artifacts.sh +0 -91
- package/benchmark/auto-resolve/scripts/test-frozen-verify-gate.sh +0 -517
- package/benchmark/auto-resolve/scripts/test-full-pipeline-pair-gate.sh +0 -610
- package/benchmark/auto-resolve/scripts/test-headroom-gate.sh +0 -401
- package/benchmark/auto-resolve/scripts/test-iter-0033c-compare.sh +0 -525
- package/benchmark/auto-resolve/scripts/test-iter-0033c-l1-summary.sh +0 -254
- package/benchmark/auto-resolve/scripts/test-lint-fixtures.sh +0 -580
- package/benchmark/auto-resolve/scripts/test-pair-candidate-frontier.sh +0 -591
- package/benchmark/auto-resolve/scripts/test-run-full-pipeline-pair-candidate.sh +0 -497
- package/benchmark/auto-resolve/scripts/test-run-headroom-candidate.sh +0 -401
- package/benchmark/auto-resolve/scripts/test-run-swebench-solver-batch.sh +0 -111
- package/benchmark/auto-resolve/scripts/test-ship-gate.sh +0 -1189
- package/benchmark/auto-resolve/scripts/test-swebench-frozen-case.sh +0 -1221
- package/benchmark/auto-resolve/shadow-fixtures/S1-cli-lang-flag/NOTES.md +0 -28
- package/benchmark/auto-resolve/shadow-fixtures/S1-cli-lang-flag/expected.json +0 -63
- package/benchmark/auto-resolve/shadow-fixtures/S1-cli-lang-flag/metadata.json +0 -10
- package/benchmark/auto-resolve/shadow-fixtures/S1-cli-lang-flag/setup.sh +0 -3
- package/benchmark/auto-resolve/shadow-fixtures/S1-cli-lang-flag/spec.md +0 -47
- package/benchmark/auto-resolve/shadow-fixtures/S1-cli-lang-flag/task.txt +0 -1
- package/benchmark/auto-resolve/shadow-fixtures/S2-cli-inventory-reservation/NOTES.md +0 -34
- package/benchmark/auto-resolve/shadow-fixtures/S2-cli-inventory-reservation/expected.json +0 -53
- package/benchmark/auto-resolve/shadow-fixtures/S2-cli-inventory-reservation/metadata.json +0 -10
- package/benchmark/auto-resolve/shadow-fixtures/S2-cli-inventory-reservation/setup.sh +0 -3
- package/benchmark/auto-resolve/shadow-fixtures/S2-cli-inventory-reservation/spec.md +0 -50
- package/benchmark/auto-resolve/shadow-fixtures/S2-cli-inventory-reservation/task.txt +0 -1
- package/benchmark/auto-resolve/shadow-fixtures/S2-cli-inventory-reservation/verifiers/duplicate-order-error.js +0 -27
- package/benchmark/auto-resolve/shadow-fixtures/S2-cli-inventory-reservation/verifiers/priority-stock-reservation.js +0 -44
- package/benchmark/auto-resolve/shadow-fixtures/S3-cli-ticket-assignment/NOTES.md +0 -34
- package/benchmark/auto-resolve/shadow-fixtures/S3-cli-ticket-assignment/expected.json +0 -55
- package/benchmark/auto-resolve/shadow-fixtures/S3-cli-ticket-assignment/metadata.json +0 -10
- package/benchmark/auto-resolve/shadow-fixtures/S3-cli-ticket-assignment/setup.sh +0 -3
- package/benchmark/auto-resolve/shadow-fixtures/S3-cli-ticket-assignment/spec.md +0 -52
- package/benchmark/auto-resolve/shadow-fixtures/S3-cli-ticket-assignment/task.txt +0 -1
- package/benchmark/auto-resolve/shadow-fixtures/S3-cli-ticket-assignment/verifiers/duplicate-ticket-error.js +0 -29
- package/benchmark/auto-resolve/shadow-fixtures/S3-cli-ticket-assignment/verifiers/priority-agent-assignment.js +0 -48
- package/benchmark/auto-resolve/shadow-fixtures/S4-cli-return-routing/NOTES.md +0 -34
- package/benchmark/auto-resolve/shadow-fixtures/S4-cli-return-routing/expected.json +0 -55
- package/benchmark/auto-resolve/shadow-fixtures/S4-cli-return-routing/metadata.json +0 -10
- package/benchmark/auto-resolve/shadow-fixtures/S4-cli-return-routing/setup.sh +0 -3
- package/benchmark/auto-resolve/shadow-fixtures/S4-cli-return-routing/spec.md +0 -55
- package/benchmark/auto-resolve/shadow-fixtures/S4-cli-return-routing/task.txt +0 -1
- package/benchmark/auto-resolve/shadow-fixtures/S4-cli-return-routing/verifiers/duplicate-return-error.js +0 -43
- package/benchmark/auto-resolve/shadow-fixtures/S4-cli-return-routing/verifiers/priority-return-routing.js +0 -70
- package/benchmark/auto-resolve/shadow-fixtures/S5-cli-credit-grant-ledger/NOTES.md +0 -37
- package/benchmark/auto-resolve/shadow-fixtures/S5-cli-credit-grant-ledger/expected.json +0 -54
- package/benchmark/auto-resolve/shadow-fixtures/S5-cli-credit-grant-ledger/metadata.json +0 -10
- package/benchmark/auto-resolve/shadow-fixtures/S5-cli-credit-grant-ledger/setup.sh +0 -3
- package/benchmark/auto-resolve/shadow-fixtures/S5-cli-credit-grant-ledger/spec.md +0 -59
- package/benchmark/auto-resolve/shadow-fixtures/S5-cli-credit-grant-ledger/task.txt +0 -1
- package/benchmark/auto-resolve/shadow-fixtures/S5-cli-credit-grant-ledger/verifiers/credit-ledger-priority.js +0 -98
- package/benchmark/auto-resolve/shadow-fixtures/S5-cli-credit-grant-ledger/verifiers/duplicate-charge-error.js +0 -38
- package/benchmark/auto-resolve/shadow-fixtures/S6-cli-refund-window-ledger/NOTES.md +0 -36
- package/benchmark/auto-resolve/shadow-fixtures/S6-cli-refund-window-ledger/expected.json +0 -56
- package/benchmark/auto-resolve/shadow-fixtures/S6-cli-refund-window-ledger/metadata.json +0 -10
- package/benchmark/auto-resolve/shadow-fixtures/S6-cli-refund-window-ledger/setup.sh +0 -3
- package/benchmark/auto-resolve/shadow-fixtures/S6-cli-refund-window-ledger/spec.md +0 -59
- package/benchmark/auto-resolve/shadow-fixtures/S6-cli-refund-window-ledger/task.txt +0 -1
- package/benchmark/auto-resolve/shadow-fixtures/S6-cli-refund-window-ledger/verifiers/duplicate-refund-error.js +0 -41
- package/benchmark/auto-resolve/shadow-fixtures/S6-cli-refund-window-ledger/verifiers/priority-refund-ledger.js +0 -65
- package/config/skills/_shared/pair-plan-schema.md +0 -298
- package/scripts/lint-fixtures.sh +0 -352
- package/scripts/lint-shadow-fixtures.sh +0 -58
- package/scripts/lint-skills.sh +0 -4265
- package/scripts/test-owner-phases.py +0 -238
|
@@ -1,56 +0,0 @@
|
|
|
1
|
-
{
|
|
2
|
-
"verification_commands": [
|
|
3
|
-
{
|
|
4
|
-
"cmd": "node --test tests/cli.test.js",
|
|
5
|
-
"exit_code": 0,
|
|
6
|
-
"stdout_contains": [],
|
|
7
|
-
"stdout_not_contains": ["not ok "]
|
|
8
|
-
},
|
|
9
|
-
{
|
|
10
|
-
"cmd": "node \"$BENCH_FIXTURE_DIR/verifiers/idempotent-close.js\"",
|
|
11
|
-
"exit_code": 0,
|
|
12
|
-
"stdout_contains": ["\"ok\":true"],
|
|
13
|
-
"stdout_not_contains": [],
|
|
14
|
-
"contract_refs": [
|
|
15
|
-
"Exact duplicate transactions are idempotent: apply the first copy once and count later exact copies in `duplicates_ignored`.",
|
|
16
|
-
"Apply accepted unique transactions in chronological order by `occurred_at` ascending, then `id` ascending.",
|
|
17
|
-
"Negative closing balances are allowed and must be reported, not rejected."
|
|
18
|
-
]
|
|
19
|
-
},
|
|
20
|
-
{
|
|
21
|
-
"cmd": "node \"$BENCH_FIXTURE_DIR/verifiers/conflicting-duplicate.js\"",
|
|
22
|
-
"exit_code": 0,
|
|
23
|
-
"stdout_contains": ["\"ok\":true"],
|
|
24
|
-
"stdout_not_contains": [],
|
|
25
|
-
"contract_refs": [
|
|
26
|
-
"Conflicting duplicate transaction ids are invalid input. A duplicate id is conflicting when any field other than `id` differs from the first transaction with that id.",
|
|
27
|
-
"Invalid input exits `2`, writes exactly one JSON error object to stderr, and writes nothing to stdout.",
|
|
28
|
-
"Conflicting duplicate transaction ids use the exact stderr JSON shape `{ \"error\": \"conflicting_duplicate\", \"id\": string }`."
|
|
29
|
-
]
|
|
30
|
-
}
|
|
31
|
-
],
|
|
32
|
-
"forbidden_patterns": [
|
|
33
|
-
{
|
|
34
|
-
"pattern": "catch\\s*\\([^)]*\\)\\s*\\{[^}]*return\\s+(null|undefined|''|\\{\\})",
|
|
35
|
-
"description": "silent catch returning fallback in ledger path",
|
|
36
|
-
"files": ["bin/cli.js"],
|
|
37
|
-
"severity": "disqualifier"
|
|
38
|
-
},
|
|
39
|
-
{
|
|
40
|
-
"pattern": "catch\\s*\\([^)]*\\)\\s*\\{\\s*\\}",
|
|
41
|
-
"description": "empty catch block",
|
|
42
|
-
"files": ["bin/cli.js"],
|
|
43
|
-
"severity": "disqualifier"
|
|
44
|
-
},
|
|
45
|
-
{
|
|
46
|
-
"pattern": "/\\*\\s*eslint-disable",
|
|
47
|
-
"description": "eslint-disable without scoped justification",
|
|
48
|
-
"files": ["bin/cli.js"],
|
|
49
|
-
"severity": "disqualifier"
|
|
50
|
-
}
|
|
51
|
-
],
|
|
52
|
-
"required_files": ["bin/cli.js", "tests/cli.test.js"],
|
|
53
|
-
"forbidden_files": [],
|
|
54
|
-
"max_deps_added": 0,
|
|
55
|
-
"spec_output_files": ["bin/cli.js", "tests/cli.test.js"]
|
|
56
|
-
}
|
|
@@ -1,10 +0,0 @@
|
|
|
1
|
-
{
|
|
2
|
-
"id": "F22-cli-ledger-close",
|
|
3
|
-
"category": "high-risk",
|
|
4
|
-
"difficulty": "high",
|
|
5
|
-
"timeout_seconds": 1500,
|
|
6
|
-
"required_tools": ["node"],
|
|
7
|
-
"browser": false,
|
|
8
|
-
"deps_change_expected": false,
|
|
9
|
-
"intent": "Add a bench-cli ledger-close command that reads account opening balances and transaction events, applies chronological idempotent ledger rules, rejects conflicting duplicate transaction ids, and prints exact closing balances and audit counts."
|
|
10
|
-
}
|
|
@@ -1,64 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
id: "F22-cli-ledger-close"
|
|
3
|
-
title: "Ledger close command"
|
|
4
|
-
status: planned
|
|
5
|
-
complexity: high
|
|
6
|
-
depends-on: []
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
# F22 Ledger close command
|
|
10
|
-
|
|
11
|
-
## Context
|
|
12
|
-
|
|
13
|
-
Add a `bench-cli ledger-close --input <path>` command that reads account
|
|
14
|
-
opening balances and transaction events, applies chronological idempotent
|
|
15
|
-
ledger rules, rejects conflicting duplicate transaction ids, and prints exact
|
|
16
|
-
closing balances and audit counts.
|
|
17
|
-
|
|
18
|
-
The command is used in finance-style reconciliation. Determinism matters more
|
|
19
|
-
than convenience: every amount is integer cents, every accepted event is applied
|
|
20
|
-
once, and duplicate ids must not silently corrupt balances.
|
|
21
|
-
|
|
22
|
-
## Requirements
|
|
23
|
-
|
|
24
|
-
- [ ] `bench-cli ledger-close --input <path>` reads JSON shaped as `{ "accounts": Array<Account>, "transactions": Array<Transaction> }`.
|
|
25
|
-
- [ ] Each account has `{ "id": string, "currency": string, "opening_cents": number }`.
|
|
26
|
-
- [ ] Each transaction has `{ "id": string, "account": string, "currency": string, "kind": "debit" | "credit", "amount_cents": number, "occurred_at": string }`.
|
|
27
|
-
- [ ] Validate before applying balances: account ids are unique, transaction ids are non-empty strings, account references exist, currencies match the referenced account, `amount_cents` is a positive integer, and `kind` is either `debit` or `credit`.
|
|
28
|
-
- [ ] Conflicting duplicate transaction ids are invalid input. A duplicate id is conflicting when any field other than `id` differs from the first transaction with that id.
|
|
29
|
-
- [ ] Exact duplicate transactions are idempotent: apply the first copy once and count later exact copies in `duplicates_ignored`.
|
|
30
|
-
- [ ] Apply accepted unique transactions in chronological order by `occurred_at` ascending, then `id` ascending.
|
|
31
|
-
- [ ] A `debit` subtracts `amount_cents`; a `credit` adds `amount_cents`.
|
|
32
|
-
- [ ] Negative closing balances are allowed and must be reported, not rejected.
|
|
33
|
-
- [ ] Invalid input exits `2`, writes exactly one JSON error object to stderr, and writes nothing to stdout.
|
|
34
|
-
- [ ] Conflicting duplicate transaction ids use the exact stderr JSON shape `{ "error": "conflicting_duplicate", "id": string }`.
|
|
35
|
-
- [ ] On success, write exactly one JSON object to stdout and no stderr. Keys: `accounts`, `applied_count`, `duplicates_ignored`.
|
|
36
|
-
- [ ] Output `accounts` sorted by account id ascending. Each row has keys `id`, `currency`, `closing_cents`.
|
|
37
|
-
- [ ] `tests/cli.test.js` is updated. Existing tests still pass AND at least two ledger tests cover one success case and one invalid-input case.
|
|
38
|
-
|
|
39
|
-
## Constraints
|
|
40
|
-
|
|
41
|
-
- **No new npm dependencies.**
|
|
42
|
-
- **No floating-money output.** All amounts are integer cents.
|
|
43
|
-
- **No silent catches.** Invalid input and file-read failures must surface as JSON errors with exit `2`.
|
|
44
|
-
- **No mutation of the input file.**
|
|
45
|
-
- **No extra stdout/stderr text** on the success path; downstream tooling parses stdout as JSON.
|
|
46
|
-
- **Touch only `bin/cli.js` and `tests/cli.test.js`.**
|
|
47
|
-
|
|
48
|
-
## Out of Scope
|
|
49
|
-
|
|
50
|
-
- Exchange rates.
|
|
51
|
-
- Decimal currency parsing.
|
|
52
|
-
- Persistence or database writes.
|
|
53
|
-
- Account creation from transactions.
|
|
54
|
-
- Touching `server/`, `web/`, or `tests/server.test.js`.
|
|
55
|
-
|
|
56
|
-
## Verification
|
|
57
|
-
|
|
58
|
-
- `node --test tests/cli.test.js` exits 0.
|
|
59
|
-
- Exact duplicate transactions are counted in `duplicates_ignored` and applied once.
|
|
60
|
-
- Conflicting duplicate transaction ids exit `2`, write one JSON error to stderr, and write no stdout.
|
|
61
|
-
- Conflicting duplicate transaction ids use the exact stderr JSON shape `{ "error": "conflicting_duplicate", "id": string }`.
|
|
62
|
-
- Transactions are applied in chronological order by `occurred_at` ascending, then `id` ascending.
|
|
63
|
-
- Negative closing balances are allowed and appear in output.
|
|
64
|
-
- `git diff --stat` shows only `bin/cli.js` and `tests/cli.test.js` touched.
|
|
@@ -1,7 +0,0 @@
|
|
|
1
|
-
Add a `ledger-close` command to `bench-cli` so users can run `bench-cli ledger-close --input <path>` with a JSON file containing account opening balances and transaction events. It should apply chronological idempotent ledger rules, reject conflicting duplicate transaction ids, and print exact closing balances and audit counts.
|
|
2
|
-
|
|
3
|
-
The input has `accounts` with `id`, `currency`, and `opening_cents`, and `transactions` with `id`, `account`, `currency`, `kind`, `amount_cents`, and `occurred_at`. Validate the full input before applying balances. Exact duplicate transactions should be applied only once and counted in `duplicates_ignored`; duplicate ids with any conflicting field should make the whole input invalid. Apply accepted unique transactions by `occurred_at` ascending, then id ascending. Debits subtract, credits add, and negative closing balances are allowed.
|
|
4
|
-
|
|
5
|
-
On success, stdout must be exactly parseable JSON and stderr must be empty. The output has `accounts`, `applied_count`, and `duplicates_ignored`; accounts are sorted by id and each row has `id`, `currency`, and `closing_cents`. Invalid input or file-read failures should exit `2`, print exactly one JSON error object to stderr, and print nothing to stdout. Conflicting duplicate transaction ids must use the exact stderr JSON shape `{ "error": "conflicting_duplicate", "id": string }`.
|
|
6
|
-
|
|
7
|
-
Update `tests/cli.test.js` so existing tests still pass and add at least two ledger tests: one success case and one invalid-input case. No new npm dependencies. Only touch `bin/cli.js` and `tests/cli.test.js`.
|
package/benchmark/auto-resolve/fixtures/F22-cli-ledger-close/verifiers/conflicting-duplicate.js
DELETED
|
@@ -1,34 +0,0 @@
|
|
|
1
|
-
'use strict';
|
|
2
|
-
const assert = require('node:assert');
|
|
3
|
-
const { spawnSync } = require('node:child_process');
|
|
4
|
-
const fs = require('node:fs');
|
|
5
|
-
const os = require('node:os');
|
|
6
|
-
const path = require('node:path');
|
|
7
|
-
|
|
8
|
-
const work = process.env.BENCH_WORKDIR || process.cwd();
|
|
9
|
-
const cli = path.join(work, 'bin', 'cli.js');
|
|
10
|
-
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'f22-conflict-'));
|
|
11
|
-
const input = path.join(tmp, 'ledger.json');
|
|
12
|
-
|
|
13
|
-
fs.writeFileSync(input, JSON.stringify({
|
|
14
|
-
accounts: [
|
|
15
|
-
{ id: 'cash', currency: 'USD', opening_cents: 1000 }
|
|
16
|
-
],
|
|
17
|
-
transactions: [
|
|
18
|
-
{ id: 'same-id', account: 'cash', currency: 'USD', kind: 'credit', amount_cents: 100, occurred_at: '2026-01-01T00:00:00Z' },
|
|
19
|
-
{ id: 'same-id', account: 'cash', currency: 'USD', kind: 'credit', amount_cents: 101, occurred_at: '2026-01-01T00:00:00Z' }
|
|
20
|
-
]
|
|
21
|
-
}), 'utf8');
|
|
22
|
-
|
|
23
|
-
const proc = spawnSync('node', [cli, 'ledger-close', '--input', input], {
|
|
24
|
-
cwd: work,
|
|
25
|
-
encoding: 'utf8'
|
|
26
|
-
});
|
|
27
|
-
assert.strictEqual(proc.status, 2);
|
|
28
|
-
assert.strictEqual(proc.stdout, '');
|
|
29
|
-
const parsed = JSON.parse(proc.stderr);
|
|
30
|
-
assert.deepStrictEqual(parsed, {
|
|
31
|
-
error: 'conflicting_duplicate',
|
|
32
|
-
id: 'same-id'
|
|
33
|
-
});
|
|
34
|
-
console.log(JSON.stringify({ ok: true }));
|
|
@@ -1,41 +0,0 @@
|
|
|
1
|
-
'use strict';
|
|
2
|
-
const assert = require('node:assert');
|
|
3
|
-
const { execFileSync } = require('node:child_process');
|
|
4
|
-
const fs = require('node:fs');
|
|
5
|
-
const os = require('node:os');
|
|
6
|
-
const path = require('node:path');
|
|
7
|
-
|
|
8
|
-
const work = process.env.BENCH_WORKDIR || process.cwd();
|
|
9
|
-
const cli = path.join(work, 'bin', 'cli.js');
|
|
10
|
-
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'f22-ledger-'));
|
|
11
|
-
const input = path.join(tmp, 'ledger.json');
|
|
12
|
-
|
|
13
|
-
const tx = { id: 't-dup', account: 'cash', currency: 'USD', kind: 'debit', amount_cents: 1200, occurred_at: '2026-01-03T00:00:00Z' };
|
|
14
|
-
fs.writeFileSync(input, JSON.stringify({
|
|
15
|
-
accounts: [
|
|
16
|
-
{ id: 'cash', currency: 'USD', opening_cents: 1000 },
|
|
17
|
-
{ id: 'receivable', currency: 'USD', opening_cents: 200 }
|
|
18
|
-
],
|
|
19
|
-
transactions: [
|
|
20
|
-
{ id: 't-late', account: 'receivable', currency: 'USD', kind: 'credit', amount_cents: 50, occurred_at: '2026-01-05T00:00:00Z' },
|
|
21
|
-
tx,
|
|
22
|
-
{ ...tx },
|
|
23
|
-
{ id: 't-early', account: 'cash', currency: 'USD', kind: 'credit', amount_cents: 300, occurred_at: '2026-01-01T00:00:00Z' }
|
|
24
|
-
]
|
|
25
|
-
}), 'utf8');
|
|
26
|
-
|
|
27
|
-
const stdout = execFileSync('node', [cli, 'ledger-close', '--input', input], {
|
|
28
|
-
cwd: work,
|
|
29
|
-
encoding: 'utf8',
|
|
30
|
-
stdio: ['ignore', 'pipe', 'pipe']
|
|
31
|
-
});
|
|
32
|
-
const parsed = JSON.parse(stdout);
|
|
33
|
-
assert.deepStrictEqual(parsed, {
|
|
34
|
-
accounts: [
|
|
35
|
-
{ id: 'cash', currency: 'USD', closing_cents: 100 },
|
|
36
|
-
{ id: 'receivable', currency: 'USD', closing_cents: 250 }
|
|
37
|
-
],
|
|
38
|
-
applied_count: 3,
|
|
39
|
-
duplicates_ignored: 1
|
|
40
|
-
});
|
|
41
|
-
console.log(JSON.stringify({ ok: true }));
|
|
@@ -1,39 +0,0 @@
|
|
|
1
|
-
# F23 CLI fulfillment wave
|
|
2
|
-
|
|
3
|
-
## Failure mode
|
|
4
|
-
|
|
5
|
-
This fixture detects plausible allocators that pass simple order tests while
|
|
6
|
-
missing production-critical interactions: priority before input order,
|
|
7
|
-
all-or-nothing rollback, FEFO lot ordering, distance tie-breaks, and
|
|
8
|
-
single-warehouse constraints.
|
|
9
|
-
|
|
10
|
-
## Pipeline phase target
|
|
11
|
-
|
|
12
|
-
PLAN must separate validation, order processing, tentative allocation, rollback,
|
|
13
|
-
and final output sorting. IMPLEMENT must avoid partial mutation leaks. VERIFY
|
|
14
|
-
should construct counterexamples where a partial allocation looks locally valid
|
|
15
|
-
but corrupts later orders.
|
|
16
|
-
|
|
17
|
-
## Why existing fixtures do not cover it
|
|
18
|
-
|
|
19
|
-
F21 covers interval scheduling. F16 covers quote arithmetic. F22 was too easy
|
|
20
|
-
for bare in the first calibration run. This fixture targets allocation rollback
|
|
21
|
-
and inventory consumption across multiple dimensions.
|
|
22
|
-
|
|
23
|
-
## Measurement status
|
|
24
|
-
|
|
25
|
-
Pair evidence passed in `20260510-f16-f23-f25-combined-proof`: bare `33`,
|
|
26
|
-
solo_claude `66`, pair `97`, margin `+31`, wall `2.25x`,
|
|
27
|
-
arm `l2_risk_probes`, verdict `pair_evidence_passed`.
|
|
28
|
-
|
|
29
|
-
## Solo-headroom hypothesis
|
|
30
|
-
|
|
31
|
-
A capable solo_claude baseline is expected to miss all-or-nothing rollback after
|
|
32
|
-
a higher-priority order consumes stock first; observable command
|
|
33
|
-
`node "$BENCH_FIXTURE_DIR/verifiers/priority-rollback.js"` exposes the miss.
|
|
34
|
-
|
|
35
|
-
## Retirement
|
|
36
|
-
|
|
37
|
-
Retire or replace if both bare and solo consistently exceed the headroom
|
|
38
|
-
thresholds, or if a later logistics fixture provides the same rollback and
|
|
39
|
-
allocation-order signal with less wall time.
|
|
@@ -1,56 +0,0 @@
|
|
|
1
|
-
{
|
|
2
|
-
"verification_commands": [
|
|
3
|
-
{
|
|
4
|
-
"cmd": "node --test tests/cli.test.js",
|
|
5
|
-
"exit_code": 0,
|
|
6
|
-
"stdout_contains": [],
|
|
7
|
-
"stdout_not_contains": ["not ok "]
|
|
8
|
-
},
|
|
9
|
-
{
|
|
10
|
-
"cmd": "node \"$BENCH_FIXTURE_DIR/verifiers/priority-rollback.js\"",
|
|
11
|
-
"exit_code": 0,
|
|
12
|
-
"stdout_contains": ["\"ok\":true"],
|
|
13
|
-
"stdout_not_contains": [],
|
|
14
|
-
"contract_refs": [
|
|
15
|
-
"Process orders by `priority` descending, then `submitted_at` ascending, then `id` ascending.",
|
|
16
|
-
"An order is all-or-nothing. If any line cannot be fully allocated, reject the order and roll back all allocations tentatively made for that order.",
|
|
17
|
-
"Accepted allocations reduce stock for later orders in the same wave. Rejected orders do not reduce stock."
|
|
18
|
-
]
|
|
19
|
-
},
|
|
20
|
-
{
|
|
21
|
-
"cmd": "node \"$BENCH_FIXTURE_DIR/verifiers/single-warehouse-fefo.js\"",
|
|
22
|
-
"exit_code": 0,
|
|
23
|
-
"stdout_contains": ["\"ok\":true"],
|
|
24
|
-
"stdout_not_contains": [],
|
|
25
|
-
"contract_refs": [
|
|
26
|
-
"For a line where `single_warehouse` is `true`, the entire line quantity must come from one warehouse. It may use multiple lots inside that warehouse, but it must not split across warehouses.",
|
|
27
|
-
"Lot choice inside a warehouse is FEFO: `expires` ascending, then lot id ascending.",
|
|
28
|
-
"`remaining` is sorted by warehouse id, then sku, then expires, then lot."
|
|
29
|
-
]
|
|
30
|
-
}
|
|
31
|
-
],
|
|
32
|
-
"forbidden_patterns": [
|
|
33
|
-
{
|
|
34
|
-
"pattern": "catch\\s*\\([^)]*\\)\\s*\\{[^}]*return\\s+(null|undefined|''|\\{\\})",
|
|
35
|
-
"description": "silent catch returning fallback in fulfillment path",
|
|
36
|
-
"files": ["bin/cli.js"],
|
|
37
|
-
"severity": "disqualifier"
|
|
38
|
-
},
|
|
39
|
-
{
|
|
40
|
-
"pattern": "catch\\s*\\([^)]*\\)\\s*\\{\\s*\\}",
|
|
41
|
-
"description": "empty catch block",
|
|
42
|
-
"files": ["bin/cli.js"],
|
|
43
|
-
"severity": "disqualifier"
|
|
44
|
-
},
|
|
45
|
-
{
|
|
46
|
-
"pattern": "/\\*\\s*eslint-disable",
|
|
47
|
-
"description": "eslint-disable without scoped justification",
|
|
48
|
-
"files": ["bin/cli.js"],
|
|
49
|
-
"severity": "disqualifier"
|
|
50
|
-
}
|
|
51
|
-
],
|
|
52
|
-
"required_files": ["bin/cli.js", "tests/cli.test.js"],
|
|
53
|
-
"forbidden_files": [],
|
|
54
|
-
"max_deps_added": 0,
|
|
55
|
-
"spec_output_files": ["bin/cli.js", "tests/cli.test.js"]
|
|
56
|
-
}
|
|
@@ -1,10 +0,0 @@
|
|
|
1
|
-
{
|
|
2
|
-
"id": "F23-cli-fulfillment-wave",
|
|
3
|
-
"category": "high-risk",
|
|
4
|
-
"difficulty": "high",
|
|
5
|
-
"timeout_seconds": 1500,
|
|
6
|
-
"required_tools": ["node"],
|
|
7
|
-
"browser": false,
|
|
8
|
-
"deps_change_expected": false,
|
|
9
|
-
"intent": "Add a bench-cli fulfill-wave command that allocates prioritized orders across warehouses and inventory lots using all-or-nothing order rollback, FEFO lot choice, distance tie-breaks, and single-warehouse line constraints."
|
|
10
|
-
}
|
|
@@ -1,76 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
id: "F23-cli-fulfillment-wave"
|
|
3
|
-
title: "Fulfillment wave allocator"
|
|
4
|
-
status: planned
|
|
5
|
-
complexity: high
|
|
6
|
-
depends-on: []
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
# F23 Fulfillment wave allocator
|
|
10
|
-
|
|
11
|
-
## Context
|
|
12
|
-
|
|
13
|
-
Add a `bench-cli fulfill-wave --input <path>` command that allocates
|
|
14
|
-
prioritized orders across warehouses and inventory lots using all-or-nothing
|
|
15
|
-
order rollback, FEFO lot choice, distance tie-breaks, and single-warehouse line
|
|
16
|
-
constraints.
|
|
17
|
-
|
|
18
|
-
The allocator prepares a batch for warehouse pickers. A plausible partial plan
|
|
19
|
-
is worse than no plan: rejected orders must not consume stock, and accepted
|
|
20
|
-
orders must be deterministic.
|
|
21
|
-
|
|
22
|
-
## Requirements
|
|
23
|
-
|
|
24
|
-
- [ ] `bench-cli fulfill-wave --input <path>` reads JSON shaped as `{ "warehouses": Array<Warehouse>, "orders": Array<Order> }`.
|
|
25
|
-
- [ ] Each warehouse has `{ "id": string, "distance": number, "lots": Array<Lot> }`.
|
|
26
|
-
- [ ] Each lot has `{ "sku": string, "lot": string, "qty": number, "expires": "YYYY-MM-DD" }`.
|
|
27
|
-
- [ ] Each order has `{ "id": string, "priority": number, "submitted_at": string, "lines": Array<Line> }`.
|
|
28
|
-
- [ ] Each line has `{ "sku": string, "qty": number, "single_warehouse": boolean }`.
|
|
29
|
-
- [ ] Validate before allocation: ids are non-empty strings, quantities are positive integers, dates parse as ISO dates, priorities are numbers, and order ids are unique.
|
|
30
|
-
- [ ] Process orders by `priority` descending, then `submitted_at` ascending, then `id` ascending.
|
|
31
|
-
- [ ] An order is all-or-nothing. If any line cannot be fully allocated, reject the order and roll back all allocations tentatively made for that order.
|
|
32
|
-
- [ ] For a normal line where `single_warehouse` is `false`, allocation may split across warehouses and lots.
|
|
33
|
-
- [ ] For a line where `single_warehouse` is `true`, the entire line quantity must come from one warehouse. It may use multiple lots inside that warehouse, but it must not split across warehouses.
|
|
34
|
-
- [ ] Warehouse choice order is `distance` ascending, then warehouse id ascending.
|
|
35
|
-
- [ ] Lot choice inside a warehouse is FEFO: `expires` ascending, then lot id ascending.
|
|
36
|
-
- [ ] Accepted allocations reduce stock for later orders in the same wave. Rejected orders do not reduce stock.
|
|
37
|
-
- [ ] If an order cannot be fully allocated, reject with `{ "id": string, "reason": "insufficient_stock" }`.
|
|
38
|
-
- [ ] Invalid input exits `2`, writes exactly one JSON error object to stderr, and writes nothing to stdout.
|
|
39
|
-
- [ ] On success, write exactly one JSON object to stdout and no stderr. Keys: `accepted`, `rejected`, `remaining`.
|
|
40
|
-
- [ ] `accepted` is ordered by processing order. Each accepted row has keys `id`, `allocations`.
|
|
41
|
-
- [ ] Each allocation row has keys `sku`, `warehouse`, `lot`, `qty` and rows are ordered in the sequence stock was chosen.
|
|
42
|
-
- [ ] `rejected` is ordered by original input order. Each rejected row has keys `id`, `reason`.
|
|
43
|
-
- [ ] `remaining` is sorted by warehouse id, then sku, then expires, then lot. Each row has keys `warehouse`, `sku`, `lot`, `qty`, `expires`.
|
|
44
|
-
- [ ] `tests/cli.test.js` is updated. Existing tests still pass AND at least two fulfill-wave tests cover one accepted allocation and one rejected all-or-nothing order.
|
|
45
|
-
|
|
46
|
-
## Constraints
|
|
47
|
-
|
|
48
|
-
- **No new npm dependencies.**
|
|
49
|
-
- **No silent catches.** Invalid input and file-read failures must surface as JSON errors with exit `2`.
|
|
50
|
-
- **No mutation of the input file.**
|
|
51
|
-
- **No extra stdout/stderr text** on the success path; downstream tooling parses stdout as JSON.
|
|
52
|
-
- **Touch only `bin/cli.js` and `tests/cli.test.js`.**
|
|
53
|
-
|
|
54
|
-
## Out of Scope
|
|
55
|
-
|
|
56
|
-
- Carrier selection.
|
|
57
|
-
- Package dimensions.
|
|
58
|
-
- Backorders or partial order acceptance.
|
|
59
|
-
- Persistence beyond stdout.
|
|
60
|
-
- Touching `server/`, `web/`, or `tests/server.test.js`.
|
|
61
|
-
|
|
62
|
-
## Verification
|
|
63
|
-
|
|
64
|
-
- `node --test tests/cli.test.js` exits 0.
|
|
65
|
-
- A higher-priority order consumes stock before a lower-priority order even when the lower-priority order appears first in the input.
|
|
66
|
-
- A rejected order rolls back all tentative allocations and does not reduce stock available to later orders.
|
|
67
|
-
- `single_warehouse: true` does not split a line across warehouses even if total stock across warehouses is enough.
|
|
68
|
-
- Lot choice is FEFO by expiry date, then lot id.
|
|
69
|
-
- `remaining` is sorted by warehouse id, then sku, then expires, then lot.
|
|
70
|
-
- `git diff --stat` shows only `bin/cli.js` and `tests/cli.test.js` touched.
|
|
71
|
-
|
|
72
|
-
## Solo-headroom hypothesis
|
|
73
|
-
|
|
74
|
-
A capable solo_claude baseline is expected to miss all-or-nothing rollback after
|
|
75
|
-
a higher-priority order tentatively allocates FEFO lots; observable command
|
|
76
|
-
`node "$BENCH_FIXTURE_DIR/verifiers/priority-rollback.js"` exposes the miss.
|
|
@@ -1,7 +0,0 @@
|
|
|
1
|
-
Add a `fulfill-wave` command to `bench-cli` so users can run `bench-cli fulfill-wave --input <path>` with a JSON file containing warehouses, inventory lots, and prioritized orders. It should allocate orders across warehouses and lots using all-or-nothing order rollback, FEFO lot choice, distance tie-breaks, and single-warehouse line constraints.
|
|
2
|
-
|
|
3
|
-
Process orders by priority descending, then submitted_at ascending, then id ascending. Warehouse choice is distance ascending, then warehouse id ascending. Lot choice inside a warehouse is FEFO: expires ascending, then lot id ascending. Normal lines may split across warehouses and lots, but a line with `single_warehouse: true` must be fully allocated from one warehouse, though it may use multiple lots inside that warehouse. If any line in an order cannot be fully allocated, reject that order and roll back every tentative allocation for that order.
|
|
4
|
-
|
|
5
|
-
On success, stdout must be exactly parseable JSON and stderr must be empty. The output has `accepted`, `rejected`, and `remaining`. Invalid input or file-read failures should exit `2`, print exactly one JSON error object to stderr, and print nothing to stdout.
|
|
6
|
-
|
|
7
|
-
Update `tests/cli.test.js` so existing tests still pass and add at least two fulfill-wave tests: one accepted allocation and one rejected all-or-nothing order. No new npm dependencies. Only touch `bin/cli.js` and `tests/cli.test.js`.
|
package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/verifiers/priority-rollback.js
DELETED
|
@@ -1,64 +0,0 @@
|
|
|
1
|
-
'use strict';
|
|
2
|
-
const assert = require('node:assert');
|
|
3
|
-
const { execFileSync } = require('node:child_process');
|
|
4
|
-
const fs = require('node:fs');
|
|
5
|
-
const os = require('node:os');
|
|
6
|
-
const path = require('node:path');
|
|
7
|
-
|
|
8
|
-
const work = process.env.BENCH_WORKDIR || process.cwd();
|
|
9
|
-
const cli = path.join(work, 'bin', 'cli.js');
|
|
10
|
-
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'f23-wave-'));
|
|
11
|
-
const input = path.join(tmp, 'wave.json');
|
|
12
|
-
|
|
13
|
-
fs.writeFileSync(input, JSON.stringify({
|
|
14
|
-
warehouses: [
|
|
15
|
-
{
|
|
16
|
-
id: 'near',
|
|
17
|
-
distance: 1,
|
|
18
|
-
lots: [
|
|
19
|
-
{ sku: 'A', lot: 'n-old', qty: 2, expires: '2026-02-01' },
|
|
20
|
-
{ sku: 'B', lot: 'n-b', qty: 1, expires: '2026-02-01' }
|
|
21
|
-
]
|
|
22
|
-
},
|
|
23
|
-
{
|
|
24
|
-
id: 'far',
|
|
25
|
-
distance: 9,
|
|
26
|
-
lots: [
|
|
27
|
-
{ sku: 'A', lot: 'f-a', qty: 3, expires: '2026-01-15' }
|
|
28
|
-
]
|
|
29
|
-
}
|
|
30
|
-
],
|
|
31
|
-
orders: [
|
|
32
|
-
{ id: 'low-first', priority: 1, submitted_at: '2026-01-01T09:00:00Z', lines: [{ sku: 'A', qty: 2, single_warehouse: false }] },
|
|
33
|
-
{ id: 'bad-middle', priority: 5, submitted_at: '2026-01-01T09:01:00Z', lines: [{ sku: 'B', qty: 1, single_warehouse: false }, { sku: 'C', qty: 1, single_warehouse: false }] },
|
|
34
|
-
{ id: 'high-second', priority: 10, submitted_at: '2026-01-01T09:02:00Z', lines: [{ sku: 'A', qty: 5, single_warehouse: false }] },
|
|
35
|
-
{ id: 'after-bad', priority: 4, submitted_at: '2026-01-01T09:03:00Z', lines: [{ sku: 'B', qty: 1, single_warehouse: false }] }
|
|
36
|
-
]
|
|
37
|
-
}), 'utf8');
|
|
38
|
-
|
|
39
|
-
const stdout = execFileSync('node', [cli, 'fulfill-wave', '--input', input], {
|
|
40
|
-
cwd: work,
|
|
41
|
-
encoding: 'utf8',
|
|
42
|
-
stdio: ['ignore', 'pipe', 'pipe']
|
|
43
|
-
});
|
|
44
|
-
const parsed = JSON.parse(stdout);
|
|
45
|
-
assert.deepStrictEqual(parsed.accepted, [
|
|
46
|
-
{
|
|
47
|
-
id: 'high-second',
|
|
48
|
-
allocations: [
|
|
49
|
-
{ sku: 'A', warehouse: 'near', lot: 'n-old', qty: 2 },
|
|
50
|
-
{ sku: 'A', warehouse: 'far', lot: 'f-a', qty: 3 }
|
|
51
|
-
]
|
|
52
|
-
},
|
|
53
|
-
{
|
|
54
|
-
id: 'after-bad',
|
|
55
|
-
allocations: [
|
|
56
|
-
{ sku: 'B', warehouse: 'near', lot: 'n-b', qty: 1 }
|
|
57
|
-
]
|
|
58
|
-
}
|
|
59
|
-
]);
|
|
60
|
-
assert.deepStrictEqual(parsed.rejected, [
|
|
61
|
-
{ id: 'low-first', reason: 'insufficient_stock' },
|
|
62
|
-
{ id: 'bad-middle', reason: 'insufficient_stock' }
|
|
63
|
-
]);
|
|
64
|
-
console.log(JSON.stringify({ ok: true }));
|
package/benchmark/auto-resolve/fixtures/F23-cli-fulfillment-wave/verifiers/single-warehouse-fefo.js
DELETED
|
@@ -1,66 +0,0 @@
|
|
|
1
|
-
'use strict';
|
|
2
|
-
const assert = require('node:assert');
|
|
3
|
-
const { execFileSync } = require('node:child_process');
|
|
4
|
-
const fs = require('node:fs');
|
|
5
|
-
const os = require('node:os');
|
|
6
|
-
const path = require('node:path');
|
|
7
|
-
|
|
8
|
-
const work = process.env.BENCH_WORKDIR || process.cwd();
|
|
9
|
-
const cli = path.join(work, 'bin', 'cli.js');
|
|
10
|
-
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'f23-single-'));
|
|
11
|
-
const input = path.join(tmp, 'wave.json');
|
|
12
|
-
|
|
13
|
-
fs.writeFileSync(input, JSON.stringify({
|
|
14
|
-
warehouses: [
|
|
15
|
-
{
|
|
16
|
-
id: 'east',
|
|
17
|
-
distance: 2,
|
|
18
|
-
lots: [
|
|
19
|
-
{ sku: 'K', lot: 'e-late', qty: 2, expires: '2026-04-01' },
|
|
20
|
-
{ sku: 'K', lot: 'e-early', qty: 1, expires: '2026-03-01' }
|
|
21
|
-
]
|
|
22
|
-
},
|
|
23
|
-
{
|
|
24
|
-
id: 'west',
|
|
25
|
-
distance: 1,
|
|
26
|
-
lots: [
|
|
27
|
-
{ sku: 'K', lot: 'w-only', qty: 2, expires: '2026-02-01' },
|
|
28
|
-
{ sku: 'Z', lot: 'w-z', qty: 1, expires: '2026-02-01' }
|
|
29
|
-
]
|
|
30
|
-
}
|
|
31
|
-
],
|
|
32
|
-
orders: [
|
|
33
|
-
{ id: 'single-ok', priority: 10, submitted_at: '2026-01-01T09:00:00Z', lines: [{ sku: 'K', qty: 3, single_warehouse: true }] },
|
|
34
|
-
{ id: 'single-reject', priority: 9, submitted_at: '2026-01-01T09:01:00Z', lines: [{ sku: 'K', qty: 3, single_warehouse: true }] },
|
|
35
|
-
{ id: 'normal-z', priority: 8, submitted_at: '2026-01-01T09:02:00Z', lines: [{ sku: 'Z', qty: 1, single_warehouse: false }] }
|
|
36
|
-
]
|
|
37
|
-
}), 'utf8');
|
|
38
|
-
|
|
39
|
-
const stdout = execFileSync('node', [cli, 'fulfill-wave', '--input', input], {
|
|
40
|
-
cwd: work,
|
|
41
|
-
encoding: 'utf8',
|
|
42
|
-
stdio: ['ignore', 'pipe', 'pipe']
|
|
43
|
-
});
|
|
44
|
-
const parsed = JSON.parse(stdout);
|
|
45
|
-
assert.deepStrictEqual(parsed.accepted, [
|
|
46
|
-
{
|
|
47
|
-
id: 'single-ok',
|
|
48
|
-
allocations: [
|
|
49
|
-
{ sku: 'K', warehouse: 'east', lot: 'e-early', qty: 1 },
|
|
50
|
-
{ sku: 'K', warehouse: 'east', lot: 'e-late', qty: 2 }
|
|
51
|
-
]
|
|
52
|
-
},
|
|
53
|
-
{
|
|
54
|
-
id: 'normal-z',
|
|
55
|
-
allocations: [
|
|
56
|
-
{ sku: 'Z', warehouse: 'west', lot: 'w-z', qty: 1 }
|
|
57
|
-
]
|
|
58
|
-
}
|
|
59
|
-
]);
|
|
60
|
-
assert.deepStrictEqual(parsed.rejected, [
|
|
61
|
-
{ id: 'single-reject', reason: 'insufficient_stock' }
|
|
62
|
-
]);
|
|
63
|
-
assert.deepStrictEqual(parsed.remaining, [
|
|
64
|
-
{ warehouse: 'west', sku: 'K', lot: 'w-only', qty: 2, expires: '2026-02-01' }
|
|
65
|
-
]);
|
|
66
|
-
console.log(JSON.stringify({ ok: true }));
|
|
@@ -1,40 +0,0 @@
|
|
|
1
|
-
# F25 CLI cart promotion rules
|
|
2
|
-
|
|
3
|
-
## Failure mode
|
|
4
|
-
|
|
5
|
-
This fixture detects checkout implementations that pass shallow cart tests while
|
|
6
|
-
missing interaction invariants: duplicate-SKU aggregation, line-promotion order,
|
|
7
|
-
order-coupon order, taxable base selection, free-shipping threshold timing, and
|
|
8
|
-
externalized catalog data.
|
|
9
|
-
|
|
10
|
-
## Pipeline phase target
|
|
11
|
-
|
|
12
|
-
PLAN must preserve ordering between aggregation, validation, line promotions,
|
|
13
|
-
coupon discount, tax, and shipping. IMPLEMENT must keep all money values in
|
|
14
|
-
integer cents without new dependencies. VERIFY should execute adversarial cart
|
|
15
|
-
examples rather than only checking a happy path.
|
|
16
|
-
|
|
17
|
-
## Why existing fixtures do not cover it
|
|
18
|
-
|
|
19
|
-
F16 covers quote tax rules, but not multiple line-promotion types plus an order
|
|
20
|
-
coupon. F21/F23 cover scheduling/allocation, not checkout interaction ordering.
|
|
21
|
-
This fixture keeps the F16-style fair visible-contract shape while testing a
|
|
22
|
-
different checkout interaction.
|
|
23
|
-
|
|
24
|
-
## Measurement status
|
|
25
|
-
|
|
26
|
-
Pair evidence passed in `20260510-f16-f23-f25-combined-proof`: bare `25`,
|
|
27
|
-
solo_claude `75`, pair `99`, margin `+24`, wall `1.65x`,
|
|
28
|
-
arm `l2_risk_probes`, verdict `pair_evidence_passed`.
|
|
29
|
-
|
|
30
|
-
## Solo-headroom hypothesis
|
|
31
|
-
|
|
32
|
-
A capable solo_claude baseline is expected to miss the interaction between
|
|
33
|
-
duplicate aggregation, line promotions, tax base, coupon order, and shipping;
|
|
34
|
-
observable command `node "$BENCH_FIXTURE_DIR/verifiers/exact-success.js"` exposes the miss.
|
|
35
|
-
|
|
36
|
-
## Retirement
|
|
37
|
-
|
|
38
|
-
Retire or replace this fixture if either `bare` or `solo_claude` consistently
|
|
39
|
-
reaches ceiling, or if a later fixture covers the same promotion-order and
|
|
40
|
-
catalog-source failure mode with cleaner full-pipeline lift.
|