@1aboveio/engineering-skills 0.0.0-stage → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (233) hide show
  1. package/README.md +28 -2
  2. package/package.json +34 -4
  3. package/skills/better-review/LICENSE +3 -0
  4. package/skills/better-review/SKILL.md +249 -0
  5. package/skills/better-review/agents/openai.yaml +9 -0
  6. package/skills/better-review/references/domain-hazards.md +53 -0
  7. package/skills/better-review/references/passes.md +147 -0
  8. package/skills/better-review/references/result.md +151 -0
  9. package/skills/better-review/release-check.mjs +93 -0
  10. package/skills/better-review/scripts/pr-context.mjs +212 -0
  11. package/skills/better-review/scripts/pr-context.test.mjs +168 -0
  12. package/skills/better-review/scripts/rereview-scope.mjs +503 -0
  13. package/skills/better-review/scripts/rereview-scope.test.mjs +587 -0
  14. package/skills/better-review/scripts/scan-diff.mjs +365 -0
  15. package/skills/better-review/scripts/scan-diff.test.mjs +414 -0
  16. package/skills/better-review/scripts/vendor-sync.test.mjs +96 -0
  17. package/skills/e2e-test/LICENSE +3 -0
  18. package/skills/e2e-test/SKILL.md +98 -0
  19. package/skills/e2e-test/assets/ci-gates.cloudbuild.yaml +24 -0
  20. package/skills/e2e-test/references/authoring/auth-flows.md +159 -0
  21. package/skills/e2e-test/references/authoring/playwright-config.md +73 -0
  22. package/skills/e2e-test/references/authoring/playwright-patterns.md +144 -0
  23. package/skills/e2e-test/references/ci-integration.md +25 -0
  24. package/skills/e2e-test/references/ci-playwright-container.md +290 -0
  25. package/skills/e2e-test/references/presentation-sweep.md +104 -0
  26. package/skills/e2e-test/references/reviewing.md +41 -0
  27. package/skills/e2e-test/references/route-manifest.md +51 -0
  28. package/skills/e2e-test/release-check.mjs +93 -0
  29. package/skills/e2e-test/scripts/detect-routes-nextjs.mjs +200 -0
  30. package/skills/e2e-test/scripts/presentation-checks.mjs +171 -0
  31. package/skills/e2e-test/scripts/presentation-checks.test.mjs +106 -0
  32. package/skills/e2e-test/scripts/presentation-perceivability.mjs +179 -0
  33. package/skills/e2e-test/scripts/presentation-perceivability.test.mjs +117 -0
  34. package/skills/e2e-test/scripts/presentation-reachability.mjs +154 -0
  35. package/skills/e2e-test/scripts/presentation-reachability.test.mjs +69 -0
  36. package/skills/e2e-test/scripts/presentation-render-health.mjs +141 -0
  37. package/skills/e2e-test/scripts/presentation-render-health.test.mjs +98 -0
  38. package/skills/e2e-test/scripts/presentation-sweep.mjs +147 -0
  39. package/skills/e2e-test/scripts/presentation-sweep.test.mjs +71 -0
  40. package/skills/e2e-test/scripts/presentation-temporal.mjs +127 -0
  41. package/skills/e2e-test/scripts/presentation-temporal.test.mjs +59 -0
  42. package/skills/e2e-test/scripts/project-route-manifest.mjs +82 -0
  43. package/skills/e2e-test/scripts/project-route-manifest.test.mjs +76 -0
  44. package/skills/e2e-test/scripts/validate-manifest.mjs +106 -0
  45. package/skills/engineering-runtime/LICENSE +3 -0
  46. package/skills/engineering-runtime/SKILL.md +11 -0
  47. package/skills/engineering-runtime/agents/openai.yaml +6 -0
  48. package/skills/engineering-runtime/coherence/workflow.json +597 -0
  49. package/skills/engineering-runtime/engineering-runtime.test.mjs +68 -0
  50. package/skills/engineering-runtime/release-check.mjs +93 -0
  51. package/skills/engineering-runtime/scripts/exact-head-artifact.mjs +131 -0
  52. package/skills/engineering-runtime/scripts/head-check-set.mjs +398 -0
  53. package/skills/engineering-runtime/scripts/invocation-policy.mjs +187 -0
  54. package/skills/engineering-runtime/scripts/invocation-policy.test.mjs +95 -0
  55. package/skills/engineering-runtime/scripts/main-module.mjs +79 -0
  56. package/skills/engineering-runtime/scripts/main-module.test.mjs +288 -0
  57. package/skills/engineering-runtime/scripts/mergify-yaml.mjs +11 -0
  58. package/skills/engineering-runtime/scripts/package-lock.json +43 -0
  59. package/skills/engineering-runtime/scripts/package.json +10 -0
  60. package/skills/engineering-runtime/scripts/required-check-plan.mjs +223 -0
  61. package/skills/engineering-runtime/scripts/required-check-plan.test.mjs +96 -0
  62. package/skills/engineering-runtime/scripts/review-completion.mjs +56 -0
  63. package/skills/engineering-runtime/scripts/workflow-coherence.mjs +595 -0
  64. package/skills/engineering-runtime/scripts/workflow-policy.mjs +189 -0
  65. package/skills/engineering-runtime/scripts/yaml-mapping.mjs +18 -0
  66. package/skills/ensure-coverage/LICENSE +3 -0
  67. package/skills/ensure-coverage/SKILL.md +129 -0
  68. package/skills/ensure-coverage/references/breadth/coverage-ledger.md +92 -0
  69. package/skills/ensure-coverage/references/breadth/inventory-contract.md +85 -0
  70. package/skills/ensure-coverage/references/breadth/surface-baseline.md +44 -0
  71. package/skills/ensure-coverage/references/breadth/surface-discovery.md +16 -0
  72. package/skills/ensure-coverage/references/depth/characterization.md +68 -0
  73. package/skills/ensure-coverage/references/depth/coverage.config.example.json +19 -0
  74. package/skills/ensure-coverage/references/depth/grading.md +35 -0
  75. package/skills/ensure-coverage/references/depth/mock-policy.md +57 -0
  76. package/skills/ensure-coverage/references/depth/test-smells.md +39 -0
  77. package/skills/ensure-coverage/references/enforcement/ci-contract.md +44 -0
  78. package/skills/ensure-coverage/references/examples/coverage-ledger.md +110 -0
  79. package/skills/ensure-coverage/references/examples/refund-flow.md +35 -0
  80. package/skills/ensure-coverage/references/presentation/axis.md +64 -0
  81. package/skills/ensure-coverage/references/process/audit-mode.md +27 -0
  82. package/skills/ensure-coverage/references/process/output-template.md +139 -0
  83. package/skills/ensure-coverage/references/process/test-strategy.md +56 -0
  84. package/skills/ensure-coverage/release-check.mjs +93 -0
  85. package/skills/ensure-coverage/schemas/coverage-config.schema.json +67 -0
  86. package/skills/ensure-coverage/schemas/coverage-file.schema.json +140 -0
  87. package/skills/ensure-coverage/scripts/README.md +99 -0
  88. package/skills/ensure-coverage/scripts/adapters/nextjs-inventory.mjs +182 -0
  89. package/skills/ensure-coverage/scripts/adapters/nextjs-inventory.test.mjs +96 -0
  90. package/skills/ensure-coverage/scripts/check-quarantine-expiry.mjs +105 -0
  91. package/skills/ensure-coverage/scripts/check-quarantine-expiry.test.mjs +47 -0
  92. package/skills/ensure-coverage/scripts/ci-audit.mjs +137 -0
  93. package/skills/ensure-coverage/scripts/ci-audit.test.mjs +86 -0
  94. package/skills/ensure-coverage/scripts/coverage-ledger.mjs +674 -0
  95. package/skills/ensure-coverage/scripts/coverage-ledger.test.mjs +447 -0
  96. package/skills/ensure-coverage/scripts/design-parity.mjs +591 -0
  97. package/skills/ensure-coverage/scripts/design-parity.test.mjs +552 -0
  98. package/skills/ensure-coverage/scripts/lint-tests.mjs +282 -0
  99. package/skills/ensure-coverage/scripts/lint-tests.test.mjs +289 -0
  100. package/skills/ensure-coverage/scripts/mock-policy-config.mjs +176 -0
  101. package/skills/ensure-coverage/scripts/mock-policy-config.test.mjs +119 -0
  102. package/skills/ensure-coverage/scripts/package.json +19 -0
  103. package/skills/golden-path/LICENSE +3 -0
  104. package/skills/golden-path/SKILL.md +230 -0
  105. package/skills/golden-path/agents/openai.yaml +3 -0
  106. package/skills/golden-path/assets/golden-paths.example.json +57 -0
  107. package/skills/golden-path/references/contract.md +239 -0
  108. package/skills/golden-path/references/migration.md +111 -0
  109. package/skills/golden-path/release-check.mjs +93 -0
  110. package/skills/golden-path/scripts/golden-path-core.mjs +1779 -0
  111. package/skills/golden-path/scripts/golden-path.mjs +6 -0
  112. package/skills/golden-path/scripts/golden-path.test.mjs +1457 -0
  113. package/skills/harness-runtime/LICENSE +3 -0
  114. package/skills/harness-runtime/SKILL.md +19 -0
  115. package/skills/harness-runtime/agents/openai.yaml +6 -0
  116. package/skills/harness-runtime/bin/discover-models.mjs +4 -0
  117. package/skills/harness-runtime/bin/model-catalog.mjs +4 -0
  118. package/skills/harness-runtime/contracts.md +15 -0
  119. package/skills/harness-runtime/discover-models.mjs +392 -0
  120. package/skills/harness-runtime/discover-models.test.mjs +363 -0
  121. package/skills/harness-runtime/fixtures/native-question-schemas.json +33 -0
  122. package/skills/harness-runtime/fixtures/question-responses.json +54 -0
  123. package/skills/harness-runtime/harness-runtime.test.mjs +967 -0
  124. package/skills/harness-runtime/index.mjs +767 -0
  125. package/skills/harness-runtime/model-catalog-seed.test.mjs +829 -0
  126. package/skills/harness-runtime/model-catalog.mjs +787 -0
  127. package/skills/harness-runtime/model-catalog.test.mjs +366 -0
  128. package/skills/harness-runtime/native-question-contracts.md +37 -0
  129. package/skills/harness-runtime/references/model-catalog-seed.json +408 -0
  130. package/skills/harness-runtime/references/model-catalog.md +57 -0
  131. package/skills/harness-runtime/release-check.mjs +93 -0
  132. package/skills/resolve-issues/LICENSE +3 -0
  133. package/skills/resolve-issues/SKILL.md +250 -0
  134. package/skills/resolve-issues/agents/openai.yaml +9 -0
  135. package/skills/resolve-issues/plugin-hooks.json +17 -0
  136. package/skills/resolve-issues/references/canary.md +28 -0
  137. package/skills/resolve-issues/references/ci.md +105 -0
  138. package/skills/resolve-issues/references/done.md +25 -0
  139. package/skills/resolve-issues/references/dropbox.md +90 -0
  140. package/skills/resolve-issues/references/expire.md +318 -0
  141. package/skills/resolve-issues/references/exploration.md +202 -0
  142. package/skills/resolve-issues/references/implementation.md +303 -0
  143. package/skills/resolve-issues/references/preflight.md +445 -0
  144. package/skills/resolve-issues/references/review.md +228 -0
  145. package/skills/resolve-issues/references/shared-modules.md +44 -0
  146. package/skills/resolve-issues/references/task-plan.md +264 -0
  147. package/skills/resolve-issues/release-check.mjs +93 -0
  148. package/skills/resolve-issues/scripts/cleanup-runs.mjs +364 -0
  149. package/skills/resolve-issues/scripts/cleanup-runs.test.mjs +280 -0
  150. package/skills/resolve-issues/scripts/discover-models.mjs +9 -0
  151. package/skills/resolve-issues/scripts/dropbox.mjs +473 -0
  152. package/skills/resolve-issues/scripts/dropbox.test.mjs +346 -0
  153. package/skills/resolve-issues/scripts/findings-reuse.mjs +98 -0
  154. package/skills/resolve-issues/scripts/findings-reuse.test.mjs +88 -0
  155. package/skills/resolve-issues/scripts/github-reviews.mjs +384 -0
  156. package/skills/resolve-issues/scripts/github-reviews.test.mjs +264 -0
  157. package/skills/resolve-issues/scripts/install-hooks.mjs +135 -0
  158. package/skills/resolve-issues/scripts/install-hooks.test.mjs +79 -0
  159. package/skills/resolve-issues/scripts/issue-tree.mjs +162 -0
  160. package/skills/resolve-issues/scripts/issue-tree.test.mjs +107 -0
  161. package/skills/resolve-issues/scripts/manual-invocation.test.mjs +14 -0
  162. package/skills/resolve-issues/scripts/model-catalog.mjs +9 -0
  163. package/skills/resolve-issues/scripts/next-actions-capacity.test.mjs +149 -0
  164. package/skills/resolve-issues/scripts/next-actions.mjs +856 -0
  165. package/skills/resolve-issues/scripts/next-actions.test.mjs +915 -0
  166. package/skills/resolve-issues/scripts/plan-sync.mjs +269 -0
  167. package/skills/resolve-issues/scripts/plan-sync.test.mjs +156 -0
  168. package/skills/resolve-issues/scripts/preflight-models.mjs +736 -0
  169. package/skills/resolve-issues/scripts/preflight-models.test.mjs +448 -0
  170. package/skills/resolve-issues/scripts/ready-check.mjs +185 -0
  171. package/skills/resolve-issues/scripts/ready-check.test.mjs +217 -0
  172. package/skills/resolve-issues/scripts/rehydrate.mjs +98 -0
  173. package/skills/resolve-issues/scripts/rehydrate.test.mjs +91 -0
  174. package/skills/resolve-issues/scripts/review-verdict.mjs +509 -0
  175. package/skills/resolve-issues/scripts/review-verdict.test.mjs +1060 -0
  176. package/skills/resolve-issues/scripts/size-check.mjs +492 -0
  177. package/skills/resolve-issues/scripts/size-check.test.mjs +458 -0
  178. package/skills/resolve-release/LICENSE +3 -0
  179. package/skills/resolve-release/SKILL.md +215 -0
  180. package/skills/resolve-release/agents/openai.yaml +8 -0
  181. package/skills/resolve-release/flows/cloud-run-canary.md +13 -0
  182. package/skills/resolve-release/flows/cloud-run.md +7 -0
  183. package/skills/resolve-release/flows/root-tag-npm.md +13 -0
  184. package/skills/resolve-release/flows/workspace-npm.md +22 -0
  185. package/skills/resolve-release/references/canary.md +51 -0
  186. package/skills/resolve-release/references/candidate.md +41 -0
  187. package/skills/resolve-release/references/cut.md +131 -0
  188. package/skills/resolve-release/references/dark-launch.md +66 -0
  189. package/skills/resolve-release/references/flows.md +150 -0
  190. package/skills/resolve-release/references/preflight.md +154 -0
  191. package/skills/resolve-release/references/prepared-metadata.md +260 -0
  192. package/skills/resolve-release/references/promotion.md +82 -0
  193. package/skills/resolve-release/references/publish.md +232 -0
  194. package/skills/resolve-release/references/workspace-npm.md +103 -0
  195. package/skills/resolve-release/release-check.mjs +93 -0
  196. package/skills/resolve-release/scripts/apply.mjs +212 -0
  197. package/skills/resolve-release/scripts/apply.test.mjs +99 -0
  198. package/skills/resolve-release/scripts/dark-flags.mjs +134 -0
  199. package/skills/resolve-release/scripts/dark-flags.test.mjs +100 -0
  200. package/skills/resolve-release/scripts/flow.mjs +500 -0
  201. package/skills/resolve-release/scripts/flow.test.mjs +340 -0
  202. package/skills/resolve-release/scripts/green-head.mjs +231 -0
  203. package/skills/resolve-release/scripts/green-head.test.mjs +101 -0
  204. package/skills/resolve-release/scripts/npm-version-absence.mjs +138 -0
  205. package/skills/resolve-release/scripts/npm-version-absence.test.mjs +426 -0
  206. package/skills/resolve-release/scripts/plan-npm.test.mjs +659 -0
  207. package/skills/resolve-release/scripts/plan-toml-boundary.test.mjs +317 -0
  208. package/skills/resolve-release/scripts/plan-uv.test.mjs +291 -0
  209. package/skills/resolve-release/scripts/plan.mjs +806 -0
  210. package/skills/resolve-release/scripts/plan.test.mjs +428 -0
  211. package/skills/rush-issues/LICENSE +3 -0
  212. package/skills/rush-issues/SKILL.md +58 -0
  213. package/skills/rush-issues/agents/openai.yaml +8 -0
  214. package/skills/rush-issues/rehydrate.mjs +10 -0
  215. package/skills/rush-issues/release-check.mjs +93 -0
  216. package/skills/rush-release/LICENSE +3 -0
  217. package/skills/rush-release/SKILL.md +35 -0
  218. package/skills/rush-release/agents/openai.yaml +8 -0
  219. package/skills/rush-release/release-check.mjs +93 -0
  220. package/skills/test-quality/LICENSE +3 -0
  221. package/skills/test-quality/SKILL.md +79 -0
  222. package/skills/test-quality/references/audit.md +80 -0
  223. package/skills/test-quality/references/lint.md +24 -0
  224. package/skills/test-quality/references/principles.md +143 -0
  225. package/skills/test-quality/references/testing-conventions.md +9 -0
  226. package/skills/test-quality/release-check.mjs +93 -0
  227. package/skills/test-quality/scripts/lint-tests.mjs +574 -0
  228. package/skills/test-quality/scripts/lint-tests.test.mjs +591 -0
  229. package/skills/test-quality/scripts/mock-policy-config.mjs +176 -0
  230. package/skills/test-quality/scripts/mock-policy-config.test.mjs +119 -0
  231. package/skills/test-quality/scripts/persist-conventions-core.mjs +117 -0
  232. package/skills/test-quality/scripts/persist-conventions.mjs +17 -0
  233. package/skills/test-quality/scripts/persist-conventions.test.mjs +104 -0
@@ -0,0 +1,366 @@
1
+ // Tests for model-catalog — pure core + CLI round-trip (exit codes), no network.
2
+ import { test } from 'node:test'
3
+ import assert from 'node:assert/strict'
4
+ import fs from 'node:fs'
5
+ import os from 'node:os'
6
+ import path from 'node:path'
7
+ import { fileURLToPath } from 'node:url'
8
+ import { spawnSync } from 'node:child_process'
9
+
10
+ import {
11
+ PRUNE_AFTER_DAYS, syncCatalog, bankEntry, readCatalog, writeCatalog, pathForHarness,
12
+ EFFORTS, DEFAULT_EFFORT, TIERS, SLOT_ELIGIBLE_TIERS, normalizeTier, operatingPoints, tierAt, toTiers,
13
+ } from './model-catalog.mjs'
14
+
15
+ const SCRIPT = fileURLToPath(new URL('./bin/model-catalog.mjs', import.meta.url))
16
+ const tmpFile = () => path.join(fs.mkdtempSync(path.join(os.tmpdir(), 'model-catalog-test-')), 'catalog.json')
17
+
18
+ const entry = (id, lastSeenAt, confirmedAt = '2026-01-01') => ({ id, family: 'x', tier: 'frontier', confirmedAt, ...(lastSeenAt ? { lastSeenAt } : {}) })
19
+
20
+ // ---- syncCatalog: the rule-4 predicate in one place ----
21
+
22
+ // @covers package:harness-runtime
23
+ // @level unit
24
+ test('sync bumps lastSeenAt for seen ids and keeps fresh unseen entries', () => {
25
+ const { catalog, bumped, pruned } = syncCatalog(
26
+ { models: [entry('a', '2026-05-01'), entry('b', '2026-06-01')] },
27
+ ['a'],
28
+ '2026-07-13',
29
+ )
30
+ assert.deepEqual(bumped, ['a'])
31
+ assert.deepEqual(pruned, [])
32
+ assert.equal(catalog.models.find((m) => m.id === 'a').lastSeenAt, '2026-07-13')
33
+ assert.equal(catalog.models.find((m) => m.id === 'b').lastSeenAt, '2026-06-01') // unseen: untouched
34
+ })
35
+
36
+ test('sync prunes strictly after PRUNE_AFTER_DAYS — day 60 kept, day 61 pruned', () => {
37
+ // 2026-05-14 is exactly 60 days before 2026-07-13; 2026-05-13 is 61.
38
+ const { catalog, pruned } = syncCatalog(
39
+ { models: [entry('kept', '2026-05-14'), entry('gone', '2026-05-13')] },
40
+ [],
41
+ '2026-07-13',
42
+ )
43
+ assert.equal(PRUNE_AFTER_DAYS, 60)
44
+ assert.deepEqual(pruned, ['gone'])
45
+ assert.deepEqual(catalog.models.map((m) => m.id), ['kept'])
46
+ })
47
+
48
+ test('sync falls back to confirmedAt when lastSeenAt is absent (pre-field entries)', () => {
49
+ const { pruned } = syncCatalog(
50
+ { models: [entry('old', null, '2026-01-01'), entry('new', null, '2026-07-01')] },
51
+ [],
52
+ '2026-07-13',
53
+ )
54
+ assert.deepEqual(pruned, ['old'])
55
+ })
56
+
57
+ test('sync bumps before pruning — a stale entry rediscovered this run survives', () => {
58
+ const { catalog, pruned } = syncCatalog(
59
+ { models: [entry('back', '2026-01-01')] },
60
+ ['back'],
61
+ '2026-07-13',
62
+ )
63
+ assert.deepEqual(pruned, [])
64
+ assert.equal(catalog.models[0].lastSeenAt, '2026-07-13')
65
+ })
66
+
67
+ // ---- bankEntry: rule-2 classification writes ----
68
+
69
+ test('bank appends a new entry stamped with confirmedAt + lastSeenAt', () => {
70
+ const c = bankEntry({ models: [] }, { id: 'm', family: 'openai', tier: 'balanced-coder' }, '2026-07-13')
71
+ assert.deepEqual(c.models, [{ id: 'm', family: 'openai', tiers: { high: 'balanced-coder' }, confirmedAt: '2026-07-13', lastSeenAt: '2026-07-13' }])
72
+ })
73
+
74
+ test('bank overwrites an existing id (human correction) and refreshes confirmedAt', () => {
75
+ const c = bankEntry(
76
+ { models: [entry('m', '2026-06-01')] },
77
+ { id: 'm', family: 'openai', tier: 'balanced-coder' },
78
+ '2026-07-13',
79
+ )
80
+ assert.equal(c.models.length, 1)
81
+ assert.deepEqual(c.models[0].tiers, { high: 'balanced-coder' })
82
+ assert.equal(c.models[0].confirmedAt, '2026-07-13')
83
+ })
84
+
85
+ test('bank rejects entries missing id/family/tier or with an unknown tier', () => {
86
+ assert.throws(() => bankEntry({ models: [] }, { family: 'x', tier: 'frontier' }, '2026-07-13'))
87
+ assert.throws(() => bankEntry({ models: [] }, { id: 'm', tier: 'frontier' }, '2026-07-13'))
88
+ assert.throws(() => bankEntry({ models: [] }, { id: 'm', family: 'x' }, '2026-07-13')) // neither tier nor tiers
89
+ assert.throws(() => bankEntry({ models: [] }, { id: 'm', family: 'x', tier: 'huge' }, '2026-07-13'))
90
+ })
91
+
92
+ // ---- operating points: tier = f(model, effort), banked once (2026-07-19) ----
93
+
94
+ test('bank accepts a tiers map — one model, two operating points', () => {
95
+ const c = bankEntry(
96
+ { models: [] },
97
+ { id: 'gpt-5.6-terra', family: 'openai', tiers: { high: 'balanced-coder', max: 'deep-reasoner' } },
98
+ '2026-07-19',
99
+ )
100
+ assert.deepEqual(c.models[0].tiers, { high: 'balanced-coder', max: 'deep-reasoner' })
101
+ assert.equal(c.models[0].tier, undefined) // the legacy single-point field never persists
102
+ })
103
+
104
+ test('a single tier UPSERTS one point; a full tiers map REPLACES all of them', () => {
105
+ let c = bankEntry({ models: [] }, { id: 'terra', family: 'openai', tier: 'balanced-coder' }, '2026-07-19')
106
+ assert.deepEqual(c.models[0].tiers, { high: 'balanced-coder' }) // no effort -> the high point
107
+
108
+ // measured later, confirmed on its own: the existing point survives
109
+ c = bankEntry(c, { id: 'terra', family: 'openai', tier: 'deep-reasoner', effort: 'max' }, '2026-07-19')
110
+ assert.deepEqual(c.models[0].tiers, { high: 'balanced-coder', max: 'deep-reasoner' })
111
+
112
+ // a full map is the human restating the whole picture — it wins outright
113
+ c = bankEntry(c, { id: 'terra', family: 'openai', tiers: { max: 'deep-reasoner' } }, '2026-07-19')
114
+ assert.deepEqual(c.models[0].tiers, { max: 'deep-reasoner' })
115
+ })
116
+
117
+ test('an upsert onto a LEGACY-shape entry leaves no stale tier/effort behind', () => {
118
+ // Regression: the strip ran on the incoming entry only, so `existing.tier` was
119
+ // spread back in with nothing to shadow it — the stored entry then claimed a tier
120
+ // that could flatly contradict its own map. Reads went through toTiers and looked
121
+ // fine; the FILE was wrong, and the file is what a human and a cross-harness
122
+ // lookup (catalog rule 5) actually read.
123
+ const legacy = { models: [{ id: 'm', family: 'f', tier: 'mid-tier', effort: 'high', confirmedAt: '2026-06-01' }] }
124
+
125
+ const upserted = bankEntry(legacy, { id: 'm', family: 'f', tier: 'deep-reasoner', effort: 'max' }, '2026-07-19')
126
+ assert.equal(upserted.models[0].tier, undefined)
127
+ assert.equal(upserted.models[0].effort, undefined)
128
+ assert.deepEqual(upserted.models[0].tiers, { high: 'balanced-coder', max: 'deep-reasoner' }) // legacy point preserved
129
+
130
+ const replaced = bankEntry({ models: [{ id: 'm', family: 'f', tier: 'frontier' }] }, { id: 'm', family: 'f', tiers: { high: 'balanced-coder' } }, '2026-07-19')
131
+ assert.equal(replaced.models[0].tier, undefined) // never "frontier" beside a balanced-coder map
132
+ assert.deepEqual(replaced.models[0].tiers, { high: 'balanced-coder' })
133
+ })
134
+
135
+ test('bank refuses an entry carrying no operating points', () => {
136
+ // `tiers: {}` passes a truthiness guard but classifies nothing: tierAt returns
137
+ // null at every effort and no later sync ever flags it. Check the POINTS.
138
+ for (const bad of [{}, [], null]) {
139
+ assert.throws(
140
+ () => bankEntry({ models: [] }, { id: 'a', family: 'f', tiers: bad }, '2026-07-19'),
141
+ /tiers|operating points/,
142
+ `tiers: ${JSON.stringify(bad)} must be rejected`,
143
+ )
144
+ }
145
+ assert.throws(() => bankEntry({ models: [] }, { id: 'a', family: 'f', tiers: 'frontier' }, '2026-07-19'), /tiers/)
146
+ })
147
+
148
+ test('sync does not emit an undefined tier for an entry with no classification at all', () => {
149
+ const { migrated } = syncCatalog({ models: [{ id: 'nocap', family: 'f', lastSeenAt: '2026-07-18' }] }, [], '2026-07-19')
150
+ assert.ok(!migrated.some((m) => m.includes('undefined')), `migration line leaked undefined: ${migrated}`)
151
+ })
152
+
153
+ test('bank rejects a point below high — cheaper means a cheaper model, not a throttled one', () => {
154
+ for (const bad of ['low', 'medium', 'none']) {
155
+ assert.throws(
156
+ () => bankEntry({ models: [] }, { id: 'm', family: 'x', tiers: { [bad]: 'balanced-coder' } }, '2026-07-19'),
157
+ /effort/,
158
+ `effort "${bad}" must be rejected`,
159
+ )
160
+ }
161
+ assert.throws(() => bankEntry({ models: [] }, { id: 'm', family: 'x', tiers: { high: 'huge' } }, '2026-07-19'), /tier/)
162
+ assert.deepEqual(EFFORTS, ['high', 'xhigh', 'max'])
163
+ assert.equal(DEFAULT_EFFORT, 'high')
164
+ })
165
+
166
+ test('operatingPoints lists every banked point, strongest tier first', () => {
167
+ const catalog = {
168
+ models: [
169
+ { id: 'terra', family: 'openai', tiers: { high: 'balanced-coder', max: 'deep-reasoner' } },
170
+ { id: 'sonnet', family: 'anthropic', tiers: { high: 'balanced-coder' } },
171
+ ],
172
+ }
173
+ // slotEligible rides along on every point (wired to SLOT_ELIGIBLE_TIERS in 2026-07-20's
174
+ // audit fix); both bands here are eligible, the scoped-coder case lives in the seed tests.
175
+ assert.deepEqual(operatingPoints(catalog, 'terra'), [
176
+ { effort: 'max', tier: 'deep-reasoner', slotEligible: true },
177
+ { effort: 'high', tier: 'balanced-coder', slotEligible: true },
178
+ ])
179
+ assert.deepEqual(operatingPoints(catalog, 'sonnet'), [{ effort: 'high', tier: 'balanced-coder', slotEligible: true }])
180
+ assert.deepEqual(operatingPoints(catalog, 'unknown'), [])
181
+ assert.deepEqual(operatingPoints(null, 'terra'), []) // absent catalog is normal
182
+ })
183
+
184
+ test('tierAt answers what a model reaches at a given effort, null where nothing is banked', () => {
185
+ const catalog = { models: [{ id: 'terra', family: 'openai', tiers: { high: 'balanced-coder', max: 'deep-reasoner' } }] }
186
+ assert.equal(tierAt(catalog, 'terra', 'max'), 'deep-reasoner')
187
+ assert.equal(tierAt(catalog, 'terra', 'high'), 'balanced-coder')
188
+ assert.equal(tierAt(catalog, 'terra', 'xhigh'), null) // not measured -> not claimed
189
+ assert.equal(tierAt(catalog, 'nope', 'high'), null)
190
+ })
191
+
192
+ test('bank accepts deep-reasoner (the band below the current top end)', () => {
193
+ const c = bankEntry({ models: [] }, { id: 'gpt-5.5', family: 'openai', tier: 'deep-reasoner' }, '2026-07-13')
194
+ assert.deepEqual(c.models[0].tiers, { high: 'deep-reasoner' })
195
+ })
196
+
197
+ // ---- tier renames (2026-07-19): prior-frontier -> deep-reasoner, mid-tier -> balanced-coder ----
198
+
199
+ test('bank accepts either legacy tier name and stores the current one', () => {
200
+ const a = bankEntry({ models: [] }, { id: 'opus', family: 'anthropic', tier: 'prior-frontier' }, '2026-07-19')
201
+ assert.deepEqual(a.models[0].tiers, { high: 'deep-reasoner' })
202
+ const b = bankEntry({ models: [] }, { id: 'sonnet', family: 'anthropic', tier: 'mid-tier', effort: 'max' }, '2026-07-19')
203
+ assert.deepEqual(b.models[0].tiers, { max: 'balanced-coder' })
204
+ assert.deepEqual(toTiers({ tiers: { high: 'prior-frontier' } }), { high: 'deep-reasoner' }) // legacy names inside a map too
205
+ assert.deepEqual(TIERS, ['frontier', 'deep-reasoner', 'balanced-coder', 'scoped-coder'])
206
+ for (const t of TIERS) assert.equal(normalizeTier(t), t) // current names pass through
207
+ })
208
+
209
+ test('sync migrates legacy tier names AND the single-point shape, reporting both', () => {
210
+ const { catalog, migrated } = syncCatalog(
211
+ {
212
+ models: [
213
+ { id: 'opus', family: 'anthropic', tier: 'prior-frontier', lastSeenAt: '2026-07-18' },
214
+ { id: 'terra', family: 'openai', tier: 'mid-tier', effort: 'max', lastSeenAt: '2026-07-18' },
215
+ { id: 'fable', family: 'anthropic', tiers: { high: 'frontier' }, lastSeenAt: '2026-07-18' },
216
+ ],
217
+ },
218
+ ['opus'],
219
+ '2026-07-19',
220
+ )
221
+ // opus: renamed value + reshaped; terra: renamed value at its banked effort;
222
+ // fable: already current -> absent from the report entirely.
223
+ assert.deepEqual(migrated, [
224
+ 'opus: {high:prior-frontier} -> {high:deep-reasoner}',
225
+ 'terra: {max:mid-tier} -> {max:balanced-coder}',
226
+ ])
227
+ assert.deepEqual(catalog.models.find((m) => m.id === 'opus').tiers, { high: 'deep-reasoner' })
228
+ assert.deepEqual(catalog.models.find((m) => m.id === 'terra').tiers, { max: 'balanced-coder' })
229
+ assert.deepEqual(catalog.models.find((m) => m.id === 'fable').tiers, { high: 'frontier' })
230
+ for (const m of catalog.models) {
231
+ assert.equal(m.tier, undefined) // the legacy fields are gone, not merely shadowed
232
+ assert.equal(m.effort, undefined)
233
+ }
234
+ })
235
+
236
+ test('sync reshapes a current-name single-point entry without claiming a rename', () => {
237
+ const { migrated } = syncCatalog(
238
+ { models: [{ id: 'sonnet', family: 'anthropic', tier: 'balanced-coder', lastSeenAt: '2026-07-18' }] },
239
+ [],
240
+ '2026-07-19',
241
+ )
242
+ assert.deepEqual(migrated, ['sonnet: tier/effort -> tiers map {high:balanced-coder}'])
243
+ })
244
+
245
+ test('sync migrates a legacy entry even when it never saw it', () => {
246
+ const { migrated } = syncCatalog(
247
+ { models: [{ id: 'unseen', family: 'x', tier: 'prior-frontier', lastSeenAt: '2026-07-01' }] },
248
+ [], // migration is not gated on discovery — it is a rename, not an observation
249
+ '2026-07-19',
250
+ )
251
+ assert.deepEqual(migrated, ['unseen: {high:prior-frontier} -> {high:deep-reasoner}'])
252
+ })
253
+
254
+ // ---- CLI round-trip ----
255
+
256
+ function run(args, input) {
257
+ return spawnSync('node', [SCRIPT, ...args], { encoding: 'utf8', input })
258
+ }
259
+
260
+ test('CLI: read/sync exit 3 on an absent catalog (absent is normal)', () => {
261
+ const file = tmpFile()
262
+ assert.equal(run(['read', '--file', file]).status, 3)
263
+ assert.equal(run(['sync', '--file', file, '--seen', 'a']).status, 3)
264
+ })
265
+
266
+ // @covers package:harness-runtime
267
+ // @level integration
268
+ test('CLI: bank creates the file; sync bumps + prunes and reports both', () => {
269
+ const file = tmpFile()
270
+ assert.equal(run(['bank', JSON.stringify({ id: 'a', family: 'x', tier: 'frontier' }), '--file', file, '--today', '2026-01-01']).status, 0)
271
+ assert.equal(run(['bank', JSON.stringify({ id: 'b', family: 'y', tier: 'balanced-coder' }), '--file', file, '--today', '2026-07-01']).status, 0)
272
+
273
+ const sync = run(['sync', '--file', file, '--seen', 'b', '--today', '2026-07-13'])
274
+ assert.equal(sync.status, 0)
275
+ assert.deepEqual(JSON.parse(sync.stdout), { bumped: ['b'], pruned: ['a'], migrated: [] }) // a: unseen since Jan — pruned
276
+
277
+ const kept = readCatalog(file)
278
+ assert.deepEqual(kept.models.map((m) => m.id), ['b'])
279
+ assert.equal(kept.models[0].lastSeenAt, '2026-07-13')
280
+ })
281
+
282
+ test('CLI: list aliases read, help exits 0', () => {
283
+ const file = tmpFile()
284
+ assert.equal(run(['bank', JSON.stringify({ id: 'a', family: 'x', tier: 'frontier' }), '--file', file, '--today', '2026-07-13']).status, 0)
285
+ const list = run(['list', '--file', file])
286
+ assert.equal(list.status, 0)
287
+ assert.equal(JSON.parse(list.stdout).models[0].id, 'a')
288
+ assert.equal(run(['help']).status, 0)
289
+ assert.equal(run(['bogus']).status, 2)
290
+ })
291
+
292
+ test('CLI: bank via --stdin round-trips through read', () => {
293
+ const file = tmpFile()
294
+ assert.equal(run(['bank', '--stdin', '--file', file, '--today', '2026-07-13'], JSON.stringify({ id: 's', family: 'x', tier: 'frontier' })).status, 0)
295
+ const read = run(['read', '--file', file])
296
+ assert.equal(read.status, 0)
297
+ assert.equal(JSON.parse(read.stdout).models[0].id, 's')
298
+ })
299
+
300
+ // ---- per-harness catalogs (one file per harness, never shared) ----
301
+
302
+ // @covers package:harness-runtime
303
+ // @level unit
304
+ test('pathForHarness maps the known harnesses and returns null for unknown ones', () => {
305
+ assert.equal(pathForHarness('claude-code', { HOME: '/h' }), '/h/.claude/model-catalog.json')
306
+ assert.equal(pathForHarness('codex', { HOME: '/h' }), '/h/.codex/model-catalog.json')
307
+ assert.equal(pathForHarness('pi', { HOME: '/h' }), '/h/.pi/agent/model-catalog.json')
308
+ assert.equal(pathForHarness('bogus', { HOME: '/h' }), null)
309
+ })
310
+
311
+ test('CLI: bank --harness stamps the file; a write from another harness is refused', () => {
312
+ const file = tmpFile()
313
+ assert.equal(run(['bank', JSON.stringify({ id: 'a', family: 'x', tier: 'frontier' }), '--file', file, '--harness', 'codex', '--today', '2026-07-13']).status, 0)
314
+ assert.equal(readCatalog(file).harness, 'codex')
315
+
316
+ // same file, different harness → refused (exit 2), catalog untouched
317
+ assert.equal(run(['sync', '--file', file, '--harness', 'pi', '--seen', 'a', '--today', '2026-07-13']).status, 2)
318
+ assert.equal(run(['bank', JSON.stringify({ id: 'b', family: 'y', tier: 'balanced-coder' }), '--file', file, '--harness', 'pi', '--today', '2026-07-13']).status, 2)
319
+ assert.deepEqual(readCatalog(file).models.map((m) => m.id), ['a'])
320
+
321
+ // matching harness → fine
322
+ assert.equal(run(['sync', '--file', file, '--harness', 'codex', '--seen', 'a', '--today', '2026-07-13']).status, 0)
323
+ })
324
+
325
+ test('CLI: bank --harness stamps a legacy (pre-field) file at its next confirmed write', () => {
326
+ const file = tmpFile()
327
+ writeCatalog(file, { models: [entry('a', '2026-07-01')] }) // no harness stamp
328
+ assert.equal(run(['bank', JSON.stringify({ id: 'b', family: 'y', tier: 'balanced-coder' }), '--file', file, '--harness', 'pi', '--today', '2026-07-13']).status, 0)
329
+ assert.equal(readCatalog(file).harness, 'pi')
330
+ })
331
+
332
+ test('CLI: --harness with no location mapping errors, path --harness resolves', () => {
333
+ assert.equal(run(['read', '--harness', 'bogus']).status, 2)
334
+ const p = run(['path', '--harness', 'pi'])
335
+ assert.equal(p.status, 0)
336
+ assert.ok(p.stdout.trim().endsWith(path.join('.pi', 'agent', 'model-catalog.json')))
337
+ })
338
+
339
+ test('writeCatalog is atomic-ish: file parses after write, tmp file removed', () => {
340
+ const file = tmpFile()
341
+ writeCatalog(file, { models: [] })
342
+ assert.deepEqual(JSON.parse(fs.readFileSync(file, 'utf8')), { models: [] })
343
+ assert.ok(!fs.existsSync(`${file}.tmp`))
344
+ })
345
+
346
+ test('scoped-coder is a bankable classification but never a slot candidate', () => {
347
+ // The band exists so a model that keeps turning up in discovery, but sits below the
348
+ // implementer floor, can be answered ONCE instead of being tagged
349
+ // "unconfirmed — please classify" every run. It must bank like any other tier...
350
+ const c = bankEntry({ models: [] }, { id: 'gemini-3.5-flash', family: 'google', tier: 'scoped-coder' }, '2026-07-20')
351
+ assert.deepEqual(c.models[0].tiers, { high: 'scoped-coder' })
352
+
353
+ // ...and must never reach a slot.
354
+ assert.ok(TIERS.includes('scoped-coder'), 'bankable')
355
+ assert.ok(!SLOT_ELIGIBLE_TIERS.includes('scoped-coder'), 'not proposable for implementer/reviewer')
356
+ assert.deepEqual(SLOT_ELIGIBLE_TIERS, ['frontier', 'deep-reasoner', 'balanced-coder'])
357
+ })
358
+
359
+ test('scoped-coder sorts below every slot-eligible band', () => {
360
+ // operatingPoints sorts by TIERS.indexOf, so a mis-ordered enum would float a
361
+ // scoped-coder above a balanced-coder in the proposal block and invite a bad pick.
362
+ assert.equal(TIERS.indexOf('scoped-coder'), TIERS.length - 1)
363
+ for (const t of SLOT_ELIGIBLE_TIERS) {
364
+ assert.ok(TIERS.indexOf(t) < TIERS.indexOf('scoped-coder'), `${t} must outrank scoped-coder`)
365
+ }
366
+ })
@@ -0,0 +1,37 @@
1
+ # Native structured question contracts
2
+
3
+ `planStructuredQuestions(capabilities, questions)` is pure. It selects an adapter only when that tool name is present in the current-turn nativeTools input; harness names, static configuration, installed extensions, and these fixtures never establish availability.
4
+
5
+ ## Pinned schemas
6
+
7
+ The machine-readable summary is `fixtures/native-question-schemas.json`; native response examples are in `fixtures/question-responses.json`.
8
+
9
+ | Adapter | Pinned source | Native constraints |
10
+ | --- | --- | --- |
11
+ | Pi `ask_user_question` | `@juicesharp/rpiv-ask-user-question@2.4.0`, `tool/types.ts` and `docs/tool-schema.md` | 1-4 questions, 2-4 options, header <=16 characters, option label <=60 characters, multi-select supported |
12
+ | Codex `request_user_input` | `openai/codex@3d4d253f8f4a812c595cd59e2c114c2c3696c293`, `request_user_input_spec.rs` | 1-3 questions, 2-3 options, header <=12 characters, stable snake-case ids, first recommended option suffixed with `(Recommended)`, no advertised multi-select field |
13
+ | Claude Code `AskUserQuestion` | `anthropics/claude-code@54cc51a08a5d3900e5abd02ad75a2ce46f3f008c`, `interactive-commands.md` | 1-4 questions, 2-4 options, header <=12 characters, multi-select supported |
14
+
15
+ When more than one supported name is advertised, priority is Pi, Codex, then Claude Code. Input order does not alter selection. Questions and recommended options preserve logical order. Codex treats the first recommendation id as the recommended choice, appends its required ` (Recommended)` label suffix in both the payload and response mapping, and leaves later recommendation ids as ordering hints. Capacity overflow produces consecutive calls and `batching: { required: true, reason: "adapter-capacity", callCount }`.
16
+
17
+ Before emitting a native call, every adapter validates the final labels after adapter-specific transformation. Each emitted label must identify exactly one neutral option id. A collision raises `invalid-question` with reason `native-option-label-collision`, the adapter, question id, native label, and colliding option ids. This includes a Codex recommendation such as `Queue` colliding with a literal `Queue (Recommended)`. Pi and Claude Code currently preserve option labels unchanged; Pi rejects labels longer than 60 characters rather than truncating them. Pi also reserves the native UI labels `Other`, `Type something.`, and `Next`; a neutral option using any of them fails with `reserved-option-label` before invocation.
18
+
19
+ Claude Code serializes multi-select answers by joining selected labels with `, `. The planner enumerates that bounded native format and rejects any string that could identify more than one option-id sequence with reason `native-answer-format-collision`. For example, separate `A` and `B` options cannot coexist in a multi-select question with a literal `A, B` option.
20
+
21
+ ## Unsupported features
22
+
23
+ | Adapter | Multi-select | Secret input |
24
+ | --- | --- | --- |
25
+ | Pi | Native | `UnsupportedQuestionError` |
26
+ | Codex | `UnsupportedQuestionError` | `UnsupportedQuestionError` |
27
+ | Claude Code | Native | `UnsupportedQuestionError` |
28
+
29
+ Secret input always fails closed because none of the advertised model-call schemas above exposes a secret field. The Codex app-server protocol has a separate `isSecret` transport property, but the advertised `request_user_input` tool input does not; the planner does not emit unadvertised fields.
30
+
31
+ Without a supported tool, one optionless, non-secret, single-select question returns a `concise-free-text` fallback decision. Textual multiple-choice emulation is forbidden: a logical question with options returns `textual-multiple-choice-forbidden` and no prose-choice payload. Multiple free-text questions, secret input, and multi-select never silently degrade.
32
+
33
+ ## Responses
34
+
35
+ `normalizeQuestionResponses(plan, responses)` accepts one native response per planned call and returns frozen `{ questionId: answers[] }` data. Recognized native labels, including Codex's emitted recommendation suffix, map back to neutral option ids; unrecognized custom text remains literal. Pi's explicit `custom` answer kind always remains literal, even when its text equals an option label. Codex responses must contain exactly one answer because its advertised input is single-select. Pi multi-select arrays and Claude Code's comma-joined multi-select response normalize to the same option-id array. A malformed plan containing duplicate native labels raises `invalid-answers` with reason `ambiguous-option-label`; normalization never chooses the first matching id.
36
+
37
+ Pi exposes cancellation in `details.cancelled`. Codex interruption and Claude Code cancellation do not provide a successful answer payload, so callers pass the explicit `{ cancelled: true }` sentinel shown in the fixtures. Cancellation of any native call cancels the whole logical gate and returns `{}`; partial gate answers are not acted on. A non-cancelled native call must answer every question in that call. Unknown or missing ids/prompts, empty answers, invalid option labels, response-count mismatches, and malformed native shapes raise `invalid-answers`.