@ikon85/agent-workflow-kit 0.41.1 → 0.42.1

package/README.md

@@ -469,6 +469,16 @@ the old way. Decision record:
 
 ## Release notes
 
+### 0.42.1
+
+- changed: `docs/agents/skills/orchestrate-wave.md`
+
+### 0.42.0
+
+- added: `docs/adr/0010-model-roster-replaces-the-optimization-dial.md`
+- added: `docs/research/agent-task-taxonomy-benchmark-coverage.md`
+- changed: `docs/adr/0006-routing-knowledge-access-and-policy-are-separate.md`
+
 ### 0.41.1
 
 - changed: `README.md`

package/agent-workflow-kit.package.json

@@ -1,5 +1,5 @@
 {
-  "kitVersion": "0.41.1",
+  "kitVersion": "0.42.1",
   "files": [
     {
       "path": ".agents/skills/ask-matt/SKILL.md",

package/docs/adr/0006-routing-knowledge-access-and-policy-are-separate.md

@@ -1,6 +1,10 @@
 # Routing knowledge, access, and policy are separate
 
-Status: accepted (2026-07-22, Program #197)
+Status: accepted (2026-07-22, Program #197) — the optimization-dial clause is
+superseded by
+[ADR-0010](./0010-model-roster-replaces-the-optimization-dial.md)
+(items 3 and the "Model preferences and optimization overrides" sentence below;
+everything else stands)
 
 Planning cannot safely persist a concrete recommended model. Provider catalogs
 change, model names age, effort behavior differs by model, agent surfaces expose

package/docs/adr/0010-model-roster-replaces-the-optimization-dial.md

@@ -0,0 +1,87 @@
+# A Model roster replaces the optimization dial
+
+Status: accepted (2026-07-27, Program #287, Welle 18 / #294) — supersedes the
+Routing-policy clause of
+[ADR-0006](./0006-routing-knowledge-access-and-policy-are-separate.md)
+
+ADR-0006 made "optimization goals and optional advanced overrides" inputs of
+the Routing policy, and setup asked the user to pick Balanced, Quality, or
+Cost. That dial cannot be made to work. The goal is always the right model for
+the job, and a cheap-but-weak model can cost more per completed task than a
+strong one — so a Cost dial does not express a preference the resolver can act
+on without knowing cost per *completed* task, which no benchmark owner
+publishes. Meanwhile the dial was never read: `readRoutingProfile` has zero
+callers and `resolveRoute` always returns `routing-infrastructure-missing`.
+
+We decided the user control is a **Model roster** — a positive list of
+model-and-effort pairs the user authorizes — plus three **Standard routes**
+that decide when no decisive evidence covers the resolved Routing intent.
+
+1. **The optimization dial is removed everywhere it appears, not only from the
+   Routing policy.** The Routing intent loses its optimization goal too. What a
+   user is willing to spend is a Routing profile choice, not a property of the
+   work; leaving the field in the intent while the resolver ignores it would
+   leave a dead concept in the domain language. The glossary entry and the
+   intent schema both drop it.
+2. **`unreachable` and `missingInfrastructure` are kept.** They are unrelated to
+   the dial, and the resolver's fallback semantics depend on them.
+3. **Transport authorization belongs to the Routing profile.** Selecting Claude
+   Code and Codex does not authorize either to drive the other's CLI. ADR-0006
+   is already explicit that a detected transport is not an approved one; this
+   ADR names where the approval is stored — the global profile document, as its
+   own interview answer, narrowable per project by intersection only.
+4. **The Routing policy is derived, not stored.** The profile is the stored
+   personal choice and carries no revision. The policy is the constraint object
+   derived from profile plus inventory for one dispatch, and it carries the
+   composed revision so a Dispatch receipt can prove which constraints applied.
+5. **Cost is displayed and used as a tiebreak inside a cohort of identical
+   currency and unit — never as the ranking motor.** This is what the Cost dial
+   was reaching for, at the only altitude where it is honest.
+6. **The Kit ships no opinion about which models are good.** Excluding a model
+   is one user's preference under one subscription, not a default truth. The
+   inventory stays unfiltered per ADR-0006; the roster authorizes.
+
+## `inherit` is retained, and the condition that would have removed it
+
+This ADR was drafted to remove `inherit` entirely if verify-spike 18c (#296)
+could not prove that the session-default model-and-effort pair is readable back
+and identifiable against the roster. **The spike came back positive, so
+`inherit` is retained** in the constrained form: inheritance is permitted only
+when the session-default pair is attested *and* inside the effective roster,
+and otherwise blocks.
+
+The condition under which `inherit` is removed entirely therefore stands as a
+standing test, not a settled negative: **if the session-default pair ceases to
+be readable back on a surface, `inherit` is removed for that surface rather
+than left as an unprovable path.** A receipt that cannot name the pair it
+inherited is not proof of anything.
+
+The spike qualifies the retention with an identification requirement. One and
+the same Claude Code session reports its default model under three different
+identifiers — `opus[1m]` in `~/.claude/settings.json`, `claude-opus-5[1m]` in
+the session's init event, and `claude-opus-5` in the server-returned
+`message.model`. The attested channel is the one that drops the context
+variant. Roster identification therefore requires an explicit normalization
+rule and a decision on whether the context variant is part of pair identity;
+until that rule exists, "identified against the roster" is not a check a
+receipt may claim to have made.
+
+## Consequences
+
+- ADR-0006's Routing-policy clause (item 3, "optimization goals and optional
+  advanced overrides") and its closing sentence "Model preferences and
+  optimization overrides remain optional advanced settings" no longer describe
+  the system. The rest of ADR-0006 — the separation of Evidence catalog, Access
+  graph, and Routing policy; detection is not authorization; the surface
+  adapter capability declaration; the Dispatch receipt requirement — stands
+  unchanged and remains the foundation this ADR builds on.
+- `advanced.optimization` is removed from the Routing profile schema, and
+  `optimization` from the Routing policy schema, each with a deterministic
+  version-aware decoder that records the drop rather than silently discarding a
+  choice the user made.
+- Setup's Balanced / Quality / Cost prompt is replaced by roster admission and
+  Standard-route nomination.
+- Model-and-effort is pair identity. Effort domains are per model — some models
+  support an effort level others do not, and some carry no effort axis at all —
+  so a pair is authorized as a pair, never as a model with an effort attached
+  afterwards.

package/docs/agents/skills/orchestrate-wave.md

@@ -2,12 +2,165 @@
 <!-- agent-workflow-kit: project-extension/v1; skill=orchestrate-wave -->
 # Project layer — orchestrate-wave
 
-These stable sections are intentionally empty until the kit has proven its own landing recipe. The skill uses its generic fallback meanwhile.
+Filled 2026-07-27 after Welle 18 (#289) ran the whole loop end-to-end — recon,
+build, integrate, verify, land, tag, publish — and proved the recipe below. The
+sections were deliberately empty until then; the generic fallback carried that
+run and cost it a `degraded` start plus a hand-derivation of every command here.
+
+This repo has **no database, no dev server, no typecheck step and no browser
+surface**, so several generic phases collapse to very little. Where that is the
+case the section says so rather than inventing a command.
 
 ## §Setup
+
+Nothing to start. No DB, no tunnel, no service, no dev server — Phase 0's
+"start what live-verify needs" is a no-op here.
+
+Two things that are *not* no-ops:
+
+- `git config core.hooksPath .githooks` once per clone (worktrees inherit it).
+- `npm install` in the wave worktree after fast-forwarding to `origin/main` —
+  the lockfile is untracked, so dependencies do not travel with the branch.
+
 ## §Builder Commands
+
+Per-slice gate, run by the implementer before reporting back:
+
+```sh
+npm test                  # test:node (node --test) + test:python (unittest)
+```
+
+The fast pre-commit lints run automatically via `.githooks/pre-commit`
+(~3 s, skill/manifest lints). A slice is not "green" on the fast lints alone.
+
 ## §Builder Hard Rules
+
+- **Never `--no-verify`.** pre-commit and pre-push are the backstop, not an
+  obstacle.
+- **English-first prose** in every published skill; the language census in the
+  maintainer suite enforces it.
+- **No hardcoded board values** — labels, headings, field IDs and status names
+  come from `docs/agents/board-sync.md`; the portability lint blocks literals.
+- **Dual-surface mirror in the same PR.** A changed `.claude/skills/<s>/SKILL.md`
+  needs its `.agents/skills/<s>/SKILL.md` mirror via `codex-adapter-sync` in the
+  same PR, never after merge.
+- **Shipped-file changes must preserve the manifest/reconcile contract.**
+
 ## §Integration Suites
+
+Both frameworks, every time — `npm test` runs both and a red in either is a red:
+
+```sh
+npm test
+```
+
+- `test:node` — `node --test` (665 tests as of 0.42.0)
+- `test:python` — `python3 -m unittest discover -s scripts -p 'test_*.py'`
+  (522 tests as of 0.42.0)
+
+There is no typecheck step in this repo; do not look for one. Several
+negative-path tests print `[FAIL] …` lines **by design** — only the runner's own
+summary and exit code decide red or green.
+
 ## §Verify Recipe
+
+Central verify is the `/local-ci` full gate, run from the wave worktree root.
+It mirrors the CI `test` job step for step:
+
+```sh
+git fetch origin main
+npm test
+npm run kit:staleness
+npm run release:guard -- --base "$(git merge-base origin/main HEAD)"
+npm pack --dry-run
+```
+
+- **No branch-name-derived guard exists here**, so the generic skill's warning
+  about an integrated verify branch blocking with no baseline does not apply.
+  No skip or override is needed.
+- **No browser, no headless login, no screenshots, no console check, no DB
+  compare, no design/brand check.** This repo ships text; "assert the AC as a
+  user-visible outcome" means reading the artefact the slice produced (an ADR,
+  a skill body, a script's output) and the tests that pin it.
+- **`release:guard` red with "shipped delta has no version bump" is not a
+  failure of the wave** — it means the wave touched the shipped consumer set and
+  the release metadata is missing. See §Landing.
+- **A flaky red is possible.** `scripts/codex-exec.test.mjs` contains
+  timing-sensitive tests that fail under CPU contention (#345). Re-run the single
+  test in isolation before treating it as a real red; a test that is green in
+  isolation and red under load is contention, not the wave.
+
 ## §Headless Login
+
+Not applicable. No application, no browser surface, no authenticated session.
+Never invent a login recipe for this repo.
+
 ## §Landing
+
+**One PR for the whole wave**, base `main`. `main` is protected by the
+`main protection` ruleset: a PR is required, the CI job **`test`** (job name, not
+the workflow name `CI`) must be green with the branch up to date, and there are
+no bypass actors — `gh pr merge --admin` does not help.
+
+- `Closes #<n>` on its own line per leaf; `Part of #<anchor>` for the anchor,
+  never `closes`.
+- Body via temp file + `--body-file`, and **run `gh` from the repo** (it resolves
+  the target repo from the cwd).
+
+**Anchor reconcile, on PR create and on merge:**
+
+```sh
+python3 scripts/board-sync.py anchor-sync <anchor#> --dry-run   # review first
+python3 scripts/board-sync.py anchor-sync <anchor#>
+```
+
+`anchor-sync` only works on a **promoted** anchor whose body carries a slice
+table between `<!-- slice-table:start -->` and `<!-- slice-table:end -->`. A
+stub-shaped anchor (a bullet list of slices, `wave-stub` label) makes it a no-op
+that exits 1 with "no slice table found — nothing to sync". That is a promotion
+gap, not a sync failure: install the table with the columns
+`| # | Status | Slice | Sub-Issue | Gate | closes/refs |`, drop the `wave-stub`
+label, then re-run. Do not hand-maintain the Status cells afterwards — they are
+regenerated from the board.
+
+**Completion, after merge:**
+
+```sh
+python3 scripts/board-sync.py add --issue <anchor#> --status-role done
+gh issue close <anchor#> --reason completed --comment "…"
+python3 scripts/board-sync.py item-of --issue <anchor#>   # re-read: status must be Done
+python3 scripts/board-sync.py parent-of <anchor#>         # Program-PRD?
+python3 scripts/board-sync.py program-sync <prd#>         # after the anchor is Done
+```
+
+Closing the issue does **not** move the board field — set the field, then re-read
+both.
+
+**New issues** go through `python3 scripts/board-sync.py create --title … --body-file …`,
+never a bare `gh issue create`.
+
+### Release lockstep
+
+There is no deploy and no migration, but there **is** a publication gate, and a
+wave that touches the shipped consumer set (anything in the kit manifest —
+including `docs/adr/*` and `docs/research/*`) drags it along.
+
+1. `npm run release:prepare -- --base origin/main` reports the delta and a
+   `recommendedBump` without writing anything.
+2. **The confirmed Semver is the one human gate.** Ask, unless an explicit
+   end-to-end mandate covers release preparation. That confirmed version then
+   authorizes metadata, merge, tag *and* publish — do not ask again after merge.
+3. `npm run release:prepare -- --version <x.y.z>`, commit the bump.
+4. Merge the PR. Merging **integrates only**; it cannot publish.
+5. Verify on canonical `main`: tip == merge commit, `package.json` version
+   matches, no existing tag. Then `git tag -a v<x.y.z> <main-tip> -m "…"` and
+   push. The annotated tag on the `main` tip is the sole publication intent.
+6. **A red release run does not prove nothing was published.** Read
+   `npm view @ikon85/agent-workflow-kit version` and `gh release view v<x.y.z>`
+   directly, plus `npm run release:status` (cache-bypassing), before reacting.
+7. **Recovery is `gh workflow run release.yml -f tag=v<x.y.z>` on the existing
+   tag** — the reconciler is idempotent. **Never recover by bumping the version.**
+
+The release workflow runs the full suite *after* the tag is pushed, so a flaky
+test strands an already-published tag. That is the recovery path above, not a
+reason to re-tag.

package/docs/research/agent-task-taxonomy-benchmark-coverage.md

@@ -0,0 +1,451 @@
+# Agent task taxonomy and benchmark coverage
+
+**Researched:** 2026-07-26
+**Question:** Which agent task classes do recognized benchmark owners actually
+distinguish, which of those classes carry evidence strong enough to pick a
+model **and** an effort level, and how does that generalize the Kit's existing
+frontend taxonomy?
+
+## Verdict
+
+Benchmark owners distinguish **more task classes than the Kit currently models,
+but far fewer domains**. Of the seven candidate classes in the brief, six are
+directly measured by an owner today; one (architecture/judgment work) exists
+only as a static knowledge quiz, not as an agent measurement.
+
+Three findings drive the proposed schema:
+
+1. **Effort identity is the scarce resource, not the task class.** Only four
+   sources publish a row that names model **and** reasoning effort **and**
+   harness: DeepSWE, the Terminal-Bench leaderboard, Artificial Analysis'
+   per-evaluation leaderboards, and Scale's SWE Atlas boards. Everything else
+   collapses at least one of the three.
+2. **Almost nobody reports per-domain scores.** Code Arena is the only source in
+   this survey that publishes a separate score per subject-matter domain. Every
+   other owner publishes one aggregate over a task mix whose composition is
+   documented but not scored separately. The Kit's `domain` segment is therefore
+   `general` for most of the taxonomy — that is a coverage fact, not a schema
+   defect.
+3. **No owner publishes cost per *completed* task.** Every cost figure found is
+   cost per *attempted* task (or per whole run). Cost per success is derivable
+   but is an inference the Kit must label as such.
+
+## 0. Method: what counts as decisive
+
+The repo already carries a per-source `decisive` boolean
+(`src/lib/frontendWorkloads.mjs`). This note evaluates it against a three-part
+test, because a single boolean turned out to hide the interesting failure mode:
+
+| Dimension | Question | Failure mode |
+|---|---|---|
+| **Triple match** | Does the owner measure this exact `workload:domain:axis`, or is it an aggregate the Kit is slicing? | Aggregate laundering |
+| **Configuration identity** | Does the published row name the reasoning effort? | Effort collapse |
+| **Harness identity** | Does the published row name the agent/scaffold and its version? | Harness collapse |
+
+**Decisive** = all three hold. **Diagnostic** = the owner measures something
+relevant but at least one dimension is collapsed, estimated, or stale.
+
+This is a stricter reading than the current boolean. Under it, two sources the
+repo currently marks `decisive: true` are decisive on the triple and on harness,
+but **not** on effort — see §4.
+
+## 1. Proposed taxonomy: `workload:domain:axis`
+
+The identity format is unchanged: three colon-free segments, exactly one axis
+per observation (`evidenceWorkloadIdentity` in `src/lib/routingIntent.mjs`).
+
+### 1.1 Containment of the existing frontend taxonomy
+
+The existing frontend vocabulary is a **strict subset**, unchanged:
+
+- workloads `frontend-greenfield`, `frontend-repository-repair` keep their
+  identifiers and their meaning (lifecycle × repository context);
+- all eight frontend domains (`general`, `reference-design`, `marketing`,
+  `analytics`, `product`, `game`, `simulation`, `editor`) keep their identifiers
+  and remain valid **only** for the two frontend workloads;
+- all five existing axes keep their identifiers. Four (`visual-preference`,
+  `visual-fidelity`, `accessibility`, `responsive`) stay frontend-scoped; one
+  (`functional`) is promoted to the general executable-verifier axis, which is
+  exactly how it is already used for `openhands-frontend`.
+
+Nothing in `FRONTEND_SOURCE_CLAIMS`, `classifyFrontendWorkload`, or
+`frontendEvidenceWorkload` needs a value change to fit under the general
+taxonomy. The generalization is additive.
+
+### 1.2 Workloads
+
+A workload is **what the agent is asked to change or produce**, defined by
+lifecycle × repository context — the same rule the frontend pair already
+follows.
+
+| Workload | Definition | Owner that measures it |
+|---|---|---|
+| `frontend-greenfield` | New UI from a prompt or prototype, isolated from a repo | [Code Arena WebDev](https://arena.ai/leaderboard/code/webdev) |
+| `frontend-repository-repair` | Fix/extend UI inside an established repo | [SWE-bench Multimodal](https://www.swebench.com/multimodal), [OpenHands Index](https://index.openhands.dev/) |
+| `repository-repair` | Issue → patch in an established non-frontend repo | [DeepSWE](https://deepswe.datacurve.ai/), [SWE-bench Verified](https://www.swebench.com/verified) |
+| `repository-comprehension` | Answer questions about an existing system without changing it | [SWE Atlas Codebase QnA](https://labs.scale.com/leaderboard/sweatlas-qna) |
+| `code-transformation` | Restructure code while preserving behavior (refactor, migration) | [SWE Atlas Refactoring](https://labs.scale.com/leaderboard/sweatlas-refactoring), [RefactorBench](https://arxiv.org/abs/2503.07832) |
+| `test-authoring` | Write tests that catch a stated regression | [SWE Atlas Test Writing](https://labs.scale.com/leaderboard/sweatlas-tw) |
+| `greenfield-application` | Build a new non-frontend app/library from a spec | [OpenHands Index](https://www.openhands.dev/blog/openhands-index) (commit0) |
+| `algorithmic-synthesis` | Self-contained competitive-programming style problems | [LiveCodeBench](https://livecodebench.github.io/) |
+| `terminal-operations` | End-to-end system/CLI work in a real environment | [Terminal-Bench](https://www.tbench.ai/leaderboard/terminal-bench/2.1) |
+| `tool-orchestration` | Multi-turn tool use with a simulated user under a written policy | [τ²-bench / τ³-bench](https://github.com/sierra-research/tau2-bench), [BFCL](https://gorilla.cs.berkeley.edu/leaderboard.html) |
+| `knowledge-deliverable` | Produce a professional artifact (doc, sheet, slides, diagram) | [GDPval](https://arxiv.org/pdf/2510.04374), [GDPval-AA v2](https://artificialanalysis.ai/evaluations/gdpval-aa) |
+| `architecture-reasoning` | Judge architectural trade-offs and system-level constraints | [SAKE](https://arxiv.org/abs/2606.29520) — **knowledge quiz only, not agentic** |
+| `long-horizon-autonomy` | Cross-suite: how long an unattended run stays correct | [METR Time Horizon 1.1](https://metr.org/blog/2026-1-29-time-horizon-1-1/) |
+| `long-context-operation` | Cross-suite: accuracy as a function of input length | [AA-LCR](https://artificialanalysis.ai/evaluations/artificial-analysis-long-context-reasoning), [NoLiMa](https://arxiv.org/abs/2502.05167) |
+
+The last two are deliberately **cross-cutting** workloads rather than axes: their
+owners run their own task suites, so an observation about them is not a slice of
+another workload's evidence.
+
+Mapping to the brief's candidate list:
+
+| Candidate class in the brief | Verified? | Maps to |
+|---|---|---|
+| mechanical / refactoring | **yes** | `code-transformation` (SWE Atlas Refactoring is live: [Scale, 2026-05-07](https://scale.com/blog/swe-atlas-complete)) |
+| algorithmics | **yes** | `algorithmic-synthesis` (LiveCodeBench scenarios) |
+| debugging / repository repair | **yes** | `repository-repair` |
+| frontend design | **yes** | `frontend-greenfield` (Code Arena domains) |
+| architecture / judgment work | **partly** | `architecture-reasoning` exists only as multiple choice (SAKE); `knowledge-deliverable` is the closest *agentic* judgment proxy |
+| long-horizon agentic work | **yes** | `long-horizon-autonomy` + `terminal-operations` |
+| tool-use / orchestration | **yes** | `tool-orchestration` |
+
+Two classes the brief did not name, but owners do distinguish and score
+separately, were added: `repository-comprehension` and `test-authoring` — both
+are separate SWE Atlas leaderboards, and `information gathering` / `software
+testing` are separate OpenHands Index categories.
+
+### 1.3 Domains
+
+A domain is a **subject-matter partition the owner scores separately**. The rule
+is deliberately strict: if the owner publishes only an aggregate, the domain is
+`general`, even when the owner documents a richer task mix.
+
+| Workload | Allowed domains | Basis |
+|---|---|---|
+| `frontend-greenfield`, `frontend-repository-repair` | the existing eight | Code Arena publishes per-domain boards derived from >250k clustered prompts ([category methodology](https://arena.ai/blog/new-categories-code-arena/)) |
+| `tool-orchestration` | `general`, `airline`, `retail`, `telecom`, `banking` | Sierra ships the domains separately ([tau2-bench](https://github.com/sierra-research/tau2-bench)); AA publishes [τ²-Bench Telecom](https://artificialanalysis.ai/evaluations/tau2-bench) and [τ³-Banking](https://artificialanalysis.ai/evaluations/tau3-banking) as separate boards |
+| everything else | `general` | owner publishes one aggregate |
+
+Documented-but-unscored partitions (record as metadata, **never** as a domain
+segment, or the Kit is laundering an aggregate):
+
+- DeepSWE: 113 tasks over 91 repos, TypeScript 35 / Go 34 / Python 34 /
+  JavaScript 5 / Rust 5 — one aggregate pass@1
+  ([DeepSWE methodology](https://deepswe.datacurve.ai/blog/deepswe)).
+- Terminal-Bench 2.x: software engineering, ML, security, data science, system
+  administration — one aggregate accuracy
+  ([tbench.ai](https://www.tbench.ai/), [arXiv 2601.11868](https://arxiv.org/abs/2601.11868)).
+- SWE Atlas Codebase QnA: architecture, root-cause analysis, code onboarding,
+  security, API integration — one aggregate resolve rate
+  ([Scale blog](https://scale.com/blog/swe-atlas-complete)).
+- GDPval: 44 occupations across 9 sectors; the AA leaderboard reports an
+  aggregate Elo ([GDPval-AA v2](https://artificialanalysis.ai/evaluations/gdpval-aa)).
+  Whether OpenAI's own release publishes per-occupation win rates is
+  **unverified** — `openai.com/index/gdpval/` returned HTTP 403 to automated
+  fetch on 2026-07-26; the paper is at
+  [arXiv 2510.04374](https://arxiv.org/pdf/2510.04374).
+- LiveCodeBench reports Easy/Medium/Hard columns. That is **difficulty, not
+  domain** — putting it in the `domain` segment would be schema abuse.
+
+### 1.4 Axes
+
+An axis is the **quality dimension measured**. Five exist; six are proposed.
+
+| Axis | Status | Meaning | Grader |
+|---|---|---|---|
+| `functional` | existing, generalized | executable verifier pass/fail | program |
+| `visual-fidelity` | existing, frontend-scoped | similarity to a reference rendering | metric / VLM |
+| `visual-preference` | existing, frontend-scoped | blinded human pairwise on rendered UI | human |
+| `accessibility` | existing, frontend-scoped | WCAG-style conformance | tool / human |
+| `responsive` | existing, frontend-scoped | correctness across a viewport matrix | program / VLM |
+| `rubric-quality` | **new** | structured rubric score where no single pass/fail exists | rubric + judge |
+| `answer-accuracy` | **new** | correctness of an answer about a system or document | judge / key |
+| `blind-preference` | **new** | blinded pairwise ranking of a non-visual deliverable | human **or** LLM judge — record which |
+| `policy-adherence` | **new** | compliance with a written domain policy during tool use | program |
+| `time-horizon` | **new** | human task length at a fixed success probability | fitted from runs |
+| `context-retention` | **new** | accuracy as a function of input length | program / judge |
+
+`visual-preference` is conceptually the frontend special case of
+`blind-preference`; it is kept as a distinct identifier for backward
+compatibility, and because its grader is always human whereas GDPval-AA v2 uses
+an LLM judge ([AA GDPval-AA](https://artificialanalysis.ai/evaluations/gdpval-aa))
+while OpenAI's own GDPval uses human expert graders
+([arXiv 2510.04374](https://arxiv.org/pdf/2510.04374)). An adapter that merges
+those two grader types into one axis is fabricating comparability.
+
+### 1.5 Cost is not an axis
+
+`validateObservation` in `src/lib/routingCatalog.mjs` already carries
+`cost.{amount,currency,unit}` on every observation. Cost must stay there. Making
+it an axis would produce identities like `repository-repair:general:cost` that
+have no owner behind them, and would let a cheap-but-failing configuration
+outrank a working one. See §6 for what `unit` may legitimately contain.
+
+## 2. Source table
+
+`Decisive` uses the §0 three-part test. `Freshness` is what was observable on
+2026-07-26; where an owner publishes no cadence statement, that is stated.
+
+| Source | Workloads claimed | Axes | Decisive? | Freshness / cadence | Machine-ingestible artifact |
+|---|---|---|---|---|---|
+| **DeepSWE v1.1** (DataCurve) | `repository-repair:general` | `functional` | **Yes** — `reasoning_effort`, `harness: mini-swe-agent`, `config`, `ci_method` all in the JSON | `generated_at` 2026-07-25T03:13Z, latest job finished 2026-07-25; **no published cadence** | [`/artifacts/v1.1/leaderboard-live.json`](https://deepswe.datacurve.ai/artifacts/v1.1/leaderboard-live.json) (HTTP 200, ~62 KB) |
+| **Terminal-Bench 2.1 leaderboard** (Laude Institute) | `terminal-operations:general` | `functional` | **Yes** — columns are Rank, Agent, Model, **Effort**, Accuracy ±CI, Date, Agent Org, Model Org, PR, Hacks, Cost | latest submissions 2026-07-11; **no cadence statement** | **None found.** `www.tbench.ai/api/leaderboard` → 404. Task registry only: [`registry.json`](https://raw.githubusercontent.com/harbor-framework/terminal-bench/main/registry.json). Run-log repos exist but are empty scaffolds ([terminal-bench-2-leaderboard](https://github.com/laude-institute/terminal-bench-2-leaderboard)) |
+| **Artificial Analysis — per-evaluation boards** | `terminal-operations`, `tool-orchestration`, `knowledge-deliverable`, `long-context-operation`, `algorithmic-synthesis` (all `:general`) | `functional`, `policy-adherence`, `blind-preference`, `answer-accuracy` | **Yes** for boards that expose effort-variant rows — verified on [Terminal-Bench v2.1](https://artificialanalysis.ai/evaluations/terminalbench-v2-1) ("GPT-5.6 Sol (xhigh)", "GPT-5.6 Terra (max)", harness named as *Terminus 2 in an e2b sandbox*) | Intelligence Index v4.1 current; Coding Agent Index v1.3 current with a May–Jul 2026 change history; **AA states no fixed refresh schedule** ([methodology](https://artificialanalysis.ai/methodology/intelligence-benchmarking)) | [Data API](https://artificialanalysis.ai/data-api/docs) — snake_case JSON; cost lives under `artificial_analysis_intelligence_index_cost.cost_per_task`. **There is no `costPerTaskUsd` field** (see §6). Attribution required at all tiers |
+| **Artificial Analysis — Coding Agent Index v1.3** | `repository-repair`, `terminal-operations`, `repository-comprehension` (`:general`) | `functional`, `answer-accuracy` | **No — effort collapse by design.** AA states: *"Unless otherwise specified, we use each agent's default reasoning settings so the benchmark reflects the default user experience"* | v1.3 current, history May–Jul 2026 | [coding-agents methodology](https://artificialanalysis.ai/methodology/coding-agents-benchmarking) |
+| **SWE Atlas** (Scale) — QnA / Test Writing / Refactoring | `repository-comprehension`, `test-authoring`, `code-transformation` (`:general`) | `answer-accuracy`, `rubric-quality` | **Yes on QnA and Test Writing** — rows read e.g. *"Opus 4.8 (Claude Code) xhigh"*; native scaffold named, effort in the label, resolve rate ± CI. **Refactoring: effort exposure unverified** — the rendered rows observed showed model + harness + `48.57±6.73` but no effort token | Refactoring board went live with [Scale's 2026-05-07 post](https://scale.com/blog/swe-atlas-complete); dataset repo pushed 2026-07-20; **no cadence statement** | Dataset + run config only, Apache-2.0: [scaleapi/SWE-Atlas](https://github.com/scaleapi/SWE-Atlas). **No results JSON found** |
+| **Code Arena WebDev** (Arena) | `frontend-greenfield:{8 domains}` | `visual-preference` | **Partly.** Triple ✓ (only source with real per-domain boards), harness ✓ (encoded in labels such as `-codex`), **effort ✗** — the board columns are Rank, Model, Organization, License, Score, Votes, Price ($/M), Context Length; effort appears only when a lab bakes it into the model label | live pairwise voting; page timestamp 2026-07-24, 477,155 votes observed on the WebDev board; changes tracked in the [leaderboard changelog](https://arena.ai/blog/leaderboard-changelog/) | **No leaderboard API found.** Ranking code is open ([arena-rank](https://github.com/lmarena/arena-rank)); raw votes are released as a HF dataset (`lmarena-ai/arena-human-preference-140k`) but that is votes, not the published board |
+| **OpenHands Index** | `repository-repair`, `greenfield-application`, `frontend-repository-repair`, `test-authoring`, `repository-comprehension` (`:general`) | `functional` | **No — effort collapse.** Harness ✓ (OpenHands Software Agent SDK), triple ✓, but no reasoning-effort dimension is reported | launched [2026-01-29](https://www.openhands.dev/blog/openhands-index); [3-months-out update 2026-05-11](https://www.openhands.dev/blog/openhands-index-3-months-out) states only *"looking to do some more frequent updates going forward"* | Board at [index.openhands.dev](https://index.openhands.dev/) (no export found); harness open-sourced at [OpenHands/benchmarks](https://github.com/OpenHands/benchmarks), pushed 2026-07-19 |
+| **SWE-bench** (Verified / Multimodal / Lite / Multilingual) | `repository-repair:general`, `frontend-repository-repair:general` | `functional` | **No.** Submissions are self-reported with heterogeneous scaffolds; the *Verified* badge is opt-in (maintainers re-run a random subset). `metadata.yaml` may carry scaffold/effort, but it is not a guaranteed, uniform column | [SWE-bench/experiments](https://github.com/SWE-bench/experiments) last commit **2026-03-29** — ~4 months stale | Per-submission `all_preds.jsonl`/`preds.json`, `metadata.yaml`, `logs/*/report.json`, `trajs/` in [SWE-bench/experiments](https://github.com/SWE-bench/experiments) |
+| **METR Time Horizon 1.1** | `long-horizon-autonomy:general` | `time-horizon` | **No — effort collapse.** Per-model p50/p80 horizons with bootstrapped CIs and a `scaffolds` list, but **no reasoning-effort field** in the entries inspected | TH1.1 published [2026-01-29](https://metr.org/blog/2026-1-29-time-horizon-1-1/); [time-horizons page](https://metr.org/time-horizons/) latest entry 2026-05-08, updated *"periodically whenever we have new measurements"*; [eval-analysis-public](https://github.com/METR/eval-analysis-public) pushed 2026-03-06 | [`benchmark_results_1_1.yaml`](https://metr.org/assets/benchmark_results_1_1.yaml) (HTTP 200, ~16 KB) — contains `p50_horizon_length`/`p80_horizon_length` with `ci_low`/`ci_high`, `average_score`, `release_date`, `scaffolds`, plus `doubling_time_in_days` |
+| **BFCL V4** (Berkeley/Gorilla) | `tool-orchestration:general` | `functional` | **No — effort collapse.** Cost is *"an estimate of the cost for the entire benchmark"*; no per-row effort | last updated **2026-04-12**, evaluated at a pinned commit; *"will be updated periodically"* — no cadence | [ShishirPatil/gorilla](https://github.com/ShishirPatil/gorilla) (repo pushed 2026-04-13); categories in [TEST_CATEGORIES.md](https://github.com/ShishirPatil/gorilla/blob/main/berkeley-function-call-leaderboard/TEST_CATEGORIES.md); [changelog](https://github.com/ShishirPatil/gorilla/blob/main/berkeley-function-call-leaderboard/CHANGELOG.md) |
+| **Aider polyglot** | `code-transformation:general` (instruction-following edit) | `functional` | **No — stale.** Effort *is* in the label (e.g. `gpt-5 (high)`) and cost is reported, but the data is ~10 months old | `polyglot_leaderboard.yml` last commit **2025-10-04** (repo itself pushed 2026-05-22) | [`polyglot_leaderboard.yml`](https://github.com/Aider-AI/aider/blob/main/aider/website/_data/polyglot_leaderboard.yml); also `refactor_leaderboard.yml`, `edit_leaderboard.yml` in the same directory |
+| **LiveCodeBench** | `algorithmic-synthesis:general` | `functional` | **No.** Board columns are Rank, Model, Pass@1, Easy, Medium, Hard — no effort, no harness, no cost | official repo [LiveCodeBench/LiveCodeBench](https://github.com/LiveCodeBench/LiveCodeBench) pushed **2025-07-16**; AA's replication uses a May 2023–May 2024 problem window | [HF org](https://huggingface.co/livecodebench/); no leaderboard JSON found |
+| **SWE-Lancer** (OpenAI) | `repository-repair:general`, `architecture-reasoning:general` (SWE Manager split) | `functional`, `answer-accuracy` | **No — frozen.** 1,400+ Upwork tasks worth $1M in real payouts; IC SWE graded by triple-verified end-to-end tests, manager tasks graded against the original hiring manager's choice | [openai/SWELancer-Benchmark](https://github.com/openai/SWELancer-Benchmark) **archived 2025-07-18**, redirected into `openai/preparedness`. No maintained leaderboard | [arXiv 2502.12115](https://arxiv.org/abs/2502.12115) |
+| **SAKE** | `architecture-reasoning:general` | `answer-accuracy` | **No — not agentic.** 2,154 expert-curated 4-option MCQs over 8 architectural categories and 4 context-length levels, 11 models, zero-/five-shot | static paper artifact, [arXiv 2606.29520](https://arxiv.org/abs/2606.29520) | open-sourced evaluation scripts + results per the paper; no live board |
+| **AA-LCR** | `long-context-operation:general` | `answer-accuracy` | **Partly** — 100 questions over 10k–100k-token documents, pass/fail via LLM judge, per-model cost/time/tokens per task. **Not segmented by context length in the published view** | part of Intelligence Index v4.1 (6% weight); no cadence statement | [AA-LCR board](https://artificialanalysis.ai/evaluations/artificial-analysis-long-context-reasoning) + Data API |
+| **NoLiMa** | `long-context-operation:general` | `context-retention` | **No.** 13 models, latent-association needle retrieval; 11 of 13 fall below 50% of their short-context baseline at 32K | static paper, [arXiv 2502.05167](https://arxiv.org/abs/2502.05167); [adobe-research/NoLiMa](https://github.com/adobe-research/NoLiMa) | repo |
+| **Chroma "Context Rot"** | `long-context-operation:general` | `context-retention` | **No.** 18 models, extended NIAH + LongMemEval (~113k tokens) + repeated-words | published **2025-07-14**, one-off report | [trychroma.com/research/context-rot](https://www.trychroma.com/research/context-rot) |
+| **RefactorBench** (Microsoft) | `code-transformation:general` | `functional` | **No — static.** 100 handcrafted multi-file refactors, 3 instruction specificities; agents solved 22% vs 87% for a time-limited human developer | ICLR 2025 paper, static | [arXiv 2503.07832](https://arxiv.org/abs/2503.07832), [microsoft/RefactorBench](https://github.com/microsoft/RefactorBench) |
+| **GDPval / GDPval-AA v2** | `knowledge-deliverable:general` | `blind-preference` | **AA board: yes on effort/cost; no on grader parity.** 220 tasks, 44 occupations, blind pairwise, Elo anchored to a human baseline of 1000, *"average cost per task (USD), broken down by input, cache hit, cache write, reasoning, and answer tokens"* plus average turns per task. OpenAI's own grading is human-expert; AA's is an LLM judge — the two are not the same axis instance | v2 current; no cadence statement | [GDPval-AA v2 board](https://artificialanalysis.ai/evaluations/gdpval-aa) + Data API; paper [arXiv 2510.04374](https://arxiv.org/pdf/2510.04374) |
+
+## 3. What owners actually distinguish (question a)
+
+Confirmed as **separately published task classes**, not inferred:
+
+- Scale splits the engineering loop into **three separate leaderboards** —
+  Codebase QnA (124 tasks), Test Writing (90), Refactoring (70) — and states the
+  design intent: *"it targets underrepresented but practically important task
+  categories, uses comprehensive category-specific evaluation protocols, and
+  adopts under-specified, agentic task formulations"*
+  ([arXiv 2605.08366](https://arxiv.org/abs/2605.08366)).
+- OpenHands splits into **five categories**: issue resolution, greenfield
+  development, frontend development, software testing, information gathering
+  ([launch post](https://www.openhands.dev/blog/openhands-index)).
+- LiveCodeBench splits into **four scenarios**: code generation, self-repair,
+  test output prediction, code execution, and states that *"model performances
+  are correlated across different scenarios"* yet relative rankings vary by task
+  type ([livecodebench.github.io](https://livecodebench.github.io/)).
+- SWE-bench splits by **repository population**, not by task type: Full, Lite,
+  Verified, Multimodal, Multilingual ([swebench.com](https://www.swebench.com/SWE-bench/)).
+  Multimodal is 517 visual-domain issues
+  ([multimodal](https://www.swebench.com/multimodal)).
+- SWE-Lancer splits **IC engineering vs managerial proposal selection**
+  ([arXiv 2502.12115](https://arxiv.org/abs/2502.12115)).
+- Code Arena splits by **subject-matter domain**, uniquely: Reference-Based
+  Design (~29% of prompts), Brand/Marketing, Data & Analytics, Consumer Product,
+  Gaming, Simulations (~15.3%), Content Creation & Editing Tools
+  ([category methodology](https://arena.ai/blog/new-categories-code-arena/)).
+- BFCL splits by **call shape and interaction mode**: `simple_{python,java,
+  javascript}`, `parallel`, `multiple`, `parallel_multiple`, `irrelevance`,
+  `live_irrelevance`, `live_relevance`, `multi_turn_{base,miss_func,miss_param,
+  long_context}`, `memory_{kv,vector,rec_sum}`, `web_search_{base,no_snippet}`,
+  `format_sensitivity` ([TEST_CATEGORIES.md](https://github.com/ShishirPatil/gorilla/blob/main/berkeley-function-call-leaderboard/TEST_CATEGORIES.md)).
+
+What owners **explicitly say they do not cover** — this is the most useful
+primary-source material in the survey:
+
+- DeepSWE: *"bug localization and refactoring are under-represented, even though
+  each is challenging in its own right"*, and *"Developers also don't use these
+  models through mini-swe-agent in practice; they use them inside more
+  sophisticated, model-native harnesses like Codex CLI, Claude Code, Cursor, and
+  Gemini CLI, none of which the current leaderboard directly reflects"*
+  ([limitations](https://deepswe.datacurve.ai/blog/deepswe)).
+- SAKE: *"their ability to reason about software architecture remains largely
+  unmeasured"* ([arXiv 2606.29520](https://arxiv.org/abs/2606.29520)).
+- Scale, on why model-plus-scaffold is the unit: native agents
+  *"(Claude Code, Codex CLI) perform 1.5-2x more exploration than generic
+  harnesses"* ([Scale blog](https://scale.com/blog/swe-atlas-complete)).
+- GDPval: automated grading reached only **66% agreement with human graders**,
+  which is why the primary metric stays human head-to-head
+  ([arXiv 2510.04374](https://arxiv.org/pdf/2510.04374)).
+
+## 4. Decisive vs diagnostic (question b)
+
+**Decisive today (all three dimensions intact):**
+
+| Identity | Source |
+|---|---|
+| `repository-repair:general:functional` | DeepSWE v1.1 JSON |
+| `terminal-operations:general:functional` | Terminal-Bench 2.1 board; AA Terminal-Bench v2.1 board |
+| `repository-comprehension:general:answer-accuracy` | SWE Atlas Codebase QnA |
+| `test-authoring:general:rubric-quality` | SWE Atlas Test Writing |
+| `knowledge-deliverable:general:blind-preference` | AA GDPval-AA v2 (LLM judge — record the grader) |
+| `tool-orchestration:{telecom,banking}:policy-adherence` | AA τ²-Telecom, AA τ³-Banking |
+
+**Diagnostic only, with the reason:**
+
+| Identity | Source | Collapsed dimension |
+|---|---|---|
+| `frontend-greenfield:{domain}:visual-preference` | Code Arena WebDev | **effort** — no effort column; only sometimes in the label |
+| `frontend-repository-repair:general:functional` | OpenHands Index | **effort** — not a reported dimension |
+| `code-transformation:general:rubric-quality` | SWE Atlas Refactoring | **effort** — unverified on the rendered rows |
+| `greenfield-application:general:functional` | OpenHands Index | **effort** |
+| `algorithmic-synthesis:general:functional` | LiveCodeBench | effort + harness + staleness (repo 2025-07) |
+| `long-horizon-autonomy:general:time-horizon` | METR TH1.1 | **effort** |
+| `long-context-operation:general:context-retention` | NoLiMa, Chroma | effort + harness + staleness |
+| `architecture-reasoning:general:answer-accuracy` | SAKE | **not agentic at all** — MCQ knowledge |
+| `repository-repair:general:functional` | SWE-bench Verified | heterogeneous self-reported scaffolds; staleness |
+
+**Direct consequence for `src/lib/frontendWorkloads.mjs`:** the two entries
+currently marked `decisive: true` (`code-arena-webdev`, `openhands-frontend`) are
+decisive on triple and harness but **not on effort**. Under the current single
+boolean, a resolver reading `decisive: true` may believe it can pick an effort
+level from those sources. It cannot. Recommendation (not applied here): replace
+the boolean with three flags — `measuresTriple`, `preservesEffort`,
+`preservesHarness` — so an effort route requires `preservesEffort`, and let the
+resolver fall back to a separate effort-curve source (DeepSWE, Terminal-Bench)
+with the cross-source inference explicitly marked. That is the same rule
+`docs/research/frontend-agent-benchmarks.md` §7.4 already states in prose; making
+it a schema field prevents it from being forgotten.
+
+## 5. Does a smaller model degrade on long runs / large context? (question c)
+
+**No primary source in this survey reports long-run or long-context degradation
+broken down by model tier or reasoning effort.** The intuition is widely held; it
+is not measured that way by any owner found.
+
+What *is* measured:
+
+- **METR** fits a logistic curve of success probability against *human* task
+  duration and reports the length at which the curve crosses 50% (and 80%). TH1.1
+  expanded the suite from 170 to 228 tasks and moved from Vivaria to Inspect;
+  8h+ tasks doubled from 14 to 31, but **only 5 of those 31 have measured human
+  baselines** — the rest are estimated
+  ([TH1.1](https://metr.org/blog/2026-1-29-time-horizon-1-1/)). CIs come from
+  bootstrapping over task families, tasks, and runs. The published YAML records
+  `scaffolds` per model but **no effort field** in the entries inspected
+  ([benchmark_results_1_1.yaml](https://metr.org/assets/benchmark_results_1_1.yaml)).
+  Reported doubling times: 187.8 days all-time, 128.7 days (CI 104.4–158.0) from
+  2023 on. **This is per model, not per tier.**
+- **Toby Ord's half-life model** ([arXiv 2505.05115](https://arxiv.org/abs/2505.05115))
+  explains the METR curve with *"a constant rate of failing during each minute"*,
+  giving each agent its own half-life. The abstract does **not** claim weaker
+  models have systematically shorter half-lives — that stratification is an
+  inference, not a result.
+- **NoLiMa** ([arXiv 2502.05167](https://arxiv.org/abs/2502.05167)): 11 of 13
+  models drop below 50% of their short-context baseline at 32K; even GPT-4o falls
+  from 99.3% to 69.7%. It compares 13 different models — **not size variants
+  within one family**.
+- **Chroma "Context Rot"** ([2025-07-14](https://www.trychroma.com/research/context-rot))
+  tested 18 models *including* three within-family size ladders (Qwen3-8B / 32B /
+  235B-A22B and GPT-4.1 / mini / nano). It is therefore the only source that
+  *could* have answered the tier question — and it explicitly does not. Its
+  size-specific remarks are behavioural quirks, not degradation curves:
+  *"We only observe non-attempts with Qwen3-8B, [which] make up 4.21% of tasks"*;
+  *"GPT-4.1 mini attempts all tasks, but sometimes generates random words for the
+  'Golden Gate Bridge'/'Golden Gate Park' combination."* Its headline finding is
+  tier-agnostic: *"model performance degrades as input length increases, often in
+  surprising and non-uniform ways."*
+- **AA-LCR** reports one aggregate over 10k–100k-token documents; the published
+  view is **not** segmented by context length
+  ([AA-LCR](https://artificialanalysis.ai/evaluations/artificial-analysis-long-context-reasoning)).
+
+**Verdict for the Kit:** "route a small model away from long unattended runs" is
+currently a **defensible heuristic with no decisive backing**. The closest real
+evidence is indirect and within-family effort, not tier: DeepSWE's
+`mean_agent_steps` / `mean_output_tokens` / `mean_duration_seconds` fields let
+the Kit observe that a low-effort configuration consumes a different trajectory
+shape at the same task set. Publishing that as a tier claim would be an
+inference. Mark it `unverified`.
+
+## 6. Cost per completed task (question d)
+
+**Verified correction to the brief: there is no `costPerTaskUsd` field.** The
+Artificial Analysis Data API is snake_case; the cost object is
+`artificial_analysis_intelligence_index_cost` containing `total_cost` and
+`cost_per_task` (itself nested into `total_cost`, `input_cost`, `reasoning_cost`,
+`answer_cost`) ([Data API docs](https://artificialanalysis.ai/data-api/docs)).
+
+**More importantly: every source found reports cost per *attempted* task, never
+cost per *completed* task.**
+
+| Source | Cost metric published | Denominator |
+|---|---|---|
+| Artificial Analysis | *"Cost per Task ... calculated by multiplying input, cached, and output token prices by tokens consumed across the workload, weighted by the relative weights of each benchmark ... then dividing by task count"* | **task count** — attempted |
+| AA Coding Agent Index | *"Cost to run: average pay per token API cost per task, based on provider token pricing rather than consumer plans"* ([methodology](https://artificialanalysis.ai/methodology/coding-agents-benchmarking)) | attempted |
+| AA GDPval-AA v2 | *"average cost per task (USD), broken down by input, cache hit, cache write, reasoning, and answer tokens"* | attempted |
+| DeepSWE | `mean_cost_usd`, `median_cost_usd` — per **attempt** | attempted |
+| Terminal-Bench 2.1 board | `Cost` in USD for the whole submission run (e.g. $552.67) | whole run |
+| BFCL V4 | *"an estimate of the cost for the entire benchmark, in USD"* | whole run |
+| Aider polyglot | total run cost, divisible by 225 exercises | attempted |
+| OpenHands Index | cost-accuracy curves per model; the [3-months-out post](https://www.openhands.dev/blog/openhands-index-3-months-out) gives only relative pricing, not absolute per-task figures | attempted |
+| SWE-Lancer | $1M in Upwork **payouts earned**, not spend | neither — it is revenue, not cost |
+
+Cost per completed task is `cost_per_attempt / pass_rate`. Both operands are
+published by DeepSWE and by AA, so the Kit **can** compute it — but it is a
+derived quantity and must be tagged as such, with the caveat that the two
+operands must come from the same row. Concretely: `cost.unit` in
+`routingCatalog.mjs` should carry `usd-per-attempt` (what owners publish) or
+`usd-per-run`, and any `usd-per-success` value must be flagged as Kit-derived.
+
+Also note the retry semantics differ and are not interchangeable: DeepSWE reports
+pass@1 over four repeated whole-benchmark runs with a 95% run-to-run interval
+(`ci_method: "95% run-to-run: SE across repeated whole-benchmark passes"`), AA
+runs 3 attempts per task, and AA retries API failures for reliability rather than
+as extra solution attempts.
+
+## 7. Gaps
+
+Task classes with **no decisive source**, and what would close each:
+
+1. **Architecture / judgment work — the largest gap.** SAKE is 2,154 multiple
+   choice questions, not an agent run; SWE-Lancer's manager split is the only
+   agentic proxy and its repo has been archived since 2025-07-18. *Needed:* an
+   owner-run board where an agent produces an architectural decision inside a
+   real repository and is graded against the decision a senior engineer actually
+   made, with effort and harness in the row. Until then the Kit's `judgment`
+   routing workload rests on **no decisive evidence at all** — its closest proxy
+   is `knowledge-deliverable` (GDPval-AA), which is not software architecture.
+2. **Mechanical / refactoring at effort granularity.** SWE Atlas Refactoring is
+   live and names the harness, but effort exposure on its rows is unverified;
+   RefactorBench is a static 2025 paper; Aider's refactor/polyglot YAMLs are ~10
+   months stale. *Needed:* one confirmation that the Refactoring board carries
+   the effort token its sibling boards do. This is the cheapest gap to close —
+   a single human look at the rendered board settles it.
+3. **Any per-domain evidence outside frontend.** Code Arena is the sole source
+   with per-domain boards. Terminal-Bench, DeepSWE, and SWE Atlas QnA all
+   document a category mix and publish one aggregate. *Needed:* per-category
+   score export from any of them; until then the Kit must not emit a non-`general`
+   domain for those workloads.
+4. **Tier- or effort-resolved long-horizon degradation.** §5. *Needed:* a source
+   that reports success against task duration *per effort level*. DeepSWE already
+   holds both halves (effort rows + per-run duration/step statistics) and is the
+   most plausible candidate to publish it.
+5. **Accessibility and responsiveness.** Unchanged from
+   `docs/research/frontend-agent-benchmarks.md`: no current, broad, model-comparative
+   board. `accessibility` and `responsive` remain axes with **zero** decisive
+   sources.
+6. **Cost per completed task.** No owner publishes it (§6). *Needed:* nothing
+   external — the Kit derives it and labels it derived.
+7. **Machine-ingestible artifacts for four otherwise-strong sources.**
+   Terminal-Bench (no results endpoint; `/api/leaderboard` → 404), SWE Atlas (only
+   dataset + run config), Code Arena (no board API), OpenHands Index (no export).
+   Only DeepSWE, Artificial Analysis, METR, Aider, and SWE-bench/experiments can
+   be ingested without scraping a rendered page. Three of those five are stale by
+   more than 3 months.
+8. **Cadence is undeclared almost everywhere.** Only two freshness statements
+   were found in the whole survey: METR's *"periodically whenever we have new
+   measurements"* and OpenHands' *"looking to do some more frequent updates going
+   forward"*. Every other owner publishes dated artifacts with no promise. The
+   catalog's `freshness.expiresAt` therefore has to be a **Kit-side policy
+   decision per source**, not a value read from the owner.
+
+## 8. Implication for the routing catalog
+
+Minimal changes implied by the above, in order of value:
+
+1. Split the `decisive` boolean into `measuresTriple` / `preservesEffort` /
+   `preservesHarness` (§4). An effort route requires `preservesEffort`; a
+   cross-source effort inference must be recorded as an inference.
+2. Constrain `domain` to `general` unless the owner publishes a separate score
+   for that domain (§1.3). Add a lint so a new adapter cannot introduce a domain
+   the owner does not score.
+3. Give `cost.unit` an explicit enum — `usd-per-attempt`, `usd-per-run`,
+   `usd-per-success-derived` — so a derived number is never mistaken for a
+   published one (§6).
+4. Treat `long-horizon-autonomy` and `long-context-operation` as first-class
+   workloads with their own suites, not as modifiers on other workloads (§1.2).
+5. Keep the tier-degradation heuristic in the policy layer as an explicitly
+   `unverified` prior, subject to override by local outcome telemetry (§5).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ikon85/agent-workflow-kit",
-  "version": "0.41.1",
+  "version": "0.42.1",
   "description": "Portable AI-agent workflow skills (plan → execute → land → learn) for Claude Code & Codex — grilling, TDD, diagnosis, two-axis code review, cross-model Codex review, design & domain-modeling, plus a skill router (ask-matt). npx init/update/diff/uninstall.",
   "type": "module",
   "bin": {