@bastani/atomic 0.9.19-alpha.1 → 0.9.19-alpha.10

package/docs/models/model-selection.md

@@ -17,12 +17,20 @@ It is a **static reference**. It does not change runtime model routing — routi
 The table below is a snapshot of the [DeepSWE](https://deepswe.datacurve.ai/) leaderboard (v1.1, highest published thinking level per model), a long-horizon coding-agent benchmark reporting `pass@1` and average dollars per task. The source reports 113 tasks and was updated September 3, 2026. DeepSWE's own default table view is **Best** — the best-scoring configuration per model — so four models show a different row there than the highest-effort row used here; the snapshot note below the table names them. Benchmarks and pricing drift and new models ship constantly, so **treat the live leaderboards as authoritative** and refresh this page from them rather than hand-maintaining scores. See [Evals](/models/evals). **Last compiled: 2026-09-03.**
 </Note>
 
-Artificial Analysis was reviewed separately on **2026-09-05** against its September 4 Intelligence Index v4.2 revision. The DeepSWE table remains the September 3 snapshot; an independent September 5 browser check confirmed its update date and Gemini 3.8 Flash row, not every configuration.
+Artificial Analysis was reviewed separately on **2026-09-08** against its **September 7 Intelligence Index v4.3** revision. The DeepSWE table remains the September 3 snapshot and was not revalidated in this refresh; the earlier September 5 browser check confirmed its update date and Gemini 3.8 Flash row, not every configuration.
+
+## Answering model-choice questions
+
+For any task-specific model recommendation, not just workflow authoring, read the [task-type picker](/models/evals#pick-by-task-type) and its per-benchmark charts, then consult [Artificial Analysis](https://artificialanalysis.ai/) for the relevant live evaluation and methodology. Recommend a task-fit candidate and explain the cost or latency tradeoff. Cite the benchmark version, retrieval date and exact model/effort configuration. An aggregate leaderboard winner is not a winner on every task.
+
+If live results cannot be retrieved, use the dated docs snapshot and say it was not refreshed. Check the configured catalog before suggesting an exact provider/model ID or effort level; catalog presence does not prove live access. Keep benchmark measurement settings separate from production effort recommendations.
 
 ## Benchmark levels are measurement settings
 
 The thinking level in brackets in the chart is the **measurement configuration used for that benchmark result**, not a universal workflow default. A score measured at `max` does not mean every stage using that model should use `max`; benchmark model identity and production thinking effort are separate choices. When authoring a workflow, choose effort from the stage role and cost of being wrong, then check the returned `availableThinkingLevels` for the configured catalog model.
 
+Practical default: use `low` or `medium` for coding, and `high` or `xhigh` for code review, test design and failure analysis, subject to catalog support. Run actual tests as tool calls. `max` is usually overkill and is not preferred in practice. These are starting recommendations to validate on your workflow, not a claim that lower effort reproduces the benchmark scores below.
+
 ## Pin model identity
 
 When a workflow needs an exact model, call `workflow({ action: "models" })` and pin a returned `fullId`. Do not pin a
@@ -32,18 +40,18 @@ reports the ambiguity. Use `--provider <provider> --model <id>` or `--model <pro
 
 ## AA cross-check for current candidates
 
-These are selected candidates, not a replacement DeepSWE frontier. The linked model pages and [rendered AA highlights](https://artificialanalysis.ai/) were retrieved **2026-09-05** under Intelligence Index **v4.2**, announced **2026-09-04**. Model-page metrics have no separate visible update timestamp. Scores are index points, not pass percentages; cost is weighted USD per **AA Intelligence Index task**, not DeepSWE cost. Speed is the model-page summary in standardized output tokens per second, not full-task latency.
+These are selected candidates, not a replacement DeepSWE frontier. The [AA leaderboard](https://artificialanalysis.ai/leaderboards/models) was retrieved **2026-09-08** under Intelligence Index **v4.3**, announced **2026-09-07**. Scores are index points, not pass percentages; cost is weighted USD per **AA Intelligence Index task**, not DeepSWE cost. Speed uses the default 10k-input workload in standardized output tokens per second, not full-task latency. The source has no separate publication timestamp for these measurements.
 
 | Model and AA measurement configuration | Intelligence Index | AA $/task | Output tokens/s | Candidate role and tradeoff |
 | --- | --- | --- | --- | --- |
-| [Claude Fable 5.1, Adaptive Reasoning, Max Effort, Default Fallback](https://artificialanalysis.ai/models/claude-fable-5-1) | 57 | $6.12 | 68.7 | Quality-first planning and judgment candidate; leads AA's displayed intelligence highlights, but has the highest task cost of this shortlist |
-| [GPT-6 Astra, max](https://artificialanalysis.ai/models/gpt-6-astra) | 55 | $2.57 | 87.5 | High-stakes planning and document-reasoning candidate with lower AA task cost than Fable 5.1 |
-| [Gemini 3.8 Flash, high](https://artificialanalysis.ai/models/gemini-3-8-flash) | 47 | $0.74 | Not reported | Cost-conscious coding candidate with strong separate DeepSWE evidence; do not invent API speed from its token price |
-| [GPT-5.6 Luna, max](https://artificialanalysis.ai/models/gpt-5-6-luna) | 43 | $0.10 | 134.8 | Budget research and worker candidate when its lower measured quality is acceptable |
+| [Claude Fable 5.1, max with default fallback](https://artificialanalysis.ai/models/claude-fable-5-1) | 53 | $7.63 | 69 | Knowledge-work planning candidate; xhigh also displays 53 at $5.98 per task |
+| [GPT-6 Astra, max](https://artificialanalysis.ai/models/gpt-6-astra) | 53 | $3.26 | 62 | Terminal and document-reasoning candidate; xhigh displays 53 at $2.31 and scores higher on those two individual evaluations |
+| [Gemini 3.8 Flash, high](https://artificialanalysis.ai/models/gemini-3-8-flash) | 41 | $1.24 | 286 | Strong historical Datacurve result, but only 20% on the new Terminal-Bench v4.0; check the intended task distribution |
+| [GPT-5.6 Luna, max](https://artificialanalysis.ai/models/gpt-5-6-luna) | 38 | $0.18 | 121 | Budget long-context candidate with verification; 12% on Terminal-Bench v4.0 |
 
-AA's [v4.2 announcement](https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v4-2) places Fable 5.1 and Opus 5 ahead on AA-Briefcase, while Astra leads GDP.pdf. The measured numbers behind that: Fable 5.1 max and Opus 5 max score 58% on AA-Briefcase and 63% and 62% on GDPval-AA v2, against 53% and 54% for Astra max; Astra max scores 33% on GDP.pdf against 26% for Fable 5.1 max and 22% for Opus 5 max. Use those task-specific signals for knowledge-work planning versus document research; the full per-evaluation tables for every catalog model are in [Evals](/models/evals#per-evaluation-scores-for-catalog-models), with a [task-type picker](/models/evals#pick-by-task-type) at the top of that page. An aggregate lead does not establish security-review reliability or the best model for every role. Keep Fable's **Default Fallback** qualifier: this is not evidence for an arbitrary no-fallback configuration.
+AA's [v4.3 announcement](https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v4-3) replaces 𝜏³-Banking with AutomationBench-AA and Terminal-Bench v2.1 with v4.0. Fable 5.1 max with default fallback scores 58% normalized Elo on AA-Briefcase and 63% on GDPval-AA v2, versus Astra max at 53% and 54%. These are transformed Elo displays, **not pass rates**. Astra xhigh scores 32% GDP.pdf All-pass and 60% Terminal-Bench v4.0; Astra max scores 31% and 59%. Use the [per-evaluation tables](/models/evals#per-evaluation-scores-for-catalog-models) and [task-type picker](/models/evals#pick-by-task-type), not an aggregate rank. A rounded lead does not establish statistical significance, security-review reliability or the best model for every role. Fable's default-fallback result is not evidence for an arbitrary no-fallback configuration.
 
-The separate [Coding Agent Index v1.4](https://artificialanalysis.ai/agents/coding-agents), also retrieved 2026-09-05, reports **Claude Code - Fable 5.1 (max) (with fallback)** at **70**, **$9.18/task** and **24.0 minutes agent wall time/task**, versus **Opencode - Gemini 3.8 Flash (high)** at **61**, **$2.04/task** and **11.9 minutes**, and **Codex - GPT-5.6 Luna (max)** at **57**, **$0.29/task** and **8.0 minutes**. Those agent-specific measurements illustrate a quality/cost/time tradeoff; neither is an Atomic result or a base-model score. Fable 5.1 is measured there even though it is absent from the Datacurve snapshot below. All fourteen rows, with their DeepSWE, Terminal-Bench and SWE-Atlas-QnA components, are tabulated in [Evals](/models/evals#coding-agent-index-v14-is-a-different-comparison); read the methodology and units there before combining evidence.
+The separate [Coding Agent Index v1.4](https://artificialanalysis.ai/agents/coding-agents), retrieved 2026-09-08, still uses Terminal-Bench v2.1 alongside DeepSWE and SWE-Atlas-QnA. It reports Claude Code + Fable 5.1 max with fallback at **70**, **$9.18/task** and **24.0 minutes/task**, Opencode + Gemini 3.8 Flash high at **61**, **$2.04/task** and **11.9 minutes/task**, and Codex + Luna max at **57**, **$0.29/task** and **8.0 minutes/task**. These are named-agent runs, not Atomic or interchangeable base-model results. All fourteen rows are in [Evals](/models/evals#coding-agent-index-v1-4-is-a-different-comparison). Fable 5.1 remains absent from the separately dated Datacurve snapshot below.
 
 ## Recommendation chart
 
@@ -81,7 +89,7 @@ The current highest-effort-config Pareto frontier is **gemini-3.8-flash** (accur
 | gemini-3.1-pro-preview [high] | 12% | $2.14 | Historical — outside the default selection | Last published measurement, re-verified unchanged; the live page labels it `gemini-3.1-pro`; removed from all chains |
 
 <Note>
-DeepSWE values above use the v1.1 results published in the September 3, 2026 snapshot, including the August 21 pricing corrections for GPT-5.6 Sol and DeepSeek V4. Sol's cost reflects OpenAI's promotional input and output price cut through at least November 21, 2026. DeepSWE uses DeepSeek's peak rates; its off-peak rates are half as much. GPT-6 Astra's DeepSWE costs are expected-launch-pricing estimates rather than billed rates; that caveat does not describe the separately sourced AA costs above. `pass@1` is rounded as on the live leaderboard and confidence intervals are omitted here — but note that the top of the board is a cluster: the top three rows span less than a point unrounded, well inside DeepSWE's published run-to-run intervals, so read a one-row lead as a tie. The highest published thinking level is a measurement choice, not a production default, and DeepSWE's own data shows effort saturation: for GPT-6 Astra, Claude Fable 5, Grok 4.6, and Gemini 3.7 Flash the best-scoring configuration is *not* the highest one. That is why DeepSWE's default "Best" table view displays four rows this table does not: `gpt-6-astra [xhigh]` at 74% for $6.52 with 29 average steps, `claude-fable-5 [xhigh]` at 70% for $13.41, `grok-4.6 [medium]` at 67% for $3.45, and `gemini-3.7-flash [medium]` at 65% for $2.03. Seven measured configurations are retained here with their last published values because DeepSWE excludes them from its default model selection, not because they were withdrawn; each was re-verified unchanged against the September 3, 2026 artifact and can be re-enabled in the site's model picker: GPT-5.6 Terra, Grok 4.5, Muse Spark 1.1, GPT-5.4, Kimi K2.7 Code, Claude Sonnet 4.6, and Gemini 3.1 Pro Preview. Atomic now ships GPT-6 Astra through its built-in OpenAI, OpenAI Codex, and Amazon Bedrock catalogs. A benchmark row still does not prove that the current account has provider access, so run `workflow({ action: "models" })` or `--list-models` before pinning Astra or another catalog model. See the live page for intervals, output tokens, steps, lower-effort configurations, and later corrections.
+DeepSWE values above use the v1.1 results published in the September 3, 2026 snapshot, including the August 21 pricing corrections for GPT-5.6 Sol and DeepSeek V4. Sol's cost reflects OpenAI's promotional input and output price cut through at least November 21, 2026. DeepSWE uses DeepSeek's peak rates; its off-peak rates are half as much. GPT-6 Astra's DeepSWE costs are expected-launch-pricing estimates rather than billed rates; that caveat does not describe the separately sourced AA costs above. `pass@1` is rounded as on the live leaderboard and confidence intervals are omitted here — but note that the top of the board is a cluster: the top three rows span less than a point unrounded, well inside DeepSWE's published run-to-run intervals, so read a one-row lead as a tie. The highest published thinking level is a measurement choice, not a production default, and DeepSWE's own data shows effort saturation: for GPT-6 Astra, Claude Fable 5, Grok 4.6, and Gemini 3.7 Flash the best-scoring configuration is *not* the highest one. That is why DeepSWE's default "Best" table view displays four rows this table does not: `gpt-6-astra [xhigh]` at 74% for \$6.52 with 29 average steps, `claude-fable-5 [xhigh]` at 70% for \$13.41, `grok-4.6 [medium]` at 67% for \$3.45, and `gemini-3.7-flash [medium]` at 65% for \$2.03. Seven measured configurations are retained here with their last published values because DeepSWE excludes them from its default model selection, not because they were withdrawn; each was re-verified unchanged against the September 3, 2026 artifact and can be re-enabled in the site's model picker: GPT-5.6 Terra, Grok 4.5, Muse Spark 1.1, GPT-5.4, Kimi K2.7 Code, Claude Sonnet 4.6, and Gemini 3.1 Pro Preview. Atomic now ships GPT-6 Astra through its built-in OpenAI, OpenAI Codex, and Amazon Bedrock catalogs. A benchmark row still does not prove that the current account has provider access, so run `workflow({ action: "models" })` or `--list-models` before pinning Astra or another catalog model. See the live page for intervals, output tokens, steps, lower-effort configurations, and later corrections.
 </Note>
 
 <Note>
@@ -89,7 +97,7 @@ DeepSWE values above use the v1.1 results published in the September 3, 2026 sna
 
 What is source-backed for `claude-fable-5-1` today, from [Anthropic's model overview](https://platform.claude.com/docs/en/models/fable-5-1/overview): a 1M-token context window and 128K maximum output; adaptive thinking that is always on, with effort `low`, `medium`, `high`, `xhigh`, and `max` and an Anthropic default of `high`; a June 2026 knowledge cutoff; and $10 input, $50 output, $12.50 five-minute cache write, $20 one-hour cache write, and $0.25 cache read per million tokens. The cache read is a quarter of Fable 5's $1.00, which is the main pricing reason to prefer it for long agentic sessions that re-read a cached prefix. Non-default `temperature`, `top_p`, and `top_k` return a 400 on every request, so Atomic omits `temperature` for this model.
 
-Atomic generates Fable 5.1 for the providers it has a matching runtime integration for. At the time of writing that is Anthropic, GitHub Copilot, three Amazon Bedrock inference profiles (`anthropic.`, `global.`, and `us.`), OpenRouter, and the Vercel AI Gateway. GitHub Copilot includes Fable 5 and Fable 5.1 in its static catalog from models.dev metadata, but its authenticated picker still decides which models each account may select. A provider "latest" alias such as OpenRouter's `~anthropic/claude-fable-latest` may also route to Fable 5.1 without naming it. That set genuinely moves — opencode zen published the model and then withdrew it while this page was being written — so run `workflow({ action: "models" })` or `--list-models` for the current list rather than trusting this one. Published catalogs also list the model on Google Vertex, Google Vertex (Anthropic), Azure, and Azure Cognitive Services; Atomic has no Claude runtime integration for those providers and generates no entries for them, which is a current limitation rather than a roadmap commitment. What does *not* vary is the invariant that matters: **Atomic's preserved-thinking handling is scoped to `provider: "anthropic"` on the `anthropic-messages` API and applies to none of the other mirrors** — including GitHub Copilot and the Vercel AI Gateway, which ride `anthropic-messages` but are deliberately excluded. See [Preserved thinking and model switches](/models#preserved-thinking-and-model-switches).
+Atomic generates Fable 5.1 for the providers it has a matching runtime integration for. At the time of writing that is Anthropic, GitHub Copilot, three Amazon Bedrock inference profiles (`anthropic.`, `global.`, and `us.`), OpenRouter, and the Vercel AI Gateway. GitHub Copilot includes Fable 5 and Fable 5.1 in its static catalog from models.dev metadata, but its authenticated picker still decides which models each account may select. A provider "latest" alias such as OpenRouter's `~anthropic/claude-fable-latest` may also route to Fable 5.1 without naming it. That set genuinely moves — opencode zen published the model and then withdrew it while this page was being written — so run `workflow({ action: "models" })` or `--list-models` for the current list rather than trusting this one. Published catalogs also list the model on Google Vertex, Google Vertex (Anthropic), Azure, and Azure Cognitive Services; Atomic has no Claude runtime integration for those providers and generates no entries for them, which is a current limitation rather than a roadmap commitment. What does *not* vary is the invariant that matters: **Atomic's preserved-thinking handling is scoped to `provider: "anthropic"` on the `anthropic-messages` API and applies to none of the other mirrors** — including GitHub Copilot and the Vercel AI Gateway, which ride `anthropic-messages` but are deliberately excluded. See [Preserved thinking and model switches](/models/reference#preserved-thinking-and-model-switches).
 </Note>
 
 <Note>
@@ -106,25 +114,26 @@ Use this table when the user has not requested a thinking level. It is a product
 
 | Stage role | Default thinking level | Why |
 | --- | --- | --- |
-| Security, identity, adversarial challenge, final approval | `max` | A wrong judgment can create a high-risk false approval or waste a full downstream loop. |
-| Codebase mapping, lifecycle analysis, compatibility, planning, synthesis, triage, repair | `high` | These stages must resolve demanding uncertainty and preserve evidence across handoffs; routine synthesis may use `medium` when evidence quality holds. |
+| Coding, implementation, routine fixes | `low` or `medium` | Keep implementation fast; use review and tests to catch defects. |
+| Code review, test design, failure analysis, security, identity, adversarial challenge, final approval | `high` or `xhigh` | Spend more reasoning on finding defects, probing edge cases and judging evidence. |
+| Codebase mapping, lifecycle analysis, compatibility, planning, synthesis, triage | `high` | These stages must resolve demanding uncertainty and preserve evidence across handoffs; routine synthesis may use `medium` when evidence quality holds. |
 | User-impact review and final reporting | `medium` | Clear evidence-backed summaries usually do not need the deepest reasoning. |
 | Deterministic checks | No model call | Run typechecks, tests, schema checks, runtime probes, and artifact checks as durable tool nodes. |
 
-Reserve `max` for a high-cost-of-error role or an explicit user request. An explicit request wins over this role default, but the requested level still must appear in the configured catalog; do not invent an unsupported suffix. For each primary and fallback, choose a level for the same stage role independently. A fallback is not a reason to inherit `max` mechanically: use the role default at a supported level, choose another catalog model when needed, or leave the stage unpinned rather than guessing.
+`max` is an exception, not a role default. Consider it only when task-specific evidence justifies the extra effort or the user explicitly requests it. An explicit request wins over these defaults, but the requested level still must appear in the configured catalog; do not invent an unsupported suffix. For each primary and fallback, choose a supported level for the same stage role independently. If `xhigh` is unavailable, use `high` rather than automatically promoting to `max`; choose another catalog model or leave the stage unpinned if neither is supported.
 
 ## Scenario-based guidance
 
-Pick by the cost of being wrong in each role, not by raw accuracy. Match the role to the benchmark that best measures it (see [Evals](/models/evals) and its [task-type picker](/models/evals#pick-by-task-type)). The percentages below are AA per-evaluation scores read on 2026-09-05 for the named configuration.
+Pick by the cost of being wrong in each role, not by raw accuracy. The AA evidence below was read on 2026-09-08; the Datacurve evidence remains the September 3 snapshot. See [Evals](/models/evals) for measurement settings, normalized Elo versus pass-rate units and source links.
 
-- **Reviewer / judgment gates** — use `max` when the reviewer makes a security, identity, adversarial, or final-approval decision whose wrong verdict discards an entire loop. `gemini-3.8-flash` is the DeepSWE accuracy ceiling and reaches it at a fraction of top-tier cost; `claude-opus-5` and `gpt-5.6-sol` are the near-peers when a different family is wanted. For code-reading judgments, the Anthropic rows separate: Claude Code + Fable 5.1 and + Opus 5 score 56% and 55% on SWE-Atlas-QnA against 51% for Codex + Astra and 38% for Opencode + Gemini 3.8 Flash. Use another family when decorrelated errors matter.
-- **Codebase mapping / planner** — start at `high` for repository mapping, lifecycle analysis, compatibility, and plans. Gemini 3.8 Flash is a cost-conscious coding candidate but near the bottom of the catalog on AA-Briefcase (35%; only Gemini 3.7 Flash is lower at 31%); for plans that resemble knowledge work, Fable 5.1 max and Opus 5 max (58%) or, on a budget, GLM-5.3-Flash (48% for $0.18 per AA task) measure better. Raise production effort only when the plan gates a high-cost loop or the user asks, and only to a level advertised by that exact catalog entry.
-- **Debugger / triage / repair** — start at `high`; deep reasoning pays off when root-causing or repairing is costly. Weight DeepSWE and Terminal-Bench together rather than treating either as a complete measure: Terminal-Bench v2.1 is flat at 88–91% across Fable 5.1 max, Astra at every effort, Sol max and xhigh, Opus 5 max through high, Terra max and Gemini 3.8 Flash high, so DeepSWE cost and steps decide among them.
-- **Research / synthesis** — use `high` for demanding research and evidence reconciliation; use `medium` for routine synthesis when the evidence is already strong. `gpt-5.6-luna` is the budget starting point for long-context work (84% on AA-LCR v1.1 for $0.10 per AA task), but it abstains only 7% of the time on AA-Omniscience, so back it with verification tool nodes; evaluate Astra (33% GDP.pdf All-pass, the leader) when document-reasoning failures justify more cost, and GLM-5.3-Flash or GLM-5.3 (72% and 70% non-hallucination) when a wrong fact is worse than an abstention. Weight AA-LCR v1.1, GDP.pdf and AA-Omniscience rather than general intelligence alone.
-- **Orchestrator / worker / cheap loops** — Luna offers the best broad cost/accuracy balance for code, but scores 31% on 𝜏³-Banking, the lowest catalog row, so prefer GLM-5.3-Flash (47%) or Muse Spark 1.3 (52%) when the loop is tool-call heavy. GLM-5.3 Flash is the cheapest live frontier point at 63% for $0.24 with 123 average steps. Gemini 3.8 Flash is the most accurate frontier point but averages 166 steps and 143k output tokens per task, which makes it a judgment-gate choice rather than an automatic worker default. DeepSeek V4 Pro and Flash are provider-diversity options, not budget-frontier choices.
-- **User-impact review / final reporting** — use `medium` for impact summaries and reports that preserve the evidence needed by the user. Do not spend `max` here unless the user explicitly requests it or the role has become a high-cost-of-error approval.
+- **Reviewer / judgment gates.** Use `high` or `xhigh` for code review and approval decisions, subject to the configured model's supported efforts. No external benchmark here establishes security-review reliability. Claude Code + Fable 5.1 max with fallback and Claude Code + Opus 5 xhigh score 56% and 55% on SWE-Atlas-QnA, versus 51% for Codex + Astra max, but those harness-specific results need validation in Atomic.
+- **Codebase mapping / planner.** Start at `high`. For knowledge-work deliverables, Fable 5.1 max with fallback scores 58% normalized Elo on AA-Briefcase, Opus 5 max 57%, and GLM-5.3-Flash 48% at $0.25 per Index task. These are candidates, not measured repository-planning pass rates. Raise effort only for the role's cost of error or the user's request.
+- **Debugger / triage / repair.** Use `high` or `xhigh` for failure analysis and test design, then `low` or `medium` to implement a diagnosed fix. The new Terminal-Bench v4.0 is no longer flat near 90%: Astra xhigh scores 60%, Fable 5.1 xhigh with fallback 55%, Opus 5 max 49%, and Gemini 3.8 Flash high 20%. Keep Datacurve cost and steps as separate evidence; do not transfer its 74% Gemini result into this benchmark.
+- **Research / synthesis.** Use `high` for demanding reconciliation and `medium` for routine synthesis. Luna max scores 84% on AA-LCR at $0.18 per Index task, but its 7% non-hallucination rate counts partial answers or not attempted among non-correct responses, not all answers. Verify factual claims. Astra xhigh leads the displayed GDP.pdf rows at 32%; GLM-5.3-Flash and GLM-5.3 max score 72% and 70% non-hallucination.
+- **Orchestrator / worker / cheap loops.** Use AutomationBench-AA for SaaS tool workflows: Astra max 68%, GLM-5.3-Flash 60%, Luna max 50%. GLM-5.3-Flash and Luna remain budget candidates on the separately dated Datacurve frontier. Gemini 3.8 Flash's 166 steps and 143k output tokens in that snapshot argue against choosing workers on pass rate alone. Validate the tradeoff on the actual workflow.
+- **User-impact review / final reporting.** Use `medium` for impact summaries and reports that preserve the evidence needed by the user. If the stage makes an approval decision, use the reviewer guidance instead.
 - **Design** — a quality-first domain not directly measured by these coding tables. Choose effort by the review or approval role. Fable 5.1's AA-Briefcase results make it a candidate for knowledge-work deliverables, not proof of product-design quality; evaluate it on the intended design tasks and do not carry Fable 5's DeepSWE row over to it.
-- **Interactive coding sessions** — use `high` for complex, multi-step coding and `medium` for routine edits; reserve `max` for a high-cost-of-error judgment or an explicit user request.
+- **Interactive coding sessions.** Use `low` or `medium` for implementation, switching to `high` or `xhigh` for code review, test design and failure analysis. Choose only levels supported by the configured model.
 - **Deterministic checks** — make typechecks, tests, schema validation, runtime probes, and artifact inspection tool nodes with no model call. Model self-report is not verification evidence.
 
 ## Related

package/docs/models/pareto-efficiency.md

@@ -13,7 +13,7 @@ The axes here are `pass@1` (accuracy) and `average dollars per task` (cost), tak
 Figures are a snapshot of DeepSWE v1.1 using the highest published thinking level for each of the 21 models displayed on the September 3, 2026 leaderboard. They include the August 21 pricing corrections for GPT-5.6 Sol and DeepSeek V4, and GPT-6 Astra's costs are DeepSWE's expected launch pricing rather than billed rates. DeepSWE's own default table view is **Best** — the best-scoring configuration per model — which picks a different row for four models; the frontier under that reading is stated below. DeepSWE publishes a live cost-vs-score scatter, so **read the frontier off the live chart** rather than trusting a static list. **Last compiled: 2026-09-03.**
 </Note>
 
-The [AA review](/models/evals) is separately dated **2026-09-05**. AA Intelligence Index v4.2 uses weighted intelligence and its own cost per task; AA Coding Agent Index v1.4 uses agent-specific runs across several benchmarks. Neither defines the Datacurve frontier on this page. A September 5 browser check confirmed the September 3 DeepSWE update and Gemini 3.8 Flash's measured row, without recomputing this entire snapshot.
+The [AA review](/models/evals) is separately dated **2026-09-08**. Intelligence Index v4.3, announced September 7, uses weighted intelligence and its own cost per task; Coding Agent Index v1.4 still uses agent-specific runs including Terminal-Bench v2.1 rather than v4.0. Neither defines the Datacurve frontier on this page. This refresh did not revalidate or recompute the September 3 DeepSWE snapshot. The earlier September 5 browser check confirmed its update date and Gemini 3.8 Flash's row.
 
 ## The frontier
 
@@ -23,13 +23,13 @@ Three displayed highest-effort model configurations sit on the frontier, from th
 - **gpt-5.6-luna [max]**: 67% for $0.61 with 102 average steps. This is the cheapest broadly-capable point.
 - **gemini-3.8-flash [high]**: 74% for $2.36 with 166 average steps and 143k output tokens. This is the current accuracy ceiling, and also the step-heaviest point on the frontier — weigh that before making it a worker default.
 
-Under DeepSWE's default **Best** view, which selects each model's best-scoring configuration instead of its highest effort, these three points still hold and `gpt-6-astra [xhigh]` joins as a fourth member and the accuracy ceiling, at 74.12% unrounded for $6.52 with 29 average steps. That is a frontier position under that reading only: at its highest published effort (`max`, 73.23% for $12.37) GPT-6 Astra is dominated by both Gemini 3.8 Flash and Claude Opus 5. The same view also shows `claude-fable-5 [xhigh]` at 70% for $13.41, `grok-4.6 [medium]` at 67% for $3.45, and `gemini-3.7-flash [medium]` at 65% for $2.03, none of which reach the frontier.
+Under DeepSWE's default **Best** view, which selects each model's best-scoring configuration instead of its highest effort, these three points still hold and `gpt-6-astra [xhigh]` joins as a fourth member and the accuracy ceiling, at 74.12% unrounded for \$6.52 with 29 average steps. That is a frontier position under that reading only: at its highest published effort (`max`, 73.23% for \$12.37) GPT-6 Astra is dominated by both Gemini 3.8 Flash and Claude Opus 5. The same view also shows `claude-fable-5 [xhigh]` at 70% for \$13.41, `grok-4.6 [medium]` at 67% for \$3.45, and `gemini-3.7-flash [medium]` at 65% for \$2.03, none of which reach the frontier.
 
 ## What changed
 
 The September 3 snapshot collapses the frontier from five members to three:
 
-- **Gemini 3.8 Flash [high]**, added September 1, 2026, arrives at 74% for $2.36 and takes the accuracy ceiling. Rounded scores cannot settle the top of this board: Gemini 3.8 Flash and Claude Opus 5 both display 74%, and only the unrounded rates — 73.83% against 73.65% — order them. The dominance holds either way, because the cheaper model is also $9.48 less per task, about one fifth of Opus 5's cost.
+- **Gemini 3.8 Flash [high]**, added September 1, 2026, arrives at 74% for \$2.36 and takes the accuracy ceiling. Rounded scores cannot settle the top of this board: Gemini 3.8 Flash and Claude Opus 5 both display 74%, and only the unrounded rates — 73.83% against 73.65% — order them. The dominance holds either way, because the cheaper model is also \$9.48 less per task, about one fifth of Opus 5's cost.
 - **claude-opus-5 [max]**, **gpt-5.6-sol [max]**, and **glm-5.3 [max]** leave the frontier. None of their numbers moved; a cheaper and more accurate point simply appeared above all three.
 - **GPT-6 Astra**, added September 3, 2026 across low, medium, high, xhigh, and max effort, does not join at its highest published effort: `[max]` scores 73% for $12.37 and is dominated by both Gemini 3.8 Flash and Claude Opus 5. DeepSWE priced it at the expected launch rate card, so treat every Astra dollar figure as projected rather than billed.
 - **glm-5.3-flash [max]** and **gpt-5.6-luna [max]** are unchanged and keep the budget end of the frontier.
@@ -75,7 +75,7 @@ Efficiency is not the only axis. A dominated model can still earn a slot when it
 - **glm-5.3 [max]** remains the best open-weights point; the new frontier ceiling is closed-weights, so the open-weights niche survives the frontier change intact.
 - **gemini-3.8-flash [high]** holds the frontier ceiling but is Google-family and step-heavy at 166 average steps; pair it with a model from another family for fallback diversity rather than routing every stage through one provider.
 - **claude-fable-5** remains useful where Anthropic-family behavior is specifically wanted, such as the quality-first, unbenchmarked design chain.
-- **claude-fable-5-1** is available in Atomic's catalog but is absent from the September 3, 2026 Datacurve snapshot, so it has no position on this frontier. It is measured by AA Intelligence Index and by AA with Claude Code, as detailed in the [September 5 AA cross-check](/models/model-selection#aa-cross-check-for-current-candidates). Those results justify evaluation for quality-first work, not importing an AA score or cost into this DeepSWE chart. Its cache pricing may matter for long sessions, but a token-price discount is not a measured cost-per-task saving or accuracy result.
+- **claude-fable-5-1** is available in Atomic's catalog but absent from the September 3, 2026 Datacurve snapshot, so it has no position on this frontier. The [September 8 AA cross-check](/models/model-selection#aa-cross-check-for-current-candidates) includes Fable 5.1's max and xhigh default-fallback measurements. Those results justify task-specific evaluation, not importing AA scores or costs into this DeepSWE chart. A token-price discount is not a measured task-cost saving.
 - **Unmeasured models** may remain operational defaults when a family lacks current DeepSWE or Artificial Analysis coverage, but they should not inherit a predecessor's score.
 
 ## How to use this