@bastani/atomic 0.9.19-alpha.6 → 0.9.19-alpha.8

package/docs/models/model-selection.md

@@ -51,7 +51,7 @@ These are selected candidates, not a replacement DeepSWE frontier. The [AA leade
 
 AA's [v4.3 announcement](https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v4-3) replaces 𝜏³-Banking with AutomationBench-AA and Terminal-Bench v2.1 with v4.0. Fable 5.1 max with default fallback scores 58% normalized Elo on AA-Briefcase and 63% on GDPval-AA v2, versus Astra max at 53% and 54%. These are transformed Elo displays, **not pass rates**. Astra xhigh scores 32% GDP.pdf All-pass and 60% Terminal-Bench v4.0; Astra max scores 31% and 59%. Use the [per-evaluation tables](/models/evals#per-evaluation-scores-for-catalog-models) and [task-type picker](/models/evals#pick-by-task-type), not an aggregate rank. A rounded lead does not establish statistical significance, security-review reliability or the best model for every role. Fable's default-fallback result is not evidence for an arbitrary no-fallback configuration.
 
-The separate [Coding Agent Index v1.4](https://artificialanalysis.ai/agents/coding-agents), retrieved 2026-09-08, still uses Terminal-Bench v2.1 alongside DeepSWE and SWE-Atlas-QnA. It reports Claude Code + Fable 5.1 max with fallback at **70**, **$9.18/task** and **24.0 minutes/task**, Opencode + Gemini 3.8 Flash high at **61**, **$2.04/task** and **11.9 minutes/task**, and Codex + Luna max at **57**, **$0.29/task** and **8.0 minutes/task**. These are named-agent runs, not Atomic or interchangeable base-model results. All fourteen rows are in [Evals](/models/evals#coding-agent-index-v14-is-a-different-comparison). Fable 5.1 remains absent from the separately dated Datacurve snapshot below.
+The separate [Coding Agent Index v1.4](https://artificialanalysis.ai/agents/coding-agents), retrieved 2026-09-08, still uses Terminal-Bench v2.1 alongside DeepSWE and SWE-Atlas-QnA. It reports Claude Code + Fable 5.1 max with fallback at **70**, **$9.18/task** and **24.0 minutes/task**, Opencode + Gemini 3.8 Flash high at **61**, **$2.04/task** and **11.9 minutes/task**, and Codex + Luna max at **57**, **$0.29/task** and **8.0 minutes/task**. These are named-agent runs, not Atomic or interchangeable base-model results. All fourteen rows are in [Evals](/models/evals#coding-agent-index-v1-4-is-a-different-comparison). Fable 5.1 remains absent from the separately dated Datacurve snapshot below.
 
 ## Recommendation chart
 
@@ -89,7 +89,7 @@ The current highest-effort-config Pareto frontier is **gemini-3.8-flash** (accur
 | gemini-3.1-pro-preview [high] | 12% | $2.14 | Historical — outside the default selection | Last published measurement, re-verified unchanged; the live page labels it `gemini-3.1-pro`; removed from all chains |
 
 <Note>
-DeepSWE values above use the v1.1 results published in the September 3, 2026 snapshot, including the August 21 pricing corrections for GPT-5.6 Sol and DeepSeek V4. Sol's cost reflects OpenAI's promotional input and output price cut through at least November 21, 2026. DeepSWE uses DeepSeek's peak rates; its off-peak rates are half as much. GPT-6 Astra's DeepSWE costs are expected-launch-pricing estimates rather than billed rates; that caveat does not describe the separately sourced AA costs above. `pass@1` is rounded as on the live leaderboard and confidence intervals are omitted here — but note that the top of the board is a cluster: the top three rows span less than a point unrounded, well inside DeepSWE's published run-to-run intervals, so read a one-row lead as a tie. The highest published thinking level is a measurement choice, not a production default, and DeepSWE's own data shows effort saturation: for GPT-6 Astra, Claude Fable 5, Grok 4.6, and Gemini 3.7 Flash the best-scoring configuration is *not* the highest one. That is why DeepSWE's default "Best" table view displays four rows this table does not: `gpt-6-astra [xhigh]` at 74% for $6.52 with 29 average steps, `claude-fable-5 [xhigh]` at 70% for $13.41, `grok-4.6 [medium]` at 67% for $3.45, and `gemini-3.7-flash [medium]` at 65% for $2.03. Seven measured configurations are retained here with their last published values because DeepSWE excludes them from its default model selection, not because they were withdrawn; each was re-verified unchanged against the September 3, 2026 artifact and can be re-enabled in the site's model picker: GPT-5.6 Terra, Grok 4.5, Muse Spark 1.1, GPT-5.4, Kimi K2.7 Code, Claude Sonnet 4.6, and Gemini 3.1 Pro Preview. Atomic now ships GPT-6 Astra through its built-in OpenAI, OpenAI Codex, and Amazon Bedrock catalogs. A benchmark row still does not prove that the current account has provider access, so run `workflow({ action: "models" })` or `--list-models` before pinning Astra or another catalog model. See the live page for intervals, output tokens, steps, lower-effort configurations, and later corrections.
+DeepSWE values above use the v1.1 results published in the September 3, 2026 snapshot, including the August 21 pricing corrections for GPT-5.6 Sol and DeepSeek V4. Sol's cost reflects OpenAI's promotional input and output price cut through at least November 21, 2026. DeepSWE uses DeepSeek's peak rates; its off-peak rates are half as much. GPT-6 Astra's DeepSWE costs are expected-launch-pricing estimates rather than billed rates; that caveat does not describe the separately sourced AA costs above. `pass@1` is rounded as on the live leaderboard and confidence intervals are omitted here — but note that the top of the board is a cluster: the top three rows span less than a point unrounded, well inside DeepSWE's published run-to-run intervals, so read a one-row lead as a tie. The highest published thinking level is a measurement choice, not a production default, and DeepSWE's own data shows effort saturation: for GPT-6 Astra, Claude Fable 5, Grok 4.6, and Gemini 3.7 Flash the best-scoring configuration is *not* the highest one. That is why DeepSWE's default "Best" table view displays four rows this table does not: `gpt-6-astra [xhigh]` at 74% for \$6.52 with 29 average steps, `claude-fable-5 [xhigh]` at 70% for \$13.41, `grok-4.6 [medium]` at 67% for \$3.45, and `gemini-3.7-flash [medium]` at 65% for \$2.03. Seven measured configurations are retained here with their last published values because DeepSWE excludes them from its default model selection, not because they were withdrawn; each was re-verified unchanged against the September 3, 2026 artifact and can be re-enabled in the site's model picker: GPT-5.6 Terra, Grok 4.5, Muse Spark 1.1, GPT-5.4, Kimi K2.7 Code, Claude Sonnet 4.6, and Gemini 3.1 Pro Preview. Atomic now ships GPT-6 Astra through its built-in OpenAI, OpenAI Codex, and Amazon Bedrock catalogs. A benchmark row still does not prove that the current account has provider access, so run `workflow({ action: "models" })` or `--list-models` before pinning Astra or another catalog model. See the live page for intervals, output tokens, steps, lower-effort configurations, and later corrections.
 </Note>
 
 <Note>
@@ -97,7 +97,7 @@ DeepSWE values above use the v1.1 results published in the September 3, 2026 sna
 
 What is source-backed for `claude-fable-5-1` today, from [Anthropic's model overview](https://platform.claude.com/docs/en/models/fable-5-1/overview): a 1M-token context window and 128K maximum output; adaptive thinking that is always on, with effort `low`, `medium`, `high`, `xhigh`, and `max` and an Anthropic default of `high`; a June 2026 knowledge cutoff; and $10 input, $50 output, $12.50 five-minute cache write, $20 one-hour cache write, and $0.25 cache read per million tokens. The cache read is a quarter of Fable 5's $1.00, which is the main pricing reason to prefer it for long agentic sessions that re-read a cached prefix. Non-default `temperature`, `top_p`, and `top_k` return a 400 on every request, so Atomic omits `temperature` for this model.
 
-Atomic generates Fable 5.1 for the providers it has a matching runtime integration for. At the time of writing that is Anthropic, GitHub Copilot, three Amazon Bedrock inference profiles (`anthropic.`, `global.`, and `us.`), OpenRouter, and the Vercel AI Gateway. GitHub Copilot includes Fable 5 and Fable 5.1 in its static catalog from models.dev metadata, but its authenticated picker still decides which models each account may select. A provider "latest" alias such as OpenRouter's `~anthropic/claude-fable-latest` may also route to Fable 5.1 without naming it. That set genuinely moves — opencode zen published the model and then withdrew it while this page was being written — so run `workflow({ action: "models" })` or `--list-models` for the current list rather than trusting this one. Published catalogs also list the model on Google Vertex, Google Vertex (Anthropic), Azure, and Azure Cognitive Services; Atomic has no Claude runtime integration for those providers and generates no entries for them, which is a current limitation rather than a roadmap commitment. What does *not* vary is the invariant that matters: **Atomic's preserved-thinking handling is scoped to `provider: "anthropic"` on the `anthropic-messages` API and applies to none of the other mirrors** — including GitHub Copilot and the Vercel AI Gateway, which ride `anthropic-messages` but are deliberately excluded. See [Preserved thinking and model switches](/models#preserved-thinking-and-model-switches).
+Atomic generates Fable 5.1 for the providers it has a matching runtime integration for. At the time of writing that is Anthropic, GitHub Copilot, three Amazon Bedrock inference profiles (`anthropic.`, `global.`, and `us.`), OpenRouter, and the Vercel AI Gateway. GitHub Copilot includes Fable 5 and Fable 5.1 in its static catalog from models.dev metadata, but its authenticated picker still decides which models each account may select. A provider "latest" alias such as OpenRouter's `~anthropic/claude-fable-latest` may also route to Fable 5.1 without naming it. That set genuinely moves — opencode zen published the model and then withdrew it while this page was being written — so run `workflow({ action: "models" })` or `--list-models` for the current list rather than trusting this one. Published catalogs also list the model on Google Vertex, Google Vertex (Anthropic), Azure, and Azure Cognitive Services; Atomic has no Claude runtime integration for those providers and generates no entries for them, which is a current limitation rather than a roadmap commitment. What does *not* vary is the invariant that matters: **Atomic's preserved-thinking handling is scoped to `provider: "anthropic"` on the `anthropic-messages` API and applies to none of the other mirrors** — including GitHub Copilot and the Vercel AI Gateway, which ride `anthropic-messages` but are deliberately excluded. See [Preserved thinking and model switches](/models/reference#preserved-thinking-and-model-switches).
 </Note>
 
 <Note>

package/docs/models/pareto-efficiency.md

@@ -23,13 +23,13 @@ Three displayed highest-effort model configurations sit on the frontier, from th
 - **gpt-5.6-luna [max]**: 67% for $0.61 with 102 average steps. This is the cheapest broadly-capable point.
 - **gemini-3.8-flash [high]**: 74% for $2.36 with 166 average steps and 143k output tokens. This is the current accuracy ceiling, and also the step-heaviest point on the frontier — weigh that before making it a worker default.
 
-Under DeepSWE's default **Best** view, which selects each model's best-scoring configuration instead of its highest effort, these three points still hold and `gpt-6-astra [xhigh]` joins as a fourth member and the accuracy ceiling, at 74.12% unrounded for $6.52 with 29 average steps. That is a frontier position under that reading only: at its highest published effort (`max`, 73.23% for $12.37) GPT-6 Astra is dominated by both Gemini 3.8 Flash and Claude Opus 5. The same view also shows `claude-fable-5 [xhigh]` at 70% for $13.41, `grok-4.6 [medium]` at 67% for $3.45, and `gemini-3.7-flash [medium]` at 65% for $2.03, none of which reach the frontier.
+Under DeepSWE's default **Best** view, which selects each model's best-scoring configuration instead of its highest effort, these three points still hold and `gpt-6-astra [xhigh]` joins as a fourth member and the accuracy ceiling, at 74.12% unrounded for \$6.52 with 29 average steps. That is a frontier position under that reading only: at its highest published effort (`max`, 73.23% for \$12.37) GPT-6 Astra is dominated by both Gemini 3.8 Flash and Claude Opus 5. The same view also shows `claude-fable-5 [xhigh]` at 70% for \$13.41, `grok-4.6 [medium]` at 67% for \$3.45, and `gemini-3.7-flash [medium]` at 65% for \$2.03, none of which reach the frontier.
 
 ## What changed
 
 The September 3 snapshot collapses the frontier from five members to three:
 
-- **Gemini 3.8 Flash [high]**, added September 1, 2026, arrives at 74% for $2.36 and takes the accuracy ceiling. Rounded scores cannot settle the top of this board: Gemini 3.8 Flash and Claude Opus 5 both display 74%, and only the unrounded rates — 73.83% against 73.65% — order them. The dominance holds either way, because the cheaper model is also $9.48 less per task, about one fifth of Opus 5's cost.
+- **Gemini 3.8 Flash [high]**, added September 1, 2026, arrives at 74% for \$2.36 and takes the accuracy ceiling. Rounded scores cannot settle the top of this board: Gemini 3.8 Flash and Claude Opus 5 both display 74%, and only the unrounded rates — 73.83% against 73.65% — order them. The dominance holds either way, because the cheaper model is also \$9.48 less per task, about one fifth of Opus 5's cost.
 - **claude-opus-5 [max]**, **gpt-5.6-sol [max]**, and **glm-5.3 [max]** leave the frontier. None of their numbers moved; a cheaper and more accurate point simply appeared above all three.
 - **GPT-6 Astra**, added September 3, 2026 across low, medium, high, xhigh, and max effort, does not join at its highest published effort: `[max]` scores 73% for $12.37 and is dominated by both Gemini 3.8 Flash and Claude Opus 5. DeepSWE priced it at the expected launch rate card, so treat every Astra dollar figure as projected rather than billed.
 - **glm-5.3-flash [max]** and **gpt-5.6-luna [max]** are unchanged and keep the budget end of the frontier.